Software Tools Design

1
Software Tools Design
2
Architecture the design and and specification
of overall system structure

• Classical architecture
• Design Representations
• plan, elevation, perspective
• drawings/models
• Architectural Styles
• Victorian
• Colonial
• Spanish
• Requirements,Constraints
• circulation patterns
• acoustics
• lighting
• air flow

• Software architecture
• Design Representations
• multiple views
• models for different roles (customer, developer)
• Architectural Styles
• Client/server
• Layered
• Object-Oriented
• Requirements,Constraints
• reliability, fault tolerance
• performance
• maintainabililty

3
Software Architecture

• As size/complexity of software increases,
  design/spec of overall system structure becomes
  more significant than the choice of
  algorithms/data structures
• Currently not a well-defined terminology
  available for software arch.
• but good software engineers do use architectural
  principles
• Software architecture defines a system in terms
  of computational components and interactions
  among them.

4
Architectural Elements

• components define computation,data storage
• clients/servers, databases, filters,...
• connectors define the interactions between
  components
• procedure calls, pipes, event multicast,
• configurations collections of interacting
  components and connectors
• diagrams of boxes and arrows, formal mathematical
  models
• constraints provide semantic understanding of
  the style requirements
• what makes the style correct, legal, efficient,
  secure, reliable

5
Architectural Styles

• Dataflow systems
• batch sequential
• pipes and filters
• Call-and-return systems
• main program and subroutine
• OO systems
• Hierarchical layers
• Independent components
• communicating processes
• event systems

6
Architectural Styles, cont.

• Virtual machines
• interpreters
• rule-based systems
• Data-centered systems (repositories)
• databases
• hypertext systems
• blackboards

7
This class covers

• Pipes and Filters Unix shell
• Object-Oriented Java
• Client/Server
• Web Applications and XML, Javascript
• CGI programming with Perl
• Applets, Servlets

8
Architectural Style Questions

• What is the design vocabulary? (components,
  connectors?)
• What are the allowable structural patterns?
• What is the underlying computational model?
• What are the essential invariants of the style?
• What are some common examples of its use?
• What are the advantages/disadvantages of using
  the style?
• What are some common specializations?

9
Common Framework

• Use a common framework in order to compare
  different arch. styles
• framework collection of computational components
  (components), description of interactions
  (connectors), set of (constraints) on how they
  can be combined
• For each style
• structural pattern
• underlying computation model
• essential invariants
• common examples
• advantages/disadvantages
• common specializations

10
Pipes Filters

• Components
• filters (processing components) incrementally
  transform data (stream to stream) from a set of
  inputs to a set of outputs.
• Connectors
• pipes conduits for the data stream. transmit
  output of one filter to input of next filter
• Invariants
• filters (components) must be independent
  entities
• little context info is utilized, no state info
  preserved between instantiations
• filters do not know the identity of their
  upstream and downstream filters.
• Examples unix shell, compilers

11
Pipes Filters

• Advantages
• neat design support reuse easy to maintain
• hierarchical decomposition
• black box approach
• easy to reuse support concurrency
• support specialized analysis (deadlock,
  throughput)
• Disadvantages
• batch organization of processing
• not interactive
• difficult to synchronize pipes, order filters
• performance penalty forces lowest common
  denominator of data (often ASCII)
• parsing/unparsing overhead
• infinite queues may be required
• possible deadlock with finite buffers

12
Data abstraction, OO

• Components objects/instances of ADT
• Connectors procedure invocation
• Variations
• allow multiple interfaces
• (member function access)
• allow concurrency
• Advantages
• encapsulation/info hiding
• can change implementation without affecting
  clients
• inheritance shared definitions of functionality
• polymorphism actual operation determined at
  runtime
• reuse/maintenance promotes separation of
  concerns
• Disadvantages
• objects must know id for interaction, change of
  id requires notification

13
Event-based, Implicit Invocation

• Components objects/processes with interfaces
  that
• define allowable incoming, outgoing events
• Connectors event procedure bindings
• procedures are registered with events
• components interact by announcing events
• announcing component does not know who registered
  for the event
• upon event occurrence, associated procedures are
  implicitly invoked
• order of invocation is non-deterministic
• Main invariant event announcer does not know who
  will respond
• Examples GUIs, databases
• Advantages
• strong support of reuse (event registration)
• eases system evolution (replace components
  without affecting interfaces)
• Disadvantages
• less control of computation, data exchange
  difficult
• difficult to reason about correctness
• harder to understand flow, test

14
Client/Server

• Components server contains processing and data
  resources required by clients
• Connectors packets of client requests and server
  replies
• Examples distributed systems, web applications
• Advantages
• strong support of reuse
• support parallel computation
• Disadvantages
• complexity
• performance (data transmission)
• security

15
The Structure Of a Unix OS

• Kernel
• Manages memory and allocates it to each process
• Schedules work done by the CPU
• Organizes transfer of data from one part of
  machine to another
• Accepts instructions from shell and carries them
  out
• Enforces access permission on the file system

• Major Unix OS features
• Kernel
• Shell
• File System

16
UNIX OS

• Shell
• Command interpreter
• Create customized environments
• Write shell scripts
• Define command aliases
• Manipulate command history
• File and command completion
• Edit the command line

17
Unix OS

• File System
• Logical method for organizing and storing large
  amounts of information.
• Easy to manage.
• File basic storage unit.
• Types
• ordinary file (stores info)
• directory (holds other files and directories)
• special file (represents physical devices like
  printers, terminals, etc)
• pipe (temporary file for command linkage)

18
Unix Operating System Structure

• OS mediates between the user and the computer

User
Application Programs
Kernel
Shell
Hardware
19
UNIX Multi-user Multi-tasking

• More than one user can run at the same time and
  more than one task can run at the same time
• In Unix, each program is started as a process.
• A process is a program in execution.
• Usually only one copy of a program, but there may
  be many processes running the same program.
• To each interactive user (window)
• only one process in foreground
• may have several processes in background

20
Unix Process

• The fundamental unit of a multi-user UNIX
  operation.
• A program in execution, doing some work.
• Multiple users and multiple processes.
• CPU is switched rapidly from one process to the
  other.
• The shell uses the UNIX process structure to
  carry out user commands.

21

• Command Execution From Shell
• Typically a shell process forks a child process
  to execute the command.
• The child process execs the utility program named
  in the command.
• Now two processes are running for the user - the
  shell (parent) and the utility (child) process.
• Running the command in the foreground shell
  waits until the child process is complete.
• Running the command in the background
• the shell prints the process id of the child
  process and waits for the next command.

22
Processes
kernel
Process 0 Kernel bootstrap. Start process 1.
kernal mode
Process 1 create processes to allow login.
user mode
/etc/init
fork exec
fork exec
inetd
lpd
condition terminal for login
/etc/getty
/etc/getty
httpd
exec
exec
check password
/bin/login
/bin/login
exec
exec
command interpreter
shell
shell
23
Unix Process

• Init process
• last step in booting procedure
• create other processes to allow the users to
  login
• Getty process
• conditions for terminal connection
• wait for user-id
• display login on the screen
• Login process
• check password with the uid
• execute .profile or .login (depends on default
  shell)
• display shell prompt
• Shell process (command line interpreter)
• Shell prompt (, )

24
UNIX Process

• Process environment
• Process id, parent-process-id, process-group-id
• Opened files
• Working directory
• File creation mask
• User ID, Group ID
• Resource limits
• Environment variables
• Code
• A child process inherits parents environment.

25
Processes

• use ps to see the processes that you are running.
• ps
• PID TTY TIME CMD
• 221 pts/4 415 netscape
• 201 pts/4 005 bash
• 215 pts/4 115 emacs-19
• use to execute a task in background
• Example sort infile gt outfile
• use ctrl-Z to suspend the foreground task, and
  then use bg.
• use fg to move a background task to foreground.

26
Shell Command Interpreter

• Bourne Shell the default shell (sh)
• original unix shell
• does not have interactive features of newer
  shells
• widely used for writing shell scripts
• standard on Unix System V
• C Shell (csh) available on BSD Unix and most
  other systems
• with syntax similar to the C language
• with many enhancement over the Bourne shell.
• Korn Shell (ksh) ATTs answer to C Shell
• combine features of Bourne and C shells
• very efficient
• Other shells tcsh, bash

27
Day-to-Day Use

• Command Function Meaning
• cat Display a file conCATenate
• cp Copies a file CoPy
• mv Renames a file or moves it MoVe
• rm Delete files ReMove
• lpr Sends a file to a printer Line Printer
• lp (Sys V)
• ls Lists the contents of a directory LiSt
• chmod Changes the mode of permissions Change
  MODe
• pwd Shows the current directory Print WorkingDir
• cd Change current directory Change Dir
• mkdir Create a directory MaKe DIR
• rmdir Delete a directory ReMove DIR
• ps Shows the processes on the system Process
  Status
• man Shows info. about commands Manual
• df Shows file system status Disk File
• du Shows the space allocation of files Disk
  Utilization
• grep Search for patterns in files

28
Standard Command Format

• command options arguments
• wc -c -m -C -lw file
• stuff in brackets is optional
• boldface words are literals (must be typed as is)
• italicized (or ltgt enclosed) words are args
  (replace appropriately)
• Options modify how the command works
• command options -- ltfilegt
• options -option white-space options
• option noargoption argoption noargoption
• noargoption letter
• argoption letter whitespace string
• cc -po zap zap.c

29
Some Examples

• ls -alRF file-list
• a for listing all files including the dot files
• l for long format (file type, permissions,
  links, owner, group, etc)
• R for recursive, list subdirectories.
• F for listing directories with a trailing /
• ps ltoptionsgt
• List the information about running processes
• Example
• ps -el print the info about all processes
  (e) in the long format (l)

30
I/O Redirection

• Redirection and Pipe
• gt redirects standard output (screen) to a file
• E.g. ls gt dirlist
• lt redirects standard input (keyboard) from a
  file
• E.g. sort lt infile gt outfile
• pipe output of program 1 to input of program
  2
• E.g. who wc
• Or getit lt in check_it process_it
  format_id gt out
• gtgt appends output to a file
• E.g. ls -l gtgt oldfile
• Exercise find out the definition of tee.

31
Sequential vs. Concurrent Process

• Sequential
• date
• ps -ef OR date ps -ef who
• who
• Concurrent
• pgm1 prgm2 gt file1 pgm3
• make gt log
• sort 1 pdir ((pr dir lpr) sort 1 local))

32
File Name Expansion

• Each shell program recognizes a set of special
  characters called meta characters.
• The metacharacters are used to create patterns to
  match filenames and command names.
• Bourne and Korn shell meta/wildcard characters
• matches any string (including null)
• ? matches any one character
• a-dA-D matches any one character in the range
• !AB123 matches a char not in this range
• \ escape
• ltusernamegt (not bourne shell) the home dir
  of the user.

33
File Name Expansion

• Assume we have the following files under the
  current directory 120, 235, 25, a.out, c22,
  c2, Doc.1, Doc.2, Doc.3, one.c,two.c, three.c
• ls .c ls c2
• ls a-z!0-9 a. .
• ls ??? cd foo
• ls

34
Filters

• Most UNIX utilities are filters
• A filter is a program which
• reads input (text) from standard input only
• writes output to standard output only
• writes error to standard error only
• may use temporary files for intermediate results
• Filters can be combined to work together using
  pipes
• Pipe takes stdout of one command and uses it as
  stdin of another command
• ls wc

35
Command Alias

• Assign your own name for a command
• Syntax is shell dependent
• alias ll ls -l C shell
• alias llls -l Korn, Bourne shell
• Displaying the value of an alias
• alias ll (displays value)

36
Unix File Systems

• File a collection of data
• Just a sequence of bytes
• no record or block structure required
• Directory
• A file that contains information for files
• distinction between a directory and a file
• system can alter the contents of a directory
• rooted tree file structure (inverted tree)
• directories can contain both files and other
  directories info.

37
Unix File System Road Map
/
/etc
/var
/bin
/tmp
/usr
/mnt
/home
/dev
passwd hosts ...
sue
john
spool
adm
lib
include
etc
5bin
bin
printer mail(in) mail(out) uucp
messages wtmp

• Special files /dev/ represents I/O devices.

38
File Systems and the I-nodes

• Each disk drive contains one or more file systems
• Each file system occupies a number of cylinder
  groups.
• Each file system has a superblock, an i-node
  table and files
• The superblock is at a fixed location relative to
  the beginning of the file system. It contains a
  description of the file system.
• One can find the location of the I-node table
  thru superblock.
• Each entry of the I-node table is an I-node,
  which uniquely represents a file in the file
  system.
• An I-node contains uid, gid, time of creation,
  modification, and access, permissions, location
  of the file on the disk.

superblock
I-node table
file1
file2
free
file3
free
39

• An I-node includes the following information
• File access permissions
• Owner and group ids
• File link count
• Time of last modification
• Time of last file access
• Location of blocks for each regular file and
  directory file
• Device id for special files
• I-node 2 contains the location of the block with
  the root directory

40
chmod

• Change the access permissions of a file
• chmod ltpermissionsgt ltfilenamegt
• permissions can be specified as 3 octal digits,
  ltuser,group,othersgt, the three bits of an octal
  means r,w,x
• Example chmod 755 test
• permissions can be specified as x, or ux, or
  gr,
• chmod s test sets the set-uid bit for file
  test.
• If a directory has x in its permission, the dir
  is searchable, ie., one can do ls on the
  directory.

41
Pathnames

• Every file and directory in the file system can
  be identified by a full path name (route from
  root to file)
• /home/sue/email/f1
• Relative path name
• location relative to current directory
• . Current directory
• .. Parent directory
• if cwd is /home/sue
• ls email
• ls ./email
• cd ..
• ls ../fred

/
home
sue
fred
email
docs
f2
f1
42
Some Details

• cp -ir file1 file2
• cp -ir file-list directory
• i for interactive. prompt use whenever a file
  will be overwritten
• r for recursive. copy a directory tree
• ls -alRF file-list
• a for listing all files including the dot files
• l for long format
• R for recrusive. list the all subdirectories.
• F for listing directories with a trailing /
• date format
• date h d, 19y

43
Some Details (cont.)

• wc file-list
• display the number of lines, words and characters
• more file-list
• Browse through text files on page at a time.
• head -n file-list
• Display the first n lines of the files
  (default10)
• tail n-n -f
• Display the last few lines in the files (default
  10)
• Example
• tail 5 foo display the last parf of foo
  starting from line 5
• tail -5 foo display the last five lines of
  foo
• tail 30 foo head -15 more display line
  30-45 of foo
• tail -f foo wait and display the new lines
  appended to foo

44
Some Details (cont.)

• cut -c list file
• cut -f list -dChar file
• Cut out selected characters or fields from each
  line of a file
• Examples
• cut -c 1-5,15-20 foo
• extract chars 1-5 and 5-20 from each line
  of foo.
• cut -f 1,3 -d moo extract field 1 and 3
  from each line of moo.
• paste file1 file2
• Concatenate corresponding lines of the given
  input files
• Example (reverse two fields of the file abc)
• cut -f1 abc gt abc1
• cut -f2 abc gt abc2
• paste abc2 abc1 gt xyz

45
Some Details (cont.)

• grep, egrep, fgrep
• grep -nv... pattern file-list
• Search the input text files for lines matcing
  the pattern
• grep Unix doc.1 Display the lines in doc.1
  that contains Unix
• grep -n Unix doc. Display the lines with line
  numbers
• grep -v Unix doc.1 Display the lines which do
  not contain Unix

46
Some Details (cont.)

• sort -tC -o outfile field-list
  file-list
• sort the files
• sort 1 list1 sort list 1 starting from field
  2 to the end of the line
• sort 2-3 list2 sort list2 on the third field
• sort -n -o list4 list3 sort list3 numerically
  and place the output in list4

47
diff

• diff file1 file 2
• Display different lines that are found when
  comparing two files
• It prints a message that users ed-lide notation
  (a - append, c - change, d -delete) to describe
  how a group of lines has changed.
• It also describes what changes (additions,
  deletions and changes) need to be made to the
  first file to make it the same as the second file.

48
diff

• Example
• file1 file2 file3
• apples apples oranges
• oranges oranges bananas
• bananas kumquat s kiwis
• peaches

49
diff (cont.)

• diff file1 file2
• 3c3
• ltbananas
• -----------------
• gtkumquats
• diff file1 file3
• 1d0
• ltapples
• 3a3,4
• gtkiwis
• gtpeaches

50
comm file1 file2

• Takes two sorted text fields and print common
  lines and lines which appear exclusively in one
  file on separate columns.
• column1 lines only in file1, column 2 lines
  only in file2 col 3 comm
• Example
• file1 file2 comm file1 file2
• apple apple apple
• banana cantaloup banana
• grape grade cantaloup
• orange kiwi grape
• lemon kiwi
• comm -12 file1 file2
• apple
• grape

51
tr -csd pattern1 pattern2

• Translate input character to output character
  based on the input and output patterns
• Example
• tr A-Z a-z ltin gtout
• xlate all letters to lower case.
• tr -s \012\011\040 \012\012\012 lt in gt out
• xlate newline, tab and blank chars to newline
  chars and squeeze (-s) consecutive newline char
  into one
• tr -cs a-zA-Z \012 lt in gt out
• change all non-aplphbetic (-c) chars to
  newline chars and squeeze consecutive newlne
  char into one.
• The (asterisk) tells tr to repeat the (number
  sign) enough times
• to make the second string as long as the
  first one.
• tr -d \040 lt in gt out
• delete all blanks.

52
find ltdir-namegt ltexpgt

• Recursively search the directory tree rooted at
  ltpathnamegt and find all files whose names satisfy
  ltexpgt
• find pathname search options action options
• There are many details in the expression.
• Examples
• find . -name \.doc -print list all files
  names ending with .doc
• find /etc/source -atime 2 -print
• print the names of the files under
  /etc/source whose lst access time was 2 days
  ago.

53
find ltdir-namegt ltexpgt

• find . -name a-z -exec rm \
• remove the files under the curent directory
  whose names begin with a lower case letter.
• find / \(-name a.out -o -name .o \) -atime 7
  -exec \
• remove the object and binary executable files
  under the root directory which have not be
  accessed more than 7 days.