TDC368 UNIX and Network Programming

1
TDC368UNIX and Network Programming

• Week 3-4
• UNIX File System
• File Representation
• File Handles
• System Calls (APIs)
• System Calls dup dup2 pipe
• Library functions popen() pclose()

• Camelia Zlatea, PhD
• Email czlatea_at_cs.depaul.edu

2
Outline

• UNIX File System files, catalogs (inodes
  types,size, permissions).
• UNIX File I/O unbuffered I/O functions (open
  ,close,seek, umask,read,write).
• Standard I/O buffering (fopen, fclose, fflush).
• File Sharing
• Communication between related processes via pipe
  (pipe, dup,dup2)
• Appendix
• UNIX system calls C library functions, C
  classes, POSIX API's.

3
UNIX File System

• Hierarchical organization
• Device independence
• compatibility among files or catalogs, devices
  and IPC
• Types of files
• regular files (byte-stream, no format)
• directory files (groups of files mapped on users
  projects)
• special files (to access peripheral devices,
  /dev/...)
• pipe files (inter process communication
  mechanisms)

4
UNIX Type of Files

• Regular Files
• Shell commands cp, rm, mv, cat, ls
• System Calls ltfcntl.hgt, ltunistd.hgt
• open, creat, read, write, close, link, unkink
• Directory Files
• Shell commands mkdir, rmdir, mv
• System Calls ltdirent.hgt
• Special Files
• mknod ( root privileges)
• Pipe Files
• mkfifo

5
File Attributes

• File Type (-,d, b,c,p)
• ls -l
• Access Permission for owner, group and others
  (r,w,x)
• File Owner ID and GID
• Number of hard links of a file
• File Size
• Last access, modify and change permission time
• INODE number
• File system ID

6
File Attributes

• Constant Attributes
• file type
• file inode number
• File System ID
• Changeable Attributes
• chmod (command and system call)
• chown, chgrp
• touch (command), utime (sys call)
• ln (command ), link(syscall)
• rm (command), unlink(syscall)
• Attributes query by
• system calls stat or fstat

7
File Permissions

• all UNIX files have associated a set of owner
  permissions bits, that are used by the OS to
  determine the access
• classes of users owner, group, other
• permissions owner group other
• rwx rwx rwx
• 111 111 111 octal 777
• 111 101 101 octal 755
• - default permissions creation_mask XOR
  umask_value
• - creation mask (set at system initialization)
  is usually 0777

8
File Permissions

• Command shell
• umask new_mask_value (display/change umask
  value)
• System call
• mode_t umask(mode_t mask)
• - to modify umask for all your processes edit
  your startup file , .login (.profile) and insert
  the line umask nnn (where nnn is the new umask
  value)
• - change permissions
• - command shell
• chmod guo?rwx file

9
File Permissions

• System calls ltsys/types.hgt
• ltsys/stat.hgt
• int chmod(const char path, mode_t mode)
• int fchmod(int fid, mode_t mode)
• Example Use system call stat and determine the
  inode information
• Name of the file is passed as programs
  argument.

10
File Permissions

• include ltsys/types.hgt
• include ltsys/stat.hgt
• void main(int argc, char argv)
• struct stat statbuf
• if (stat(argv1, statbuf)-1)
• / print error message /
• exit(1)
• else
• if (statbuf.st_mode S_IFDIR)
• printf( s is a directory \n, argv1)
• printf(user ID is s \n, stat_buf.st_uid)

11
File Owner Information

• real user ID (UID)
• real group ID (GID)
• - information is obtained from the password file
  entry (/etc/passwd)
• group ID (GID) numbers should map to group names
  from /etc/group
• effective user ID (EUID)
• effective group ID (EGID)
• - usually UID identical with EUID

12
File Owner Information

• Example
• command passwd (change password) need to have
  access to a system file (as a root)
• a non-privileged user is able to run a command
  temporarily as root if suid permission is set
• rws r-x r-x root /etc/passwd

13
File Owner Information

• Command shell id is used to display owner
  information (UID, GUID, EUID, EGID)
• System Calls
• ltsys/types.hgt
• ltunistd.hgt
• uid_t getuid(void)
• uid_t geteuid(void)
• gid_t getgid(void)
• gid_t getegid(void)

14
File Representation

• UNIX File System Organization BOOT BLOCK
• SUPERBLOCK
• INODE TABLE
• DATA BLOCK
• Boot Block
• contains a program that loads UNIX kernel file
  from the file system
• transfer control to the start address of the
  kernel code
• if the system is already running and the disk is
  accessed by mount, boot block is ignored
•  SUPERBLOCK
• - contains all information that defines the
  current state of the file system
• - size of i-list Iinode list table)
• - size in blocks of the files system
• - number of free blocks available in the file
  system
• - the address of the free inodes list
• - number of total free blocks in the system
• - files system name and file system disk name
• - superblock is read when mounting the device 

15
File Representation

• INODE - identifies a UNIX file
• Inode Structure
• file attributes
• physical disk address where file date is located
• Inode Table (UNIX System V)
• an inode entry for each file in the file system
• Directory File - maps file name to inodes numbers

16
File Representation

• Inode Table
• inode structure
• File Information
• - size (bytes)
• - owner information (UID, GID, EUID, EGID)
• - relevant times
• - number of links
• - number of blocks
• - permissions
• Data Blocks
• - direct pointers to beginning of file blocks
• - indirect pointers
• - double indirect pointers
• - triple indirect pointers
• Data Block
•  .
• Data Block

17
Directory Structure
/bin /etc unix /dev
/usr /tmp
Absolute path name /usr/project/bin/a.out Relat
ive path name ./a.out ../../a.out File
name NAME_MAX in ltlimits.hgt Path name
PATH_MAX -- Hard Link - the path name of a
file ln /usr/myproj/a.out /usr/project/bin/a.out
18
Directory Representation

• Directory Representation
• UNIX directory - a file containing a
  correspondence between filenames and inodes
• the inode does not contain file name
• data blocks of the directory file are directory
  entries
• a directory entry contains inode number file
  name
• command as rename/move (mv) affect the name filed
  from directory entry,
• only one physical copy of the file exists on disk
  (file may have several names on different
  directories or the same name)
• for a newly created directory (ex. mkdir
  dir_name) OS makes a new directory entry for it
  and assign an inode to it

19
Accessing Files

• Example of Directory file
• Inode Number File Name
• 78 .
• 109 ..
• 115 sample
• 240 test.c
• 300 a.out
• 115 sample_data_l

20
Directory Files

• Record-oriented files
• file name
• inode number
• Record data type is the structure dirent in
  SV/POSIX
• ltdirent.hgt
• direct in BSD
• ltsys/dir.hgt

21
Directory Functions

• Opens a directory file, return a handle of type
  DIR
• opendir
• Reads next record from directory file
• readdir
• Closes a directory file
• closedir
• Sets file pointer at the beginning
• rewinddir
• Random directory access
• telldir, seekdir

22
Hard vs. Symbolic Links

• Hard Links ln fileA fileB
• pathnames of a file (directory entries)
• NO new inode is created
• cannot link files across file systems
• cannot link directories (as reg.user)
• increase the hardlink counter per inode
• ln /dirA/name1 /dirB/name2
• Effect creates a hard link, both names point to
  the same inode structure)
• Symbolic Links ln -s fileA fileB
• Symbolic link is a file containing the name of
  another file(directory)
• a new inode is created
• can link files across file systems
• can link directories (as reg.user)
• ln -s /dirB/name2 /dirA.name1
• Effect if there is an entry 12345 name1 in
  /dirA
• creates new inode 56789
• new entry 56789 name2 in /dirB
• the block of date from inode 56789 contains
  /dirA/name1

23
UNIX File System Calls (APIs)

• Header file ltsys/types.hgt
• ltfcntl.hgt
• open - opens a file for data access
• Header file ltsys/types.hgt
• ltunistd.hgt
• creat - creates regular file
• read - reads data from a file
• write - writes data to a file
• close - terminate the access to a file

24
UNIX File System Calls (APIs)

• Header files ltsys/types.hgt
• ltunistd.hgt
• lseek - random access to a file
• link - creates a hard link to a file
• unlink - removes a hard link of a file
• chown - changes UID or GID of a file
• chmod - changes access perm. of a file
• Header files ltsys/types.hgt
• ltsys/stat.hgt
• stat, fstat - queries attributes of s file

25
File Handles

• C stream pointers (FILE )
• Conversions, header file ltstdio.hgt
• int fileno(FILE stream_pointer)
• FILE fdopen(inf fd, char open_mode)
• C Library Func. UNIX System Call
• fopen open
• fread, fget, fgets, fscanf read
• fwrite,fput,fputs,fprintf write
• fseek, ftell, frewind lseek
• fclose close

26
FILE handle

• include ltstdio.hgt
• FILE fp
• if (( fp fopen(test.dat, w) NULL)
• fprintf(stderr, cannot open file\n)
• else
• fprintf(fp, This is a text)

fp
This is a text)
3
User area
Kernel area
27
Buffering

• Minimize of write/read calls
• Types of Buffering
• Fully / Line / No Buffering
• Change Buffering Type
• include ltstdio.hgt
• void setbuf(FILE fp, char buf)
• void setbuf(FILE fp, char buf, int mode,
  size_t size)
• mode _IOFBF, _IOLBF, _IONBF
• size default is BUFSIZ
• Flush a buffer
• int fflush(FILE fp)

28
Parent Process opens file test.dat before fork
29
Parent Process opens file test.dat after fork
30
Pipe

• Think at a pipe as being a special file that can
  store a limited amount of data in a FIFO manner
• Size 512 K on most Unix systems (ltlimits.hgt or
  ltsys/param.hgt)
• Interaction rule one process writes to the pipe
  (as it were a file) and other process reads from
  the pipe
• Data are written at one end of the pipe and are
  read at the other end
• System keeps track of the current location of
  last read/write

31
Process synchronization using pipes

• If a writing process attempts to write to a full
  pipe the system blocks it until pipe is able to
  receive data
• If a process attempts to read from an empty pipe
  the system blocks it until data is available
• A process will block if it tries to read from a
  pipe that has been opened for reading but no
  process has opened it for writing
• If a write is made to a pipe not open for reading
  SIGPIPE is generated and errno is set to broken
  pipe''

32
write() System Call

• Data are written to a pipe using unbuffered I/O
  write
• Include files ltunistd.hgt
• Calling pattern
• size t write (int fileds, void buf, size t
  nbytes)
• fileds is a file descriptor
• Return values
• On success bytes written
• On failure 1
• Set errno? Yes

33
Writing to a pipe

• Each write request is appended to the end of the
  pipe
• Write requests of P IPE BUF size or less are
  guaranteed not to be interleaved with other write
  requests to the same pipe
• Write request may cause the process to block the
  constants
• O NONBLOCK, O NDELAY in !sys/fcntl.h? can be set
  by fcntl() to nonblocking strategy
• When O NONBLOCK, O NDELAY are set and the request
  to write P IPE BUF size bytes or less fails the
  value returned by write is shown in the next
  slide Table gt

34
Setting the constants that control piping strategy

• The table shows what write returns in function of
  constants
• Blocking constants for write()
• If a write is made to a pipe not open for reading
  SIGPIPE is generated and errno is set to broken
  pipe''

35
read() system call

• Data are read from a pipe using unbuffered I/O
  read
• Include files ltsys/types.hgt,
• ltsys/uio.hgt,
• ltunistd.hgt
• Calling pattern
• size t read (int fileds, void buf, size t
  nbyte)
• fileds is a file descriptor
• Return values
• On success bytes read
• On failure 1
• Set errno? Yes

36
Reading from a pipe

• Read actions are initiated from the current
  position
• If pipe is open for writing by another process
  but it is empty read returns as shown bellow
• Blocking constants for read
• If the pipe is not opened for writing by another
  process read returns 0 (as in case of O NDELAY
  set)

37
Types of pipes

• unnamed pipes, used with related processes
  (example parent/child or child/child)
• named pipes, used with unrelated process

38
Unnamed pipes pipe() system call

• An unnamed pipe is constructed with pipe() system
  call,
• Include files ltunistd.hgt
• Calling pattern int pipe ( int filedes2 )
• Return values
• On success 0
• On failure 1
• Set errno? Yes
• Functionality
• pipe() operates on a fullduplex or halfduplex
  basis
• Writing/reading from a pipe

fullduplex
halfduplex
39
Actions performed by pipe()

• If successful, it returns two file descriptors,
  filedes0, filedes1 that reference two data
  streams
• Historically pipes were nonduplex (i.e.,
  unidirectional) if twoway communication was
  needed two pipes were opened, one for reading and
  the other for writing
• Currently file descriptors returned by pipe() are
  fullduplex (i.e., bidirectional), both opened
  for reading and writing.
• In a full duplex if the process writes to
  filedes0 then filedes1 is used for reading,
  and viceversa
• In a halfduplex, filedes0 is used for reading
  and filedes1 for writing
• an attempt to use filedes1 for reading or
  filedes0 for writing produces an error

40
Example a parent process sends to a child the
first argument from its command line and
child acknowledges the receipt

• / Using a pipe to send data from parent to child
  /
• include ltstdio.hgt
• include ltunistd.hgt
• include ltstdlib.hgt
• include ltstring.hgt
• include lterrno.hgt
• int main (int argc, char argv)
• int fdes2
• static char messageBUFSIZ
• if (argc ! 2) fprintf(stderr,''Usage s
  messagen'', argv) exit(1)
• if (pipe(fdes) 1) perror(''Pipe'')
  exit(2)
• switch (fork())
• case 1 perror(''Fork'') exit(3)

41
Example a parent process sends to a child the
first argument from its command line and
child acknowledges the receipt

• / Child process closes fdes1 and read fdes0
  to obtain the message /
• case 0
• close(fdes1)
• if (read(fdes0, message, BUFSIZ) ! 1)
• printf(''Message received by child is
  sn'', message)
• fflush(stdout)
• else perror(''Read'') exit(4)
• break

42
Example a parent process sends to a child the
first argument from its command line and
child acknowledges the receipt

• / Parent closes fdes0 and write message to
  fdes1 /
• default
• close(fdes0)
• if (write(fdes1, argv1, strlen(argv1)) !
  1)
• printf(''Message sent by parent is sn'',
  argv1)
• fflush(stdout)
• else perror(''Write'') exit(5)
• break
• exit(0)

43
Redirecting stdin and stdout

• Examples
• cat gt test.dat

2
2
0
0
1
1 test.dat
44
Redirecting stdin and stdout dup() system call

• Create a pipe
• Associate stdin and stdout with the created pipe
  using dup() system call
• Include files ltunistd.hgt
• Calling pattern
• int dup ( int fdes )
• Return values
• On success next available file descriptor
• On failure 1
• Set errno? Yes or No

45
dup() functionality

• dup() duplicates an original open file
  descriptor
• new descriptor references SFT (System File Table)
  entry for the next available nonnegative file
  descriptor
• new descriptor shares the same file pointer
  (offset), has the same access mode as original
  and remain open across exec.
• since actions necessary for redirection are as in
  the code
• int fdes2 pipe(fdes) close(fileno(stdout))dup
  (fdes1)
• there is a chance that descriptor returned by
  dup() is not the one closed

46
dup2() system call

• Using dup2() system call for file redirection
• Include files ltunistd.hgt
• Calling pattern int dup2 ( int fdes1, int fdes2
  )
• Return values
• On success next available file descriptor gt0
• On failure 1
• Set errno? Yes
• dup2() functionality
• dup2() closes the file and duplicates the file
  descriptor atomically
• therefore dup2() avoid the limitation of dup()
• both dup() and dup2() can be implemented with
  fcntl()

47
Redirecting stdin and stdout

• Example
• int fd
• mode_t fd_mode S_IRUSR S_IWUSR S_IRGRP
  S_IROTH
• fd open(test.dat, O_WRONLY O_CREAT,
  fd_mode)
• dup2(fd, STDOUT_FILENO)
• ..
• close(fd)

Process FDT after close
Process FDT after dup2
Process FDT after open
48
Example implementing ''last sort '' command

• / A home grown last sort command /
• include ltstdio.hgt
• include ltunistd.hgt
• include ltstdlib.hgt
• include lterrno.hgt
• void main (void)
• int fdes2
• if (pipe(fdes) 1) perror(''Pipe'')
  exit(1)
• switch (fork())
• case 1 perror(''Fork'') exit(2)
• case 0
• dup2(fdes1, fileno(stdout))
• close(fdes0)
• close(fdes1)
• execlp(''last'', ''last'', 0)
• exit(2)

default dup2(fdes0,fileno(stdin))
close(fdes0) close(fdes1)
execlp(''sort'', ''sort'', 0) exit(4)
49
Per Process FDT and SFT before dup2()
50
Per Process FDT and SFT after dup2()
51
Per Process FDT and SFT after close()
52
Summary for communication via unnamed pipes

• Create the pipe(s) needed
• Generate the child process(es)
• Close/duplicate file descriptors to properly
  associate the ends of the pipe
• Close the unneeded ends of the pipe
• Perform the communication activities
• Close any remaining open file descriptors
• If appropriate, wait for child processes to
  terminate

53
popen() library function

• popen() encapsulates the sequence of activities,
• Generate a pipe
• Fork a child process
• Duplicate file descriptors created by pipe
• Passes command execution information from one
  process to another
• Include files ltstdio.hgt
• Calling pattern
• FILE popen (char command, char type )
• Return values
• On success ptr to a FILE
• On failure NULL pointer
• Set errno? no

54
popen() functionality

• create a child process
• child process exec's a Bourne shell which will
  execute the shell command passed to popen()
• Input to and output from the child process is
  accomplished via pipe
• if mode type in popen is w'' parent process can
  write to the standard input of the shell command
  run by the child and child process can read the
  data at its standard input
• if mode type in popen is r'' parent process can
  read from the standard output of the shell
  command run by the child and child process can
  write the data at its standard output

55
pclose() library function

• pclose() closes a data stream opened by popen(),
  
• Include files ltstdio.hgt
• Calling pattern int pclose (FILE stream )
• Return values
• On success Exit status of command
• On failure 1
• Set errno? no

56
Example using popen and pclose

• / Using popen and pclose /
• include ltstdio.hgt
• include ltunistd.hgt
• include ltstdlib.hgt
• include ltlimits.hgt
• int main (int argc, char argv)
• FILE fin, fout
• char bufferPIPE_BUF
• int n
• if (argc ! 3) fprintf(stderr, ''Usage s cmd1
  cmd2\n'', argv0) exit(1)
• fin popen(argv1, ''r'')
• fout popen(argv2, ''w'')
• while ((n read(fileno(fin), buffer, PIPE_BUF))
  gt 0)
• write (fileno(fout), buffer, n)
• pclose (fin)
• pclose (fout)
• exit(0)

57
Example using popen and pclose

• Observations
• This program requires two command line arguments,
  i.e., two shell commands whose standard
  output/input will be redirected via pipes
  generated by popen()
• The first popen() call with option r'' directs
  the system to fork a child that will execute
  first shell command whose output will be
  redirected so it can be read by the parent using
  fin.
• The second popen() call with option w'' directs
  the system to fork a new child that will execute
  the second shell command whose input will be the
  data written by parent using fout.
• The while() loop is used to copy data from the
  output end of one pipe to the input end of the
  other pipe.