File Commands

1
File Commands

• Working with files is one of the strong points of
  Unix
• Unix contains many commands for manipulating
  files
• Commands that change file characteristics
• Commands for viewing file contents
• Commands for comparing two or more files
• Commands that modify file contents
• Commands that change the format of a file

2
File Characteristics

• Commands that change file characteristics change
  things such as
• Permissions
• Default permissions
• Owner
• Access/modification times
• chmod
• chown
• umask
• touch

3
chmod

• Change the permissions of a file or files
• Since directories are a file, it also works on
  them
• Only owner or superuser can change file
  permissions
• Can operate in either Absolute or Symbolic mode
• chmod 700 filename
• Gives owner read, write, and execute permissions
• Removes permissions for group and other
• chmod urwx filename
• Gives owner read, write, and execute permissions
• Does not affect group and other permissions
• -R option makes it recursively descend through
  directory arguments

4
Directory Execute Permission

• What the heck does "Execute" permission mean for
  a directory?
• Another term often used is "Search permission"
• All directories in the current branch of the
  directory structure must have execute permission
• Execute also allows you to cd to the directory

5
chown

• Change the owner of a file
• Owner may be specified as either a decimal user
  ID (see the /etc/passwd file) or the login name
• Only the superuser can change owner
• -R option also works with chown to recursively
  change owner of all files and subdirectories when
  the command line argument is a directory

6
umask

• umask displays the file creation mask
• This is used to set the default file and
  directory permissions
• umask value sets the file creation mask to the
  value, where value is given in octal notation
• value is XORed with the permissions 666 for files
  and 777 for directories to determine the
  permissions for new files
• Common values for umask are 002 and 022
• 002 gives owner and group full permissions and
  others read and directory search permissions
• 022 gives user full permissions but restricts
  group and others to read and directory search

7
touch

• touch lets you update the access and modification
  times of a file
• This is often used when doing program development
  with make
• touch can either change the times to the current
  time or a user specified time
• touch can also create a file if the filename
  given as an argument does not exist

8
Viewing Files

• What good is having a file if you can't look at
  its contents?
• Commands for viewing files include cat, more, and
  the various editors
• Specialized commands
• head
• tail
• od

9
head, tail

• head - displays the first few lines of a file
• Number of lines can be specified by the user,
  defaults to 10
• When more than one file name is specified, the
  start of each file is preceded by its file name
• tail displays the last part of a file
• tail has more options than head
• Output can be counted by lines, characters, or
  blocks
• Output can be listed in reverse
• If the input file is not a pipe, tail can loop
  endlessly, sleeping for a second after each
  output
• Great for monitoring a file that's being updated
  by a process

10
od - octal dump

• When you really just have to know exactly what
  characters are in a file, od comes to the rescue
• od can display the contents of a file in octal,
  hexadecimal, decimal, floating point, or
  characters
• Default format is octal
• od will dump either from the beginning of the
  file or you can give it an offset from the
  beginning of file to begin dumping
• od can also suppress multiple instances of
  identical display lines

11
Comparing Files

• Finding differences between multiple versions of
  files is tedious, at best
• Again, Unix comes to the rescue with file
  comparison tools
• uniq
• diff
• diff3
• comm

12
uniq

• Remove or report adjacent duplicate lines
• uniq reads the input file, comparing adjacent
  lines
• The default action is to suppress the output of
  multiple instances of duplicate lines
• Options allow you to
• Output only lines that are not duplicated (ie
  truly are unique)
• Skip the first n characters or fields when
  performing the comparisons

13
diff, diff3

• diff displays line by line differences between
  pairs of test files
• diff tells which lines must be changed in the two
  files to bring them into agreement
• Options exist to
• Write an ed compatible script file to actually
  make the changes
• Ignore case
• Expand TAB characters
• Ignore all blanks (TABS and SPACE characters)
• List differences within context

14
diff3

• diff3 performs line-by-line comparisons between
  three files
• diff3 compares the three files and shows
  disagreeing ranges of text flagged with
• All three files differ
• 1 file1 is different
• 2 file2 is different
• 3 file3 is different
• Options include producing various scripts to
  incorporate the changes to the files in a variety
  of combinations

15
comm

• Displays lines in common and not in common
  between two sorted lists
• Reads two files, which must be sorted in ASCII
  order, and produces three columns of output
• Lines that occur only in file1
• Lines that occur only in file2
• Lines common to both files
• Options let you suppress any individual or
  combination of output columns

16
Modifying Files

• Even though Unix files are normally standard
  ASCII, often we want to change the specific
  layout of the file or extract some specific
  portion of the file
• Examples
• fold, paste, join
• expand
• nl
• cut
• split, csplit

• Wrap long lines
• Changes TABS to spaces
• Combine multiple lines

• Number lines
• Extract some portion of the file

17
fold, paste, join

• fold takes takes long lines from a file and folds
  them at a specific length for display
• Line length can be user specified, defaults to 80
• May not work correctly if underlining is present
• paste joins corresponding lines of several files
  or subsequent lines of a single file
• If multiple file names are given, paste treats
  each file as a column and joins them together
  horizontally
• If a single file name is given, paste treats each
  line as a column and joins them together
  horizontally
• In both cases, the joins are done using TAB
  characters as a separator (or with characters
  from an optionally specified list

18
join

• join is based on the relational data base command
  join
• Like in a relational database, join combines
  records from two different files or tables that
  have a common field
• Both files must be sorted in increasing ASCII
  order on the fields on which they are to be
  joined
• By default, this is the first field but is user
  definable
• Default field separator is SPACE and TAB but can
  be specified using the -t option

19

• One line is output for each pair of lines in
  file1 and file2 that have identical join fields
• Output line is normally the common field,
  followed by the rest of the line from file1
  followed by the rest of the line from file2
• What is output can be user specified using the -o
  option
• join is just a touch tricky to use so be sure to
  check the man page
• Fields are numbered starting with 1 rather than 0
  as in sort

20
expand, unexpand

• expand and unexpand expand TABS to SPACE
  characters and vice versa
• Number of spaces to use for each TAB is user
  definable

21
nl - number lines

• nl is a line numbering filter
• Reads lines from stdin or a filename, numbers
  them, and sends them to stdout
• nl by default does not number empty lines or
  lines containing only blanks
• If the file contains page markers, nl can, and by
  default will, start the line number sequence over
  for each new page

22
cut

• cut extracts selected fields from each line of a
  file
• Used to extract columns from a table or fields
  from each line of a file
• Fields can be fixed or variable length and can be
  optionally marked by a field delimiter character
• Fixed length fields are specified by character
  position with "-" specifying a range
• cut -10,12-14 extracts from each line the
  contents of columns 1 through 10 and 12 through
  14
• Variable length fields use a default (TAB) or
  user specified field delimiter and are specified
  by field number
• cut 2,6,9 extracts fields 2, 6, and 9

23
split, csplit

• split splits a file into multiple pieces by lines
• Default is 1000 lines per subfile, although the
  user can specify a different number
• Output files are named by either a user specified
  output filename with aa appended for the first
  file, ab for the second, ac for the third, and so
  on
• If no output file name is given, "x" is used
• xaa, followed by xab, then xac, etc.

24
csplit - Context Split

• Like split, csplit also splits a file into
  multiple pieces
• csplit breaks up the file according to context
  instead of line counts
• Arguments, consisting of regular expressions or
  line numbers control where the file will be split
• The number of output files cannot exceed 100
• By default they are named xx00, xx01, xx02, xx99
  but the user can specify a different prefix

25
Changing a File's Format

• Sometimes we need to actually change the contents
  of a file to another format
• Examples
• Sending a binary file through email
• Compressing a file to save space
• Once again, Unix supplies tools to solve these
  problems
• pack
• compress
• uuencode

26
pack, unpack

• pack and unpack compress and expand files using
  Huffman codes, a minimum redundancy code
• The input file is replaced by a packed file named
  filename.z, retaining the original access modes,
  access and modification dates
• Typically, large text files are reduced to 60-75
  of original size
• Since a decoding tree must be included as part of
  the packed file, files smaller than 1.5K
  probably aren't worth packing

27
compress, uncompress

• compress and uncompress do a tasks similar to
  pack and unpack but use a different form of
  compression that is both faster and more
  efficient
• Compression ratios of 50-60 are common for large
  text files
• Output files are named filename.Z
• Like pack, small files may actually produce a
  larger compressed file than the original
• The difference is that compress will not produce
  a compressed file that is larger than the
  original unless forced by the user

28
uuencode, uudecode

• Encodes binary files into 7-bit ASCII
  representation
• Primarily used for sending binary attachments in
  email
• Produces a file that is 35 larger than the
  original (each 3 bytes of original file produce 4
  bytes in uuencoded file, plus control
  information)
• uudecode strips off any leading and trailing
  lines added by mailer programs and converts the
  file back to the original binary with the owner,
  filename and attributes encoded in the uuencoded
  file's header

29
here document

• The shell provides a mechanism that allows you to
  store the standard input for a command in the
  script along with the command
• It is a form of I/O redirection and looks like
• cmd ltlt - word
• ..
• ..
• word
• The ltlt redirects the standard input of the
  command to the here document
• The optional - tells the shell to discard leading
  tabs
• This is used to allow to make your code look nice
  in your script

30

• This works by taking the lines following the
  redirection and feeding them to cmd until word is
  encountered on a line by itself
• word can be any sequence of non-blank characters
  and should be something that won't accidentally
  be encountered elsewhere
• Example
• cat ltlt EOD
• This is an example of a here document.
• It is a form of input redirection.
• I'll bet most of your Sys Admins won't have
• a clue what it is.
• EOD

31

• If you want to use additional redirection or
  piping with a here document, it needs to be on
  the same line as the command, not after the here
  document
• Example
• cat ltlt EOD more
• EOD