CS 205 Linux

1
CS 205 - Linux

• Chapter 4
• UNIX/Linux File Processing

2
Objectives

• Explain UNIX and Linux file processing.
• Use basic file manipulation commands to create,
  delete, copy, and move files and directories.
• Employ commands to combine, cut, paste,
  rearrange, and sort information in files.
• Create a script file.
• Use the join command to link files using a common
  field.
• Use the awk command to create a
  professional-looking report.

Practice Exercises
None.
Additional References
None
A thorough understanding of the join and awk
commands will require significant practice and
study. We will focus on a high-level overview of
the commands and practice some usage.
3
Reviewing File Types

• Use the ls command to see the properties of
  files. For example
• ls l
• Regular files (-)
• Text files containing printable ASCII characters.
• Binary files containing nonprintable characters.
• Directories (d)
• Special files
• Character special files (c), block special
  files (b)
• Use the chmod command to change permissions. For
  example
• chmod ugorwx myfile

Practice Exercises
Execute the ls command with the l option.
Additional References
None
The first character in the permissions indicates
the file type. The next 9 characters show
permissions in groups of 3 User, Group, and
Others.
4
File Structures

• Different ways to structure files
• Flat ASCII file created, manipulated, and used
  to store data (e.g., letters, product reports)
• Record structure
• Variable-length record typically separated by a
  delimiter
• Fixed-length record each field has a specified
  length

Practice Exercises
None.
Additional References
None
Variable length records need some kind of
delimiter. Most examples in the textbook will use
the colon as a delimiter but other characters can
also be used.
5
Processing files

• stdin standard input
• Keyboard
• stdout standard output
• Monitor or console
• stderr standard error
• Screen

Practice Exercises
None.
Additional References
None
Input, output, and errors can be redirected using
gt, gtgt, 2gt, lt, and ltlt. Anyone wish to guess the
purpose of the 2gt? No, skip to next slide
6
Using Input and Error Redirection

• Use gt and gtgt to redirect output
• Example ls gt homedir.list
• Use lt and ltlt to redirect input
• Example vi testfile lt commands
• Use 2gt to redirect commands or program error
  messages
• Example ls Fellowese 2gt errors

Practice Exercises
None.
Additional References
None
Several of the projects at the end of the chapter
will focus on redirection.
7
Manipulating Files

• This chapter will focus on the following ways to
  manipulate files
• Create files
• Delete files
• Remove directories
• Copy files
• Move files
• Find files
• Combine files
• Combine files through pasting
• Extract fields in files through cutting
• Sort files

Practice Exercises
None.
Additional References
None
If anything is not clear, review the appropriate
projects at the end of the chapter.
8
Creating Files

• Two simple ways to create empty files
• gt accounts
• touch accounts
• Primary purpose of touch is to change a files
  time stamp and date stamp.

Practice Exercises
Create a file using the 1st technique. Display
the details of the file ls l accounts Wait a
minute or so and execute the 2nd command. Display
the details of the file again.
Additional References
None
These techniques create empty files. To create
files containing data, you may wish to use the vi
editor or the cat command.
9
Deleting Files

• Delete a file using the rm (remove) command
• rm accounts

Practice Exercises
Delete the file previously created using the rm
command.
Additional References
None
Be careful with wildcard characters when removing
files. What happens if you execute the command
rm?
10
Removing Directories

• Use rm or rmdir to remove an empty directory
• Use rm -r to remove a non-empty directory

Practice Exercises
None.
Additional References
None
Be careful with the r option. You may
accidentally delete a directory containing files
that you wish to keep.
11
Copying Files

• Use cp for copying files
• Examples
• cp /home/csis/linuxuser/corp_phones1 corp_phones1
• cp /home/csis/linuxuser/corp_phones2 corp_phones2
• cp /home/csis/linuxuser/products products
• cp /home/csis/linuxuser/vendors vendors

Practice Exercises
Execute each of the examples then use the cat
command to see the contents.
Additional References
None
The linuxuser directory should have permissions
allowing everyone to read the files. These files
are used in several of the projects at the end of
the chapter.
12
Moving Files

• To move a file, use mv (move) along with the
  source file name and destination name.
• As insurance, a file is copied before it is
  moved.
• Moving and renaming a file are the same operation.

Practice Exercises
Try these commands touch testfile1 ls test mv
testfile1 testfile2 ls test What would have
happened had you use the cp command instead of
the mv command?
Additional References
None
You can move a file from one directory to another
and keep the same file name or change it. To
change a file name in the same directory, use the
move command.
13
Finding Files

• To search for files with a specified name, use
  find .

Practice Exercises
Create several files with similar names and use
the find and ls commands. What are the
differences in the commands.
Additional References
None
The ls command can also be used to find files but
will show completely different information about
the files.
14
Combining Files with the cat Command

• So far, we have used the cat command to create a
  file using redirection. For example
• cat gt corp_phones1
• Or to display the contents of a file. For
  example
• cat corp_phones1
• You can also use the cat command to combine
  files. For example
• cat corp_phones1 corp_phones2 gt phones3

Practice Exercises
If you copied the files corp_phone files in an
earlier exercise, combine now using the cat
command. Display the contents of the combined
file also using the cat command.
Additional References
None
Remember that cat stands for concatenate so the
primary purpose of the command is to concatenate
files. If you have redirection with no files,
input comes from the keyboard.
15
Combining Files with the paste Command

• The cat command combines files by appending one
  to the other. The paste command combines files
  side-by-side.
• For example, two files (vegetables and bread)
• Can be pasted using paste vegetables bread gt food

Practice Exercises
Create two files called vegetables and bread and
paste them together using the example.
Carrots Spinach Lettuce Beans
Whole wheat White bread Sourdough Pumpernickel
Additional References
None
You can also use the paste command to combine
more than two files.
16
Extracting Fields Using the cut command

• The cut command allows you to specify fields,
  delimiter characters, and files from which to cut
  fields of data.

Practice Exercises
None.
Additional References
None
Remember, the output of commands can also be
redirected to a file.
17
Sorting files

• The sort command can be used to sort a file and
  display the results on the screen or redirect the
  sorted output to another file.

Practice Exercises
If you have the products file from an earlier
exercise, try to sort it with the following
command sort k 2 t products What happened?
Additional References
None
Note the option used for the delimiter is t, not
d as in other commands.
18
Creating Script Files

• DOS and Windows users create batch files with an
  extension of bat. Batch files contain lists of
  commands to be executed in sequence.
• UNIX/Linux users create shell scripts. Shell
  scripts contain lists of commands to be executed
  in sequence.
• Steps
• Create the script using vi or another editor.
• Make the file executable using chmod.
• Execute the script
• ./myscript

Practice Exercises
Create a file called myscript containing the cal
command. Change the permissions of the file with
the chmod command chmod ugox myscript Execute
the script ./myscript
Additional References
None
Remember the ls l command? The next 9 characters
show permissions in groups of 3 User, Group, and
Others. Use chmod to add or subtract permissions.
19
Using the Join Command

• Use join to associate lines in two files on the
  basis of a common field in them.
• For examples, the two files
• Can be combined to a third file.

Practice Exercises
See homework.
Brown8253,000 Anders11032,000 Caplan17441,00
0 Crow9536,000
BrownLaVerneFAccounting Department444-7508 And
ersCarolMSales Department444-2130 CaplanJason
RPayroll Department444-5609 CrowLorrettaLShi
pping Department444-8901
Additional References
BrownLaVerneAccounting Department53,000 Anders
CarolSales Department32,000 CaplanJasonPayroll
Department41,000 CrowLorrettaShipping
Department36,000
None
The join command is similar to the join in SQL,
in that it is used to join to files based on a
common value to form a third file.
20
Using the Join Command
Practice Exercises
See homework.
Additional References
None
The join command is similar to the join in SQL,
in that it is used to join to files based on a
common value to form a third file.
21
A Brief Introduction to the AWK Command

• awk pattern-scanning and processing language
• Helps to produce reports that look professional
• Inventors A. Aho, P. Weinberger, and B.
  Kernighan
• Some of the tasks you can do with awk include
• Manipulate fields and records in a data file
• Use variables
• Use arithmetic, string, and logical operators
• Execute commands from a shell script
• Use classic programming logic, such as loops
• Process/organize data into well-formatted reports
• Example
• awk -F printf "s\t s\n", 1, 2 datafile

Practice Exercises
See homework.
Additional References
None
The awk command is a c-like report writer
included with UNIX/Linux.
22
Summary

• UNIX/Linux support regular files, directories,
  character special files, and block special files.
• touch updates a files time/date stamp and to
  create empty files.
• rmdir removes an empty directory. Use rm r to
  remove non-empty directories.
• cut extracts specific columns or fields from a
  file
• paste combines two or more files
• sort sorts a files contents
• Create shell scripts to automate tasks
• join extracts information from two files sharing
  a common field
• Awk is a pattern-scanning and processing language
  used to create a formatted report.

Practice Exercises
None
Additional References
None
23
Command Summary
Practice Exercises
None
Additional References
None
24
Command Summary
Practice Exercises
None
Additional References
None