Filters Using Regular Expressions: Grep and Sed

1
Filters Using Regular Expressions Grep and Sed

• Brian C. Hoffman
• CINS 142

2
grep - Finding an Expression

• As we saw in our lesson on common commands, grep
  takes the
• form.
• grep options pattern file list
• Options
• c, count number of lines with matches
• l, display names of files with matches only
• n, display line numbers along with lines
• i, ignore case
• h, omit filenames when handling multiple files
• v, display lines that do not match

3
When grep Fails

• If the grep command fails to find a string
  matching the given pattern it simply returns to
  the command prompt with no message.
• However, grep does return a failure value to a
  shell variable (?) that can be used within a
  shell script.

4
Using Regular Expressions

• Our goal is to make grep more flexible and search
  for strings that match general patterns instead
  of only looking for specific ones.
• To accomplish our goal we will use regular
  expressions, strings that contain meta-characters
  that can be used to indicate a various patterns.
  
• In the following slides we will examine the set
  of meta-characters one can use in these
  expressions. These characters are fairly
  universal and are also used in Perl.
• See the table on p441 in the book for helpful
  hints.

5
Quoting in Expressions

• When using multiword strings, you must quote the
  entire expression or every word after the first
  will be treated as file names.
• You can use either single () or double
  quotes(), but if the expression requires an
  apostrophe(), you are forced to use double
  quotes.
• If you require any kind of command substitution
  (see p249) or variable evaluation, you should use
  double quotes.

6
The Meta-Character

• To begin, in a regular expression does not work
  the same as the wildcard. They are used in
  different contexts and have very different
  meanings.
• The meta-character refers to the character that
  directly precedes it. It matches 0 or more
  occurrences of that character.
• Examples
• myg will match my, myg, mygg, myggg,
  mygggggg, mygggggg
• E will match the null string and e,
  ee, eee, eeee, eeeee,
• trueman will match both truman and trueman. It
  will also match
• trueeman

7
The Character Class

• Regular expression may also use character
  classes. These work much the same way as they did
  in the lesson on shells.
• Character classes are indicated by and will
  allow any character in the included range to take
  its place.
• ijk replace this spot with an i,j,or k
• a-f replace this spot with character in range
  a-f
• 1-3 replace this spot with character in the
  range 1-3
• Character classes can be negated using
• ijk replace this spot with any character
  but i,j,or k
• a-f replace this spot with a character not
  in range a-f
• Notice that the negation character is and not !

8
The . Meta-Character

• The . character will match any single character.
  It plays the same role for a regular expression
  that ? plays for the shell.
• Examples,
• wom.n will match women, woman, wombn, womzn,
  
• ..b will match afb, dzb, ffb, abb,
• Per. will match perm, pern, pert, perp,
• . is a very special combination. It indicates
  any possible combination of characters or none.
  For example, wood. will match wood, woods,
  woodsman, or woody.

9
The and Meta Characters

• The and characters are used to indicate
  position in the line.
• indicates the front
• indicates the back
• For example, 2 will search for a 2 followed by 3
  characters anywhere in the line, but 2 will only
  look for lines that start with a 2.
• Other Examples
• fred find lines that
  begin with fred
• fred find lines that end
  with fred
• find lines that
  contain nothing
• ls -l grep ..w finds files where group
  has write permission

10
Other Useful Expressions

• Here are some other examples from the table on
  p441.
• \m\ look for m occurences of the previous
  character Ex. .\9\nobody look
  for nobody after skipping 9 characters
  from the
  front
• \m,\ look for at least m occurences of
  previous character
• \m,n\ look for between m and n occurences of
  the previous character

11
egrep - An Extended Grep

• egrep adds some additional features to the
  standard grep command
• You can include multiple expressions to match
  against by using the character.
• For example, egrep woodwoodswoodsman will
  print matches for any of the three components.
• You can load the patterns from a file using the
  -f option. The patterns must be stored using the
  separator.
• For example, egrep -f pat.dat will search for
  all matches to the patterns in pat.dat in the
  current directory.
• Using () will create an OR condition. (x1x2)x3
  means x1x3 or x2x3 but not both
• You can use some additional meta-characters (see
  table p447)
• like but must have at least one occurrence
• ? matches a single instance or nothing

12
fgrep - Fast Grep

• fgrep is a fast version of grep, but it doesnt
  use any regual expression characters. It is best
  for rapidly finding simple expressions.
• fgrep can search for multiple strings like egrep,
  but the different expressions must be placed on
  separate lines instead of being separated by a .
• Example
• fgrep hello
• fred
• william

13
sed - The Stream Editor

• Sed is a non-interactive editor that allows the
  user to modify a file by using locators and
  commands.
• The general format of the sed command is
• sed options locator action
  fname(s)
• Notice that the locator and the action both go
  within the same set of single quotes. You can use
  double quotes in those cases where you need
  command substitution or variable evaluation.

14
Line Addressing

• One of the ways of specifying a location is to
  use line addressing.
• The general format of a line address is
  startLine,endLinewhere startLine and endLine are
  integers that give the relevant line numbers.
• Examples
• sed 20q file prints the first 20
  lines and quits
• sed -n 1-20p file also prints the first 20
  lines
• sed 10,20p file prints lines 10-20

15
Line Addressing Continued

• The -n option
• By default sed prints every line in the file
  unless the user quits. A command lik sed 1-20p
  will print the first 20 lines and then print the
  whole file. To suppress the printing of the whole
  file use -n.
• Reversing line selection
• The ! Character indicates that one should do the
  reverse of what is stated. In the case of line
  selection it will work will all but the indicated
  linesex sed n 3,!p emp.lst will print
  all but lines 3-end

16
Selecting Multiple Chunks

• The e option can be used to combine several
  instructions onto one line. For example,
• sed n 1,2p e 7,9p e 20,25p emp.lst
• will print lines 1,2,7,8,9,20,21,22,23,24,25

17
Context Addressing

• Context addressing find locations to act on by
  searching for a pattern.
• The general form of a context address is /expr/
  where expr is a literal or regular expression one
  can search for.
• Multiple expressions can be joined using commas
  /expr1/,/expr2/,/expr3/
• For example the following will print out all from
  lines in the mailbox
• sed n /From /p HOME/mbox

18
Some Sed Commands

• q quit after reading up to
  addressed line
• p print the lines to the screen
• i,a,c insert, append, change
• d delete lines
• s/s1/s2 replace first occurrence of s1 in
  all lines with s2
• s/s1/s2/g replace all occurrences of s1 with
  s2
• For some good examples see tabe on p449 and read
  through the
• text.