sed

1
sed
2
Class Issues

• vSphere Issues
• root only until lab 3

3
sed

• sed stream editor
• Allows you make changes to text files on the fly
  on a line by line basis
• Basic syntax is
• sed s/pattern/replacement/ filename
• Works on files, default output is to the screen

4
Basic Examples

• Replace
• sed s/1/ONE/g file.txt (would replace the first
  1 with ONE in file.txt)
• Swap
• sed 'y/abc/ABC' file.txt (replace a with A, b
  with B, c with C)
• Show line numbers then lines
• sed file.txt
• Lowercase sed 's/A-Z/\l/ file.txt

5
Special Replacements

• \L - turn the replacement to lowercase until a \U
  or \E is found,
• \l - turn the next character to lowercase
• \U - turn the replacement to uppercase until a \L
  or \E is found
• \u - turn the next character to uppercase,
• \E - stop case conversion started by \L or \U.

6
Remember the g?

• First Character Lower Case
• sed 's/A-Z/\l/' addresses.txt
• All Characters Lower Case
• sed 's/A-Z/\l/g' addresses.txt
• 5th Character Lower Case
• sed 's/A-Z/\l/5' addresses.txt

7
Useful sed command line

• -f means read sed commands from file
• sed f mysed.txt ltfile listgt

8
Gotchas

• Dont do
• sed 's/A-Z/\l/3-5' addresses.txt gt
  addresses.txt
• Create a new file and mv it back.

9
awk
10
awk

• Programming language for data streams usually
  files or subsets of files
• Developed at Bell Labs in the 70s by Alfred
  Aho, Peter Weinberger, and Brian Kernighan
• Like sed but does the following as well
• specify particular fields of a matching line to
  manipulate and/or output
• use variables to store values temporarily
• perform calculations on those variables.

11
How to run?

• Usually
• awk ltawk commandsgt files to parse
• Example (a cat replacement)
• awk print 0

12
Fields in awk

• By default lines are delimited by white space
  space characters or tabs
• 0 whole line
• 1 is first field in line
• 2 is 2nd field in line
• NF number of fields on line
• No error reported if field doesnt exist, simply
  outputs a blank line

13
Basic Pattern Matching

• /pattern/ do something
• Example
• /A/ print This line has an A
• No pattern means do the work on every line

14
Multiple patterns

• Just list them
• awk
• /A/ print this line has an A
• /B/ print this line has a B
• print this gets printed always
• my_file
• Dont forget the closing

15
Forming Patterns

• Can use most of the extended regular expressions,
  except
• Can do the relational expressions from other
  languages lt, gt, , !, lt, gt

16
Comparisons

• Numeric example
• ls -l awk '5 gt 512 print 0'
• String example
• ls -l awk '6 Sep print 0'

17
Variables

• Variables are allowed
• ls -l awk '5 gt 512 size 5
• Variables dont have to be initialized
• However, we wont ever see size output!

18
BEGIN/END

• BEGIN run before any lines are read
• END run after every line has been read
• You can have one, both, neither
• Example
• ls -l awk '5 gt 512 size 5
• END print size

19
Begin Example

• awk BEGIN printf(Im pretending to be cat\n)
  
• print 0 my_file
• printf is C/Java like

20
Useful BEGIN command

• Remember BEGIN goes before reading of lines
  begins
• Want the first line of your file as a header? Use
  getline, but no pattern matching or processing of
  the lines read
• awk BEGIN getline print 0 my_file

21
Useful awk variables

• FILENAME current file being parsed
• NF number of fields on the current line
• FNR current record in the file (usually means
  current line)
• NR current record in the stream
• There are many, many others as well, read the man
  page

22
Standalone awk scripts

• Add the following to top of file of awk commands
  !/usr/bin/awk f
• Dont forget to chmod x
• Example
• !/usr/bin/awk -f
• /.\// dots 1
• // blank 1
• /l/ sym 1
• /d/ dir 1
• /-/ file 1
• total 1
• END printf("Dotsd\nBlank Linesd\nSym
  Linksd\nDirectoriesd\nFilesd\nTotald\n",do
  ts, blank, sym, dir, file, total)

23
Intro to Shell Scripting
24
Shell Script 101

• List of commands interpreted by a shell
• cd /var/tmp
• mkdir boks
• cd boks
• gzip -dc /var/tmp/boks_install.tar.gz tar xf -
• ./install.jvss0001
• cd /var/tmp
• rm -r boks
• cd /etc/rc2.d
• mv S99boksm _S99boksm

25
Why need?

• Add users
• List bad perms
• Parsing log files
• A menu system for users
• Startup of system (.bashrc is a script)
• etc
• etc

26
Tests (if then else)
if count ne 0 then avg((sum/count)) else
avg0 fi
27
Looping (for)
for var in _at_ do echo var done
28
Variables in Scripts
I0 JI E((CD)) X((X1)) Notice the
lack of spaces!
29
Input/Output/Assignment

• Input
• read
• Output
• echo
• printf

30
Comparisions

• Numeric
• -eq, -ne, -lt, -le, -gt, -ge
• Strings
• , !
• And is
• Or is

31
Comparisions II

• File testing
• -d is the item a directory?
• -e does the file already exist?
• -f does the file exist and is it a regular
  file?
• -h (or L) is the item a symbolic link?
• (current user)
• -r does the file exist and is it readable?
• -w does the file exist and is it writable?
• -x does the file exist and is it executable?

32
Sample Comparisons

• -f my_file (dont forget spaces)
• x ge 0
• 1 start
• 1 empty string test!
• 1x x better empty string test!

33
Command Line Parameters

• Accessed via parameters
• 0 is script name
• 1, 2 9 parameters on the command line
• _at_ or are all parameters
• is total number passed
• Assignments say receive as parameter, assume its
  this one!

34
Accessing Parameters
!/bin/bash echo The command name is 0. echo
The number of command line arguments passed as
parameters are . echo Yet another way to show
total number . echo The value of the command
line arguments are 1 2 3 4 5 6 7 8
9. echo Another way to display values of all of
the arguments _at_. exit 0
35
Debugging?

• Set -vx shows info
• -v verbose
• -x lines after run