Linux Intermediate

1
Linux Intermediate

• ITS Research Computing Center
• C. D. Poon, Ph.D.
• Email cdpoon_at_unc.edu

2
Class Material

• Point web browser to http//its.unc.edu/Research
• Click on Training on the left column
• Click on ITS Research Computing Training
  Presentations
• Click on Linux Intermediate

3
Outline

• Linux Command Category
• Stdout/Stdin/Stderr, Pipe and Redirection,
  Wildcards
• Linux Command Review
• Break
• Tips and Tricks
• Conclusion
• Exercise

4
Linux Command Category
5
Linux Command Category

• Communication
• ssh scp
• File/Directory Management
• cat cd chmod cp ln ls mkdir more less mv pwd
  dirs rm head tail wc
• Comparisons
• diff

6
Linux Command Category Contd

• Searching
• grep find locate
• Archiving
• compress uncompress gzip gunzip zcat tar
• Text Processing
• cut paste sort sed awk

7
Linux Command Category Contd

• System Status
• chgrp chown date df du env who w uptime
• Miscellaneous
• bc cal clear man

8
Stdout/Stdin/StderrPipe and RedirectionWildcards
9
stdout stdin stderr

• Output from commands
• usually written to the screen
• referred to as standard output (stdout)
• Input for commands
• usually come from the keyboard (if no arguments
  are given
• referred to as standard input (stdin)
• Error messages from processes
• usually written to the screen
• referred to as standard error (stderr)

10
Pipe and Redirection

• Pipe () stdout of one command to stdin of
  another command
• Output Redirection (gt) stdout of a command to a
  file
• Output Appending (gtgt) stdout of a command
  appending to a file
• Input Redirection (lt) stdin of a command from a
  file

11
Wildcards

• Multiple filenames can be specified using special
  pattern-matching characters. The rules are
• ? matches any single character in that position
  in the filename
• matches zero or more characters in the
  filename.
• Characters enclosed in square brackets
  match any name that has one of those characters
  in that position
• Note that the UNIX shell performs these
  expansions before the command is executed.

12
Linux Command Review
13
ssh

• Log on to remote machine
• ssh cdpoon_at_emerald.isis.unc.edu
• ssh emerald.isis.unc.edu l cdpoon
• ssh topsail
• ssh X cedar.isis.unc.edu
• ssh Y tarheelgrid.unc.edu

14
ssh using SecureCRTin Windows

• Using ssh, login to Emerald, hostname
  emerald.isis.unc.edu
• To start ssh using SecureCRT in Windows, do the
  following.
• Start -gt Programs -gt Remote Services -gt SecureCRT
• Click the Quick Connect icon at the top.
• Hostname emerald.isis.unc.edu
• Login with your ONYEN and password

15
scp

• Copy files and directories to and from remote
  computers
• scp file1 topsail.isis.unc.edu/ifs1/home/cdpoon/
  .
• scp zircon.its.unc.edu/home/cdpoon/file2 .
• scp r dir1 emerald.isis.unc.edu/netscr/cdpoon/.
  
• scp r topsail.isis.unc.edu/ifs1/scr/cdpoon/dir2
  dir3
• scp emerald/netscr/cdpoon/f topsail/ifs1/scr/cd
  poon/.

16
cat

• Read one or more files and print the on stdout
• cat file1
• cat file1 file2 file3 gt file_all
• cat file4 gtgt file_all
• Append file4 to file_all
• cat gt file5
• Create file at stdin, end with EOF (cntl-d
  normally, use stty a to find out)
• cat gt file6 ltlt STOP
• Create file at stdin, end with STOP

17
cd

• Change directory, build-in shell command
• cd /afs/isis/home/c/d/cdpoon
• cd ../../ Change directory to 2 levels up
• cd .. Change directory to 1 level up
• cd Change directory to Home
• cd Change directory to Home
• cd Change to previous directory

18
chmod

• Change the access mode of one or more files
• chmod ux file1
• chmod go-w file2
• chmod urwx, grx, ox file3
• chmod 751 file3 Same as above, 7rwx, 5rx,
  1x
• chmod r file4
• chmod 444 file4 Same as above, 4r, 2w, 1x

19
cp

• Copy a file/dir to another file/dir
• cp file1 file2 Copy to the same directory and
  change filename
• cp file1 ../dir/file2 Copy to different
  directory and change filename
• cp file1 ../dir/. Keep the same filename
• cp r dir1 dir2 Copy directory recursively
• cp r dir1 new_dir/dir2
• Copy directory recursively to another
  directory
• cp p file3 file4 Preserve the modification time
  and permission modes

20
ln

• Create links for file/dir and allow them to be
  accessed by different names
• ln file1 file2 Hard link for file
• ln dir1 dir2 Hard link not allowed for
  directory
• ln s dir1 dir2 Soft link for directory, dir2
  -gt dir1
• ln s file3 file4 Soft link, file4 -gt file3
• ln s dir/file5 file6 Soft link, file6 -gt
  dir/file5

21
ls

• List all files and directories in the current
  directory
• ls
• ls a List files/directories starting with .
  too
• ls l Long listing
• ls lh List file sizes in human readable format
• ls F Flag filenames by appending / to
  directories, to executables files,
• and _at_ to symbolic links

22
mkdir

• Create one of more directories
• mkdir dir1
• mkdir p dir1/dir2/dir3
• Create intervening parent directories if they
  dont exist
• Same as mkdir dir1 cd dir1 mkdir dir2 cd
  dir2 mkdir dir3 cd ../../

23
more

• Display files on a terminal, one screen at a time
• more file1 Hit space bar for another page, q to
  quit
• more d file2 Display the prompt Press space to
  continue, q to quit
• more c file3 Page through the file by clearing
  each window instead of
• scrolling

24
less

• Works like more but allows backward and forward
  movement
• less file1 Hit space bar for another page, q to
  quit
• Hit b to scroll backward one window
• Hit /pattern to highlight pattern in the
  text
• Hit Return to scroll one line at a time

25
mv

• Move files and directories within the same
  machine and/or rename them
• mv file1 dir1/file1 Move file1 to dir1, Same as
  mv file1 dir1/
• mv file3 file4 Rename file3 to file4
• mv dir1 dir2 Rename directory dir1 to dir2
• mv dir3 dir4/dir5/dir6
• Rename directory dir3 to dir6 and move to
  dir4/dir5 directory

26
pwd dirs

• Print the full pathname of the current directory
• pwd
• dirs C shell built-in command, works like pwd
• dirs l Print working directory in long listing

27
rm

• Delete one or more files and directories
• Delete empty directory with rmdir
• rm file1
• rm file Remove all files with filename
  starting as file
• rm i file Prompt for y (remove the file) or n
  (do not remove the file)
• rm r dir1 Delete directory dir1 and its
  content

28
head tail

• Print first/last few lines of one or more files
• head file1 Print the first 10 lines of file
  file1
• head n100 file2 Print the first 100 lines of
  file file2
• tail file Print the last 10 lines of files
  with filename starting as file
• tail f file3 Print the last 10 lines of file
  file3 and follow file as it grows

29
wc

• Print a character, word, and line count for files
• wc c file1 Print character count for file
  file1
• wc l file2 Print line count for file file2
• wc w file3 Print word count for file file3

30
diff

• Report lines that differ between file1 and file2,
  with file1 text flagged by lt and file2 by gt
• diff file1 file2 Show difference between file1
  and file2

31
grep

• Search for lines that match a regular expression
• grep abc file1
• Print line(s) in file file1 with abc
• grep i abc file2
• Print line(s) in file file2 with abc
  ignoring uppercase and lowercase distinctions

32
find

• Find particular groups of files
• find . name temp Find file named temp in
  current directory
• find /etc name rc Find file(s) in /etc
  directory with name starting with rc
• find /usr/share/man type d name man
• Find directories in /usr/share/man with name
  starting with man
• See Handout

33
locate

• Find files with matching pattern in database
  prepared by updatedb, Database needed to be
  updated daily
• locate which Find files named with pattern
  which in the OS
• locate c which Count number of files named
  with pattern which in the OS
• locate i which Find files named with pattern
  which in the OS ignoring case
  distinctions
• See Handout

34
compress uncompress

• Reduce or expand the size of one or more files
  using adaptive Lempel-Ziv coding
• Use uncompress to expand data
• compress file1 Reduce the size of file1 and
  create new file named file1.Z
• compress f file2 Force to reduce the size of
  file2 and create new file named file2.Z
• uncompress file3.Z Expand file3.Z and restore
  file3

35
gzip gunzip

• Reduce or expand the size of one or more files
  using Lempel-Ziv coding (LZ77)
• Use gunzip to expand data
• gzip file1 Reduce the size of file1 and create
  new file named file1.gz
• gzip f file2 Force to reduce the size of file2
  and create new file named file2.gz
• gunzip file3.gz Expand file3.gz and restore
  file3

36
zcat

• Expand the size of one or more files created by
  compress or gunzip
• List file contents to stdout without deleting the
  .Z or .gz file
• zcat file1.Z Expand file1.Z and list the content
  of file1 in stdout
• zcat file2.gz Expand file2.gz and list the
  content of file2 in stdout

37
tar

• Archive files and directories
• Create a single file with extension .tar
• tar cvf file123.tar file1 file2
  file3 Create archive file named file123.tar
  in verbose mode with contents, file1, file2,
  and file3
• tar xvf file123.tar Expand file123.tar
  in verbose mode and generate the original
  files and directories back

38
cut

• Remove sections from each line of files
• cut d -f1,5 /etc/passwd Use field
  delimiter to locate fields 1 and 5 from file
  /etc/passwd to extract usernames and real
  names
• cut c4 file1 Take character 4 out from
  each line of file1 and display in stdout
• See Handout

39
paste

• Merge lines of files
• cat file1
• 1
• 2
• cat file2
• a
• b
• c

paste file1 file2 1 a 2 b
c paste s file1 file2 1 2 a
b c
40
sort

• Sort lines of text files
• sort fd file1
• Alphabetize lines (-d) in file1 and ignore
  lower and upper cases (-f)
• sort t -k3 -n /etc/passwd Take column 3 of
  file /etc/passwd separated by and sort in
  arithmetic order
• See Handout

41
sed

• Edit one or more files without user interaction
  using stream editor
• sed s/xx/yy/g file1
• Substitude all occurrences of xx in file1
  with yy and display on stdout
• sed /abc/d file1 Delete all lines containing
  abc in file1
• sed /BEGIN/,/END/s/abc/123/g file1
• Substitute XYZ on lines between BEGIN and
  END with xyz in file1
• See Handout

42
awk

• Process files by pattern-matching
• awk F print 1 /etc/passwd
• Extract the 1st field separated by in
  /etc/passwd and print to stdout
• awk /abcde/ file1
• Print all lines containing abcde in file1
• awk /xyz/i ENDprint i file2
• Find pattern xyz in file2 and count the
  number
• awk length lt 1 file3
• Display lines in file3 with only 1 or no
  character
• See Handout

43
chgrp

• Change the group ownership of one or more files
  or directories
• chgrp employee file1
• Change group ownership to employee for file
  file1
• chgrp R student dir1
• Change group ownership to student for
  directory dir1 including
• subdirectories recursively

44
chown

• Change the ownership of one or more files or
  directories
• chown employee file1
• Change ownership to employee for file file1
• chown R student dir1
• Change ownership to student for directory
  dir1 including
• subdirectories recursively

45
date

• Print the current date and time in certain format
• Set the current date and time
• date
• Print the current date and time
• date D
• Print the current date and time in mm/dd/yy
  format
• date 1201160108
• Set the current date and time to Dec 01 401pm
  2008
• date d fri
• Show the date of the coming Friday

46
df

• Report the number of used and free disk block on
  all mounted file systems
• df
• Print used and free disk block on all mounted
  file system
• df -k
• Print used and free disk block in kilobyte

47
du

• Print disk usage of directories and its
  subdirectories
• du dir1 Print disk usage in kilobyte of
  directory dir1
• du -block-size1M dir2
• Print disk usage in megabyte of directory
  dir2

48
env

• Display the current environment variables or set
  new values
• env Display all of the current environment
  variables

49
who

• Display information about the current status of
  the system
• who Display the names of users currently logged
  in to the system
• who b Report information about the last reboot
• who am I Print the username of the invoking user

50
w

• Print summaries of system usage, currently
  logged-in users, and what they are doing
• w Print summaries of system usage, currently
  logged-in users
• w s Display in short form

51
uptime

• Print the current time, amount of time logged in,
  and the system load averages
• uptime Print a one line display of the current
  time, how long the system has been running,
  how many users are currently logged on, and
  the system load averages for the past 1, 5, 15
  minutes

52
bc

• Interactively perform arbitrary-precision
  arithmetic or convert numbers from one base to
  another, type quit to exit
• bc Invoke bc
• 12 Evaluate an addition
• 56/7 Evaluate a multiplication and division
• ibase8 Change to octal input
• 20 Evaluate this octal number
• 16 Output decimal value
• ibase10 Change back to decimal input

53
cal

• Print calendar of a month or all months in a year
• cal Print calendar of the current month
• cal 2 2009 Print calendar of February 2009
• cal 2009 Print calendar of all months in 2009
• cal -3 Display previous/current/next months

54
clear

• Clear the terminal display and have the prompt
  locate at the top of the terminal window
• clear Clean up the current terminal display

55
man

• Display information from the online reference
  manuals
• man man Display the manual for the command
  man
• man k link compile Display commands related to
  linking and compiling using a keyword search

56
Break
57
Tips and Tricks
58
Tips and Tricks 1

• Show files changed on a certain date in all
  directories
• ls l grep Sep 26
• Show long listing of file(s) modified on Sep
  26
• ls lt grep Dec 18 awk print 9
• Show only the filename(s) of file(s) modifed
  on Dec 18

59
Tips and Tricks 2

• Sort files and directories from smallest to
  biggest or the other way around
• du k s sort n
• Sort files and directories from smallest to
  biggest
• du ks sort nr
• Sort files and directories from biggest to
  smallest

60
Tips and Tricks 3

• Change timestamp of a file
• touch file1
• If file file1 does not exist, create it, if
  it does, change the timestamp of it
• touch t 200902111200 file2
• Change the time stamp of file file2 to
  2/11/2009 1200

61
Tips and Tricks 4

• Find out what is using memory
• ps ely awk print 8,13 sort k1 nr
  more

62
Tips and Tricks 5

• Remove the content of a file without eliminating
  it
• cat /dev/null gt file1

63
Tips and Tricks 6

• Backup selective files in a directory
• ls a gt backup.filelist
• Create a file list
• vi backup.filelist
• Adjust file backup.filelist to leave only
  filenames of the files to be backup
• tar cvf archive.tar cat backup.filelist
• Create tar archive archive.tar, use backtics
  in the cat command

64
Tips and Tricks 7

• Get screen shots
• xwd out screen_shot.wd
• Invoke X utility xwd, click on a window to
  save the image as screen_shot.wd
• display screen_shot.wd
• Use ImageMagick command display to view the
  image screen_shot.wd
• Right click on the mouse to bring up menu,
  select Save to save the image to other
  formats, such as jpg.

65
Tips and Tricks 8

• Sleep for 5 minutes, then pop up a message Wake
  Up
• (sleep 300 xmessage near Wake Up)

66
Tips and Tricks 9

• Count number of lines in a file
• cat /etc/passwd gt temp cat temp wc l rm
  temp
• wc l /etc/passwd

67
Tips and Tricks 10

• Create gzipped tar archive for some files in a
  directory
• find . name .txt tar c T - gzip gt
  a.tar.gz
• find . name .txt tar cz T - -f a.tar.gz

68
Tips and Tricks 11

• Find name and version of Linux distribution,
  obtain kernel level
• uname -a
• head n1 /etc/issue

69
Tips and Tricks 12

• Show system last reboot
• last reboot head n1

70
Tips and Tricks 13

• Combine multiple text files into a single file
• cat file1 file2 file3 gt file123
• cat file1 file2 file3 gtgt old_file
• cat find . name .out gt file.all.out

71
Tips and Tricks 14

• Create man page in pdf format
• man t man ps2pdf - gt man.pdf
• acroread man.pdf

72
Tips and Tricks 15

• Remove empty line(s) from a text file
• awk NFgt0 lt file.txt
• Print out the line(s) if the number of fields
  (NF) in a line in file
• file.txt is greater than zero
• awk NFgt0 lt file.txt gt new_file.txt
• Write out the line(s) to file new_file.txt if
  the number of fields (NF)
• in a line in file file.txt is greater than
  zero

73
Conclusion
74
Conclusion

• Many ways to do a certain thing
• Unlimited possibilities to combine commands with
  , gt, lt, and gtgt
• Even more powerful to put commands in shell
  script
• Slightly different commands in different Linux
  distributions
• Emphasized in System V, different in BSD

75
Exercise
76
Exercise

• cd Change to home directory
• cat /etc/passwd gt temp Save the file /etc/passwd
  into a new file called temp
• more temp wc l Find out the number of lines
  in file temp
• awk /home/ temp Show line(s) with the word
  home
• awk /nologin/i ENDprint i temp
• Count number of word nologin in file
  temp
• sed s/no/yes/g temp gt temp1 Replace all no with
  yes in temp and save in temp1
• awk /nologin/i ENDprint i temp1
• Count number of work nologin in file
  temp1

77
Exercise Contd
awk /yeslogin/i ENDprint i
temp1 Count number of work yeslogin in
file temp1 grep i daemon temp Display
line(s) with word daemon regardless of
case more temp grep i daemon Display line(s)
with word daemon regardless of case sort t
-k3 n temp Sort numerically over the 3rd column
in file temp separated by rm i
temp Delete the file(s) with filename starting
with temp interactively