Scommands Tutorial

1
Scommands Tutorial
By Roman Olschanowsky roman2u_at_sdsc.edu
2
Scommands

• Command line access to the SRB
• Included in TeraGrid CTSS (current v. 3.4.1)
• Download then compile from http//www.sdsc.edu/srb
  /tarfiles/main.html (make sure you get
  non-encrypted client only, as well as correct
  version that matches server version)
• Login to a machine with Scommand binaries
• via ssh to a nix machine
• Win32 binaries from command window

3
Scommand Features

• Command line interface - SCRIPTING
• Available for all of the most popular UNIX
  flavors and DOS
  
• S-commands are the most flexible and powerful of
  the clients
  
• They are the fastest, and most reliable
• They are multithreaded for big gains in data
  flow
  
• They are great for scripts, perl wrappers, batch
  jobs, etc
  
• Installed man pages via man Scommand
• man Sput

4
Common Scommands (69 total)

• Sinit
• Senv
• Spwd
• Sls
• Scd
• Sget
• Sput
• Ssh

• Scp
• Smv (logical)
• Sphymove (physical)
• Srm
• Smkdir
• Srmdir
• Serror
• Schmod

• Dont forget to Sexit!

5
Scommand Notes

• Shelp
• Gives list of commands with brief summary
• Scommand gives usage info (usually)
  or try h flag
  
• Sinit establishes a session
• Senv displays connection information
• Spwd display current working directory
• Sexit ends session

6
SRB Connection parameters

• Verify/Create /.srb/.MdasEnv
• Verify/Create /.srb/.srbAuthFile
• OR
• Set environment variables

7
Teragrid SRB

• All Teragrid accounts are given a SDSC SRB
  Teragrid account
  
• The username is the same as your SDSC UNIX
  account name
  
• Your SRB domain is teragrid
• You must register your DN string with SDSCs
  grid-mapfile or request a SRB password to
  activate your SRB account. Instructions to do so
  are here http//www.sdsc.edu/user_services/srb/ac
  count_form.html

8
/.srb/.MdasEnv File

• mdasCollectionHome '/home/roman2u.teragrid'
• srbUser 'roman2u'
• mdasDomainName teragrid
• AUTH_SCHEME 'GSI_AUTH'
• AUTH_SCHEME 'ENCRYPT1
• srbHost srb.sdsc.edu'
• srbPort 7321'
• defaultResource sfs-tape-tgd
• SERVER_DN '/CUS/ONPACI/OUSDSC/CNStorage
  Resource Broker/USERIDsrb'

9
Environment Variables

• !/bin/sh
• export mdasCollectionName"/home/roman2u.teragrid"
  
  
• export srbHostsrb.sdsc.edu"
• export srbPort7321"
• export srbUserroman2u"
• export mdasDomainNameteragrid"
• export defaultResourcesfs-tape-tgd"
• export AUTH_SCHEMEGSI_AUTH
• export SERVER_DN'/CUS/ONPACI/OUSDSC/CNStorage
  Resource Broker/USERIDsrb'
  

10
/.srb/.srbAuthFile

• Sauth
• Enter your current SRB password
• Successfully wrote /.srb/.srbAuthFile
• .CS2/j6A"8

11
Other authentication methods

• AUTH_SCHEME
• 'ENCRYPT1' random message encrypted with
  your password between clients servers.
  
• 'GSI_AUTH' - Use the Globus GSI
  authentication scheme.
  
• 'GSI_DELEGATE' - Use the GSI Delegation
  (proxy) certificate for authentication. The
  advantage is that this certificate can be passed
  from server to server whereby the user's identity
  continues to be maintained across servers and
  across zones. This scheme solves the cross zone
  authentication issues.
• 'GSI_SECURE_COMM' - Use the GSI
  authentication scheme and use the GSI I/O library
  for all socket communication between client and
  server.

12
Step by Step

• 1. Setup
• First!, we need to be on a UNIX machine
• ssh to tg-login.sdsc.edu
• soft add globus-4.0.1
• grid-proxy-init
• gx-map -interactive
• soft add srb-client-3.4.1-r1
• mkdir .srb
• cd .srb
• cp /usr/local/apps/srb-client-3.4.1-r1/doc/tg.Mdas
  Env .MdasEnv

13
Step by Step

• 2. Connect
• edit your .MdasEnv file, making it correct for
  your srb_username SRB account
  
• mdasCollectionHome '/home/srb_username.teragrid'
• srbUser srb_username'
• Sinit -v
• ls al notice the new .MdasEnv.?????? file,
  this file is associated with the current shell
  and saves your SRB session state (Like your
  current working dir).

14
Step by Step

• 3. Some basics
• Sls
• Smkdir test
• Sls
• Scd test
• Spwd
• cat .MdasEnv.?????
• mdasCollectionName '/home/roman2u.teragrid/test'
• mdasCollectionHome '/home/roman2u.teragrid'
• mdasDomainName teragrid'
• srbUser roman2u'
• srbHost srb.sdsc.edu'
• srbPort 7321'
• mcatZone 'demozone'
• defaultResource sfs-tape-tgd
• Sexit, ls al, notice it is now gone

15
Step by Step

• 4. Some basics
• pwd, cd
• Sinit
• Spwd (notice you are back in your SRB home dir)
• Senv
• Shelp
• Serror
• Smkdir /home/Test
• Serror -3220

16
Step by Step

• 5. Wait, how many are there?
• Scd /home/Demo/SRB-Tutorial/files-2
• Sls
• How many Doc.txt files are there???
• Sls l
• How many are there now?
• They are all the same file, the file has been
  replicated to different resources which is
  different than copied to different resources.
  
• The number after the owner column is the
  replication number, followed by the physical
  resource.
  

17
Step by Step

• 6. Lets try one
• Scd to go to your SRB home dir
• Smkdir tutorial
• Scd tutorial
• Scp /home/Demo/SRB-Tutorial/comics/bc.gif .
• Sls l
• There should be just one replica of bc.gif
• Sreplicate S sfs-tape-sdsc bc.gif
• Sls l
• Is it replicated?
• Why replicate? Fault tolerance, redundancy and
  faster access
  
• SgetR will list all resources
• Smv only does logical move (renaming or path
  change)
  
• Sphymove will move the file physically

18
Step by Step

• Best Practices for S-command uploads and
  downloads
  
• Low Latency
• Just upload, or recursive r option
• High Latency
• If its one big file, use Sput or Sget with M
  option
  
• If its a directory with lots of small files, use
  b option

19
Step by Step

• 7. Download and Upload a directory
• Scd /home/Demo/SRB-Tutorial
• Sget r comics
• Scd
• time Sput r comics
• time Sput b comics comics2
• time Sput b comics comics3
• Sls l comics Sls l comics2
• time Srm r comics2
• time Srm rf comics3

20
Step by Step

• 8. SRB Trash system
• Spwd
• Add /trash to the front
• Sls /trash/home/roman2u.teragrid
• Thats why it was so fast, it did not erase it,
  it moved it (logically). Great news! You can
  recover your data, just use Smv command to put
  data you want back.
• /trash/home/roman2u.teragrid/comics22005-07-26-1
  0.39.07
  
• Adds timestamp to name to avoid conflicts, and to
  inform you when data was trashed

21
Step by Step

• 9. Some user defined meta data
• Sufmeta
• Sufmeta Q Make Ford
• Sufmeta c Contents Funny comics
• Sufmeta Parts 4 comics/bc.gif
• Sufmeta Rating 5 star comics/bc.gif
• Sufmeta comics/bc.gif
• Smeta comics/bc.gif
• You will see that each row of user defined
  metadata is a portion of 9 possible strings and 2
  possible integers. 99 of the time, string0 is
  the attribute, string1 is the value, and string2
  are the units. But feel free to use more!

22
Step by Step

• 10. Put your mark on the tutorial
• vi .txt
• Contents of file Picture of my car
• Scd /home/Demo/SRB-Tutorial/cars
• Sput .txt
• Sls C .txt
• Schmod r npaci groups .txt
• Sls C .txt
• Sufmeta Make .txt
• Sufmeta Model .txt
• Sufmeta Year .txt

23
Step by Step

• 11. Copying and Replicating
• Scd
• Scp S sfs-tape-sdsc comics/bc.gif .
• Sls l
• Sreplicate bc.gif
• Sreplicate S sfs-tape-tgd bc.gif
• Sls l
• vi comics/myComic.txt
• Srsync -r comics scomics
• Sbkupsrb -r -S sfs-tape-sdsc comics
• Sls l
• Why did we use Sbkupsrb instead of Sreplicate?
  Would have worked fine the first time, but
  subsequent calls will keep making more replicas.

24
Step by Step

• 12. Rodger.Signing off
• Sexit
• Thanks!
• Questions?
• www.sdsc.edu/srb
• srb_at_sdsc.edu