CS555: Get started with Globus in dt

1
CS555 Get started with Globus in dt

• Manolis Stamatogiannakis ltmstamat_at_csd.uoc.grgt
• Charalampos Gikas lthargikas_at_csd.uoc.grgt
• Spyros Antonatos ltantonat_at_csd.uoc.grgt
• Department of Computer Science, University of
  Crete

2
Set-up your path get certified

• csh/tcsh user shall add the following lines to
  their .cshrc file source hy555/globus/globusen
  v.sh
• Create a directory named .globus in your home
  directory.
• Run grid-cert-request to create a certificate
  request.Be sure you will remember your pass
  phrase!!!
• The request is located in usercert_request.pem
  file in your .globus directory. Email the request
  to hy555_at_csd.uoc.gr
• Copy the certificate (that will be mailed to you)
  to file usercert.pem in your .globus directory.
• Currently the grid consists of the following
  hostsio, semeli, danae

3
Running jobs globus-job-run

• First you have to login into the grid. The login
  lasts 12 hours by default. To login issue the
  globus-proxy-init command.
• The simplest way to run a program to the grid is
  the globus-job-run command. The syntax
  isglobus-job-run lthostnamegt ltprogramgt
  ltargumentsgt
• The program parameter must refer to the absolute
  path of the program. You can avoid this using the
  -s option before the program name. With -s
  globus automatically transfers the program to the
  host that it will be executed.
• You can start a multi-request using the -
  delimiter. If you want all programs to have
  common parameters, use the -args
  optionglobus-job-run args 1 1024 \- semeli
  s ./foo \- europe s ./bar

4
Batch jobs - globus-job-submit

• Batch jobs are executed in the background. You
  can remotely submit a job, logout and come back
  later to collect the results.
• You submit jobs using globus-job-submit. The
  syntax isglobus-job-submit lthostnamegt/jobmanager
  ltprogramgt ltargumentsgt
• The program must refer to an absolute path. You
  cannot use the -s option with
  globus-job-submit.
• After a successful submission a job handle is
  returned. Note down this handle! You can perform
  the following actions by using the handle as the
  parameter of the corresponding command
• Check the status of a job globus-job-status
  lthandlegt
• Retrieve the output of a job globus-job-get-outpu
  t lthandlegt
• Cancel a job globus-job-cancel lthandlegt
• Clear the files produced by a job
  globus-job-clear lthandlegt

5
Globus RSL scripts

• RSL scripts are an easy way to control the grid
  and its resources to run your programs.
• globus-job-run and globus-job-submit actually
  generate and execute RSL scripts. You can see the
  generated script by inserting the -dumprsl
  option just after the command.
• RSL scripts have the following syntax(relation
  1)(relation 2)(relation n)
• Each relation specifies a different detail of the
  job to be run.
• A full list of the available relations can be
  found on http//www.globus.org/gram/gram_rsl_para
  meters.html

6
Commonly used RSL relations

• (executablestring)Defines the program to be
  executed. The string is the absolute path to the
  program.
• (argumenslist)Specifies the arguments that will
  be supplied to the executable.
• (environmentlist)Environment variables needed
  for the job. The list is consisted of space
  separated pairs enclosed in parentheses. E.g. (CC
  gcc)
• (directorystring)Specifies the jobs active
  directory.

• (stdinstring)(stdoutstring)(stderrstring)Pro
  vide redirection for the standard I/O streams.
  The string can be a file or URL.
• (countinteger)Specifies the number of processes
  to be run.
• (resourceManagerNamestring)Specifies the grid
  machine where the job will be submitted by
  default.
• (maxCpuTimeinteger)Specifies the maximum CPU
  run time in minutes.

7
Resource co-allocation

• You can allocate multiple resources for your job,
  by grouping RSL expressions as shown below.
  ((relation 1.1)(relation 1.2)(relation
  1.n))((relation 2.1)(relation 2.2)(relation
  2.n))((relation m.1)(relation m.2)(relation
  m.n))
• This way you can run a job with many different
  executables, many different hosts, many different
  parameters etc.

8
Running RSL scripts - globusrun

• You run Globus RSL scripts using the globusrun
  command. The syntax is globusrun s r
  lthostnamegt -f ltscriptgt
• The s option is not the same with the one in
  globus-job-run command. It starts a GASS server
  (described later) for moving files.
• The f option can be omitted. In this case
  globusrun interprets the string following the
  hostname as the RSL script to be run.
• To submit batch jobs, add the b option after
  the script name. After successful submission a
  handle is returned. To check the status/kill a
  job, use globusrun status/-kill lthandlegt
• Issue globusrun -help for the complete list of
  globusrun options.

9
Running sh Scripts globus-sh-exec

• To allow users to run shell scripts without
  worrying for system-dependent program locations,
  Globus provides the globus-sh-exec command. The
  syntax isglobus-sh-exec ltscriptgt
• The script path can be relative as well as
  absolute.
• globus-sh-exec does nothing more, but to set an
  environment variable to the correct locations for
  each programs in a list of commonly used
  programs.
• You can check the available environment variables
  by issuing globus-sh-exec l
• It is expected that a job is started in the
  executed script using one of the commands
  described previously.
• For more detailshttp//www.globus.org/details/pr
  ograms/globus-sh-exec.html

10
Accessing Remote Data

• To allow remote clients to access on a host,
  start globus-gass-server -c as background job in
  that host. When it starts a URL is printed in the
  standard output.
• You can shut down a remote globus-gass-server by
  issuing using globus-gass-server ltURLgt. You
  cant do this if you dont use the -c option.
• Issue globus-gass-server help for a complete
  list of options.
• You can copy a file from host A to host B you
  usingglobus-url-copy ltfromURLgt lttoURLgtThe URLs
  are consisted of a server URL concatenated with a
  local absolute path. The server on A must be
  started including the -r option and on B
  including the -w option.
• For one-time copies globus-rcp is usually
  preferred because you do not have to start a gass
  server manually.

11
Grid Information Service

• Globus Grid Information Service (GIS) allows user
  to gather information for the grid resources. The
  GIS commands have the form grid-info- . The
  most commonly used of these commands, is
  grid-info-search .
• With the current configuration the user must get
  the info for each host separately. The syntax of
  grid-info-search isgrid-info-search h lthostgt
  -p 2137 ltfiltergtIf the host matches the filter,
  then the full host information are displayed. To
  show the information of a computer host, use the
  filter (objectclassGlobusComputeResource)
  (including the quotes). This is due to change.
• Wildcards can be used in the filters.