Practicals on gLite Workload Management Systems

1
Practicals on gLite Workload Management Systems

• SHI Jingyan
• Computing Center, IHEP
• EUChinaGrid 3rd Tutorial
• Beijing, 25.11.2006

2
Workload Management System

• In the glite middleware a user can submit and
  cancel jobs, query their status, and retrieve
  their output. These tasks go under the name of
  Workload Management.
• There are two different User Interfaces to
  accomplish these tasks. One is the Command Line
  Interface and the other is the Graphical User
  Interface.

3
Command Line Interface

• Job Submission
• Perform the job submission to the Grid.
• edg-job-submit options ltjdl_filegt
• where ltjdl filegt is a file containing the job
  description, usually with extension .jdl.

--vo ltvo namegt perform submission with a
different VO than the UI default one. --output,
-o ltoutput filegt save jobId on a
file. --resource, -r ltresource valuegt specify
the resource for execution. --nomsgi neither
message nor errors on the stdout will be
displayed.
4

• If the request has been correctly submitted this
  is the tipical output that you can get
• edg-job-submit test.jdl
• 
  
• JOB SUBMIT OUTCOME
• The job has been successfully submitted to the
  Network Server.
• Use edg-job-status command to check job current
  status. Your job identifier (edg_jobId) is
• - https//gilda05.ihep.ac.cn9000/IlaDMNZXlB1IOCI
  zHH4Qkw
• 
  
• In case of failure, an error message will be
  displayed instead, and an exit status different
  form zero will be retured.

5
If the command returns the following error
message Error API_NATIVE_ERROR
Error while calling the "NSClientmulti"
native api AuthenticationException Failed to
establish security context... Error
UI_NO_NS_CONTACT Unable to contact any
Network Server it means that there are
authentication problems between the UI and the
Network Server (check your proxy or contact the
site administrator).
6
It is possible to see which CEs are eligible to
run a job specified by a given JDL file using the
command edg-job-list-match test.jdl Selected
Virtual Organisation name (from --vo option)
gilda Connecting to host gilda05.ihep.ac.cn, port
7772

COMPUTING ELEMENT IDs LIST The following
CE(s) matching your job requirements have been
found CEId
dgt01.ui.savba.sk2119/jobmanager-lcgpbs-infinite
dgt01.ui.savba.sk2119/jobmanager-lcgpbs-lo
ng dgt01.ui.savba.sk2119/jobmanager-lc
gpbs-short gilda01.ihep.ac.cn2119/jobma
nager-lcgpbs-infinite gilda01.ihep.ac.cn211
9/jobmanager-lcgpbs-long


7
After a job is submitted, it is possible to see
its status using the glite-job-status command.
edg-job-status https//gilda05.ihep.ac.cn9000/WZT
HO5VI5FRyF81a8RcELg
BOOKKEEPING
INFORMATION Status info for the Job
https//gilda05.ihep.ac.cn9000/WZTHO5VI5FRyF81a8R
cELg Current Status Ready Status Reason
unavailable Destination
gildace.oact.inaf.it2119/jobmanager-lcgpbs-long S
ubmitted Wed Jun 7 111828 2006
CST

8
The option -i ltfile pathgt can be used to specify
a file with a list of job identifiers (saved
previously with the -o option of
glite-job-submit). edg-job-status -i jobs.list
------------------------------------------------
------------------ 1 https//gilda05.ihep.ac.cn
9000/9Zi9d4Cc0P8zqFKm-hxuyw 2
https//gilda05.ihep.ac.cn9000/xliVmKVJxoYL_wMusl
GDnA a all q quit ----------------------------
-------------------------------------- Choose
one or more jobId(s) in the list - 1-2all If
the - -all option is used instead, the status of
all the jobs owned by the user submitting the
command is retrieved.
9
The --status ltstategt (-s) option makes the
command retrieve only the jobs that are in the
specified state, and the --exclude ltstategt (-e)
option makes it retrieve jobs that are not in the
specified state. This two lasts options are
mutually exclusive, although they can be used
with --from and --to. Example All jobs of the
user that are in the state DONE or RUNNING are
retrieved. edg-job-status --all -s Done -s
Running
10
A job can be canceled before it ends using the
command glite-job-cancel. edg-job-cancel
https//gilda05.ihep.ac.cn9000/WZTHO5VI5FRyF81a8R
cELg Are you sure you want to remove specified
job(s)? y/nn y
glite-job-cancel Success
The cancellation request has been successfully
submitted for the following job(s) -
https//gilda05.ihep.ac.cn9000/WZTHO5VI5FRyF81a8R
cELg

11
After the job has finished (it reaches the DONE
status), its output can be copied to the UI
edg-job-output https//gilda05.ihep.ac.cn9000/Df
dDGhhNmw6zObV0CAZLWQ Retrieving files from host
gilda05.ihep.ac.cn ( for https//gilda05.ihep.ac.c
n9000/DfdDGhhNmw6zObV0CAZLWQ )

JOB GET
OUTPUT OUTCOME Output sandbox files for the
job - https//gilda05.ihep.ac.cn9000/DfdDGhhNmw
6zObV0CAZLWQ have been successfully retrieved
and stored in the directory /tmp/glite/glite-ui/
shijy_DfdDGhhNmw6zObV0CAZLWQ

By default, the output is stored under
/tmp, but it is possible to specify in which
directory to save the output using the - -dir
ltpath namegt option.
12
Exercise 1
13
Run an ls command on a resource
Create or modify ls.jdl and ls.sh as follow
Executable "ls.sh" Arguments "-al"
StdError "stderr.log" StdOutput
"stdout.log" InputSandbox "ls.sh"
OutputSandbox "stderr.log", "stdout.log"
Create and make executable with chmod x ls.sh
script !/bin/sh /bin/ls 1
14
Exercise 2
15
This exercise allows user to submit a C program.

• Modify c_sample.c file as follow
• include ltstdio.hgt
• int main(int argc, char argv)
• printf("\n\n\n")
• printf("Hello !\n")
• printf("Welcome to the GRID tutorial for
  users and system administrators Beijing 15-16
  Junel - 2006 \n\n\n")
• exit(0)

16

• Compile your script with gcc o c_sample
  c_sample.c
• Submit the c_sample.jdl job to the grid
• Executable /bin/sh
• Arguments start_c_sample.sh
• StdOutput std.out"
• StdError std.err"
• InputSandbox "c_sample,start_c_sample.sh
• OutputSandbox std.err",std.out"

17

• Create the start_c_sample.sh script as follow
• !/bin/sh
• chmod 777 c_sample
• ./c_sample
• Inspect the status and retrieve its output when
  the job is finished.

18
Exercise 3
19

• Modify c_sample.c file as follow
• include ltstdio.hgt
• int main(int argc, char argv)
• char name argv1
• printf("\n\n\n")
• printf("Hello s!\n,name)
• printf("Welcome to the GRID tutorial for users
  and system administrators Beijing 03-07 April
  - 2006 \n\n\n")
• exit(0)

20
Compile your script with gcc o c_sample
c_sample.c Modify the start_c_sample.sh script
as follow !/bin/sh chmod 777
c_sample ./c_sample 1 Modify c_sample.jdls
Arguments as follow Arguments
"start_c_sample.sh ltYour Namegt" Submit, inspect
the status and retrieve its output when the job
is finished.