Introduction to Ganga 1

1
Ganga Introduction

• Object Orientated Interactive Job Submission
  System
• Written in python
• Based on the concept of a job object
• Developed by Atlas LHCb
• With dedicated (very helpful) support team
• Used widely - including outside HEP (e.g.
  biomedics)
• Ensures that it is an experiment-neutral
  framework
• All experiment specific extensions are plug-ins.
• Very easy automated install
• It just takes two commands and lt 5 minutes!
• Multiple backends
• local, psub, condor as well as GRID
  (glite-wms-)
• A very powerful feature run the same job as
  another process on the same machine or on the
  GRID just by changing its backend attribute
• Both standalone and a framework
• Is a complete ready to run system for submitting
  jobs
• Also API on which additional features can be
  layered
• Using local backend is great for development
  before switching to GRID
• e.g. MINOS have developed a fault tolerant batch
  system

2
Ganga The central concept A job

• A job is the central object
• Jobs are created, configured, submitted and when
  complete, examined.
• They are persistant
• Saved automatically into a Registry
• Quit Ganga at any time and come back later and
  resume
• Creation
• On creation each gets unique ID
• This is a serial number that can be used to
  access the job
• Configuration
• At any time can configure all aspects -
  executable, environment, input and output
  sandboxes, backend
• Cloning
• Having configured one job it can be cloned to
  create others
• Submission
• Job gets submitted to the configured backend
• Polling
• A job poller runs in the background and monitors
  submitted jobs
• It reports state changes.
• When job is complete it retrieves output and
  places in the jobs output directory

3
Ganga Two Interfaces
Command line

• Example
• To create job, configure, submit, check status
  and examine output-my_job Job()my_job.applica
  tion Executable(exeFile('/somescript'),
  args'1','2','3')my_job.backend LCG()
  my_job.submit()my_job.statusmy_job.peek(stdout
  )
• Ganga comes with ipython
• help with objects - they list methods and state
  e.g. help(my_job)
• tab completion - just type start of data or
  function and hit tab e.g. my_job.p lttabgt gives
  my_job.peek

GUI
4
Ganga Other Features

• Subjobs
• Have a master job and a dataset it is to operate
  on
• Job Splitter creates subjobs each taking part of
  the dataset
• Submitting the master job submits all the subjobs
• Subjobs can also be accessed individually e.g.
  fix and resubmit errors.
• When all subjobs complete Job Merger recombines
  the output
• Trees
• Initially job repository is a flat structure
• Can create folder hierarchy each with own jobs
• Can be used to perform global operations on
  collections of jobs
• Templates
• Configurable like jobs but cannot be submitted
• Used to create preconfigured jobs
• Authentication management
• GRID certificate will ask to re-authenticate
  when expired
• AFS token
• Configuration of all aspects
• By resource file read on start up
• Interactively during execution