Grid computing at BIFI: Description of resources

1
Grid computing at BIFIDescription of resources
examples of use
seminar 8th June 2005 Guillermo Losilla Anadón
2
BIFI grid-related projects

• EGEE ("Enabling Grids for E-sciencE")
• European grid for scientific use open 24
  hours-a-day
• IrisGrid
• National initiative
• I4Grid

3
EGEE predecessors

• DataGrid
• First European initiative to create a
  pan-European Grid

• LCG (LHC Computing Grid)
• Built to deal with the huge computing needs
  required to process the data that will be
  produced by the LHC

taken from GridCafé
4
EGEE project structure

• Extremely hierarchized
• Three main areas organised into 11 activities
• NA Networking Activities (5 acts)
• SA Specific Service Activities (2 acts)
• JRA Joint Research Activities (4 acts)
• 11 partner federations
• BIFI SA1 SWE federation(Spain Portugal)

5
EGEE current status

• Currently using LCG grid middleware
• EGEE grid midleware (gLite) still in development
• Some figures
• taken from GSTAT (http//goc.grid.sinica
  .edu.tw/gstat/) 1901h 7th June 2005
• EGEE is currently the most powerful Grid
  infrastructure but...
• ...it is not enough for LHC!!!

6
EGEE/LCG architecture services

• WMS (Workload Management System)
• Manages the whole process of submitting a job to
  the Grid
• Uses JDL (Job Description Language) for
  describing jobs
• DMS (Data Management System)
• Manages the storing of data in the Grid
• Identification(GUID, LFN, SURL) copy of files
• RLS(Replica Location Service) LFC(LCG File
  Catalog)

7
EGEE/LCG architecture services

• IS (Information System)
• Provides information about the Grid resources and
  their status
• Compliant with the GLUE schema
• MDS (Monitoring Discovery System) vs RGMA
  (Relational Grid Monitoring Architecture)
• Source of information for Monitoring Accounting
  services

8
EGEE/LCG node types

• UI (User Interface) point of entrance to the
  Grid
• CE (Computing Element) access point to a farm of
  computing nodes
• WN (Worker node) computing
• SE (Storage Element) access to storage element
  (disk, tape...)
• RB (Resource Broker) allocation and
  monitorization of jobs
• MON-BOX Information System (RGMA-based)
• BDII (Berkeley DB Information Index) Information
  System cache
• PS (Proxy Server) proxy certificates renewals

9
EGEE/LCG resources at BIFI

• 1 UI ui-egee.bifi.unizar.es
• 1 CE ce-egee.bifi.unizar.es
• 1 SE 1 Mon-Box se-egee.bifi.unizar.es
• 1 WN
• Using RB, PS BDII from other sites(PIC IFAE)

10
EGEE security authorization

• Most services are built on top of GSI (Grid
  Security Infrastructure)
• Every user must own, not only an user account
  in a UI node, but a valid digital certificate
  (issued by a suported CA) to be able to use the
  Grid
• Authorization enabled by Virtual Organization
  Organizations
• Every user must join a virtual organization
  (CMS, Atlas, Alice, LHCb, SWETEST...)

11
Examples of use Acess to a UI

• guillermo_at_lxbifi08gt ssh -X guillermo_at_ui-egee.bif
  i.unizar.es
• guillermo_at_ui-egee guillermo grid-proxy-init
• Your identity /CES/ODATAGRID-ES/OBIFI/CNGuill
  ermo Losilla Anadon
• Enter GRID pass phrase for this identity
• Creating proxy ...................................
  ....................... Done
• Your proxy is valid until Wed Jun 8 093911
  2005

12
Examples of use job submission

• Simplest job submission

• guillermo_at_ui-egee guillermo globus-job-run
  ce-egee.bifi.unizar.es /bin/pwd
• /home/swetest011

• Complex job Submission

guillermo_at_ui-egee tests more
testJob.jdl Executable "testJob.sh" StdOutput
"testJob.out" StdError "testJob.err" InputSa
ndbox "./testJob.sh" OutputSandbox
"testJob.out","testJob.err"
13
Examples of use job submission

• guillermo_at_ui-egee tests edg-job-list-match
  --vo swetest testJob.jdl
• Selected Virtual Organisation name (from --vo
  option) swetest
• Connecting to host lcgrb02.ifae.es, port 7772
• 
  
• COMPUTING ELEMENT IDs
  LIST
• The following CE(s) matching your job
  requirements have been found
• CEId
• ce00.inta.es2119/jobmanager-lcgpbs-swetest
• ce01.lip.pt2119/jobmanager-lcgpbs-swetest
• ce2.egee.cesga.es2119/jobmanager-lcgpbs-swetest
• lcg-ce.usc.cesga.es2119/jobmanager-lcgpbs-swetes
  t
• lcg2ce.ific.uv.es2119/jobmanager-lcgpbs-short
• ramses.dsic.upv.es2119/jobmanager-pbs-swetest
• lcg2ce.ific.uv.es2119/jobmanager-lcgpbs-swetest
• lcg2ce.ific.uv.es2119/jobmanager-lcgpbs-swetestL
  
• 
  

14
Examples of use job submission

• guillermo_at_ui-egee tests edg-job-submit --vo
  swetest testJob.jdl
• Selected Virtual Organisation name (from --vo
  option) swetest
• Connecting to host lcgrb02.ifae.es, port 7772
• Logging to host lcgrb02.ifae.es, port 9002
• 
  
• JOB SUBMIT OUTCOME
• The job has been successfully submitted to the
  Network Server.
• Use edg-job-status command to check job current
  status. Your job identifier (edg_jobId) is
• - https//lcgrb02.ifae.es9000/6scdvJ4Y1QU8yeMjH1
  zBhw
• 
  
• guillermo_at_ui-egee tests edg-job-status
  https//lcgrb02.ifae.es9000/6scdvJ4Y1QU8yeMjH1zBh
  w
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//lcgrb02.ifae.es
  9000/6scdvJ4Y1QU8yeMjH1zBhw

15
Examples of use job submission

• guillermo_at_ui-egee tests edg-job-get-output
  https//lcgrb02.ifae.es9000/6scdvJ4Y1QU8yeMjH1zBh
  w
• Retrieving files from host lcgrb02.ifae.es ( for
  https//lcgrb02.ifae.es9000/6scdvJ4Y1QU8yeMjH1zBh
  w )
• 
  
• JOB GET OUTPUT OUTCOME
• Output sandbox files for the job
• - https//lcgrb02.ifae.es9000/6scdvJ4Y1QU8yeMjH1
  zBhw
• have been successfully retrieved and stored in
  the directory
• /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw
• 
  
• guillermo_at_ui-egee tests ll /tmp/jobOutput/guill
  ermo_6scdvJ4Y1QU8yeMjH1zBhw/
• total 4
• -rw-rw-r-- 1 guillermo guillermo 0 jun
  7 2202 testJob.err
• -rw-rw-r-- 1 guillermo guillermo 168 jun
  7 2202 testJob.out

16
Examples of use data management

• guillermo_at_ui-egee tests edg-gridftp-ls
  gsiftp//se-egee.bifi.unizar.es/storage
• swetest
• alice
• atlas
• cms
• dteam
• lhcb
• sixt
• guillermo_at_ui-egee tests edg-gridftp-mkdir
  gsiftp//se-egee.bifi.unizar.es/storage/swetest/pr
  ueba
• guillermo_at_ui-egee tests edg-gridftp-ls
  gsiftp//se-egee.bifi.unizar.es/storage/swetest/
• prueba

17
Examples of use data management

• lcg-cr -l mi_fichero_en_el_Grid
• -d sfn//se-egee.bifi.unizar.es/storage/swetest/te
  st/mi_test
• --vo swetest file//pwd/mi_test
• Guid guid5def36aa-324e-4206-a631-a56093b8b02b
• lcg-lr --vo swetest guid5def36aa-324e-4206-a631
  -a56093b8b02b
• Location sfn//se-egee.bifi.unizar.es/storage/swe
  test/test/mi_test
• lcg-lr --vo swetest mi_fichero_en_el_Grid
• Location sfn//se-egee.bifi.unizar.es/storage/swe
  test/test/mi_test
• lcg-cp --vo swetest guid5def36aa-324e-4206-a631
  -a56093b8b02b file//pwd/mi_fichero_recuperado

18
Examples of use Information System

• guillermo_at_ui-egee tests lcg-infosites --vo
  swetest se
• 
  
• These are the related data for swetest (in terms
  of SE)
• 
  
• Avail Space(Kb) Used Space(Kb) Type SEs
• --------------------------------------------------
  --------
• 51354616 21660820 disk
  se01.lip.pt
• 11781208 134239228 disk
  se00.inta.es
• 17971988 10066312 disk
  aker.dsic.upv.es
• 64635064 2473188 disk
  lcg-se.usc.cesga.es
• 5503300 2331648 disk
  se2.egee.cesga.es
• 1000000000000 500000000000 mss
  castorgrid.ific.uv.es
• 0 500000000000 mss
  castorsrm.ifae.es
• 1000000000000 500000000000 mss
  castorsrm.ific.uv.es
• guillermo_at_ui-egee tests ldapsearch -LLL -x -H
  ldap//ce-egee.bifi.unizar.es2170 -b
  "mds-vo-namebifi,ogrid"

19
Useful links

• Grid computing at BIFI http//bifi.unizar.es/rese
  arch/computing/grid/gridcomputing.html
• EGEE project http//public.eu-egee.org/
• LCG project http//lcg.web.cern.ch/LCG/
• LCG-2 User Guide https//edms.cern.ch/file/454439
  //LCG-2-UserGuide.html