First experience with gLite testbed prototype

1
First experience with gLite test-bed prototype

• Julia Andreeva CERN
• APROM meeting
• 27.08.2004

2
Short overview

• Introduction to EGEE middleware architecture
• gLite first partial implementation of EGGE
  middleware
• Main components
• Use for CMS distributed analysis
• Currents status
• Short term plans

3
Main principles of EGEE middleware architecture

• EGEE middleware is supposed to be developed
  following Service Oriented Architecture model,
  delivering application functionality as services
  with the additional emphasis to loose coupling
  between interacting services. A service is a
  function which is well-defined, self-contained
  and does not depend on the context or state of
  other services.
• The services communicate with each other
• through well-defined interfaces and protocols
  (data passing or coordination of activities)

4
Main principles of EGEE middleware
architecture(continuation)

• Based on WEB service application that exposes its
  features using standard Internet protocol. WEB
  services interact by exchanging messages using
  Simple Object Access Protocol (SOAP) standard.
• Web Service Definition Language (WSDL) is used to
  specify the interface a service exposes.
• Grid Access Service (GAS) provides a common
  framework by which the user may gain access to
  Grid services. There is also API (which is not a
  service by itself) which provides a way of
  direct access to the services

5
What is gLite? Main componentsexposed to the
users

• First ( not complete ) implementation of the EGEE
  middleware
• Lightweight GRID implementation
• Distributed file catalogue built on top of RDBMS
  with user interface that mimics the file system
• Authentication module which supports various
  authentication methods
• Jobs queue which holds jobs to be executed in the
  system (commands, inputs and outputs are all
  registered in catalogue)
• Metadata catalogue
• Package Manager
• Services that support above components

6
Test-bed configuration

• Machine Type
  Status
• llxb2023.cern.ch VOMS/myProxy
  
  installed
• lxb2025.cern.ch SE (dCache)
  
  installed
• lxb2024.cern.ch SE (Castor)
  
  installed
• lxb1432.cern.ch SE (Castor, full
  namespace)
  installed
• lxn5210.cern.ch CEi
  
  I installed
• lxn5211.cern.ch WN
  
  installed
• xb1435.cern.ch WN (SLC3)
  
  installed
• lxb2028.cern.ch LRC and RMC catalogs
  (SLC3)
  installed
• lxn5216.cern.ch web portal
  
  installed
• lxb2022.cern.ch R-GMA registry/MON
  
  installed
• xb2031.cern.ch GAS
  
  installed
• lxb2021.cern.ch core services (task
  queue,optimizer, FTD) installed
• lxb2041.cern.ch database (mysql), proxy,
  ldap
  installed
• lxn5220.cern.ch Metadata catalog
  
  installed
• arda-01.cs.wisc.edu CE, WN and SE (castor)
  
  installed
• arda-02.cs.wisc.edu SE (dCache)
  
  being installed

7
Authentication and logging in

• Possible ways to authenticate to gLite shell
• - Certificates
• - SSH keys
• Test status
• Straight forward, except some problems in the
  beginning caused by the fact that users had not
  been informed about VOMS
• naming convention chosen by the VOMS manager
  for user names.

8
Authentication and logging in(continuation)

• Another possibility to access gLite through Grid
  Access Service (GAS).GAS will start services with
  user credentials on the remote machine. Then only
  a small subset of the system has to be installed
  by the client. Using myproxy-init command user
  create a proxy of his certificate and allows GAS
  to retrieve this proxy
• Test status
• Currently limited functionality is available
  while accessing gLlite through GAS

9
gLite shell

• Looks like UNIX shell
• Provides a set of UNIX-like commands
• cat,cp,echo,execute,exit,find,help,kill,ln,ls,mkdi
  r,ps,
• pwd,quit,rm,rmdir,whereis,whoami .
• Some commands are gLite-specific
• add,register,addTag,duid2lfn,lfn2guid,deleteMirror
  ,
• showMirror,submit,preFetch.
• Some commands have slightly different
  behavior/output than UNIX ones
• top,whereis
• Test status
• User friendly environment
• Developmet team is providing bug fixes,
  improvements, implementation of the suggestions
  from the users

10
Interfacing gLite services from outside gLite
shell

• Using perl API
• C library developed by Andreas Peters, Derek
  Feichtinger and Birger Koblitz from ARDA group
  (for file I/O there is also a POSIX style C
  library). Provides native shell commands having
  the same functionality as gLite-shell internal
  commands.
• Will be deployed very soon
• From UNIX shell
• glite exec ltglite-commandgt
• Not recommended, since would pass authentication
  procedure for every command
• Through web portal
• (currently mainly for getting status information
  about test-bed,
• but in future should be possible for the
  user to submit/resubmit jobs, for administrator
  to configure the system, check syslog,
• update distribution)

11
gLite catalog

• gLite catalog plays an essential role in the
  system since not only files but commands (job
  specification) as well as job input and output ,
  as well as tags are all stored in the catalog
• gLite LFN name space is organized as a directory
  tree structure, which allows to
• - record in LFNs some file META
  attributes,
• - define cleanup and access policy
• - optimize access performance

12
Example of the output of the tree command

• /egee/user/j/jandreeva/ gttree
• - - ./
• - - bin/
• - - dar
• - - dar.pm
• - - Install
• - - packages/
• - -ORCA/
• - -8_1_3/
• - - Linux-i686
• .
• /egee/user/j/jandreeva/ gtwhereis bin/dar
• And the file is in EGEECERNSRM
  file//lxb2024.cern.ch8000/afs/cern.ch/user/j/jul
  ia/scratch1/DAR/dar

13
Catalog test status

• Populated catalog with data for one dataset
  stored at CERN castor
• /egee/user/j/jandreeva/ gt cd cms
• /cms/ gt ls
• DigiPU04
• DST04
• Hit04
• /cms/ gt cd DigiPU04
• /cms/DigiPU04/ gt ls
• bt_2x1033PU761_TkMu_g133_CMS
• /cms/DigiPU04/gt cd bt_2x1033PU761_TkMu_g133_CMS
• /cms/DigiPU04/ bt_2x1033PU761_TkMu_g133_CMS gt ls
• bt03_ttH130_6j1l
• /cms/DigiPU04/bt_2x1033PU761_TkMu_g133_CMS gt cd
  bt03_ttH130_6j1l
• /cms/DigiPU04/bt_2x1033PU761_TkMu_g133_CMS/bt03_tt
  H130_6j1l gt ls
• 86800001
• 86800005
• 86800009
• .

14
Catalog test status (continuation)

• No real stress testing yet, since access to CERN
  castor from the gLite worker nodes was not
  available for a long while . Under pressure of
  the ARDA group it was finally resolved but still
  does not work with stable performance
• We are planning to use gLite catalog for defining
  of data location for gLite resource broker and
  for file management (moving/replication files) .
  Somehow CMS executables use POOL and need POOL
• catalog as an input, which creates an issue
  of synchronization between XML POOL catalog
  (which has to be sent with the job) and gLite
  catalog

15
META data catalog

• Modern file systems have META data attached to
  the file/directory
• gLite has a similar concept
• Information containing file properties (file META
  attributes) can be defined in a tag attached to a
  directory in the file catalog.
• Any arbitrary number of TAG tables can be
  attached to the corresponding directory table.
• Has to be discussed between CMS and ARDA
  how/whether this
• approach can be used for CMS
• Test status
• Stress test of the META data catalog was
  implemented by ARDA group, showed some scaling
  problem.
• gLite development team is working on
  improving scalability

16
Job submission

• Steps required for submitting of user job to
  gLite
• Register executable in the user bin directory
• Create JDL file where executable, required
  packages, input and output files, possibly some
  additional requirements are defined
• Run submit command providing JDL file as an input
• Test status
• Job submission is rather simple

17
JDL file

• In the JDL file user can define input, output
  files, files to be staged locally by gLite to a
  working directory before the job starts, package
  to be used (possibly first installed) by the job,
  and a set of requirements to be satisfied by a
  resource broker while taking a decision where the
  job would run .
• For example
• Run my job close to a given set of data
• or (one does not want to rely on RB choice)
• Run my job on a given farm

18
Testing of the Glite job queue
19
Package manager

• Currently provide a possibility to register a
• tar distribution of a given package (of a
  given version) either in the user or VO package
  directory. Package can contain a sort of setup
  script which by default would be run by gLite
  when user job required a given package starts
  running.
• If user defines in JDL file a given
  package/version to be used, package should be
  checked in the user/VO directory on the worker
  node , if it is not there should be fetched and
  unpacked there by a package manager and a setup
  script (if included in distribution) should be
  run before user executable starts.

20
Testing of the package manager

• Package manager on Glite testbed was released
  juts recently, for the moment was tried with DAR
  ORCA distribution
• Somehow for a heavy ORCA distribution (3,3GB)
  created through cmsi , including SCRAM and
  allowing to build executables, present
  functionality of the gLite package manager is not
  sufficient.
• Requirements for the package manager are
  currently discussed with the development team.

21
Running CMS analysis test jobs on gLite farm

• First prototype allowing to submit CMS analysis
  jobs, defining as an input dataset-owner name of
  a given collection, providing that this
  collection is published in RefDB/PubDB and data
  is available at CERN is under development by the
  ARDA group.
• First tests of the simple jobs reading CMS data
  (using ExDSTStatistics executable) were
  successfully run on the Glite farm.
• Real analysis jobs using executable provided by
  Lucia , were also run on gLite test-bed but with
  a small number of events. Currently jobs with a
  normal trigger failed due to the troubles of
  reading files located in castor.
• More work required to improve access to data
  files located in castor.

22
First histograms!?!
23
EGGE middleware deploymentissues

• Project started just in April, so it is quite
  impressive that we could have first prototype for
  playing already at the end of May, though with a
  very limited functionality
• Current test-bed prototype can not be used
• for testing analysis framework in a real scale
  (farm consists just of several PCs)
• We hope to have a preproduction test-bed
• available at the end of October to run tests
  in a more reasonable scale

24
Next ARDA work shop

• 20-22 of October is an open workshop
• which is held at the time of the transition
  from the closed development environment to a
  preproduction test bed which should result on the
  expansion of the user community.
• Future deployment of EGEE middleware
• will be discussed there.

25
Summary

• First prototype of gLite testbed is available at
  CERN and Wisconsin.
• Still very fragile and unstable, some components
  are delivered just recently and their
  functionality is rather limited.
• The existing system allows to develop prototype
  for CMS analysis , try it by the users and give a
  feedback to the gLite development team
• While CMS decision for ARDA activities is
  pending we hope to have a first prototype
  available next month and we will very appreciate
  some volunteers among CMS users to try it.