NorduGrid%20Architecture%20and%20tools

1
NorduGrid Architecture and tools

• CHEP2003 UCSD
• Anders Wäänänen waananen_at_nbi.dk

2
NorduGrid project

• Launched in spring of 2001, with the aim of
  creating a Grid infrastructure in the Nordic
  countries.
• Idea to have a Monarch architecture with a Nordic
  tier 1 center
• Partners from Denmark, Norway, Sweden, and
  Finland
• Initially meant to be the Nordic branch of the EU
  DataGrid (EDG) project
• 3 full-time researchers with few externally funded

3
Motivations

• NorduGrid was initially meant to be a pure
  deployment project
• One goal was to have the ATLAS data challenge run
  by May 2002
• Should be based on the the Globus Toolkit
• Available Grid middleware
• The Globus Toolkit
• A toolbox not a complete solution
• European DataGrid software
• Not mature in the beginning of 2002
• Architecture problems

4
Architecture requirements

• No single point of failure
• Should be scalable
• Resource owners should have full control over
  their resources
• As few site requirements as possible
• Local cluster installation details should not be
  dictated
• Method, OS version, configuration, etc
• Compute nodes should not be required to be on the
  public network
• Clusters need not be dedicated to the Grid

5
NorduGrid Features at glance

• Dynamic Information System, Brokering,
  Monitoring
• Independence on Globus GASS cache (and its bug)
  
• Own GridFTP server, pluggable with job submission
• Stable and tested Grid testbed
• Not Nordic or HEP specific
• Tested on RedHat 6.2, 7.2 (also Alpha), Mandrake,
  Debian, Slackware
• Can share resources with non Grid applications
• Has been running ATLAS data challenges since May
  2002

6
NorduGrid components

• Grid Manager Mange Grid jobs in cluster
• Job control and data management
• Information system
• Patched Globus MDS with improved schema
• User interface
• Job submission and personal broker
• Grid monitor
• Web based interface to information system
• Globus replica catalog

7
Grid manager features 1

• Staging of executables and input/output data
• Supported protocols
• Local files, gridftp, ftp, http(s), Replica
  Catalog, Replica Location Services
• Data transfer control including retries
• Caching of input data
• Cache size control
• Private (per UNIX user) and shared caches
• Data access control based on users credentials
• Support for runtime environment (eg. Software
  installations)
• Full job information available for auditing,
  accounting and debugging

8
Grid manager features 2

• Globus building blocks used
• GridFTP fast, reliable and secure data access
• GASS transfer http(s) like data access protocol
• Replica catalog
• Replica Location Service (with EDG)
• RSL expandable Resource Specification Language
• Limitations
• Data handling is currently only supported at job
  start and job end when cluster nodes are on a
  private network

9
Grid Manager architecture
Frontend
Computing node
LRMS
LRMS

NFS
Job session directory
Job session directory
Cache
Link or copy
10
User interface

• The NorduGrid user interface provides a set of
  commands for interacting with the grid
• ngsub for submitting jobs
• ngstat for states of jobs and clusters
• ngcat to see stdout/stderr of running jobs
• ngget to retrieve the results from finished
  jobs
• ngkill to kill running jobs
• ngclean to delete finished jobs from the system
• ngcopy to copy files to, from and between file
  servers and replica catalogs
• ngremove to delete files from file servers and
  RCs

11
Information system

• The nerve system of the Grid - information is a
  critical resource!
• Complications
• Large number of resource -gt scalability
• Heterogeneous resources -gt characterization
• Decentralized
• Efficient access to dynamic data
• Quality and reliability of information
• Compromise between
• Up to date data vs. load on the Grid

12
NorduGrid information system

• Use Globus MDS
• Improved schemas with natural representation of
  resources
• Clusters (queues, jobs and users)
• Storage elements
• Replica Catalogs
• Use efficient providers
• Each resource runs a GRIS
• GRISs are organized into a dynamic country based
  GIIS hierarchy.
• Have enough information to do brokering

13
DIT of a cluster
cluster
queue
jobs
users
user-03
user-02
user-01
job-05
job-04
14
Cluster entry
15
DIT of a cluster
cluster
queue
jobs
users
user-03
user-02
user-01
job-05
job-04
16
Queue entry
17
DIT of a cluster
cluster
queue
jobs
users
user-03
user-02
user-01
job-05
job-04
18
Job entry
job status monitoring information system query
19
Another job entry
- the job entry is generated on the execution
cluster - when the job is completed and the
results are retrieved the job disappears from
the information system
20
DIT of a cluster
cluster
queue
jobs
users
user-03
user-02
user-01
job-05
job-04
21
Personalized information

• user based information is essential on the Grid
• users are not really interested in the total
  number of cpus of a cluster, but how many of
  those are available for them!
• number of queuing jobs are irrelevant if the
  submission gets immediately executed
• instead of total disk space the user's quota is
  interesting
• nordugrid-authuser objectclass
• freecpus
• diskspace
• queuelength

22
User entry
23
GIIS Hierarchy
Hierarchy of GRISes/GIISes
24
Grid Montior
25
Brokering job submission

• Searches through the NorduGrid Testbed for
  available clusters
• Loops through all the clusters and selects those
  queues (possible targets) where
• The user is authorized to run
• Job requirements can be satisfied
• Selects a job destination from the matching
  targets
• Randomly selects among the free resources (where
  user-freecpusgt0)
• In case there are no free matching resources some
  of the load attributes (i.e. user-queuelength)
  are taken into account

26
Implementation

• Portability
• Support toolkit on variety of platforms as needed
  by sites
• Open for new architectures (already support ia32,
  ia64 and alpha)
• Always choose the best technical workable
  solution
• Not afraid to throw things away
• Not tied to solutions because of political
  reasons
• Open for new ideas
• Start with something simple that works and
  proceed from there

27
NorduGrid job submission
Gatekeeper GridFTP
Grid Manager
28
Quick client installation/job run

• As a normal user
• retrieve nordugrid-standalone-0.3.17.rh72.i386.tgz
  
• tar xfz nordugrid-standalone-0.3.17.rh72.i386.tgz
• cd nordugrid-standalone-0.3.17
• source ./setup.sh
• Maybe get a certificate
• grid-cert-request
• install certificate per instructions
• grid-proxy-init
• ngsub '(executable/bin/echo)(arguments"Hello
  World")

29
Future development or integration

• Better Authorization
• Accounting
• Optimize brokering
• More intelligent data management and replication
  service
• Handle network requests from running jobs on
  private networks
• Grid portal interface in testing
• Move towards Grid services and improved community
  compatibility

30
Future

• The committee of Nordic natural science ministers
  NOS-N has decided to fund a new common Nordic
  Grid Project based on the work done by the
  NorduGrid project. This project should work on a
  proposal/recommendation for a Nordic DataGrid
  facility.
• Support for the toolkit in the future
• This will be supported in each country by local
  Grid initiatives
• Collaboration with the Nordic computing centers
  have already been initiated with the deployment
  of the toolkit on several large centers.
• Use it for future ATLAS production in the Nordic
  countries
• Move towards OGSA and better community
  compatibility

31
Resources

• Documentation and source code are available for
  download
• Main Web site
• http//www.nordugrid.org/
• Repository
• ftp//ftp.nordugrid.org/pub/nordugrid/

32
The NorduGrid core group

• ????????? ????????????
• Balázs Kónya
• Mattias Ellert
• ?????? ????????
• Jakob Langgaard Nielsen
• Trond Myklebust
• Anders Wäänänen