Technical Aspects of GRID Technologies

1
Technical Aspects ofGRID Technologies Antun
BalaSCL, Institute of Physics
2
Science and technology today are team sports
3
Unifying concept Grid
Resource sharing and coordinated problem solving
in dynamic, multi-institutional virtual
organizations.
4
What types of problems is the Grid intended to
address?

• Too hard to keep track of authentication data
  (ID/password) across institutions
• Too hard to monitor system and application status
  across institutions
• Too many ways to submit jobs
• Too many ways to store access files/data
• Too many ways to keep track of data
• Too easy to leave dangling resources lying
  around (robustness)

5
Requirements

• Security
• Monitoring/Discovery
• Computing/Processing Power
• Moving and Managing Data
• Managing Systems
• System Packaging/Distribution

What end users need?Secure, reliable, on-demand
access to data, software, people, and other
resources (ideally all via a Web Browser!)
6
Set of basic Grid services

• Job submission/management
• File transfer (individual, queued)
• Database access
• Data management (replication, metadata)
• Monitoring/Indexing system information

7
Multi-institution issues
Certification
Certification
Authority
Authority
Domain B
Domain A
Policy
Policy
Authority
Authority
Task
Server Y
Server X
Sub-Domain A1
Sub-Domain B1
8
Why Grid security is hard

• Resources being used may be valuable the
  problems being solved sensitive
• - Both users and resources need to be careful
• Dynamic formation and management of virtual
  organizations
• - Large, dynamic, unpredictable
• VO Resources and users are often located in
  distinct administrative domains- Cant assume
  cross-organizational trust agreements
• - Different mechanisms credentials

9
Why Grid security is hard 2

• Interactions are not just client/server, but
  service-to-service on behalf of the user
• - Requires delegation of rights by user to
  service
• - Services may be dynamically instantiated
• Standardization of interfaces to allow for
  discovery, negotiation and use
• Implementation must be broadly available
  applicable- Standard, well-tested,
  well-understood protocols integrated with
  wide variety of tools
• Policy from sites, VO, users need to be combined
• - Varying formats
• Want to hide as much as possible from
  applications!

10
Grid solution use of VOs
No Cross- Domain Trust
Certification
Domain A
Federation
Service
GSI
Virtual
Organization
Domain
11
Effective policy governing access within a
collaboration
12
Use delegation to establish dynamic distributed
system
ComputingCenter
Service
Rights
VO
ComputingCenter
13
GSI implementation
SSL/WS-Security with Proxy Certificates
Services (running on users behalf)
Authz Callout
Access
ComputeCenter
Rights
VOUsers
Rights
VO
Local Policy on VO identity or attribute authority
MyProxy
Rights
KCA
14
Logging on to the Grid

• To run programs, authenticate to Grid
• grid-proxy-init
• Enter PEM pass phrase
• Creates a temporary, local, short-lived proxy
  credential for use by our computations
• Delegation remote creation of a (second level)
  proxy credential, which allows remote process to
  authenticate on behalf of the user

15
Middleware

• LCG Large Hadron Collider Computing Grid
• LCG infrastructure running LCG-2 is EGEE-0
• In parallel producing new web-service-oriented
  middleware (gLite), which will replace LCG-2 as
  production facility this year

16
User view of the Grid
User Interface
User Interface
Grid services
17
What really happens
User interface
Replica Catalogue
Resource Broker
Input sandbox
DataSets info
Information Service
Output sandbox
Job Submit Event
SE CE info
Auth. Auth.
Input sandbox Broker Info
Job Status
Output sandbox
Job Status
Computing Element
Logging Book-keeping
18
Workload Management System (WMS)

• Distributed scheduling
• multiple UIs where you can submit your job
• multiple RBs from where the job can be sent to a
  CE
• multiple CEs where the job can be put in a
  queuing system
• Distributed resource management
• multiple information systems that monitor the
  state of the grid
• Information from SE, CE, sites

19
Authentication and Authorization

• Authentication
• User obtains certificate from CA
• Connects to UI by ssh
• Downloads certificate
• Invokes Proxy server
• Single logon to UI - then Secure Socket Layer
  with proxy identifies user to other nodes
• Authorization - currently
• User joins Virtual Organisation
• VO negotiates access to Grid nodes and resources
  (CE, SE)
• Authorization tested by CE, SE gridmapfile maps
  user to local account

20
User Interface (UI)

• UI is the users interface to the Grid -
  Command-line interface to
• Proxy server
• Job operations
• To submit a job
• Monitor its status
• Retrieve output
• Data operations
• Upload file to SE
• Create replica
• Discover replicas
• Other grid services
• To run a job user creates a JDL (Job Description
  Language) file

21
Computing Element (CE)
A CE is a grid batch queuewith a grid gate
front-end
Job request
I.S.
Logging
Logging
Info system
Gatekeeper
gridmapfile
Grid gate node
Local resource management systemCondor / PBS /
LSF master
Homogeneous set of worker nodes
22
Storage Element (SE)

• Storage elements hold files write once, read
  many
• Replica files can be held on different SE
• close to CE share load on SE
• Replica Catalogue - what replicas exist for a
  file?
• Replica Location Service - where are they?

File transfer
Requests
Logging
GridFTP
EventLogging
Gatekeeper
Info system
Local Info
Disk arrays or tapes
23
Resource Broker

• Run the Workload Management System
• To accept job submissions
• Dispatch jobs to appropriate Compute Element (CE)
  
• Allow users
• To get information about their status
• To retrieve their output
• A configuration file on each UI node determines
  which RB node(s) will be used
• When a user submits a job, JDL options are to
• Specify CE
• Allow RB to choose CE (using optional tags to
  define requirements)
• Specify SE (then RB finds nearest appropriate
  CE, after interrogating Replica Location Service)

24
Logging and Bookkeeping

• Who did what and when?
• Whats happening to my job?
• Usually runs on RB node

Information System

• Receives periodic (5 min) updates from CE, SE
• Used by RB node to determine resources to be used
  by a job
• Currently BDII is used

25
Summary

• Grid structure is complicated but hidden from
  end-users, enabling all the comfort they need
• Users just need to join the VO and obtain
  certificates we already have the SEE-GRID VO and
  will have AEGIS VO
• Use of Grid is then just as easy as the use of a
  computer cluster