James Casey CERN [e-mail]

1
Next-Generation EU DataGrid Data Management
Services

• James CaseyCERNJames.Casey_at_cern.ch
• On behalf of EU DataGrid WP2

2
Talk Outline

• Introdution to EU DataGrid workpackage 2
• WP2 Service Design and Interactions
• Spitfire
• Replication Services
• Security
• Conclusions and outlook

Authors Diana Bosio, James Casey, Akos Frohner,
Leanne Guy, Wolfgang Hoschek, Peter Kunszt, Erwin
Laure, Levi Lucio, Heinz Stockinger, Kurt
Stockinger - CERN Giuseppe Andronico, Federico
DiCarlo, Andrea Domenici, Flavia Donno, Livio
Salconi INFN William Bell, David Cameron, Gavin
McCance, Paul Millar, Caitriona Nicholson
PPARC, University of Glasgow Joni Hahkala, Niklas
Karlsson, Ville Nenonen, Mika Silander, Marko
Niinimaki Helsinki Institute of Physics Olle
Mulmo, Gian Luca Volpato Swedish Research
Council
3
Grid middleware architecture hourglass

• Current Grid architectural functional blocks

CMS
ATLAS
CMS
LHCb
Specific application layer
Grid Application Services (LCG)
Common application layer
EU DataGrid middleware
High Level Grid Services
Basic Grid Services
GLOBUS 2.2
OS, Storage Network services
4
EU DataGrid WP2Data Management Work Package

• Responsible for
• Transparent data location and secure access
• Wide-area replication
• Data access optimization
• Metadata access
• NOT responsible for (but partially relying on
  other WPs for)
• Data storage
• Proper Relational Database bindings
• Remote I/O
• Security infrastructure

5
WP2 Service Paradigms

• Choice of technology
• Java-based servers using Web Services
• Tomcat, Oracle 9iAS
• Interface definitions in WSDL
• Client stubs for many languages (Java, C, C)
• Axis, gSOAP
• Persistent service data in Relational Databases
• MySQL, Oracle
• Modularity
• Modular service design for pluggability and
  extensibility
• No vendor specific lock-ins
• Evolvable
• Easy adaptation to OGSA foreseen, based on the
  same technology
• Largely independent of underlying OS, RDBMS

6
Spitfire Grid-enabling RDBMS

• Capabilities
• Simple Grid enabled front end to any type of
  local or remote RDBMS through secure SOAP-RPC
• Sample generic RDBMS methods may easily be
  customized with little additional development,
  providing WSDL interfaces
• Browser integration
• GSI authentication
• Hooks in place for local authorization
• Status current version 2.1
• Used by EU DataGrid Earth Observation and
  Biomedical applications.
• Not suitable for the retrieval of LARGE result
  sets

7
Replication Services Basic Functionality
Each file has a unique Grid ID. Locations
corresponding to the GUID are kept in the Replica
Location Service.
Users may assign aliases to the GUIDs. These are
kept in the Replica Metadata Catalog.
Files have replicas stored at many Grid sites on
Storage Elements.
Replica Metadata Catalog
Replica Location Service
Replica Manager
The Replica Manager provides atomicity for file
operations, assuring consistency of SE and
catalog contents.
Storage Element
Storage Element
8
Higher Level Replication Services
The Replica Subscription Service issues
Replication commands automatically, based on a
set of subscription rules defined by the user.
Hooks for user-defined pre- and post-processing
for replication operations are available.
Replica Subscription Service
The Replica Manager may call on the Replica
Optimization service to find the best replica
among many based on network and SE monitoring.
Replica Metadata Catalog
Replica Location Service
Replica Manager
Replica Optimization Service
Storage Element
Storage Element
SE Monitor
Network Monitor
9
Interactions with other Grid components
Virtual Organization Membership Service
User Interface or Worker Node
Resource Broker
Replica Subscription Service
Information Service
Replica Metadata Catalog
Replica Location Service
Replica Manager
Replica Optimization Service
Applications and users interface to data through
the Replica Manager either directly or through
the Resource Broker. Management calls should
never go directly to the SE.
Storage Element
Storage Element
SE Monitor
Network Monitor
10
Replication Services Status

• Current Status
• All components are deployed right now
• Initial tests show that expected performance can
  be met
• Need proper testing in a real user environment
  EDG2 LCG1
• Features for next release
• Currently Worker Nodes need outbound connectivity
  Replica Manager Proxy Service needed. Needs
  proper security delegation mechanism.
• Logical collections support
• Service-level authorization
• Subscription Service does not handle individual
  users due to missing delegation.

11
Security Infrastructure for Java-based Web
Services

• Trust Manager
• Mutual client-server authentication using GSI (ie
  PKI X509 certificates) for all WP2 services
• Supports everything transported over SSL
• Authorization Manager
• Supports coarse grained authorization Mapping
  user-gtrole-gtattribute
• Fine grained authorization through policies, role
  and attribute maps
• Web-based Admin interface for managing the
  authorization policies and tables
• Status
• Fully implemented, authentication is enabled on
  the service level
• Delegation implementation needs to be finished
• Authorization needs more integration, waiting for
  deployment of VOMS

12
Conclusions and outlook

• The second generation Data Management services
  have been designed and implemented based on the
  Web Service paradigm
• Flexible, extensible service framework
• Deployment choices robust, highly available
  commercial products supported (eg. Oracle) as
  well as open-source (MySQL, Tomcat)
• First experiences with these services show that
  their performance meets the expectations
• Real-life usage will show its strengths and
  weaknesses on the LCG-1 and EDG2.0 testbeds
  during the rest of this year.
• Thanks to the EU and our national funding
  agencies for their support of this work