Open Grid Service Architecture - Data Access

1
Open Grid Service Architecture - Data Access
Integration (OGSA-DAI)

• Dave Pearson
• OGSA-DAI Programme Manager
• dave.pearson_at_oracle.com

2
OGSA-DAI Motivation

• UK Database Taskforce
• Grid data requirements scoping study
• Data Access and Integration BoF at GGF4
• Exploit emerging Grid computing concepts (OGSA)
• Web Services adoption by DBMS vendors
• Meet needs of UK e-Science community and Grid
  application developers
• Address e-Science trends

3
UK Grid Pilot Projects
Name Industry Participants Focus
MyGrid AstraZeneca, Mersk,GSK,IBM eScience workbench, Bioinformatics
DAME Rolls Royce Distributed Aircraft Maintenance Engineering
CombiChem Combinatorial (bio-)chemistry
RealityGrid Schlumberger Interactive steering of data compute intensive analysis chemistry, earth sciences
DiscoveryNet Bioinformatics, Environmental Earth sciences
Geodise BAe Engineering Computational Fluid Dynamics
AstroGrid Virtual Observatory
4
OGSA-DAI Motivation

• e-Science Trends
• In Silico experimentation and simulation
• Collaborative working in distributed
  multidisciplinary teams
• Information based discovery
• Integrated data visualisation, analyses, mining,
  interpretation
• Database adoption
• Complex domain and application specific data
  models
• Volatile data during information based discovery
• DBMS capability
• Commoditisation of hardware, storage and network
  components
• Grid computing concepts
• Interface abstraction (WDSL)
• Protocol, language, hosting platform independence
• Web service extensions
• Resource virtualisation

5
OGSA-DAI Requirements

• Transient and persistent data -
  Data sources Resources
• Maintain complex data models -
  Information/Knowledge
• Record and maintain data about data - Metadata
• Establish reliability and quality of data -
  Provenance
• Make data more accessible - Publishing and
  Discovery
• Restrict who can read and modify data - Access
  control
• Get data in a ready to use state -
  Retrieval/Transformation
• Personalise data - Analysis/Interpretation
• Quality of Service - Managing data

6
OGSA-DAI Programme of Work

• Draws on work of DBTF and ATF
• Grid data service reference implementations
• OGSA compliant
• DB2, MySQL, ORACLE, XINDICE
• Deliverables targeted at UK e-Science Community
• Nominated early adopters AstroGrid, MyGrid
• Two phases
• Phase 1 Feb Sept 2002
• Phase 2 Oct 2002 Jul 2003

7
OGSA-DAI Programme of Work

• Scope Objectives
• OGSA compliant Grid data services in common
  design framework
• Input to DAIS on GGF standards recommendations
• Ensure Grid model supports requirements for data
  access integration
• Reference implementations that can be deployed in
  current and planned Grid applications
• Performant, scalable, reliable, manageable grid
  data services
• Integration with Globus toolkit

8
Participating Organisations
9
Phase 1 Key Deliverables

• Draft design specification
• Functional scope for Phase Two
• XML prototypes on Apache Axis/Tomcat Platform
• OGSA Grid Data Service for Xindice
• OGSA Grid Data Service Factory
• OGSA Grid Registry Service
• Command line client to access the services
• Globus 2 RDBMS prototype
• MySQL
• Document style
• Distributed query prototype v0.1

10
Phase 2 Key Deliverables

• Three reference implementation releases
• January 15th 2003
• End Q1 2003
• July 2003
• Distributed Query Prototype in July
• Supporting material with each release
• Installation notes, System User documentation
• Training for each release at NeSC in Edinburgh
• Papers at end Phase 2

11
Open Grid Services Architecture

• Service orientation to virtualize resources
• From Web services
• Standard interface definition mechanisms
  multiple protocol bindings, multiple
  implementations, local/remote transparency
• To Grid Services
• State management in a distributed environment
• Builds on Globus Toolkit
• Grid service semantics for service interactions
• Management of transient instances ( state)
• Factory, Registry, Discovery, other services
• Reliable and secure transport
• Multiple hosting targets J2EE, .NET, C,

12
The Grid Service
Binding properties - Reliable invocation -
Authentication
Implementation
Hosting environment/runtime (C, J2EE, .NET, )
13
OGSA Port Types
14
OGSA-DAI Port Types
15
OGSA-DAI Positioning - Vision
OGSA-DAI Distributed Query
OGSA-DAI Basic Services
Data Grid Infrastructure Location, Delivery,
Replication
Resource Grid Infrastructure OGSA
Database, Communication, OS Technology
16
OGSA-DAI Positioning - Today
OGSA-DAI Distributed Query
OGSA-DAI Basic Services
GDSR
GDSF
GDS
Delivery
Database, Communication, OS Technology
17
OGSA-DAI Behaviours
Registry GDSR
1a. Request to Registry for sources of data about
x
1b. Registry responds with Factory handle
2a. Request to Factory for access to database
Factory GDSF
Analyst
2c. Factory returns handle of GDS to client
2b. Factory creates GridDataService to manage
access
3a. Client queries GDS with SQL, XPath, XQuery etc
Database (Xindice MySQL Oracle DB2)
3c. Results of query returned to client as XML
Grid Data Service GDS
OR3d. Results of query delivered to consumer as
XML
3b. GDS interacts with database
Consumer
18
Grid Data Service Registry

• Implements OGSA portTypes
• Registration
• GridService
• NotificationSource
• Provides standard operations for
• Registering and unregistering services.
• Querying of
• GDSR service data.
• Information provided by registered services.
• Subscribing to changes in GDSR state.

19
GDSR - Future Capabilities

• Validation of Registering Services
• Dereferencing GSHs of registering services.
• Identifying whether service is a
• GDS.
• GDSF.
• Client Authorisation
• Restrict ability of clients/services to
• Register services.
• Unregister services.
• Query registered services.
• QueryByGridDataServiceXPath
• Searching of registration information using XPath
  queries.
• Notification.

20
GridDataServiceFactory

• Implements OGSA PortTypes
• GridService
• Factory
• NotificationSource
• Provides services for
• Creating new Grid Data Services from predefined
  configurations
• Querying of service data
• GDSF configuration data
• services and configurations available for
  creation
• Subscribing to changes in the GDSF configuration
• Is a persistent Grid Service.

21
GDSF - Future Capabilities

• Transient GDSFs.
• More detailed service data.
• Client authorisation Restrict abilities of
  clients to create services.
• Creation of GDS from user-defined configuration
  data.
• Creation of other grid data services.
• Aggregation of the Registration PortType.

22
Statement Operations

• Receives statement
• Performs statement against a data resource
• Returns results

Statement Component and Database Connections
Statement
Results
Data Resource
23
Credential Role Mapping

• The Rolemapper takes credentials and a database
  name and returns a dbRole
• dbRole contains username and password

RoleMapper
dbRole
username
Credentials
Database Name
password
24
Architecture
Engine/GDS-S
Statement
Statement Activity
Statement Handlers
JDBC Statement Handlers
XMLDB Statement Handlers
JDBC Connection
XMLDB Connection
Relational DB
XML DB
JDBC API
XMLDB API
25
Release 1

• Dependencies
• Globus TP4 (including modified Axis 1.0)
• Tomcat 4.1.12 (using 4.1.18)
• Ant 1.5.x (using 1.5.1)
• Java (using J2SE 1.4.1 - some issues)
• Capabilities
• Release 1 supports MySQL and Xindice
• Simple synchronous interaction with a data source
  using a GDS as a proxy
• Uses JDBC and XMLDB
• Databases can be local or remote

26
Releases 2 3

• Proposed Release 2
• Runtime compatible with GT3 alpha
• More metadata, e.g.schema characteristics
• GridFTP
• Basic scripting
• Proposed Release 3
• Anything not in proposed release 2
• Scriting/workflow
• Transport, streaming and multicast
• Notification, including data change
• Transactions (not distributed)
• DQP prototype

27
Release 2 and 3

• Asynchronous delivery Pull
• Asynchronous delivery Push

28
GDS Composition
GDS
GDS
GDS
GDS
GDS
GDS
GDS
GDS
GDS
GDS
GDS
29
OGSA-DAI Summary

• Reference implementation for DIAS standards
  recommendations
• Meet needs of e-Science
• Current phase provide basic building block
• Proposal for future work in progress
• Automation, intelligence, higher level services
• Release 1 available for download from
• www.ogsa-dai.org.uk