Grid Computing and the Globus Toolkit

1
Grid Computing and the Globus Toolkit

• Bill Allcock
• Jennifer M. Schopf
• Argonne National Lab
• (National eScience Centre)
• http//www.mcs.anl.gov/jms/Talks/

2
What is a Grid

• Resource sharing
• Computers, storage, sensors, networks,
• Sharing always conditional issues of trust,
  policy, negotiation, payment,
• Coordinated problem solving
• Beyond client-server distributed data analysis,
  computation, collaboration,
• Dynamic, multi-institutional virtual orgs
• Community overlays on classic org structures
• Large or small, static or dynamic
• Made more difficult by the lack of central
  control, shared resources, and increased need for
  communication

3
Globus is an Hour Glass
Higher-Level Services and Users

• Local sites have an their own policies, installs
  heterogeneity!
• Queuing systems, monitors, network protocols, etc
• Globus unifies
• Build on Web services
• Use WS-RF, WS-Notification to represent/access
  state
• Common management abstractions interfaces

Standard GT4 Interfaces
Local heterogeneity
4
Globus is a Building Block

• Basic components for grid functionality
• Highest-level services are often application
  specific, we let applications concentrate there
• Easier to reuse than to reinvent
• Compatibility with other Grid systems comes for
  free
• We provide basic infrastructure to get you one
  step closer

5
GT Domain Areas

• Core runtime
• Infrastructure for building new services
• Security
• Apply uniform policy across distinct systems
• Execution management
• Provision, deploy, manage services
• Data management
• Discover, transfer, access large data
• Monitoring
• Discover monitor dynamic services

6
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
7
dev.globus.org

• Created to facilitate contributions and open
  development
• Individual development efforts organized as
  projects
• Consensus-based decision making
• Control over each project in the hands of its
  most active and respected contributors
  (committers)
• Similar to Apache Jakarta process

8
General Globus Help and Support

• Each component also has mailing lists
• -user_at_globus.org for user issues
• -dev_at_globus.org for developer issues
• Bugzilla
• bugzilla.globus.org

9
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
10
GT4 Web Services Runtime

• Supports both GT (GRAM, RFT, Delegation, etc.)
  user-developed services
• Redesign to enhance scalability, modularity,
  performance, usability
• Leverages existing WS standards
• WS-I Basic Profile WSDL, SOAP, etc.
• WS-Security, WS-Addressing
• Adds support for emerging WS standards
• WS-Resource Framework, WS-Notification
• Java, Python, C hosting environments
• Java is standard Apache

11
What does Core give you?

• Reference implementation of WSRF and WS-N
  functions
• Naming and bindings (basis for virtualization)
• Every resource can be uniquely referenced and has
  one or more associated services for interacting
• Lifecycle (basis for resilient state management)
• Resources created by svcs following a factory
  pattern
• Resource destroyed immediately or scheduled
• Information model (basis for monitoring
  discovery)
• Resource properties associated with resources
• Operations for querying and setting this info
• Asynchronous notification of changes to
  properties
• Service groups (basis for registries collective
  svcs)
• Group membership rules and membership management
• Base fault type

12
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
13
Globus Security

• Control access to shared services
• Address autonomous management, e.g., different
  policy in different work-groups
• Support multi-user collaborations
• Federate through mutually trusted services
• Local policy authorities rule
• Allow users and application communities to set up
  dynamic trust domains
• Personal/VO collection of resources working
  together based on trust of user/VO

14
Virtual Organization (VO) Concept

• VO for each application or workload
• Carve out and configure resources for a
  particular use and set of users

15
GT4 Security

• Public-key-based authentication
• Transport- and message-level authentication
• Extensible authorization framework based on Web
  services standards
• SAML-based authorization callout
• Integrated policy decision engine
• XACML policy language, per-operation policies,
  pluggable

16
Security Tools

• Basic Grid Security Mechanisms
• Certificate Generation Tools
• Certificate Management Tools
• Getting users registered to use a Grid
• Getting Grid credentials to wherever theyre
  needed in the system
• Authorization/Access Control Tools
• Storing and providing access to system-wide
  authorization information
• Credential management service
• MyProxy (One time password support)

17
Other Security Services Include

• MyProxy
• Simplified credential management
• Web portal integration
• Single-sign-on support
• KCA kx.509
• Bridging into/out-of Kerberos domains
• SimpleCA
• Online credential generation
• CAS
• Community Authorization Service

18
A Cautionary Note

• Grid security mechanisms are tedious to set up
• If exposed to users, hand-holding is usually
  required
• These mechanisms can be hidden entirely from end
  users, but still used behind the scenes
• These mechanisms exist for good reasons.
• Many useful things can be done without Grid
  security
• It is unlikely that an ambitious project could go
  into production operation without security like
  this
• Most successful projects end up using Grid
  security, but using it in ways that end users
  dont see much

19
GT Authorization Framework
20
GT-XACML Integration

• eXtensible Access Control Markup Language
• OASIS standard, open source implementations
• XACML sophisticated policy language
• Globus Toolkit ships with XACML runtime
• Included in every client and server built on GT
• Turned-on through configuration
• that can be called transparently from runtime
  and/or explicitly from application
• and we use the XACML-model for our Authz
  Processing Framework

21
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
22
Execution Management (GRAM)

• Common WS interface to schedulers
• Unix, Condor, LSF, PBS, SGE,
• More generally interface for process execution
  management
• Lay down execution environment
• Stage data
• Monitor manage lifecycle
• Kill it, clean up
• A basis for application-driven provisioning

23
GRAM - Basic Job Submission and Control Service

• A uniform service interface for remote job
  submission and control
• Includes file staging and I/O management
• Includes reliability features
• Supports basic Grid security mechanisms
• Available in Pre-WS and WS
• GRAM is not a scheduler.
• No scheduling
• No metascheduling/brokering
• Often used as a front-end to schedulers, and
  often used to simplify metaschedulers/brokers

24
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
25
Monitoring and Discovery System(MDS4)

• Grid-level monitoring system
• Aid user/agent to identify host(s) on which to
  run an application
• Warn on errors
• Uses standard interfaces to provide publishing of
  data, discovery, and data access, including
  subscription/notification
• WS-ResourceProperties, WS-BaseNotification,
  WS-ServiceGroup
• Functions as an hourglass to provide a common
  interface to lower-level monitoring tools

26
Information Users Schedulers, Portals, Warning
Systems, etc.
WS standard interfaces for subscription,
registration, notification
Standard Schemas (GLUE schema, eg)
27
MDS4 Components

• Information providers
• Monitoring is a part of every WSRF service
• Non-WS services are also be used
• Higher level services
• Index Service a way to aggregate data
• Trigger Service a way to be notified of changes
• Both built on common aggregator framework
• Clients
• WebMDS
• All of the tool are schema-agnostic, but
  interoperability needs a well-understood common
  language

28
Working with TeraGrid

• Large US project across 9 different sites
• Different hardware, queuing systems and lower
  level monitoring packages
• Starting to explore MetaScheduling approaches
• GRMS (Poznan)
• W. Smith (TACC)
• K. Yashimoto (SDSC)
• User Portal
• Need a common source of data with a standard
  interface for basic scheduling info

29
Data Collected

• Provide data at the subcluster level
• Sys admin defines a subcluster, we query one node
  of it to dynamically retrieve relevant data
• Can also list per-host details
• Interfaces to Ganglia, Hawkeye, CluMon, and
  Nagios available now
• Other cluster monitoring systems can write into a
  .html file that we then scrape
• Also collect basic queuing data, some TeraGrid
  specific attributes

30
(No Transcript)
31
(No Transcript)
32
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
33
GT4 Data Management

• Stage/move large data to/from nodes
• GridFTP, Reliable File Transfer (RFT)
• Alone, and integrated with GRAM
• Locate data of interest
• Replica Location Service (RLS)
• Replicate data for performance/reliability
• Distributed Replication Service (DRS)
• Provide access to diverse data sources
• File systems, parallel file systems, hierarchical
  storage GridFTP
• Databases OGSA DAI

34
GridFTP

• A high-performance, secure, reliable data
  transfer protocol optimized for high-bandwidth
  wide-area networks
• FTP with well-defined extensions
• Uses basic Grid security (control and data
  channels)
• Multiple data channels for parallel transfers
• Partial file transfers
• Third-party (direct server-to-server) transfers
• Reusable data channels
• Command pipelining
• GGF recommendation GFD.20

35
Striped GridFTP Service

• A distributed GridFTP service that runs on a
  storage cluster
• Every node of the cluster is used to transfer
  data into/out of the cluster
• Head node coordinates transfers
• Multiple NICs/internal busses lead to very high
  performance
• Maximizes use of Gbit WANs

36
GridFTP in GT4
Disk-to-disk onTeraGrid

• 100 Globus code
• No licensing issues
• Stable, extensible
• IPv6 Support
• XIO for different transports
• Striping ? multi-Gb/sec wide area transport
• Pluggable
• Front-end e.g., future WS control channel
• Back-end e.g., HPSS, cluster file systems
• Transfer e.g., UDP, NetBLT transport

37
RFT - File Transfer Queuing

• A WSRF service for queuing file transfer requests
• Server-to-server transfers
• Checkpointing for restarts
• Database back-end for failovers
• Allows clients to requests transfers and then
  disappear
• No need to manage the transfer
• Status monitoring available if desired

38
Reliable File TransferThird Party Transfer

• Fire-and-forget transfer
• Web services interface
• Many files directories
• Integrated failure recovery
• Has transferred 900K files

RFT Client
SOAP Messages
Notifications(Optional)
RFT Service
GridFTP Server
GridFTP Server
39
Replica Location Service

• Identify location of files via logical to
  physical name map
• Distributed indexing of names, fault tolerant
  update protocols
• GT4 version scalable stable
• Managing 40 million files across 10 sites

Index
Index
40
What is GridFTP?

• A secure, robust, fast, efficient, standards
  based, widely accepted data transfer protocol
• A Protocol
• Multiple independent implementations can
  interoperate
• This works. Both the Condor Project at Uwis and
  Fermi Lab have home grown servers that work with
  ours.
• Lots of people have developed clients independent
  of the Globus Project.
• We also supply a reference implementation
• Server
• Client tools (globus-url-copy)
• Development Libraries

41
Striped GridFTP Service

• A distributed GridFTP service that runs on a
  storage cluster
• Every node of the cluster is used to transfer
  data into/out of the cluster
• Head node coordinates transfers
• Multiple NICs/internal busses lead to very high
  performance
• Maximizes use of Gbit WANs

42
GridFTP The Protocol

• FTP protocol is defined by several IETF RFCs
• Start with most commonly used subset
• Standard FTP get/put etc., 3rd-party transfer
• Implement standard but often unused features
• GSS binding, extended directory listing, simple
  restart
• Extend in various ways, while preserving
  interoperability with existing servers
• Striped/parallel data channels, partial file,
  automatic manual TCP buffer setting, progress
  monitoring, extended restart

43
GridFTP The Protocol (cont)

• Existing standards
• RFC 959 File Transfer Protocol
• RFC 2228 FTP Security Extensions
• RFC 2389 Feature Negotiation for the File
  Transfer Protocol
• Draft FTP Extensions
• GridFTP Protocol Extensions to FTP for the Grid
• Grid Forum Recommendation
• GFD.20
• http//www.ggf.org/documents/GWD-R/GFD-R.020.pdf

44
wuftpd based GridFTP

• Functionality prior to GT3.2
• Security
• Reliability / Restart
• Parallel Streams
• Third Party Transfers
• Manual TCP Buffer Size
• Partial File Transfer
• Large File Support
• Data Channel Caching
• Integrated Instrumentation
• De facto standard on the Grid

• New Functionality in 3.2
• Server Improvements
• Structured File Info
• MLST, MLSD
• checksum support
• chmod support (client)
• globus-url-copy changes
• File globbing support
• Recursive dir moves
• RFC 1738 support
• Control of restart
• Control of DC security

45
New GT4 GridFTP Implementation

• NOT based on wuftpd
• 100 Globus code. No licensing issues.
• Striping support has been added
• Has IPV6 support included (EPRT, EPSV), but we
  have limited environment for testing.
• Extremely modular to allow integration with a
  variety of data sources (files, mass stores,
  etc.)
• Based on XIO
• wuftpd specific functionality, such as virtual
  domains, will NOT be present

46
Striped Server

• Multiple nodes work together and act as a single
  GridFTP server
• An underlying parallel file system allows all
  nodes to see the same file system and must
  deliver good performance (usually the limiting
  factor in transfer speed)
• I.e., NFS does not cut it
• Each node then moves (reads or writes) only the
  pieces of the file that it is responsible for.
• This allows multiple levels of parallelism, CPU,
  bus, NIC, disk, etc.
• Critical if you want to achieve better than 1 Gbs
  without breaking the bank

47
(No Transcript)
48
Globus XIO
eXtensible Input Output library

• Overview
• Motivation
• Basic Architecture
• Globus XIO User API
• Data Types
• Example usage
• Driver Development
• Framework support
• Interface Functions

49
Grid IO

• Geographically Disperse Resources
• Super Computers / Clusters
• Large Data Stores
• Specialized Scientific Devices
• APS, Telescopes, Environmental Sensors
• Collaborative Sessions
• AccessGrid
• Varying Network Protocols
• GridFTP, HTTP, etc.

50
Typical Approach
Network Protocol
Network Protocol
Protocol API
Network Protocol
Application
POSIX IO
Proprietary API
51
Example Application
Collaborative Display
Remote Data Store
Proprietary API
Grid FTP Client Library
Application
HPSS Client Library
Proprietary API
Video Input Stream
HPSS
52
Problems

• Development Time
• Application must learn to use many different
  APIs.
• Proprietary APIs can be complex
• Each API has its own semantics (and bugs).
• Asynchronous/Synchronous
• Threaded/non-thread
• Different programming models in the same
  application.
• Scalability and Compatibility
• New devices or protocols
• Application must be modified to work with new
  devices
• Application must keep up with orthogonal research
  issues.

53
Observations

• Data Stream IO Abstraction
• Many Grid IO needs can be treated as a streams of
  bytes.
• open/close/read/write functionality satisfies
  most requirements.
• Protocol details
• Rarely does the application need (or want) to
  deal with protocol details.
• Most needs can be satisfied at initialization
  time.

54
Solution

• Globus XIO user API
• Single API/Single Set of semantics.
• Simple open/close/read/write API
• Driver Abstraction
• Hides protocol details
• Allows for extensibility
• Drivers can be selected at runtime

55
Globus XIO Approach
Network Protocol
Network Protocol
Driver
Network Protocol
Globus XIO
Driver
Application
Driver
56
Example Application With Globus XIO
Collaborative Display
Remote Data Store
Globus XIO (display driver)
Globus XIO (GridFTP driver)
Application
Globus XIO (HPSS Driver)
Globus XIO (video capture driver)
Video Input Stream
HPSS
57
Drivers

• Make 1 API do many types of IO
• Specific drivers for specific protocols/devices
• Transform
• Manipulate or examine data
• Do not move data outside of process space
• Compression, Security, Logging
• Transport
• Moves data across a wire
• TCP, UDP, File IO, Device IO
• Typically move data outside of process space

58
Stack
Example Driver Stack

• Transport
• Exactly one per stack
• Must be on the bottom
• Transform
• Zero or many per stack
• Control flows from user to the top of the stack,
  to the transport driver.

Compression
Logging
TCP
59
Handles

• Associated with a single stack
• When a handle is opened it is bound to a specific
  immutable stack.
• Handle is bound to stack at runtime.
• Used in all user IO operations
• open/close/read/write
• Contains the state of the connection
• Initialization
• Driver specific options.

60
Example Handle Use

• Handle is bound to the stack.
• User performs data operations on the handle.
• The data operation is passed down the stack.
• The data is compressed by the first driver.
• The logging driver logs the exchange in syslog.
• The TCP driver sends the compressed data across
  the wire.

User
Handle
Driver Stack
Compression
Logging
TCP
61
Globus XIO Framework

• Moves the data from user to driver stack.
• Manages the interactions between drivers.
• Assist in the creation of drivers.
• Asynchronous support.
• Close and EOF Barriers.
• Error checking
• Internal API for passing operations down the
  stack.

User API
Driver Stack
Transform
Transform
Framework
Transport
62
Driver Development Warning

• For power users only
• Assumption is you know what you are doing
• Weaker error reporting
• For the sake of efficiency.
• Possible to trip assertions.
• If the internal API is misused.

63
Interface functions

• A set function signatures
• Implemented by the driver
• Registered with Globus XIO
• Semantics
• Globus XIO makes calls to these functions
  expecting specific behaviour
• Ex the read() interface function should produce
  some data, and the write() interface function
  should consume data.

64
Existing Drivers

• Transport
• TCP, UDP, File
• Transform
• GSI, HTTP, GSSAPI_FTP
• Advanced
• UDT, MODE E, GridFTP
• Coming Soon
• Your driver?

65
Performance

• XIO was designed to be highly efficient.
• Comparisons of the old globus_io and the old
  GridFTP server show an improvement of maybe
  10-15 (cant guarantee it is statisitically
  significant)
• Had one individual tell us that his protocol
  implementation actually ran faster when he wrote
  a driver due to our efficient threading and event
  handling.

66
XIO Summary

• User API
• Simple open/close/read/write
• An abstraction to drivers
• Runtime configurable stacks
• Drivers
• Drivers do all the work
• Transport and Transform
• More information
• http//www.globus.org/toolkit/docs/4.0/common/xio/

67
Server Data Storage Interface (DSI)
68
New Server Architecture

• GridFTP (and normal FTP) use (at least) two
  separate socket connections
• A control channel for carrying the commands and
  responses
• A Data Channel for actually moving the data
• Control Channel and Data Channel can be
  (optionally) completely separate processes.
• A single Control Channel can have multiple data
  channels behind it.
• This is how a striped server works.
• In the future we would like to have a load
  balancing proxy server work with this.

69
New Server Architecture

• Data Transport Process (Data Channel) is
  architecturally, 3 distinct pieces
• The protocol handler. This part talks to the
  network and understands the data channel protocol
• The Data Storage Interface (DSI). A well defined
  API that may be re-implemented to access things
  other than POSIX filesystems
• ERET/ESTO processing. Ability to manipulate the
  data prior to transmission.
• currently handled via the DSI
• In V4.2 we to support XIO drivers as modules and
  chaining
• Working with several groups to on custom DSIs
• LANL / IBM for HPSS
• UWis / Condor for NeST
• SDSC for SRB

70
The Data Storage Interface (DSI)

• Conceptually, the DSI is very simple.
• There are a few required functions (init,
  destroy)
• Most of the interface is optional, and you can
  only implement what is needed for your particular
  application.
• There are a set of API functions provided that
  allow the DSI to interact with the server itself.
• Note that the DSI could be given significant
  functionality, such as caching, proxy, backend
  allocation, etc..

71
Reliable File TransferThird Party Transfer

• Fire-and-forget transfer
• Web services interface
• Many files directories
• Integrated failure recovery
• Has transferred 900K files

RFT Client
SOAP Messages
Notifications(Optional)
RFT Service
GridFTP Server
GridFTP Server
72
Globus Toolkit Open Source Grid Infrastructure
Globus Toolkit v4 www.globus.org
Data Replication
Replica Location
Grid Telecontrol Protocol
CredentialMgmt
Data Access Integration
Community Scheduling Framework
Delegation
Python Runtime
WebMDS
Reliable File Transfer
CommunityAuthorization
Trigger
C Runtime
Workspace Management
GridFTP
Authentication Authorization
Grid Resource Allocation Management
Index
Java Runtime
Data Mgmt
Security
CommonRuntime
Execution Mgmt
Info Services
73
For More Information

• Bill Allcock
• allcock_at_mcs.anl.gov
• www.mcs.anl.gov/allcock
• Jennifer Schopf
• jms_at_mcs.anl.gov
• www.mcs.anl.gov/jms
• Globus Alliance
• www.globus.org
• GlobusWORLD 2006
• September 10-14, Washington, D.C.
• Jointly located with GGF and GridWorld

74
(No Transcript)
75
Testing Overview

• Nightly builds and tests
• TestGrid at USC/ISI
• Stand up services for several weeks
• Perform stress tests
• TestGrid at LBNL
• Focus on WS Core performance and interoperability
  tests
• Performance and reliability testing is a major
  focus
• Component-specific approaches mostly
• Calls for Community Testing near release time -
  we welcome new testing help!

76
Documentation

• Current document significantly more detailed than
  earlier versions
• http//www.globus.org/toolkit/docs/4.0/
• Tutorials available for those of you building a
  new service
• http//www-unix.globus.org/toolkit/tutorials/BAS/

• Globus Toolkit 4 Programming Java Services (The
  Morgan Kaufmann Series in Networking), by Borja
  Sotomayor, Lisa Childers (Available through
  Amazon, 19.99 or 20)

77
Credits

• Globus Toolkit v4 is the work of many talented
  Globus Alliance members, at
• Argonne Natl. Lab U.Chicago
• USC Information Sciences Corporation
• National Center for Supercomputing Applns
• U. Edinburgh
• Swedish PDC
• Univa Corporation
• Other contributors at other institutions
• Supported by DOE, NSF, UK EPSRC, and other sources

78
Tested Platforms

• Debian
• Fedora Core
• FreeBSD
• HP/UX
• IBM AIX
• Red Hat
• Sun Solaris

• SGI Altix (IA64 running Red Hat)
• SuSE Linux
• Tru64 Unix
• Apple MacOS X (no binaries)
• Windows Java components only

• List of binaries and known platform-specific
  install bugs at
• http//www.globus.org/toolkit/docs/4.0/admin/
  docbook/ ch03.html

79
WSRF vs XML/SOAP

• The definition of WSRF means that the Grid and
  Web services communities can move forward on a
  common base
• Why Not Just Use XML/SOAP?
• WSRF and WS-N are just XML and SOAP
• WSRF and WS-N are just Web services
• Benefits of following the specs
• These patterns represent best practices that have
  been learned in many Grid applications
• There is a community behind them
• Why reinvent the wheel?
• Standards facilitate interoperability

80
OGSA-DAI

• Web services interface for accessing structured
  data resources
• Highly extensible and customisable
• Supports
• relational databases, xmldbs, structured files
• SQL, XPath, XQuery, text matching
• Runs on GT4, OMII_2, Axis 1.2, GRIA, UNICORE
• single client side API, multiple service flavours

81
OGSA-DAI functions

• Interact with data resources
• Queries and updates
• Data transformation / compression
• Data delivery
• Extensible set of activities
• 30 predefined, framework for writing your own
• Customize for your project using
• Additional Activities
• Client Toolkit APIs
• Data Resource handlers
• A base for higher-level services
• federation, mining, visualization,
• http//www.ogsadai.org.uk/

82
The OGSA-DAI Framework
Application
Client Toolkit
OGSA-DAI service
Engine
SQLQuery
Activities
GZip
GridFTP
XPath
readFile
XSLT
JDBC
Data Resources
XMLDB
File
MySQL
DB2
XIndice
SWISS PROT
SQL Server
Data- bases