XCAT3: A Framework for CCA Components as OGSA Services

1
XCAT3 A Framework for CCA Components as OGSA
Services

• Sriram Krishnan Dennis Gannon
• Extreme! Computing Lab
• Indiana University

2
Outline

• Literature Survey Motivation
• Design Implementation of XCAT3
• Usage Sample Application
• Conclusions Future Work

3
What is the Grid?

• A set of resources that provide seamless,
  dependable, pervasive access to high-end
  computational capabilities
• Seamless Standard protocols and interfaces
• Dependable Non-trivial Qualities of Service
• Pervasive Decentralized, Heterogeneous Resources
  
• Open Grid Services Architecture (OGSA)
• Initial effort to standardize Grid functionality
• A programming model consistent with trends in
  commercial sector
• Use of Web service concepts and technologies

4
Web Services

• Software entities accessible over the Internet
• Described, Registered, Invoked over the Web
• Description
• Web Service Definition Language (WSDL)
• Registries
• Universal Description, Discovery Integration of
  Web Services (UDDI)
• Remote Invocation
• XML-based messages using standard Internet
  protocols, e.g. SOAP over HTTP

5
The Grid Web Services

• Open Grid Services Infrastructure (OGSI)
• The core infrastructure on which OGSA is built
• OGSI defines Grid Services Specification (GSS)
• A Programming Model for the Grid based on Web
  services
• Standard Interfaces and Behaviors for Grid
  services
• Creation and Lifetime Management
• Dynamic Service Introspection Service Data
  Elements (SDE)
• Multiple Level Naming Grid Service Handles (GSH)
  and References (GSR)

6
Orchestrating Distributed Applications on the Grid
The Problem 1 Launch simulation at Y 2. Launch
simulation at Z 3. Link both simulations 4.
Execute both simulations 5. Store results at X
X
Z
Y
The Grid
7
What is missing?

• Grid and Web services are usually independent
  software entities
• Provide pre-defined functionality over the
  Internet
• No clean way to interconnect Grid/Web services
  together
• Workflow tools perform a sequence of operations
  to compose a set of Grid services Composition in
  Time
• Also needed is an ability to perform Composition
  in Space
• XCAT3 A Component Framework for Grid Applications

8
Component Architectures

• A Component Architecture consists of two parts
• Components
• Software objects that implement a set of required
  behaviors
• Frameworks
• A runtime environment
• A set of services used by components
• Benefits
• Encapsulation, modular construction of programs
  (via composition), reuse
• Component Architectures have been adopted in
  various domains
• Business EJB, CCM, COM/DCOM
• Scientific Computing CCA

9
CCA

• The U.S. Dept of Energy DOE2000 project
• The Common Component Architecture (CCA)
• Lawrence Livermore National Lab
• Sandia Labs
• Argonne National Labs
• Los Alamos National Labs
• Universities Utah, NCSA, Indiana
• A specification for component design for parallel
  and distributed applications

10
Common Component Architecture

• Ports the public interfaces of a component
• Defines the different ways we can interact with a
  component and the ways the component uses other
  services and components.

setImage(Image I)
Uses Ports - interface of a service used by
component
Image Processing Component
Image getImage()
adjustColor()
calls doFFT()
setFilter(Filter)
Provides Ports - interfaces functions provided by
component
11
Building Applications by Composition

• Connect Uses Ports to Provides Ports.

Image database component
setImage()
Image Processing Component
getImage()
Acme FFT component
doFFT()
adjustColor()
Image tool graphical interface component
12
Towards a OGSI Compliant CCA Framework

• Composition in Space for distributed applications
  on the Grid
• Components use the same standard protocols to
  interact with Grid resources, and vice versa
• Components inherit desirable features on the Grid
  as specified by OGSI
• Can utilize research in Web/Grid community on
  technologies such as Workflows (BPEL), Service
  Level Agreements (SLA)

13
XCAT3 CCA on the Grid

• Synergistic coexistence of CCA and OGSI
• Issues that need attention
• Compatibility with the two standards
• Creation of Remote Components
• Composition of Components
• Communication between the Components

14
CCA Compatibility

• CCA Specification is a set of interfaces defined
  in SIDL
• SIDL Scientific Interface Definition Language
• These interfaces provide an ability to implement,
  instantiate and connect components
• Implementation of these interfaces are framework
  specific
• Currently, CCA does not address interoperability
  between various frameworks
• To conform to CCA, need to implement all required
  interfaces

15
CCA Compatibility

• The Babel Toolkit used to generate the Java
  interfaces from the CCA SIDL
• Babel Language Interoperability Toolkit from
  LLNL that provides procedure calls across
  language boundaries
• Current shortcomings of Babel
• Babel provides language interoperability within
  the same address space
• Had to modify generated code in order to make
  remote invocations instead of local ones
• Babel only provides a binding for client side in
  Java
• Had to implement Java server side by hand

16
OGSI Compatibility

• Grid Service Extensions (GSX) Toolkit used for
  OGSI Compatible Grid services
• GSX is a lightweight implementation of the OGSI
  specification that uses the XSOAP/XSUL Toolkit
• Standard protocols used by Grid services SOAP,
  HTTP
• http//www.extreme.indiana.edu/xgws/GSX/
• Can interoperate with other OGSI implementations,
  e.g. GT3
• A Component is represented as a set of Grid
  services

17
OGSI Compatibility

• Representation for Provides ports
• In traditional Grid/Web services, multiple ports
  of the same portType are semantically equivalent
• CCA allows multiple ports of the same type
• CCA ports can not be mapped to Web service ports!
• Hence, every Provides port is mapped as a
  separate Grid service
• A single portType containing the Provides port
  interface
• Representation for Uses ports
• Clients of Grid services (Provides ports)
• Connections to Provides ports made at runtime

18
OGSI Compatibility

• Every Component contains a ComponentID
• A ComponentID uniquely identifies a Component
  instance
• Representation for the ComponentID
• Also a Grid service
• Acts as a Manager for the other Provides ports
• Contains SDEs containing GSH/GSRs for the various
  Provides ports
• The Provides ports and ComponentID services, and
  the Uses ports communicate via shared state

19
Component Creation and Composition

• Builder Service Responsible for creating
  Component instances and connecting them together
• createInstance()
• connect(), disconnect()
• Composition Tools GUIs, Portals, Jython scripts
• Use the Builder service for instantiation of
  components and composition in space
• Provide Composition in Time

20
Component Creation and Composition

• Component Creation
• Components need to be instantiated on demand on
  the Grid
• Several protocols supported
• Globus GRAM
• SSH
• Local in-process execution
• Component Connection
• Connection between Uses and Provides ports for
  Composition in Space
• A connect() results in storing the GSH of the
  Provides port inside the Uses port stub

21
Deployment of Remote Components

• XML Deployment descriptors for Components
• List of hosts that the component can be deployed
  on
• List of protocols supported
• Description of ports
• Other environment information
• Executable name
• Working directories
• Standard arguments
• Composition Tools parse the Deployment
  descriptors before invoking methods on the
  Builder service

22
Component Communication

• Use of a Remote Procedure Call Mechanism
• XSOAP
• Implementation of Simple Object Access Protocol
  (SOAP)
• http//www.extreme.indiana.edu/xgws/xsoap
• In-process calls
• Events/Messages
• Objects encoded as XML documents
• WS-Notification
• Proteus
• Multi-protocol Messaging and RMI

In progress
23
Writing Components

• Defining Port Interfaces
• Extending XCATPort
• Implementing Ports
• Extending BasicPortImpl
• Implementing the Component
• Adding Uses and Provides ports
• Other Control/Data flow
• Composing the application
• Creating instances, connecting them

24
Sample Application Chemical Engineering

• The Problem Simulation of Copper
  Electrodeposition
• Simulation driven by Finite Difference
  Application
• Multiple Monte Carlo Simulations dynamically
  launched
• The applications exchange data after every
  iteration
• The Proposed Solution
• Implemented using XCAT Master/Worker Application
  Managers
• The Master Manager manages the Finite Difference
  Application
• The set of Worker Managers manage the Monte Carlo
  Simulations
• Data exchanged between the Managers using XCAT
  Port calls

25
Code linkage in more detail
Composition Script
Master Manager
Worker Manager
Worker Manager
Worker Manager
Worker Manager
26
Code linkage in more detail
Composition Script
Finite Difference
Master Manager
Worker Manager
Worker Manager
Worker Manager
Worker Manager
Monte Carlo
Monte Carlo
Monte Carlo
Monte Carlo
27
Code linkage in more detail
Composition Script
Finite Difference
Master Manager
Worker Manager
Worker Manager
Worker Manager
Worker Manager
Monte Carlo
Monte Carlo
Monte Carlo
Monte Carlo
28
Code linkage in more detail
Composition Script
Finite Difference
Master Manager
Worker Manager
Worker Manager
Worker Manager
Worker Manager
Monte Carlo
Monte Carlo
Monte Carlo
Monte Carlo
29
Conclusions

• XCAT3 A Component Framework for the Grid
• Compatibility with CCA and OGSI specifications
• Enables the use of novel Grid standards within a
  typical Component framework
• Enables orchestration of complex applications on
  the Grid
• Composition in Space
• Composition in Time
• http//www.extreme.indiana.edu/xcat/

30
Future Research

• Fault Tolerance for Distributed Components
• Distributed User-level Checkpointing
• User implements methods to load and store state
• System ensures that the checkpoints are globally
  consistent
• System can restart from globally consistent
  snapshots if any of the components fail
• Adaptability to dynamic environments
• Migration of individual components
• A component can migrate during execution even if
  it may be connected to other components
• System ensures that no message sent to the
  migrating component is lost, and that all
  components can rediscover the migrated component