GSFL: A Workflow Framework for Grid Services

1
GSFL A Workflow Framework for Grid Services

• Sriram Krishnan
• Patrick Wagstrom
• Gregor von Laszewski

2
Outline

• Grid Workflow Requirements
• Technology Survey
• GSFL Overview
• Implementation Details
• Grid Workflow Example
• Current Future Work
• Conclusions

3
Grid Workflow Requirements

• Description of the interactions between a set of
  cooperating Grid Services
• Presentation of an agglomeration of such Grid
  Services as a Grid Service
• Enabling recursive composition of Grid Services

D
B
A
E
C
4
Grid Workflow Requirements

• Ability to export entry points that will trigger
  a set of other activities
• Enabling peer-to-peer and decentralized data
  transfer between Grid Services

Entry Point
1
B
A
2
3
Peer-to-Peer Data Transfer
5
Grid Workflow Requirements

• Ability to manage the lifecycle for the
  individual services in the workflow
• All services that are part of the workflow need
  not be alive at the same time
• Need the ability to instantiate the services as
  and when necessary

A
C
B
Time
D
6
Technology Survey

• Web Services Flow Language (WSFL)
• Flow Model describes the orchestration of the
  various activities in the workflow
• Global Model describes how these activities are
  implemented by the services
• PlugLinks describe the connection between WSDL
  operations having dual signatures
• Service Providers that can be located in a number
  of ways
• Static, Local, UDDI, Dynamic

7
Technology Survey

• WSFL Pros
• Exporting individual activities as part of the
  workflow
• Recursive composition of Web services
• WSFL Cons
• Inadequate peer-to-peer communication mechanism
  between services
• Due to the need for solicit-response and
  notification operations in WSDL
• The workflow engine has to mediate at every step,
  and can be a bottleneck
• Simplistic lifecycle management for the services
• No implementation available

8
Technology Survey

• XLANG
• Modeling of business processes as Autonomous
  Agents
• The unit of actions is a Service Process, which
  consists of a set of operations according to a
  defined sequence
• Sequential and parallel control flow constructs
• Transaction support
• Custom correlation of messages
• Flexible handling of exceptions
• Dynamic service referral
• Contracts to agglomerate services

9
Technology Survey

• XLANG Pros
• Defined as an extension to WSDL
• Availability of complicated control structures
  and exception handling
• XLANG Cons
• Does not enable exporting activities of
  individual services as part of the workflow
• Does not explicitly handle recursive composition
  of Web Services
• Relies on the ambiguous outgoing operations of
  WSDL
• Does not address lifecycle for individual services

10
Technology Survey

• Web Services Conversation Language (WSCL)
• Sequencing of interactions between operations of
  a single interface
• Document Type Descriptions
• Types of documents to be exchanged
• Interactions
• The actions of a conversation between
  participants
• Transitions
• Ordering relationships between interactions
• Conversations
• Set of interactions and transitions

11
Technology Survey

• WSCL Pros
• Fills the gap between mere IDLs that do not
  define any choreography and more complex flow
  languages that describe multi-party conversations
• WSCL Cons
• Does not define multi-party conversations
• Does not handle recursive composition of services
• Too simplistic for our goals

12
Technology Survey

• XCAT (Indiana University)
• Application Factories for Grid based Components
• Peer-to-Peer communication between components
  based on the CCA based Provides and Uses Ports
• Condor DAGMan
• Use of a DAG to describe inter-dependencies
  between jobs
• Does not deal with either Web or Grid based
  services

13
Grid Services Flow Language

• Language that allows specification of workflow
  for Grid Services in the OGSA framework
• Defined using an XML Schema
• Consists of
• Service Providers, that are part of the workflow
• Activity Model, containing the activities that
  belong to the workflow
• Composition Model, describing interactions
  between the services
• Lifecycle Model, describing the lifecycle of
  activities and services

14
Service Providers

• List of services that are part of the workflow
• Name Identifier for the service in the GSFL
  document
• Type Name of the service, as specified in the
  WSDL for the service
• Locator Information needed to locate the
  service
• Static The service is up and running the GSH
  is provided
• Factory The service has to be instantiated the
  GSH of the factory is provided
• Registry The service is up and registered with
  a Registry Service the GSH of the registry is
  provided

15
Activity Model

• List of activities that are part of the workflow
• Name Identifier for the activity in the GSFL
  document
• Source A reference to the operation which
  implements the activity
• The service that acts as the source
• The port that contains the operation
• The operation that implements the activity

16
Composition Model

• Describes the composition of individual services
  into a new Grid service
• Describes the flow of control and data between
  the various operations of the services
• Describes the direct communication between
  services in a peer-to-peer fashion
• Consists of
• Export Model, that models the activities that
  have to be exported as operations of the workflow
• Notification Model, that models the direct
  communication between the services

17
Export Model

• List of activities to be exported as operations
  of the workflow process
• Each exported operation triggers a set of
  activities implemented by the individual services
• For the exported operations
• Control Model defines the flow of control between
  activities of the workflow
• Data Model defines the flow of data between the
  individual activities
• Information to infer the messages and types for
  the exported operations

18
Export Model

• Service Providers A, B
• Exported Operation X
• Control Model P -gt Q, Q -gt R
• Data Model P -gt Q

Data
A
B
P
Q
R
Control
Exported Port
X
19
Notification Model

• Models the direct communication between the
  services
• Uses the Notification Sources Sinks provided by
  OGSA
• Set of Notification Links connecting individual
  services, for particular topics
• Unlike WSFL, there is no need to use poorly
  defined solicit-response and notification
  operations

20
Lifecycle Model

• Models the lifecycle of the services and
  activities that are part of the workflow
• Service Lifecycle
• Ordering of services as a DAG, using Precedence
  Links
• All services need not be active at the same time
• Activity Lifecycle
• Possible ordering of the activities for the
  exported workflow, using Precedence Links
• Additional semantics to the exported activities,
  in a similar vein as WSCL

21
GSFL DEFINITION
Name, Target Namespace, Scope
IMPORTS

List of Imports Namespace, Location
SERVICE PROVIDERS

List of Providers Name, Type, Locator
ACTIVITY MODEL

List of Activities Name, Source
COMPOSITION MODEL
EXPORT MODEL
NOTIFICATION MODEL
Exported Activities
Activity Info
CONTROL MODEL
DATA MODEL
Notification Links
Control In
Data In, Data Out
Control Links
Data Links
LIFECYCLE MODEL
Service Lifecycle
Activity Lifecycle
Precedence Links
Precedence Links
22
Implementation Details

• Built on top of the OGSI Technology Preview
• Implementation of the Grid Services Specification
• Tools AXIS, .NET, Tomcat, WSDL4J, Java CoG
• Important components
• GSFL Parsing and data binding
• WSDL auto-generation for workflow instance
• Generic GSFL Coordinator service, which serves as
  the back-end

23
Implementation Details

• GSFL Parsing Data Binding
• Use of Castor to auto-generate Java source from
  XML schemas
• Castor is an open source data binding framework
  for Java
• Beans generated for every element of the schema
  (appropriate get and set methods)
• Had to add wrappers to certain generated classes
• WSDL auto-generation for workflow instance
• All information needed for auto-generation is
  available in the GSFL document, and individual
  WSDLs
• Tools used WSDL4J, Castor, Xerces

24
Implementation Details

• GSFL Back-end
• Every workflow instance is an instance of the
  generic GSFLCoordinator
• WSDL for the GSFLCoordinator is auto-generated
• Specialized GSFLProvider implemented as an
  extension to the OGSA RPCURIProvider
• The GSFLProvider intercepts all calls to the
  GSFLCoordinator, and dispatches them to the
  generic marshaler operation
• The marshaler operation of the GSFLCoordinator
  uses the GSFL information to route the calls to
  the appropriate services

25
Implementation Details
Client instantiates a workflow instance with a
GSFL document
Client
Hosting Environment
Hosting Environment instantiates a
workflow instance
GSFL Provider
Client calls a method on the workflow instance
Marshaler
Exported Virtual Port
Generic GSFL Coordinator
26
Grid Workflow Example

• Service Providers
• Job Queue Service
• Resource Manager Service
• Pool of Execution Services
• Activities
• Queue/Dequeue Job Queue Service
• GetResource Resource Manager Service
• ExecuteJob Execution Service

27
Grid Workflow Example

• Composition Model
• Export Model Exported Operation - execute
• Identical Control and Data Models
• Dequeue Get next job
• GetResource Get the best resource for the job
• ExecuteJob Execute the job on that resource
• Notification Model
• Each of the Execution services send resource
  information to the Resource Manager service using
  periodic asynchronous soft-state notifications
• Set of Notification Links connecting the
  Execution Services to the Resource Manager service

28
Grid Workflow Example
Execute
SubmitJob
Job Queue
Resource Manager
Dequeue
Get Resource
Queue
Notification Link
ExecuteJob
Execution Service
Execution Service
29
Current and Future Work

• This is a work in progress
• Not in release shape yet
• Continuous updates to keep up with upcoming
  technologies
• Investigate the applicability of BPEL4WS
• Addition of other features, such as
• Exception handling
• Constructs for sequential and parallel control
  flow
• Integration of a graphical workflow editor to
  construct workflow instances visually

30
Conclusions

• GSFL provides a framework for defining workflow
  for Grid services within the OGSA framework
• Recursive composition of Grid services
• Effective peer-to-peer communication between the
  services
• Lifecycle management for individual services
• Leverages ideas from emerging Web service
  technologies
• More information gregor_at_mcs.anl.gov