Use of Workflow Techniques for Grid Management

1
Use of Workflow Techniques for Grid Management

• Junwei Cao (???)
• CC Research Laboratories
• NEC Europe Ltd.
• Germany

2
CCRLE

• Bonn, Germany
• 20 research staffs
• 3 working teams
• Numerical simulation
• MPI development
• Grid Computing
• GEMSS

3
My Previous Experience

• PACE performance prediction of parallel and
  distributed systems
• Titan prediction based job scheduling on
  clusters and multiprocessors
• ARMS agent-based resource management for grid
  computing
• GridFlow workflow management for grid computing
  (CCGrid 2003)

4
Outline

• Background
• GEMSS objectives
• Medical simulation applications
• Service performance prediction
• Workflow simulation and scheduling
• Dynamic service federation
• Summary

5
Grid Workflow Management

• Workflow Definition
• WPDL, BPEL4WS, GSFL, ASCI Grid
• Workflow Systems
• WebFlow, Symphony, GridAnt, BPWS4J, TENT
• Component-based Systems
• CCA/XCAT, SCIRun, CXML
• Other Systems
• Condor DAGMan, UNICORE, USC Grid failure handling

6
GEMSS Objectives

• Demonstrate that the grid can improve
  pre-operative planning near- real-time surgical
  support by providing access to advanced
  simulation and image-processing services.
• Build middleware on existing or developing grid
  technology standards to provide support for
  authorization, workflow, security and Quality of
  Service aspects.
• Develop, evaluate and validate a test-bed for the
  GEMSS system, including its deployment in the
  end-users working environment.
• Anticipate privacy, security and other legal
  concerns by examining and incorporating into its
  grid services the latest laws and EU regulations
  related to providing medical services over the
  Internet.

7
Medical Applications

• Medical simulation supports the optimization of
  operation procedures and the planning of
  therapeutic strategies.

Maxillo-facial surgery
Post- surgery
Pre- surgery
8
Image Pre-processing

• An intensity based algorithm (adaptive fuzzy
  C-means algorithm) is used to provide good
  quality segmentations for structures of the human
  head.

Identification of substructures
CT Images
9
Numerical Modeling
Next to the image processing step follows the
geometric modeling of the structures suitable for
Finite Element simulations.
Mesh manipulation using the halo positioning tool
Mesh generation with Or without smoothing
10
HPC Simulation Visualization
The geometric face change
Calculated deformation of the skull
11
The Problem

• The application includes the use of a complete
  chain of tools necessary for the entire process
  from geometric model generation from scan data
  (segmentation, mesh generation and mesh
  manipulation) to computer simulation and
  visualisation.

12
Application Workflow
Image Segmentation
Computation intensive / Semi interactive
Mesh Generation
Computation intensive / parallel
Mesh Manipulation
Full interactive
Finite Element Simulation
Computation intensive / parallel
Visualization
Data intensive / Full interactive
13
Grid Enabling

• Interface definition of each module using the
  WSDL
• Implementing each module as a web service using
  Apache Axis or a grid service using the GT3
• Definition of the whole process using the BPEL4WS
• Using a BPEL4WS engine for service invocation
• Applied only to Computation intensive parts

14
Challenges

• Could you finish the process in 1 hours? QoS
  and adaptation support is becoming the most
  active research topic in grid computing
  community.
• Service performance prediction
• Workflow simulation and scheduling
• Dynamic service federation

15
Grid Performance Services

• Building performance services as high-level grid
  services based on OGSA core and base services
• Managing performance-related data
• Defining performance metrics
• Developing performance analysis algorithms
• Developing new APIs for grid service performance
  prediction

16
Performance-related Data

• Application parameters that have an impact on
  application performance
• System status e.g. CPU load, job queue and
  network bandwidth
• Managing performance-related data using OGSA
  service data support

17
Performance Metrics Analysis Algorithms

• Performance metrics
• Execution time
• Memory usage
• Price, and more
• Performance analysis historic information based
• Statistical analysis algorithms
• Self organizing mapping
• More

18
Workflow Simulation, Scheduling Rescheduling
/ 5 / 5
/ 7 / 12
19
Dynamic Service Federation

• Using the BPEL4WS service references to select
  and assign actual partner service dynamically.
• Extending BPEL4WS ltpartnergt with some kind of
  ltcandidategt element to indicate candidates of a
  grid service partner.

20
The Solution
Grid Service (WSDL) Performance
Services Service data Application portTypes
UDDI
WSDL Workflow Simulation Engine Workflow Executi
on Engine
Grid Service (WSDL) Performance
Services Service data Application portTypes
User
21
Other Aspects

• Accuracy of performance prediction
• Workflow execution monitoring
• Security and legal issues
• Grid workflow GUIs
• Grid data management
• Enabling interactive applications
• Using workflow techniques for business process
  management in GEMSS

22
Summary

• Programming models for the grid workflow
  specification a candidate?
• QoS support and application adaptability
  performance prediction workflow simulation gt
  dynamic service federation a solution?
• Medical simulation applications a right target
  application of the grid?
• AgileGrid agile computing on business grids a
  next generation computing paradigm?