Porting%20applications%20to%20Globus%20Toolkit%203.0%20and%20designing%20an%20OGSI-based%20architecture

1
Porting applications to Globus Toolkit 3.0 and
designing an OGSI-based architecture

• Pawel Plaszczak and Rich Wellner

Perfringo Grid Solutions http//perfringo.com
Tutorial materials http//perfringo.com/events
2
About the authors

• From Fosters Globus team in Argonne
• Co-founded Perfringo Grid Solutions
• Focus industry level Grid applications
• Currently NeesGRID - distributed earthquake
  simulations
• pawel_at_perfringo.com, rw2_at_perfringo.com

3
Prerequisities

• Necessary distributed systems software design,
  purpose of Grid
• Useful Java, Web services, XML, Grid services
  (will be briefly explained)
• Not necessary coding experience
• We will speak on decision making and design. No
  hands-on programming

4
Plan

• Part I Grid services - why and when?
• Part II OGSI and OGSA technology review
• Part III Use case NTCP
• Part IV Porting to Grid services and GT3

5
Plan of Part I Grid services - why and when?

• Overview Globus Alliance and Toolkit
• Overview GT2 vs GT3, OGSA
• Why go Grid?
• Why comply with OGSA?
• Why port to GT3?

6
The Globus Alliance

• Globus Project , since 1996
• Ian Foster (Argonne National Lab),
• Carl Kesselman (University of Southern
  Californias Information Science Institute)
• Develop protocols, middleware and tools for Grid
  computing
• Globus Alliance, since Sept 2003
• International scope
• University of Edinburghs EPCC
• Swedish Center for Parallel Computers (PDC)
• Advisory council of Academic Affiliates from
  Asia-Pacific, Europe, US

7
Globus Toolkit

• GT2 (2.4 released in 2002) reference
  implementation of Grid fabric protocols
• GRAM for job submissions
• MDS for resource discovery
• GridFTP for data transfer
• GSI security
• GT3 (3.0 released July 2003) redesign
• OGSI based
• Grid services, built on SOAP and XML

8
OGSI and OGSA

• OGSI Open Grid Service Infrastructure
• Specs from GGF OGSI working group
• Defines what makes a Grid service
• Based on Web service
• Naming, life cycle, state, notification
• portTypes definitions, WSDL 1.2 draft
• OGSA Open Grid Service Architecture
• Specs from GGF OGSA working group
• Defines a list of fundamental Grid services, and
  how they cooperate
• Work in progress

9
Whats new in GT3.0

• Emphasis on OGSI compliant Grid services
• OGSA implemented
• Uniform SOAP/XML communication layer
• Focus on Java implementation
• Old GT2 services redesigned
• Backward compatibility where possible

10
Meaning of GT3 in the community

• Most commonly referred project
• Traditionally de facto standard
• Acknowledged leadership by academia and industry
  (IBM,)
• BSD style license allows for commercial usage
• However, it is only a reference implementation.
  Now standards GGF
• GT undergoes constant changes
• With business entering grid, commercial
  implementations may soon catch up

11
Alternatives to GT3

• Protocol level interoperability
• Grid compliant gt implements OGSI
• Other OGSI implementations
• OGSI.NET (U.Virginia)
• pyGlobus (LBNL)
• .NET (U.Edinburgh)
• PERL (U. Manchester)
• UNICORE (Fujitsu)
• Commercial OGSI compliant products by
• Avaki, Platform, Data Synapse,
• Web service alternative Grid App Framework
  http//www.neresc.ac.uk/ws-gaf

12
Part I contd.

• Overview Globus Alliance and Toolkit
• Overview GT2 vs GT3, OGSA
• Why?
• Why go Grid?
• Why comply with OGSA?
• Why port to GT3?
• upgrading from GT2

13
Why go Grid?

• Hot subject
• Try it, experience it to learn the potential
• Will enable true ubiquitous computing in future
• Today, proven in some areas intraGrids
• But still long way to World Wide Grid
• State of art techniques, tools are difficult
• Short term goals? Use another technology
• Does your system have Grid characteristics?
• Distributed users, large scale and heterogeneous
  resources, across domains

14
OGSI architecture why and when?

• Commonly accepted Grid standard
• But there are alternatives WS-GAF, GT2,
• Extends Service-Oriented Architecture
• Modular, scalable, maintainable, interoperable
• Extensions use only when needed
• Uniform communication layer
• Be compatible and inter-operable with other grid
  systems (thats what grids are about!)

15
Porting to GT3 why and when?

• Implementing OGSI is plenty of work!
• Piggyback on GT3 for OGSI/A compliance (watch
  commercial products)
• Most serious middleware to consider
• Language bindings traditionally Unix/C, now Java
  leads
• But unstable, new versions bring changes
• Traditionally, for hard core hackers
• Traditionally, documentation was lacking
• recent signs of improvement, also IBM redbook

16
GT3 stability

• GT3 took only 1.5 year to develop. It will change
  many times!
• Anticipated or considered changes
• Refactor GridFTP control channel into XML, DAI
• Security add CASS
• GRAM ongoing improvements
• OGSA-C hosting environment to come
• OGSA specs is draft
• OGSI specs are completed (v. 1.0)

17
GT3.2

• Expected January 2004
• Features
• CAS (Community Authorization Service)
• Index Service reimplemented
• GridFTP server reimplemented
• GridFTP java client connection reuse, MLST
• GRAM performance improvements
• CRL support (certificate revocation list)
• several bug fixes in core

18
Java CoG Kit status

• Before Java client for GT2
• Now integrated with GT3
• Provides low level API for GT3 services
  security, GridFTP
• Experimental features to come soon (not part of
  GT3)
• Workflow management, compliant with GT2 and GT3
• Comes with example Jetspeed portlets

19
roadmap

• Part I Grid services - why and when?
• Part II OGSI and OGSA technology review
• Part III Use case NTCP
• Part IV Porting to Grid services and GT3

20
Plan of Part II OGSI and OGSA technology review

• Web service
• XML and SOAP
• Service Oriented Architecture
• Grid service (OGSI)
• OGSA
• How does GT3 implement OGSA
• Examples

21
What is a Web service

• Not much to do with HTML
• Interface to a well-explained, self-contained
  function (module, component)
• Usu. small grained
• Defined in WSDL
• Communication XML and SOAP

Web service client
SOAP/XML
OnlineStore Web service
queryItems() orderItem()
22
More on Web services

• Specs
• W3 Consortium www.w3.org, IBM, Microsoft
• OASIS (WS security)
• For developers Building Web services with Java
  by Steve Graham et al
• For managers Web services and Service-Oriented
  Architectures by Doug Barry (the savvy managers
  guide)
• Stay tuned IBM developerWorks articles
• http//www-106.ibm.com/developerworks/webservices/

23
Service-Oriented Architecture

• System composed of several components exhibiting
  WS interfaces
• Discovery through a well-known directory, such as
  UDDI
• External services use internal ones
• Modular, scalable, maintainable, interoperable

24
Service-Oriented Architecture in practice
OnlineStore WS
Web service client
queryItems() orderItem()
query
order
StockMgr WS
OrderMgr WS
query() ship()
order() bill()
ship
warehouse
25
What is a Grid service

• Defined by OGSI (GGF working group)
• Is a Web service with additions, which are
• Name (handle GSH, reference GSR)
• Lifetime management (factories, persistent and
  transient services)
• State (Service Data)
• Notification as well as querying
• WSDL 1.2 draft (gwsdl namespace)
• Definitions of portTypes

26
Grid services in practice notifications and
service data
Notification source
Subscribe tell me when you have it on stock
OnlineStore GS
Grid service client
query() subscribe()
Notify it just arrived.
Notification sink
StockItem SDE
Service data element
27
Grid services in practice naming and life cycle
OnlineStore Persistent GS
Grid service client
1. query
queryItems()
4. query
2. Get reference
StockItemAF45X Transient GS
3. create
StockMgr Persistent GS
getItem()
Item SDEs dimension, price
Factory
28
What is OGSA

• Open Grid Service Architecture, being defined by
  GGF OGSA working group
• In ubiquitous Grid platform, there is common need
  for some basic services
• OGSA defines this set of base services
• OGSA builds on OGSI

29
How does GT3 implement OGSA

• Java implementation, C to follow
• Container approach host small-grained services
• Hosting environment takes care of basic GS
  functions
• GT3 provides core and base services, users to
  implement their own
• Transient and persistent GS API
• Tools to generate Java lt-gt GWSDL
• Axis engine used for SOAP handling
• Run with Tomcat or standalone

30
roadmap

• Part I Grid services - why and when?
• Part II OGSI and OGSA technology review
• Part III Use case NTCP
• Part IV Porting to Grid services and GT3

31
Plan of part III NTCP

• NeesGRID project overview
• NTCP requirements high level design
• Choosing Grid services for NTCP subsystem
• Choosing Grid services for Matlab subsystem
• Not choosing Grid services for ShoreWestern
  subsystem
• Deployment
• On-going work

32
NeesGRID

• Make collective use of geographically distributed
  experiment facilities, heterogeneous hardware
• Users dislocated from facilities
• Applying Grid to earthquake science
• 3 year project involving several US institutes
• Distributed simulation of a building in an
  earthquake
• Scale models costs millions of and several
  months to build. Some experiments are not
  repeatable (model used once)

33
The MOST experiment
Slide courtesy of Bill Spencer, UIUC
34
MOST requirements

• Distributed simulation of a building in an
  earthquake
• 3 experimental sites
• Hybrid experiment 2 physical and 1 computational
  simulations
• Results at one time-step used to calculate inputs
  at the next time-step
• Pseudodynamics order of 10 ms per time-step
• Successfully run on 30th July 2003

35
NTCP NEESGrid Teleoperation Control Protocol

• Vital component of NeesGRID and MOST
• Telecontrol system, used to coordinate the
  experiment with commands
• Output data collected with other NEESGrid systems
  like NSDS

36
NTCP system architecture
Coordinator (NTCP client)
NTCP subsystem
NTCP
NTCP
NTCP
NTCP server
NTCP server
NTCP server
Local protocol
Local protocol
Local protocol
local subsystems
Backend ShoreWestern
Backend Matlab sim.
Backend Matlab XPC
NCSA
U Colorado
UIUC
37
NTCP protocol

• Coordinator follows the script with recorded
  earthquake data
• Coordinator orders servers to simulate one
  time-step at a time (pseudodynamics)
• Servers can refuse if parameters out of range
• Core commands
• Propose() negotiates an action
• Execute() starts an action
• Cancel() issued when negotiation failed
• Other get/setParameter, open/closeSession

38
Sequence diagram (simplified)
NTCP Server A
NTCP Server B
coordinator
openSession
openSession
propose
(accepted)
execute
1 time step
(started)
execution
results
propose
39
NTCP subsystem choosing Grid services and GT3

• Long term goals
• System characteristics distributed,
  heterogeneous resources across administrative
  domains
• Interoperability, manageability, scalability
• Risk GT3 released early July, MOST scheduled end
  of July

40
NTCP server simplified design
41
NTCP server as a Grid service

• Front end Permanent Grid service
• Operates on real hardware only admin can restart
• methods
• Open/closeSession, propose, execute, cancel,
  get/setParameter, getControlPoint
• Service data Transaction state
• Notification Execution results
• Back end LocalControlPlugin interface for
  implementing adapters to the local non-OGSI
  technologies

42
Sequence diagram revisited in Grid service terms
Coordinator (GS client)
NTCP Grid service
Propose transaction
Transaction Service Data
create
(accepted)
Subscribe for transaction status
1 time step
Execute transaction
execution
(started)
update
Transaction results notification
results
43
Matlab subsystem plugin and backend
Coordinator (NTCP client)
NTCP
NTCP
NTCP
NTCP server
NTCP server
NTCP server
Local protocol
Local protocol
Local protocol
Backend ShoreWestern
Backend Matlab sim.
Backend Matlab XPC
44
Matlab subsystem characteristics

• Command flow
• Coordinator
• NTCP server frontend
• NTCP server Matlab control plugin (Mplugin)
• Backend machine running Matlab simulation
• Matlab single threaded, not event driven
• Information pull model enforced
• Matlab has Java interface, fully compliant with
  Java 1.3. Can use Java class library, third party
  jar files, multiple threads

45
Matlab subsystem choosing Grid services

• Choose network communication technology between
  Mplugin and backend
• TCP/IP, RMI, Corba, Web services
• Although no Grid characteristics why not Grid
  services?
• Pulling scenario -gt server on the same machine
  as the NTCP server
• OGSA container is already there
• Ockham razor one technology instead of many

46
Typical Service-Oriented Arch compared to NTCP
Matlab
47
Client, server and backendin Grid service model

• Methods
• hasCommand, pullCommand, pushResult
• No service data, notification
• Could as well be a Web service
• Reversed information flow, as compared to typical
  Service-Oriented
• Both NTCP and commandBuffer services are
  persistent must match at start-up

48
Testing the NTCP and Matlab subsystem

• Junit and TestServer from GT3
• 3 test suites, 4 test units
• Some units implement same interfaces
• 3 simulators, some deployed inside gar

49
ShoreWestern subsystem
Coordinator (NTCP client)
NTCP
NTCP
NTCP
NTCP server
NTCP server
NTCP server
Local protocol
Local protocol
Local protocol
Backend ShoreWestern
Backend Matlab sim.
Backend Matlab XPC
50
ShoreWestern subsystemplugin and backend

• ShoreWestern hardware driver to operate physical
  experiment facility
• Push communication OK
• ShoreWestern already speaks a proprietary
  protocol based on TCP/IP, equivalent in
  functionality to NTCP

51
TCP/IP based prototype of NTCP reused for
ShoreWestern
Coordinator
NTCP server
Grid service Front end
TCP/IP
TCP/IP
ShoreWestern ctrl plugin
Backend sim. ShoreWestern
TCP/IP
Same protocol
TCP/IP
Backend sim. Matlab
Backend sim. Matlab
ShoreWestern backend
52
ShoreWestern subsystemnot choosing Grid
services

• Backend has server role
• Reuse existing TCP/IP protocol
• Ockham razor (again!)
• ShoreWestern plugin adapter to the old protocol

53
Complete NTCP system for the MOST experiment
Coordinator (NTCP client)
OGSI
OGSI
OGSI
OGSA container
OGSA container
OGSA container
NTCP Grid service
NTCP Grid service
NTCP Grid service
ShoreWestern ctrl plugin
CmdBuffer Grid service
CmdBuffer Grid service
TCP/IP
OGSI
OGSI
Backend e.g. ShoreWestern
Backend e.g. Matlab XPC
Backend e.g. Matlab sim.
54
MOST experiment deployment

• First GT3-based application deployed in
  production
• 3 remote sites
• NCSA, UIUC, University of Colorado
• 5.5 hours and almost 1500 steps
• Monitored by several engineers and scientists
• Remote observers through advanced Grid
  technology multiple life video, life data, chat,
  all via Web interface
• Output data to be analyzed by earthquake
  scientists

55
NTCP ongoing work

• Very high requirements
• Reliability
• Safety
• Error checking
• Security
• Involved Information Science Institute, Argonne
  National Lab, Perfringo Grid Solutions

56
NTCP Acknowledgements

• Bill Spencer, UIUC (NeesGRID PI)
• Laura Pearlman, ISI (NTCP lead)
• Erik Johnson, USC (Matlab engineering)
• Ravi Madduri, Mike DArcy (Globus team developers)

57
roadmap

• Part I Grid services - why and when?
• Part II OGSI and OGSA technology review
• Part III Use case NTCP
• Part IV Porting to Grid services and GT3

58
Plan of Part IV Porting to Grid services and GT3

• All depends on your starting point
• Choose the best approach
• High or low abstraction level?
• Study early experience with working systems
• Stay tuned

59
All depends on your starting point

• Already have a GT2 based system?
• Porting GT2 -gt GT3
• Already have Web services based system?
• Porting Web services -gt Grid services
• Have a system in other technology?
• Starting from scratch?

60
Porting GT2 -gt GT3 GridFTP

• GT2 GridFTP is still there.
• Consider using Java client API from CoG
  (org.globus.ftp aka jftp)
• For 3rd party, consider 3-tier architecture with
  reliable file transfer (RFT)
• Uncertain long term protocol stability

Grid service Client API
RFT
jftp client API
data
GridFTP server
GridFTP server
61
Porting GT2 -gt GT3 GRAM

• GT2 GRAM now refactored into XML
• Server side new XML front end, old backend
  submission scripts
• New XML clients Java or C
• Client side API-level backward compatible old
  apps only need recompiling

62
Porting GT2 -gt GT3 MDS

• GT3 Information Services, aka MDS3
• fundamental change in technology
• Old GRIS now Service Data
• Old GIIS now Index Service
• Old LDAP/LDIF now SOAP/XML
• RFC2252 MDS schema now SDD in GWSDL
• Porting old data providers
• Substantial amount of work
• Generally, need an adapter Grid service

63
Porting GT2 -gt GT3 the rest

• GT2 GSI security still there
• New GT3 allows creating services not based on
  GridFTP, GRAM and MDS. Think about redesigning
  your architecture

64
Porting Web services to Grid services theory

• Grid service is a Web service
• Naming need to introduce handle and reference
• Life cycle
• Web services are like persistent Grid services
• Service data - not necessary
• Get/set methods can now be implemented as SDE
• Notification - not necessary
• Discovery
• GT3 uses Index Service as directory
• Web services sometimes use UDDI

65
Porting Web services to Grid services your
options

• Implement from scratch
• Need to implement (at least) handle and reference
  management
• Can retain your old Web service code
• Implement with GT3
• Get complete OGSI feature set for granted
• Hidden porting work
• Is the WS already in Axis? Handler based approach
• WSDL -gt GWSDL
• Different procedure to deploy in container
• GSI security (builds on top of WS-security)
• Dependencies, versioning
• Use non-GT3 OGSI middleware

66
Experience of NEReSC CGM team workflow service

• Courtesy of Saras Parastatidis and Paul Watson,
• http//www.neresc.ac.uk/projects/CGM/web

Web Service
Grid Service
67
Experience of NEReSC CGM team notification
service

• Courtesy of Saras Parastatidis and Paul Watson,
• http//www.neresc.ac.uk/projects/CGM/web
• Direct port of servlet proved too difficult
• Dependencies on mysql, OpenJMS, Tomcat
• Versioning nightmare
• Architecture converted to 3-tier
• New Grid service adapter constructed
• Adapter forwards to old Web service
• Client adaptation costed time

68
Experience of NEReSC CGM team conclusions

• Courtesy of Saras Parastatidis and Paul Watson,
• http//www.neresc.ac.uk/projects/CGM/web
• Web and Grid services can coexist
• Adapter pattern is one solution
• Alternative to OGSI Grid Application Framework
  based on Web Services Specifications and
  Practices document (http//www.neresc.ac.uk/ws-gaf
  )

69
Choose the best approach

• Writing Grid services from scratch
• Redesigning legacy system in Grid service
  architecture
• Refactoring communication layer
• Façade pattern approach

70
High or low abstraction level

• Adapt to lowest necessary abstraction
• Web services?
• OGSI? (what features needed?)
• OGSA? (what services needed?)
• Reuse highest possible abstraction
• track GGF progress
• track Globus plans (announcements)
• Ongoing work OGSA-DAI, OGSA-DQP, workflows, OGSA
  itself, WS-Agreement

71
Data access integration

• Global Grid Forum Data Access and Integration
  Services (DAIS) working group
• Reference implementation OGSA-DAI by Edinburgh
  Parallel Computing Center (EPCC)
• middleware to assist with access and integration
  of data from separate data sources via the grid
• First stage ended July 2003 software available
  http//www.ogsadai.org.uk/

72
Hints

• Grid services can be small grained
• One idea for the next GridFTP server is to
  represent each file as a Grid service
• Existence of a handle does not imply existence of
  a service! It should apear as a service, but
  implementation can be custom
• Represent abstractions as services
• Agreement, transaction
• Method or SDE?
• Get/set functionality can be faster to implement
  as methods, but more elegant as service data

73
Experience with working systems

• NeesGRID
• Quertermaster
• NEReSC Core Grid Middleware
• other

74
Stay tuned

• Quarterly GGF gridforum.org
• Globus World, January 20-23 2004, San Francisco
• GridToday, May 2004 to focus on commercial
  applications
• Perfringo events
• Training events, tutorials, newsletter
  subscribe_at_perfringo.com
• Questions and comments? pawel_at_perfringo.com

75
Your feedback

• Slides available for download
• http//perfringo.com/events
• Please send feedback to pawel_at_perfringo.com
• What did you learn?
• What did you expect?
• What could be added / improved?

76
Thanks!

• Laura Pearlman (NeesGRID, Globus)
• Bill Spencer (NeesGRID, UIUC)
• Savas Parastatidis (NEReSC CGM)
• Lisa Childers, Jarek Gawor, Gregor von Laszewski
  (Globus team)