The Globus Toolkit Status and Plans

1
The Globus ToolkitStatus and Plans

• Bill Allcock
• Argonne National Laboratory

2
Outline

• Web Services Resource Framework
• What, when, and why
• Immediate Plans for the Future
• Terminology
• GT3.2
• GT4.0
• Longer Term Priorities
• Important Points

3
Perspectives

• Why is WSRF important?
• How does WSRF relate to the Open Grid Services
  Infrastructure (OGSI)?
• How does WSRF relate to the Open Grid Services
  Architecture (OGSA)?
• What will the Globus Alliance do with WSRF?
• What does WSRF mean for Globus Toolkit users?

4
ContextOpen Grid Services Architecture

• Define a service-oriented architecture
• the key to effective virtualization
• to address vital Grid requirements
• AKA utility, on-demand, system management,
  collaborative computing
• building on Web services standards
• extending those standards where needed

5
Open Grid Services Architecture(www.ggf.org/ogsa-
wg)
Domain-Specific Services
Program Execution
Data Services
Core Services
Open Grid Services Infrastructure
Web Services Messaging, Security, Etc.
6
Grid and Web ServicesConvergence?
Grid
Web
However, despite enthusiasm for OGSI, adoption
within Web community turned out to be problematic
7
Three Major Web Services Concerns about OGSI

• Too much stuff in one specification
• Does not work well with existing Web services
  tooling
• Too object oriented

8
Grid and Web ServicesConvergence Yes!
Grid
Web
The definition of WSRF means that Grid and Web
communities can move forward on a common base
9
Concerns Addressed

• Too much stuff in one specification
• WSRF partitions OGSI v1.0 functionality into a
  family of composable specifications
• Does not work well with existing Web services
  tooling
• WSRF tones down the usage of XML Schema
• Too object oriented
• WSRF makes an explicit distinction between the
  service and the stateful resources acted upon
  by that service

10
From OGSI to WSRFRefactoring and Evolution
Draft document at www.globus.org/wsrf this week
11
Open Grid Services Architecture
Domain-Specific Services
Program Execution
Data Services
Core Services
Open Grid Services Infrastructure
WS-Resource Framework
Web Services Messaging, Security, Etc.
12
Globus Toolkit andWS-Resource Framework
2004
2005
Note We are not waiting for finalization of WSRF
specs
13
Implications forthe Globus Community

• Production deployments based on GT pre-OGSI
  components
• These components will be included in 3.2 and 4.x,
  and we will continue to support you
• Projects based on GT OGSI components
• Changes are regretted but promise ubiquity
• We will work to ease transition to WSRF
• Similarities between OGSI and WSRF imply that
  most changes will be straightforward

14
Summary of WSRF Issues

• Why is WSRF important?
• WSRF completes Grid/Web convergence
• How does WSRF relate to OGSI?
• WSRF restates OGSI concepts in WS terms
• How does WSRF relate to OGSA?
• WSRF mechanisms will enable OGSA
• What will Globus Alliance do with WSRF?
• WSRF-based GT4.0 planned for Q3 2004
• What does WSRF mean for GT3.0 users?
• For the most only minor changes

15
For More Information

• Specifications, architecture documents, FAQ, and
  other information
• http//www.globus.org/wsrf
• Discussion forum
• http//www.ggf.org/ogsi-wg
• GlobusWORLD Web Site
• Wednesday, 430p WSRF Technical Details
• Other presentations

16
Outline

• Web Services Resource Framework
• What, when, and why
• Immediate Plans for the Future
• Terminology
• GT3.2
• GT4.0
• Longer Term Priorities
• Important Points

17
GTX is not Grid Services

• There is nothing special about GT3
• GT2 was GT1.1.3 with feature enhancements to the
  existing components and a lot of new
  functionality (GridFTP and RC)
• GT3 is all of GT2.4 with feature enhancements to
  the existing components and a lot of new
  functionality (OGSI compliant components)
• GT4 will be a little of an exception. WSRF will
  replace OGSI, but other than that, business as
  usual.

18
Terminology

• Everyone (including us) has a bad habit of saying
  GT3 to mean Grid Services.
• I suspect we will do the same with GT4 and WSRF ?
• To be clear we should refer to OGSI or non-OGSI
  compliant services.
• One confusing point
• For the first time, there are two ways to do some
  things in the toolkit.
• I.e., non-OGSI job submission .vs. OGSI compliant
  job submission

19
Transition

• We use the Linux versioning convention
• Even number releases, I.e. 3.0, are stable
  releases (bug fixes only)
• Odd number releases, I.e., 3.1 are experimental
  (feature additions, possible interface changes)
• Our support policy has not changed since GT 2.0.
• We support the current and one previous stable
  release

20
The Question

• How long will we support GT2.x?
• That is the wrong question
• We invested substantial work in packaging so that
  we could upgrade components separately.
• We can also End-Of-Life (EOL) them independently
  as well.
• This is what we will do

21
The Right QuestionAnd the Answer

• When will we EOL specific Components
• Depends on the component, but what we do know is
  this
• All components will be present in 3.2
• The earliest we could EOL something is 3.4
  (though we have no plans for this yet)
• Therefore, support for all existing components AT
  LEAST until the release of 3.6, probably longer
• This means probably late 2004, and possibly
  well into 2005

22
Evolution

• How a specific project makes this transition is a
  huge question depending on a lot of details.
• Largely depends on how much you have already
  invested in Grid development
• Obviously, HEP has a significant investment,
  which makes it harder
• However, there are some techniques that should
  help.

23
Outline

• Web Services Resource Framework
• What, when, and why
• Immediate Plans for the Future
• Terminology
• GT3.2
• GT4.0
• Longer Term Priorities
• Important Points

24
GT Releases in 2003Evolve 2.x, Introduce 3.0
2003
2004
gt10K downloads/month
25
GT 3.2

• Much sooner than anticipated
• WSRF is a major change so, 4.0
• Had some good work done on OGSI that needed to
  get out
• Had other features that needed to get out also.
• Beta mid to late February
• Release mid to late March

26
GT3.2 Data Management

• GridFTP (still the wuftpd based version)
• New server pulled out
• Bug Fixes
• CAS Enabled
• Structured Listings
• Checksums (whole or partial file)
• Chmod support
• globus-url-copy
• File globbing, directories, recursive, restart,
  data channel security, read from file, partial
  file transfers, RFC 1738

27
GT3.2 Data Management

• eXtensible IO system
• Protocol abstraction GridFTP over non-TCP
• Easy access to GridFTP for 3rd Party apps
• Easy to modify server to custom data source
• Added timeouts (US-CMS problem)
• Reliable File Transfer
• Bug Fixes
• Schema redesign for scalablity
• Directory support
• Data Channel reuse
• Client can sink notifications

28
GT3.2 Data Management

• Replica Location Service
• Bug Fixes
• Hierarchical Index Nodes (RLIs)
• OGSA DAI Tech Preview
• Grid Data Access
• Just Relational for now
• May not be integrated, I.e. a separate tarball

29
GT3.2 Security

• Community Authorization Service
• Ameliorates the MxN problem
• Allows the enforcement of community policy
• GSI
• Bug fixes
• Improved Doc
• Support for Authz callouts
• OpenSSL v0.9.7
• Simple CA

30
GT3.2 Info Services

• Index Service that publishes content as an OGSI
  Service Group
• Enhanced Registrant side support
• ogsi-find-service-data C client that supports
  XSLT
• Software version info published through host-info
  providers
• Ganglia host status provider (tentative)

31
GT3.2 Resource Management

• Managed Job Factory
• Bug fixes (some important ones)
• Performance improvements
• Improved scheduler scripts
• Deactivation of MJSs (scalability)
• Prep work for Globalization
• IPV6 compliant
• Grim Usability Improvements

32
GT3.2 Core

• Implementation of V1 draft 33 of OGSI spec
• Improved deactivate semantics
• Persistent grid service programming model
  improvements
• Support for Certificate Revocation Lists
• Updated security deployment descriptor
• C Bindings improvements
• Python Bindings??

33
Outline

• Web Services Resource Framework
• What, when, and why
• Immediate Plans for the Future
• Terminology
• GT3.2
• GT4.0
• Longer Term Priorities
• Important Points

34
GT4.0

• WSRF
• In simplest terms, replace OGSI core with WSRF
  core
• There will be bug fixing and possibly some minore
  feature changes to existing services
• For everyones sake (including ours), we will
  make the interface changes as non-intrusive as
  possible.

35
GT4.0 other changes

• The new GridFTP server will be released
• No wuftpd code
• Will include striping
• HPSS support (hopefully)
• More Windows support (including XIO)
• Community Scheduler Framework
• Further useability / scalability improvements.

36
Outline

• Web Services Resource Framework
• What, when, and why
• Immediate Plans for the Future
• Terminology
• GT3.2
• GT4.0
• Longer Term Priorities
• Important Points

37
A Grid

• Coordinates distributed resources
• using standard, open, general-purpose protocols
  and interfaces
• to deliver required qualities of experience

38
The Grid of the FutureRoutine, Ubiquitous,
Predictable

• Users and enterprises
• live in a capability-rich environment that adapts
  easily to changing requirements
• Services
• provide the advanced capabilities needed to meet
  user requirements
• Resources
• power services and enable services to meet user
  quality of experience needs

39
Integration and Management as Fundamental
Challenges
40
Guiding Principles Architecture

• Grid definition implies that we must be able to
• Talk about resources in uniform ways
• Ubiquitous SOA base for modeling resources
• Govern the behavior of resources that are not
  under our control /or oversubscribed
• Negotiate agreements among entities
• Exploit commonality across resources so as to
  reduce integration complexity
• Factor across multiple resource types

41
Grid Architecture Hourglass( Anatomy of the
Grid 2001)
Applications and domain-specific services
App
Integration coordination of diverse distributed
services
Collective
Modeling of, access to, data, compute, etc.
Resource
Basic abstractions protocols
Connectivity
Compute, Data, Storage Resources

Structured Data
Fabric


-

Relational
XML
Semi-struct.
Distributed
42
Uniform Resource Management

• We need a ubiquitous service-oriented base for
  modeling managing resources
• This requirement has beena primary driver of our
  workover the last five years
• We are now well positioned to
• Model an increasingly rich resource set
• Model increasingly sophisticated resource
  management lifetime functions

43
Negotiate Agreements

• All interesting interactions will be based on
  agreements between requestors services
• Encode a negotiated quality of experience
• Challenge is to establish the framework within
  which agreements can be
• Established in an oversubscribed environment
• Transformed, composed, decomposed
• Managed like any other resource
• Evolved as a result of faults
• WS-Agreement is the next step on this path

44
Exploit Commonality

• Identify factor commonalities across a broad
• spectrum of applications resources, e.g.
• Task management
• Jobs, data transfers, workflows,
• Data movement
• Files, database replication,
• Registries
• Discovery, metadata, assertions
• Faults management
• Applications, computers, networks

45
Important Points

• WSRF was painful, but necessary
• WSRF will NOT be compatible with OGSI
• But we will minimize the difference as much as
  possible
• We will still be supporting the pre-OGSI
  components
• GT3 is not synonymous with OGSI, nor is GT4 with
  WSRF
• We will EOL components on a component by
  component basis