Grid Resource Management Working Group

1
Grid Resource ManagementWorking Group

• Volker Sander
• Forschungszentrum Juelich GmbH
• v.sander_at_fz-juelich.de
• GGF 4 BOF

2
Goal of the BOF

• Discuss the intention of a Grid Resource
  Management Working group
• Charter settlement
• Identify people willing in participating in the
  documentation process
• Set milestones

3
Charter

• So far
• This working group has the goal to produce a set
  of documents describing a common resource
  management protocol for Grid environments.
• What to add?
• Relation to the view OGSA of Services
• We should add more focus to the intention
• Basic building block for implementing a
  super-scheduler
• service

4
History -- Grid RM Protocol Requirements

• Expand from Advance Reservation to Grid
  Resource Management
• GGF-2 consensus you basically have to do most of
  a general RM protocol to do Advance Reservations
  anyway
• Idea was to create a set of documents
• Grid RM Protocol Requirements
• Grid RM Protocol Operations
• Capabilities and semantics
• WSDL document plus state machine?
• Leverage Other Grid Services Standards
• Transport services / OGSA
• Security
• Language
• Grid RM Protocol Bindings

5
Goal of the BOF

• Discuss the intention of a Grid Resource
  Management Working group
• Charter settlement
• Identify people willing in participating in the
  documentation process
• Set milestones

6
Grid Resource Management GGF-3

• Grid Resource Management Protocol Requirements
• SchedWD 12.1
• Presented revised document (SchedWD 12.1) focused
  on requirements derived from usage scenarios
• Good feedback and discussion on the overall space
  of issues
• A very first starting point
• Not all issues have to be related to a specific
  protocol capability

7
RequirementsSchedWD 12.1

• Control
• Extensibility
• Notification
• Reliability
• Protocol timers
• Negotiation
• Hierarchy
• Secure messaging
• Security language
• Resource language

8
1. Control

• Clients should be able to control their remote
  resource consumption
• Users should be able to express their desired
  services
• Support for adaptation
• Service Providers must be able to control what
  they want to delegate to whom
• To a super-scheduler, broker or user
• Causes additional requirements
• Co-allocation
• Delegation
• Notification
• Reservation and Consumption

9
2. Extensibility

• New Resource types
• Experimental devices
• Higher-level Services
• We need a generalized resource object
• New features and standards
• Upgradeability
• Policy languages
• New access modes
• Do we have to specify a lower-level protocol
  binding?

10
3. Notification

• We are in a highly-dynamic environment
• The ability to subscribe to specific interesting
  events
• Base for dynamic and adaptive applications
• Your Reservation has started
• You exceed your reservation
• State change of capacity space, adapt your
  reservation whenever you want
• Vehicle for Schedulers
• Asynchronous as well as synchronous

11
4. Reliability

• Any resource acquisition and utilization must be
  consistent
• All or nothing within the given policy constrains
• Acquisition should be reversible (Two-Phase
  Commit, Soft-State model)
• Might be associated with costs
• But also durable once it was committed
• Important for Co-allocation
• Barriers on the application layer are an
  additional requirement
• Requests must be persistently named

12
5. Protocol timers

• Soft-state model
• Keep-alive messages
• Could be a way for implementing schedulers
• Ask all candidate resources
• But only refresh reservation of those you select
• Good for failure situations
• Providing a timeout semantics to automatically
  reclaim or cancel orphaned resource assignments
• What if client has gone?
• Does this basically mean that we rely on agents
• User space
• Services

13
6. Negotiation

• More or less a result of asking for an extendable
  protocol
• Should allow more flexibility for resource
  managers and agents
• Upgradeability
• Adapts to the capability of the local RMs
• Grid Scheduling Tokens Document SchedWD10.5
• Web Services Inspection Language
• Do we need a specific handshake step?
• Capability attributes in messages/RPCs
• Out-of-Band

14
7. Hierarchy

• Resource utilization can be related to multiple
  reservations
• Starting two jobs on two different hosts might
  involve the instantiation of a network
  reservation
• Task of the resource specification language?
• Cascaded Reservation Object/Service Request in a
  message?

15
8. Secure messaging9. Security language

• Extremely important!
• Scheduling group handed out a security
  requirement document SchedWD6.9.pdf
• Lost in Space
• http//www-unix.mcs.anl.gov/schopf/ggf-sched/hist
  ory.html
• Control !!!!!
• Who is allowed to do what, what capabilities do I
  delegate to whom
• Policies, Limitations, Delegations
• Do we need more?
• Mutual authentication whenever impersonation
  occurs
• Least privilege principle
• Opaque fields for policy language?
• What about data protection?
• Integrity cant forge it (Mandatory)
• Protection (confidentiality) cant see it
  (Optional)

16
10. Resource language

• What do we need?
• Does it imply the expression of policies?
• Message elements respectively RPC parameters vs.
  RL fields
• Impact on actual description of the protocol
• Impact of language (XML-gtSOAP-gtWSDL)
• OGSA
• WSDL document with state machine
• Register to service factory

17
Other topics in the document

• Generalizing acquisition modes
• Service guarantees
• Quantification of service request
• How much do I want
• We need a service specification object
• Access schedule
• ASAP
• Deadline request (file staging)
• Attribute describing the access schedule (part of
  the service specification)
• Delayed commitment
• Simplification of cost model for cancellation
• Dynamic binding
• Generalized acquisition is bounded to a specific
  object during instantiation

18
Further Intention of the document

• Clarify the level of abstraction we need
• Wire protocol vs. RPCs vs. Object messages
• We need not necessarily have to rely on a
  specific protocol binding, but should provide
  documents on different bindings
• State machine
• Open Grid Services Architecture
• Launch requirement documents for other WGs (and
  vice versa)

19
Goal of the BOF

• Discuss the intention of a Grid Resource
  Management Working group
• Charter settlement
• Identify people willing in participating in the
  documentation process
• Set milestones

20
Milestones

• Grid RM Protocol Requirements
• Grid RM Protocol Operations
• Capabilities and semantics
• Leverage Other Grid Services Standards
• Transport services / OGSA
• Security
• Language
• Grid RM Protocol Bindings
• Who is willing to participate in the
  documentation process?

21
(No Transcript)
22

• Notes from bof

23
Is this worthwhile?

• Yes - if you can get good info back from the AR
  system
• globus had error hierarchies for example, but
  experiences werent too good
• what was meant wasnt errors but additional info
• NO - what if you have a system that says it gives
  you an AR and just gives you garbage?
• What is an advance reservation?
• Note - there will never be a perfect AR

24

• What about policy management?
• Very difficult over a grid
• protocol could quite likely be defined without
  getting into specific policy info

25
Is scope too broad?

• Yes, probably- how to focus it?
• Suggestion 1 - Protocol for general service
  request? WSDL document?
• Suggestion 2 - What about AR for subset of
  resources? - NW, CPU, file staging

26

• Needed -
• cluster-wide reservations (ex. EDG work)
• Premium class of service
• co-allocation of cpu, viz and steering
  functionality (control of apparatus)
• Maybe a research group
• what is advance reservable at all?

27

• Two choices
• research group, broad scope
• working group, requirements doc.
• Could include scenarios (are there some in the
  API doc?)
• What about use cases?
• Pbs pro, ccs, etc are doing this
• www.mcs.anl.gov/jms/ggf-sched/

28
Working group, advance reservation protocol

• Need for 2-phase commit
• Jon and Wolfgang second this motion
• next step - charter to send around group, then
  forward to SG

29
(No Transcript)