Agreementbased Distributed Resource Management - PowerPoint PPT Presentation
1 / 19
Title:
Agreementbased Distributed Resource Management
Description:
Resource owner goals vs. application goals. An Open Architecture to Manage Resources ... Idiom: enables one-shot job submission. Agreement::renegotiate() Fine-grained ... – PowerPoint PPT presentation
Number of Views:46
Avg rating:3.0/5.0
Title: Agreementbased Distributed Resource Management
1
Agreement-based Distributed Resource Management
- Alain Andrieux
- Karl Czajkowski
2
Overview
- The Resource Management Problem
- Decentralized resource coordination
- Resource owner goals vs. application goals
- An Open Architecture to Manage Resources
- Agreement-based negotiation model
- Several scenarios
- WS-Agreement (GGF GRAAP-WG)
- Status work in progress
- Agreements using OGSI concepts
3
Distributed Resource Management
- 1. Discovery
- What resources are relevant to interest?
- Finds service providers
- 2. Inspection
- Whats happening to them now?
- Compare/select service providers
- 3. Agreement
- Will they provide what I need?
- The core Resource Management problem
- Process can iterate due to adaptation
4
Social/Policy Conflicts
- Resource Consumers/Applications Goals
- Users deadlines and availability goals
- Applications need coordinated resources
- Localized Resource Owner Goals
- Policies distinguish users
- Community Goals Emerge As
- Global optimization goals
- aggregate user/application and/or resource
- Reconcile demands via Agreement
5
Early Co-Allocation in Grids
- SF-Express (1997-8)
- Real-time simulation
- 12 supercomputers, 1400 processors
- Required advance reservation
- Brokered by telephone!
- Practical use requires automation
- Complex fault environment
- Over 45 minutes to recover from failure
- Reservations cannot prevent faults
6
Traditional Scheduling
- Closed-System Model
- Presumption of global owner/authority
- Sandboxed applications with no interactions
- Toss job over the fence and wait
- Utilization as Primary Metric
- Deep batch queues allow tighter packing
- No incentives for matching user schedule
- Sub-cultures Counter Site Policies
- Users learn tricks for gaming their site
7
An Open Negotiation Model
- Resources in a Global Context
- Advertisement and negotiation
- Normalized remote client interface
- Resource maintains autonomy
- Automated Agents Bridge Resources
- Drive task submission and provisioning
- Coordinate acts across domains
- Community-based Mediation
- Agents coordinating for collective interest
8
Community Schedulers
- Individual users
- Require service
- Have application goals
- Community schedulers
- Broker service
- Aggregate scheduling
- Individual resources
- Provide service to clients
- Have policy autonomy
9
Intermediaries And Policy
Community
Client
Resource
Resource
advertise
advertise
control
Scheduler
Application
Manager
request
request
respond
respond
User Policy
Resource Policy
Community Policy
- Resource virtualization can
- Abstract details of underlying resource(s)
- Map between different resource description
domains - Policies from different domains influence
agreement negotiations with intermediaries
10
Heterogeneity of Service
- Many Kinds of Task
- Data stored file, data read/write
- Compute execution, suspended job
- Many Kinds of Resource
- Hardware disks, CPU, memory, networks, display
- Capabilities space, throughput
- Coordination Problem is much the same
11
Specialization File Transfer
- Single goal
- Reliable deadline transfer
- Specialized scheduler
- Brokers basic services
- Synthesizes new service
- Fault-handling logic
- Distributed resources
- Storage space
- Storage bandwidth
- Network bandwidth
J1
S1
J3
J2
R1
R3
R2
12
Technical Challenges
- Complex Security Requirements
- Global Scalability
- Similar ideals to Internet
- Interoperable infrastructure
- Policy-configurable for social needs
- Permanence or Evolve in Place
- Cannot take World off-line for service
- Over time upgrade, extend, adapt
- Accept heterogeneity
13
WS-Agreement Components
14
WS-Agreement Model
- Generic/extensible negotiation model
- Agreement wraps domain-specific terms
- Agreement supports extensible monitoring
- Reuse OGSI mechanisms
- Specializes ogsiFactory pattern
- Flexible lifetime negotiation for Agreements
- ServiceData for monitoring/introspection
15
Negotiation Interfaces
- AgreementFactory
- Persistent service
- Ex façade to scheduler(s)
- Creates Agreement services
- Agreement
- Transient service
- Ex job entry virtualized into a service
- Encapsulates state of negotiation
- Terms, service status, relationship to other
Agreements - Lifetime maps to lifetime of terms of service
16
Two-level Negotiation
- AgreementFactorycreateService()
- Coarse-grained
- Conventional fault/response model
- Batch negotiation of complex terms
- Idiom enables one-shot job submission
- Agreementrenegotiate()
- Fine-grained
- Allows complex multi-message negotiation
- Admits adaptation of provisioning terms
17
Agreement-based Jobs
- Agreement represents queue entry
- Commitment with job parameters etc.
- Job structure
- Wide range of QoS guarantees
- Point for monitoring/control of job
- Service is the Job computation
- Agreement-specific computation
- May or may not communicate with clients
- Advance Reservation is pre-agreement
- Facilitates future job negotiation
18
Agreement Terms
- Real Agreements mix-in domain terms
- Composed by logical grouping
- Combined with negotiability mark-up
- Each domain term brings a semantics
- Unambiguous service-provisioning concept
- Yamount of RAM allocable to process
- Agreement contextualizes domain term
- (Y gt 512 MB) AND (Y lt 1024 MB)
19
The End
- WS-Agreement is just beginning
- GRAAP-WG at GGF
- Work on core negotiation model
- Work on reusable term meta-language
- Domain Terms needed
- Job submission
- Data management
- Accounting/Economic trading?
Recommended
CrystalGraphics Presentations
