Grid(Lab) Resource Management System

1
Grid(Lab) Resource Management System
and general Grid Resource Management
Jarek Nabrzyski et al.
naber_at_man.poznan.pl
Poznan Supercomputing And Networking Center
2
GridLab

• EU funded project, involving 11 European and 3
  American partners (Globus and Condor teams),
• January 2002 December 2004
• Main goal to develop a Grid Application Toolkit
  (GAT) and set of grid services and tools...
• resource management (GRMS),
• data management,
• monitoring,
• adaptive components,
• mobile user support,
• security services,
• portals,
• ... and test them on a real testbed with real
  applications

3
GridLab Members

• PSNC (Poznan) - coordination
• AEI (Potsdam)
• ZIB (Berlin)
• Univ. of Lecce
• Cardiff University
• Vrije Univ. (Amsterdam)
• SZTAKI (Budapest)
• Masaryk Univ. (Brno)
• NTUA (Athens)
• Sun Microsystems
• Compaq (HP)
• ANL (Chicago, I. Foster)
• ISI (LA, C.Kesselman)
• UoWisconsin (M. Livny)

• collaborating with
• Users!
• EU Astrophysics Network,
• DFN TiKSL/GriKSL
• NSF ASC Project
• other Grid projects
• Globus, Condor,
• GrADS,
• PROGRESS,
• GriPhyn/iVDGL,
• CrossGrid and all the other European Grid
  Projects (GRIDSTART)
• other...

4
Its Easy to ForgetHow Different 2003 is From
1993

• Ubiquitous Internet 100 million hosts
• Collaboration resource sharing the norm
• Ultra-high-speed networks 10 Gb/s
• Global optical networks
• Enormous quantities of data Petabytes
• For an increasing number of communities, gating
  step is not collection but analysis
• Huge quantities of computing 100 Top/s
• Ubiquitous computing via clusters
• Moores law everywhere 1000x/decade
• Instruments, detectors, sensors, scanners

Courtesy of Ian Foster
5
And Thus,The Grid Problem (or Opportunity)

• Dynamically link resources/services
• From collaborators, customers, eUtilities,
  (members of evolving virtual organization)
• Into a virtual computing system
• Dynamic, multi-faceted system spanning
  institutions and industries
• Configured to meet instantaneous needs, for
• Multi-faceted QoX for demanding workloads
• Security, performance, reliability,

Courtesy of Ian Foster
6
Integration as a Fundamental Challenge
Courtesy of Ian Foster
7
Grid Scheduling

• Current approach
• Extension of job scheduling for parallel
  computers
• Resource discovery and load-distribution to a
  remote resource
• Usually batch job scheduling model on remote
  machine
• But actually required for Grid scheduling is
• Co-allocation and coordination of different
  resource allocations for a Grid job
• Instantaneous ad-hoc allocation not always
  suitable
• This complex task involves
• Cooperation between different resource
  providers
• Interaction with local resource management
  systems (interfaces and functions of many LRM
  differ from each other)
• Support for reservations and service level
  agreements
• Orchestration of coordinated resource allocations

8
Allocation for Grid Job Example
time
Data
Storing Data
Data Access
Network 1
Data Transfer
Data Transfer
Computer 1
Loading Data Parallel
Computation Providing Data
Communication for Computation
Network 2
Task of the Grid Resource Management!
Computer 2
Parallel Computation
Software License
Software Usage
Storage
Data Storage
Network 3
Communication for Visualization
VR-Cave
Visualization
9
Local Scheduling Systems

• Observation
• Local resource management (LRM) systems exist
• require extension for Grids by additional
  software or
• will directly support Grids in the future
• DRMAA is available today for some LRM systems
• Different LRM systems will be part of the Grid
  and will perform a lower-level scheduling
• In addition the Grid will require some
  higher-level scheduling for coordinating the
  users jobs.
• Multi-level scheduling model

10
Multi-level Grid Scheduling Architecture
Higher-levelGrid Scheduling
Grid- Scheduler
Grid-User
Grid SchedulingArchitecture
Scheduler
Scheduler
Scheduler
Lower-levelScheduling
time
time
time
Schedule
Schedule
Schedule
localJob-Queues
localJob-Queues
localJob-Queues
Resource 1
Resource 2
Resource n
Courtesy of Ramin Yahyapour
11
User Objective

• Local computing typically has
• A given scheduling objective as minimization of
  response time
• Use of batch queuing strategies
• Simple scheduling algorithms FCFS, Backfilling
• Grid Computing requires
• Individual scheduling objective
• better resources
• faster execution
• cheaper execution
• More complex objective functions apply for
  individual Grid jobs!

12
Provider/Owner Objective

• Local computing typically has
• Single scheduling objective for the whole system
  
• e.g. minimization of average weighted response
  time or high utilization/job throughput
• In Grid Computing
• Individual policies must be considered
• access policy,
• priority policy,
• accounting policy, and other
• More complex objective functions apply for
  individual resource allocations!
• User and owner policies/objectives may be subject
  to privacy considerations!

13
Grid Economics Different Business Models

• Cost model
• Use of a resource
• Reservation of a resource
• Individual scheduling objective functions
• User and owner objective functions
• Formulation of an objective function
• Integration of the function in a scheduling
  algorithm
• Market-economic approaches
• Application of computational intelligence
• Resource selection
• The scheduling instances act as broker
• Collection and evaluation of resource offers

14
Scheduling Model

• Using a Brokerage/Trading strategy

Consider individual userpolicies
Higher-levelscheduling
Coordinate Allocations
Submit Grid Job Description
Select Offers
Discover Resources
Collect Offers
Query for Allocation Offers
Consider communitypolicies
Generate Allocation Offer
Lower-levelscheduling
Consider individual owner policies
Analyze Query
15
Properties of Multi-Level Scheduling Model

• Multi-level scheduling must support different RM
  systems and strategies.
• Provider can enforce individual policies in
  generating resource offers
• User receives resource allocation optimized to
  the individual objective
• Different higher-level scheduling strategies can
  be applied
• Multiple levels of scheduling instances are
  possible
• Support for fault-tolerant and load-balanced
  services

16
Negotiation in Grids

• Multilevel Grid scheduling architecture
• Lower level local scheduling instance
• Implementation of owner policies
• Higher level Grid scheduling instance
• Resource selection and coordination
• (Static) Interface definition between both
  instances
• Different types of resources
• Different local scheduling systems with different
  properties
• Different owner policies
• (Dynamic) Communication between both instances
• Resource discovery
• Job monitoring

17
GGF WG Scheduling Attributes

• Define the attributes of a lower-level scheduling
  instance that can be exploited by a higher-level
  scheduling instance.
• Attributes of allocation properties
• Guaranteed completion time of allocation,
• Allocations run-to-completion,
• Attributes of available information
• Access to tentative schedule,
• Exclusive control,
• Attributes for manipulating allocation execution
• Preemption,
• Migration,
• Attributes for requesting resources
• Allocation Offers,
• Advance Reservation,

18
Towards Grid Scheduling

• Grid Scheduling Methods
• Support for individual scheduling objectives and
  policies
• Economic scheduling methods to Grids
• Multi-criteria scheduling models (most general)
• Architectural requirements
• Generic job description
• Negotiation interface between higher- and
  lower-level scheduler
• Economic management services
• Workflow management
• Integration of data and network management
• Interoperability is a key!

19
Data and Network Scheduling

• Most new resource types can be included via
  individual lower-level resource management
  systems.
• Additional considerations for
• Data management
• Select resources according to data availability
• But data can be moved if necessary!
• Network management
• Consider advance reservation of bandwidth or SLA
• Network resources usually depend on the selection
  of other resources!
• Coordinate data transfers and storage allocation
• User empowered/owned lambdas!

20
Example of a Scheduling Process
Example 40 resources of requested type are
found. 12 resources are selected. 8 resources
are available. Network and data dependencies are
detected. Utility function is evaluated. 6th
tried allocation is confirmed. Data/network
provided and job is started

• Scheduling Service
• receives job description
• queries Information Service for static resource
  information
• prioritizes and pre-selects resources
• queries for dynamic information about resource
  availability
• queries Data and Network Management Services
• generates schedule for job
• reserves allocation if possibleotherwise selects
  another allocation
• delegates job monitoring to Job Manager
• Job Manager/Network and Data Management service,
  
• monitor and initiate allocation

21
Conclusions for Grid Scheduling

• Grids ultimately require coordinated scheduling
  services.
• Support for different scheduling instances
• different local management systems
• different scheduling algorithms/strategies
• For arbitrary resources
• not only computing resources, also
• data, storage, network, software etc.
• Support for co-allocation and reservation
• necessary for coordinated grid usage (see data,
  network, software, storage)
• Different scheduling objectives
• cost, quality, other
• Grid resource management services are a key to
  success of the Grid vision!
• so are the applications that could show the Grid
  benefit!

22
Integration of a Grid Scheduling System

• Globus as de-facto standard
• but no higher-level scheduling services available
• Many projects include scheduling requirements
• Focus on a running implementation for a specific
  problem
• No attempt to generate a general solution
• Grid scheduling cannot be developed by single
  groups
• Requirements for several other services
• Community effort is key!
• Requires open Grid standards that enables Grid
  scheduling
• Support for different implementations while being
  interoperable

23
Activities

• Core service infrastructure
• OGSA/OGSI
• GGF hosts several groups in the area of Grid
  scheduling and resource management.
• Examples
• WG Scheduling Attributes (finished)
• WG Grid Resource Allocation Agreement Protocol
  (active)
• WG Grid Economic Services Architecture (active)
• WG Scheduling Ontology (proposed)
• RG Grid Scheduling Architecture (proposed)
• Network of Excellence CoreGRID (proposed)
• define the software infrastructure for Next
  Generation Grids

24
What are Basic Blocks for a Grid Scheduling
Architecture?
Basic Blocks and Requirements are still to be
defined!
Courtesy of Ramin Yahyapour
25
Conclusion

• Resource management and scheduling is a key
  service in an Next Generation Grid
• In a large Grid the user cannot handle this task
• Nor is the orchestration of resources a provider
  task
• System integration is complex but vital
• Individual results may be of limited benefit
  without being embedded in a larger project
• Basic research is required in this area.
• No ready-to-implement solution is available
  (although EDG, CrossGrid, GridLab etc. work on
  it)
• New concepts are necessary

A significant joint effort is needed to support
Grid Scheduling! Also research is still necessary!
26
RM in GridLab What our users want...

• Two primary applications Cactus (simulations)
  and Triana (data mining/analysis)
• other application communities are also being
  engaged,
• Application oriented environment
• Resources (grid) on demand
• Adaptive applications/adaptive scenarios
  adaptive grid environment
• job checkpoint, migration, spawn off a new job
  when needed,
• Open, pervasive, not even restricted to a single
  Virtual Organization
• The ability to work in a disconnected environment
• start my job on a disconnected laptop migrate it
  to grid when it becomes available
• from laptops to fully deployed Virtual
  Organizations
• Mobile working
• Security on all levels

27
What our users want... (cont.)

• The infrastructure must provide capabilities to
  customise choice of service implementation (e.g.
  using efficiency, reliability, first succeeding,
  all)
• Advance reservation of resources,
• To be able to express their preferences regarding
  their jobs on one hand and to understand the
  resource and VO policies on the other hand,
• Policy information and negotiation mechanisms
• what is a policy of usage of the remote
  resources?
• Prediction-based information
• How long will my job run on a particular
  resource?
• What resources do I need to complete the job
  before deadline?

28
Coalescing Binary Scenario
Controller
Email, SMS notification
Logical File Name
GW Data Distributed Storage
GAT (GRMS, Adaptive)
GW Data

• Submit Job
• Optimised Mapping

GAT (Data Management)
CB Search
GridLab Test-bed
29
GridLab RMS approach

• Grid resources are not only the machines, but
  also databases, files, users, administrators,
  instruments, mobile devices, jobs/applications
  ...
• Many metrics for scheduling throughput, cost,
  latency, deadline, other time and cost metrics...
• Grid resource management consists of job/resource
  scheduling, security (authorization
  services,...), local policies, negotiations,
  accounting, ...
• GRM is both, user and resource owner driven
  negotiation process and thus, multicriteria
  decision making process
• WS-Agreement is badly needed
• Two ongoing implementations production keep-up
  and future full-feature

30
GRMS - the VO RMS

• GRMS is a VO (community) resource management
  system
• Component and pluggable architecture allows to
  use it as a framework for many different VOs
• Components include
• Resource discovery (now MDS-based, other
  solutions easy to be added now adding the
  GridLab testbed information system see Ludeks
  talk tomorrow)
• Scheduling (Multicriteria, economy, workflow,
  co-scheduling, SLA-based scheduling) - work in
  progress
• Job Management
• Workflow Management
• Resource Reservation

31
GRMS - the plan
Information Services
Data Management
Authorization System
Adaptive
Resource Discovery
File Transfer Unit
Jobs Queue
BROKER
Job Receiver
Execution Unit
Monitoring
SLA Negotiation
Scheduler
Workflow Manager
Resource Reservation
Prediction Unit
GRMS
GLOBUS, other
Local Resources (Managers)
32
Current implementation

• submitJob - submits new job,
• migrateJob - migrates existing job,
• getMyJobsList - returns list of jobs belonging to
  the user,
• registerApplicationAccess - registers application
  access,
• getJobStatus - returns GRMS status of the job,
• getHostName - returns host name, on which the job
  is/was running
• getJobInfo - returns a structure describing the
  job,
• findResources - returns resources matching user's
  requirements,
• cancelJob - cancels the job,
• getServiceDescription - returns description of a
  service.

33
GRMS - overview
Resource Management System
Grid Environment
User Access Layer
Resource Discovery
GridLab Services

• Data Management
• Adaptive Components
• GridLab Authorization Service

(GAT) Application
Broker
GridLab Portal
Globus Infrastructure

• MDS
• GRAM
• GridFTP
• GASS

Job Manager
34
GRMS detailed view
DB
Resource Discovery
GridLab Services
Web Service Interface
Job Queue
Workflow Mgmt.
Broker
User AccessLayer
System Layer Services
Task Registry
Job Manager
35
GRMS detailed view
DB
Resource Discovery
Web Service Interface
GridLab Services
Web Service Interface
Job Queue
Workflow Mgmt.
Web Service Interface
Web Service Interface
Web Service Interface
Broker
Web Service Interface
User AccessLayer
System Layer Services
Task Registry
Web Service Interface
Job Manager
Web Service Interface
36
GRMS functionality

• Ability to choose the best resources for the job
  execution, according to Job Description and
  chosen mapping algorithm
• Ability to submit the GRMS Task according to
  provided Job Description
• Ability to migrate the GRMS Task to better
  resource, according to provided Job Description
• Ability to cancel the Task
• Provides information about the Task status
• Provides other information about Tasks (name of
  host where the Task is/was running, start time,
  finish time)

37
GRMS functionality (cont.)

• Provides list of candidate resources for the Task
  execution (according to provided Job
  Description)
• Provides a list of Tasks submitted by given user
• Ability to transfer input and output files
  (GridFTP, GASS, WP8 Data Management System)
• Ability to contact Adaptive Components Services
  to get additional information about resources
• Ability to register a GAT Application callback
  information
• Ability to submit a set of tasks with precedence
  constraints (work-flow of tasks and input/output
  files)

38
GRMS modules

• Broker Module
• Steers process of job submittion
• Chooses the best resources for job execution
  (scheduling algorithm)
• Transfers input and output files for job's
  executable
• Resource Discovery Module
• Finds resources that fullfills requirements
  described in Job Description
• Provides information about resources, required
  for job scheduling

39
GRMS modules (cont.)

• Job Manager Module
• Ability to check current status of job
• Ability to cancel running job
• Monitors for status changes of runing job
• Workflow Management Module
• Creates workflow graph of tasks from Job
  Description
• Put tasks to Job Queue
• Controls the tasks execution according to
  precedence constraints

40
GRMS modules (cont.)

• Web Service Interface
• Provides GSI enabled web service interface for
  Clients (GAT Application, GridLab Portal)
• Job Queue
• Allows to put the tasks into the queue
• Provides way for getting tasks from queue
  accorging to configured algorithm (FIFO)
• Task Registry
• Stores information about the task execution
  (start time, finish time, machine where executed,
  current status, Job Description)

41
Job Description

• Task executable
• file location
• arguments
• file argument (files which have to be present in
  working directory of running executable)
• environment variables
• standard input
• standard output
• standard error
• checkpoint files

42
Job Description (cont.)

• Resource requirements of executable
• name of host for job execution (if provided no
  scheduling algorithm is used)
• operating system
• required local resource management system
• minimum memory required
• minimum number of cpus required
• minimum speed of cpu
• other parameter passed directly to Globus GRAM

43
Job Description new elements

• Job Description consists of one or more Task
  descriptions
• Each Task can have a section which denotes parent
  tasks

44
Job Description - example

• lt grmsjob appid MyApplicationgt
• lttask id1gt
• ltresourcegt
• ltosnamegt Linux lt/osnamegt
• ltmemorygt 128 lt/memorygt
• ltcpucountgt 2 lt/cpucountgt
• lt/resourcegt  
• ltexecutable type"single" count"1"gt
• ltfile name"String" type"in"gt
• lturlgt gsiftp//rage.man.poznan.pl//Apps/MyApp
  lt/urlgt
• lt/filegt
• ltargumentsgt
• ltvaluegt 12 lt/valuegt
• ltvaluegt abc lt/valuegt
• lt/argumentsgt
• ltstdingt
• lturlgt gsiftp//rage.man.poznan.pl//Apps/appstd
  in.txt lt/urlgt
• lt/stdingt
• ltstdoutgt

45
Job Description example 2
lt grmsjob appid MyApplicationgt lttask
idtask1gt ltresourcegt ... lt/resourcegt
  ltexecutable type"single" count"1"gt ...
lt/ executable gt lt/taskgt lttask
idtask2gt ltresourcegt ... lt/resourcegt
  ltexecutable type"single" count"1"gt ...
lt/ executable gt ltworkflowgt ltparentgttask1lt/par
entgt lt/workflowgt lt/taskgt lt/grmsjob gt
46
Research focus of GRMS

• Focus on the infrastructure is not enough for the
  efficient GRM
• Focus on policies
• Focus on multicriteria aspects of the GRM
• users, their preferences and applications
• resource owners preferences
• preference models, multicriteria decision making,
  knowledge will be crucial for efficient resource
  management
• Focus on AI techniques for GRM
• Focus on business models, economy grids
• Cost negotiation mechanisms could be part of the
  SLA negotiation process (WS-Agreement)

contradictory in nature
47
GRMS and SLA
48
GRMS and SLA (cont.)
49
STAKEHOLDERS OF THE GRID RESOURCE MANAGEMENT
PROCESS

• End-users (consumers)
• having requirements concerning their applications
  (e.g. expect a good performance of their
  applications, expect a good response time)
• have requirements concerning resources (e.g.
  prefer machines with a big storage, machines with
  certain configurations)
• Resource Administrators and Owners (providers)
• share resources to achieve some benefits
• VO Administrator (criteria and preferences must
  be secure)
• requires robust and reliable provisioning of
  resources
• manages and controls VO by making global policies
  (community policies)

50
Multicriteria RM in GridLab

• Gathering of information
• apps requirements (resource requirements,
  environment, etc.)
• user preferences (which criteria and how
  important)
• user support, preference modeling tools,
• Selection phase
• choose the best resources (schedule) based on the
  information provided and on the resource
  availability (estimates, predictions)
• from simple matchmaking to multiple optimisation
  techniques
• Execution phase
• file staging, execution control, job monitoring,
  migration, usually re-selection of resources,
  application adaptation (application managers,
  adaptive services from GridLab)

51
Multicriteria approach in GRMS

• Many different research fields, e.g.
  Multicriteria Optimization, Project Scheduling,
  Artificial Intelligence, Decision Support Systems
• We consider a resource management problem as a
  multicriteria decision making process with
  various stakeholders
• Various stakeholders have different point of
  views and criteria (along with preferences)
• We need to aggregate somehow (negotiation and
  agreement processes are required) various
  criteria and stakeholders preferences
• We focus on a compromise solution rather then the
  optimal one (does it exist?)
• We want to satisfy many stakeholders rather than
  the particular one

52
MULTICRITERIA (1)
End user 1
End user 2
Application 1 (e.g. Data analysis )
Application 2 (e.g. Data mining)
Hard constraints (e.g. RSL) ltMem 100MBgt,
ltStorage 1Ggt
Memory
Memory
???
???
R1
R1
R2
R2
R3
R3
R4
R4
Storage
Storage
53
MULTICRITERIA (2)
End user 1
End user 2
Application 1 (e.g. Data analysis )
Application 2 (e.g. Data mining)
MAX Z 1Mem 2Storage (where z is the
objective function)
MAX Z 2Mem 1Storage (where z is the
objective function)
Memory
Memory
R1
R1
R2
R2
R3
R3
R4
R4
Storage
Storage
54
PROPOSALS

• We have added only two parameters to hard
  constraints priority and min/max (optimization
  direction)
• NEW ltMem 100MBgtltPriority 2gtltOpt Maxgt
  ltStorage 1GgtltPriority 1gt ltOpt Maxgt
• End users are able to express their preferences
  in a very simple way
• End users preferences are taken into account
  (compromise solutions are chosen)

55
MULTIPLE CRITERIA

• Concerning particular resources (e.g. memory,
  flops) or schedules (e.g. estimated processing
  time, maximum lateness)
• Specific for end-users (e.g. mean response time,
  mean tardiness, cost of computations), resource
  owners (e.g. machine idleness) and administrators
  (e.g. throughput, makespan)
• Time criteria (e.g. mean response time, makespan,
  mean tardiness), cost criteria (e.g. weighted
  resource consumption, cost of computations) and
  resource utilization criteria (e.g. load
  balancing, machine idleness)
• But... (in practice -(,
• Lack or limited set of low level mechanisms which
  support e.g. advanced reservation,
• Negotiation protocols and agreements
• Prediction tools which provide advanced analysis
  and estimations (e.g. execution time, queue wait
  time),
• Reliable information sources which describe
  behaviors of applications and resources, etc.

56
PREFERENCE GATHERING

• As a function
• by means of parameters used in an utility
  function in order to model the importance of
  criteria
• expressed using the resource specification
  language (e.g. JSDL) or gathered from
  stakeholders during an interactive process (e.g.
  WS-Agreement)
• As a relation
• input parameters such as weights and thresholds
  are provided by stakeholders
• As logic statements (decision rules)
• - if conditions on criteria then decision,
• - future directions we need to follow (machine
  learning methods, learning from examples and
  previous actions)...
• More details in the Kluwers book...

57
STEPS OF MULTICRITERIA RESOURCE MANAGEMENT

• Our approach extends and is based on many
  scheduling strategies...
• Gathering of information
• applications requirements (resource requirements,
  environment, etc.)
• users preferences (which criteria and how
  important)
• user support, preference modeling tools
• Selection
• choice of the best resources or schedules based
  on the information provided and on the resource
  availability (estimates, predictions)
• from simple matchmaking to multiple optimization
  techniques
• Execution
• file staging, execution control, job monitoring,
  migration, usually re-selection of resources,
  application adaptation

58
MC in JOB SCHEDULING

• One central scheduler exists for multiple
  applications (e.g. LSF, PBS, Sun Grid Engine)
• The goal is to match a set of application
  requirements to all available resources on the
  basis of various criteria
• Usually multicriteria techniques are used for the
  evaluation of resource co-allocations (external
  mechanisms )

59
MC in APPLICATION LEVEL SCHEDULING

• Each application is scheduled by an internal
  scheduler and forms a self-contained system
• The goal is to match particular application
  requirements with one (or some) good resource(s)
  based on various criteria
• Multicriteria techniques are used for the
  evaluation resources as well as resource
  co-allocations (internal mechanisms)

60
SIMPLE EXAMPLE (THEORY)

• Mathematical models
• Assumptions
• R 7 resources
• NEU 3 end-users (eu1, eu2,eu3)
• tasks t11, t21, t22, t31
• resources are maintained by NRO 3 resource
  owners (ro1, ro2, ro3),
• each task has to be assigned to one resource
• stakeholders preferences are expressed in the
  form of utility functions

More details in the Kluwers book...
61
SELECTED CRITERIA

• The average execution time (T) - an average
  execution time of all tasks submitted by a given
  end-user
• The cost of resource usage (C) - the sum of
  costs of the resources usage by end-user's tasks
• The income (I) - the total income for a given
  resource owner

More details in the Kluwers book...
62
HARD CONSTRAINTS

• Due to many hard constraints appearing in
  practice (specified by means of resource
  description language, policy rules in a
  scheduler, etc.), the number of feasible
  solutions can be decreased significantly.
• In our case, due to specific requirements of
  tasks (t11, t21, t22, and t31) the set of
  available resources is limited to r1, r2, r3 and
  r4. In addition, the manager's task t11 is
  assigned automatically to the dedicated resource
  r3 (because invoking appropriate administration
  rules) so only r1, r2, and r4 are considered.
• Once all hard constraints are met, we can examine
  the soft constraints and multicriteria analysis.

More details in the Kluwers book...
63
SOFT CONSTRAINTS AND MULTICRITERIA ANALYSIS

• Gathering preferences from all stakeholders

• Evaluation of solutions (schedules) for each
  stakeholder using local utility functions

• Aggregation of local evaluations (in the example
  using global utility function)

• The best compromise solution is chosen (in the
  example with the best value of utility function)

More details in the Kluwers book...
64
GRID SERVICES AND OGSI-AGREEMENT

• OGSA defines Grid as a service-oriented
  environment
• OGSA model represents all entities of Grid as a
  Grid service (resources, brokers, applications)
• Interoperability between high-level community
  schedulers and local resource managers is
  supported,
• There must be agreement on how these entities
  will interact with each other (e.g. many
  different resource managers could provide
  different functionalities)

65
MULTICRITERIA AND WS-AGREEMENT
End Users
VO Administrator
Resource Owners

• The goal is to negotiate the best contracts for a
  set of applications based on various additional
  criteria e.g.
• level of service capability, quality, deployment
  and dynamic reconfiguration
• level of commitment, service agreement and
  satisfaction
• Multicriteria techniques are used for the
  evaluation of resources/services and brokering
  negotiations

66
SIMPLE EXAMPLE (MC in JSDL)

• In order to implement multicriteria scenarios
  (simple) we have to add some new tags (describing
  various criteria and preferences) to the existing
  job specifications JSDL example
•  
• ltjsdlnameOfTerm wspUsage"wspRequired"gt
•   ltjsdlcriterion jsdltypetypegt
• ltjsdlpreferencevalue/gt or
  jsdlpreferences agtb, bltd, cgtd, ac
• lt/jsdlcriteriongt
• lt/jsdlnameOfTermgt
• where
• jsdltype is one of two types jsdlminimization
  or jsdlmaximization
• jsdlpriority expresses an end users preference
• More complicated and advanced MC scenario could
  be implemented, new tags and mechanisms are
  required)

67
FUTURE RESEARCH WORK

• Use of prediction techniques to support high
  level criteria
• Prediction of
• a state of resources
• application requirements (e.g. memory, storage
  space)
• application performance (e.g. execution time)
• Dealing with prediction errors (e.g. AI methods)
• Active participation in GRAAP and JSDL Working
  Groups under GGF
• Implement our new ideas in existing resource
  management systems (e.g. GridLab GRMS,
  Progress, SGI Grid, Clusterix)

68
Summary

• GridLabs RM is focused on the multicriteria
  aspects of resource management (local vs. global
  policies, user vs. resource owner etc.)
• GRMS is currently deployed on the GridLab testbed
• It supports the migration because of the bad
  performance scenario
• new emerging scenarios we have in mind
• Other deployments include SGI Grid, Progress,
  HPC Europa, Clusterix (future), GriPhyN (workflow
  execution on Globus)
• more info www.gridlab.org -gt WP9

69
The GRM book

• Published in October 2003 by Kluwer
• www.kap.nl

70
Thank you!