Superscheduling and Resource Brokering

1
Superscheduling and Resource Brokering

• Sven Groot (0024821)

2
Grid Information Service

• Not all information available
• Grid Information System
• Globus Monitoring and Discovery Service (MDS2)
• Grid Monitoring Architecture (GMA)
• Common features
• Organise sensors
• Static vs. Dynamic data
• Extensible
• Agreed upon schema

3
Stages of Grid Scheduling

• Phase 1 Resource Discovery
• Authorization filtering
• Application Requirement Definition
• Minimal requirement filtering

4
Stages of Grid Scheduling (2)

• Phase 2 System Selection
• Dynamic information gathering
• System Selection

5
Stages of Grid Scheduling (3)

• Phase 3 Job Execution
• Advance Reservation (optional)
• Job Submission
• Preparation Tasks
• Monitoring Progress
• Job Completion
• Cleanup Tasks

6
Application requirements
7
Application Requirements (2)

• General requirements
• Compute-related requirements
• Data-related requirements
• Network-related requirements

8
Application Requirements (3)

• Challenges
• Application deployment
• Metacomputing
• Predicting performance
• Theoretical prediction
• History based prediction
• Testcase-based prediction
• Adaptive brokering

9
Application Requirements (4)

• Related issues
• Application frameworks
• Virtual Organizations
• Security requirements
• Accounting policies
• User preferences

10
Scheduling in GrADS

• Scheduling phases
• Launch-time scheduling
• Rescheduling
• Meta-scheduling

11
GrADS
12
GrADS (2)

• Focus applications
• ScaLAPACK
• Cactus
• FASTA
• Iterative applications
• Jacobi method
• Game of Life
• Fish

13
GrADS Launch-time scheduling
14
GrADS Launch-time scheduling (2)

• Configurable Object Program
• Application requirements definition
• AART
• ClassAds
• Redline

15
ClassAds sample
16
GrADS Launch-time scheduling (3)

• Performance model
• General method
• develop an analytic model for well-understood
  aspects of applicatio or system performance
• test the analytic model against achieved
  application performance
• develop empirical models for poorly-understood
  aspects of application or system behavior
• Some application specific methods
• Implemented as shared libraries

17
GrADS Launch-time scheduling (4)

• Mapper
• Maps data and/or tasks to resources
• Different mapping methods
• Equal allocation
• Time balancing
• Data locality

18
GrADS Launch-time scheduling (5)

• Search procedure
• General steps
• identify a large number of sets of resources that
  may be good platforms for the application
• use the application-specific mapper and
  performance model to generate a data map and
  predicted execution time for those resource sets
• select the resource set that results in the
  lowest predicted execution time

19
GrADS Launch-time scheduling (6)

• Resource-aware search

20
GrADS Launch-time scheduling (6)

• Simulated Annealing

21
GrADS Rescheduling

• Additional complexities
• Lack of built-in mechanisms
• Need to distinguish processors that are
  running/not running the current process
• Overheads can be high

22
GrADS Rescheduling (2)
23
GrADS Rescheduling (2)

• Rescheduling methods
• Application migration
• Process swapping

24
GrADS Metascheduling
25
Grid Service Level Agreements

• Contract
• Provide some capability
• Perform some task
• Types of SLAs
• Resource Service Level Agreements
• Task Service Level Agreements
• Binding Service Level Agreements

26
Grid SLAs (2)
27
Grid SLAs (3)

• Motivating scenarios
• Community Scheduler Scenario

28
Grid SLAs (4)

• Motivating scenarios (contd)
• File transfer scenario

29
Grid SLAs

• Resource virtualization

30
Multicriteria

• Basic definitions
• Pareto Dominance
• Pareto Optimality
• Pareto-optimal set
• Pareto Front

31
Multicriteria (2)

• Motivations
• Various stakeholders and their preferences
• Job scheduling
• Application-Level scheduling
• Hard constraints and soft constraints

32
Multicriteria (3)

• Approach
• Criteria
• Related to stakeholders
• Related to entire system
• Time criteria
• Cost criteria
• Resource utilization criteria
• Modeling preferences

33
Multicriteria (4)

• Selection method
• Rule-based system requirements
• Expression of policies
• Execution of different scheduling procedures
• Adaptation to the environment
• Selection of the best solution
• Multicriteria optimization

34
(No Transcript)
35
Example (contd)
36
Example (contd)

• Aggregate criteria
• End user satisfaction
• Resource Owner Satisfaction
• VO overall performance

37
Example (contd)