Scheduling Architecture and Algorithms within ICENI

1
Scheduling Architecture and Algorithms within
ICENI

• Laurie Young, Stephen McGough, Steven Newhouse,
  John Darlington
• London e-Science Centre
• Department of Computing, Imperial College London

2
Contents

• ICENI
• Scheduling Architecture
• Scheduling Algorithms
• Variety of different algorithms
• Experimental Results
• Different policies
• Different Grid sizes
• Different Application Profiles

3
ICENI
The Iceni, under Queen Boudicca, united the
tribes of South-East England in a revolt against
the occupying Roman forces in AD60.

• IC e-Science Networked Infrastructure
• Developed by LeSC Grid Middleware Group
• Collect and provide relevant Grid meta-data
• Use to define and develop higher-level services
• Interaction with other frameworks OGSA, Jxta
  etc.

4
Component Applications

• Each job is composed of multiple components.
• Each runs on a different resource
• Each component is connected to at least one other
  component.
• Data is passed along these connections

5
ICENI Scheduling Architecture
ICENI Scheduling Services
Launching Framework Pluggable Launchers (SGE,
Globus, Condor, ICENI)
Scheduling Framework Pluggable Schedulers
(Simulated Annealing, Game Theory Random, Best
of n Random)
Performance Framework Pluggable Performance
Repositories (Perf. Models, Statistical
Analysis)
6
Schedule Evaluation

• Use a Benefit Function.
• Also called a Utility Function or Evaluation
  Function.
• A Benefit Function maps the metrics we are
  interested in to a single Benefit Value.
• Different benefit functions represent different
  optimisation preferences.
• Can set benefit to 0 if constraints (e.g. Budget)
  exceeded.

7
Random / Best of n Random

• Random Scheduler
• Randomly selects a schedule
• Checks schedule can be executed
• Produces schedules very quickly
• Best of n Random
• Produces multiple random schedules
• Returns the best one
• Still very fast
• Better results than the random schedules

8
Simulated Annealing

• Monte Carlo method
• Generate schedule at random
• Modify current schedule
• Accept new schedule if better
• If worse, accept with probability proportional to
  temperature and inversely proportional to
  benefit change
• Repeat, while reducing temperature
• Stop when no modifications to schedule accepted

9
Game Theory

• Each component is a Player
• Each player has to choose best strategy (Grid
  resource)
• Each strategy has a benefit, depending on the
  strategy chosen by all other players.
• Players identify, then remove strategies
  guaranteed to never be optimal strictly
  dominated strategies
• Produces the Nash Equilibrium

10
Experiments
11
Results (Cost Optimisation)
12
Results (Cost Optimisation)
13
Results (Time Optimisation)
14
Summary

• ICENI Scheduling Architecture
• Comprised of 3 services, using a pluggable
  architecture to allow different implementations
  to be used
• Launcher implementations allow launching to
  different underlying execution environments.
• Performance service enables execution time
  predictions
• Scheduling service operates on information
  provided by other two services

Decouples scheduler from application and
environment
15
Summary

• Scheduling Algorithms
• Four algorithms examined while varying
• Grid Sizes
• Applications
• Policies
• Simulated Annealing generally the best algorithm
  tested
• Larger applications take longer to schedule and
  return
• More choice in resources leads to
• cheaper computation for users
• Longer return times for applications

Increasing the Grid size can reduce or improve
the quality of service experienced by the user
16
Acknowledgements

• Director Professor John Darlington
• Technical Director Dr Steven Newhouse
• Research Staff
• Anthony Mayer, Nathalie Furmento
• Stephen McGough, James Stanton
• Yong Xie, William Lee
• Marko Krznaric, Murtaza Gulamali
• Asif Saleem, Laurie Young, Gary Kong
• Contact
• http//www.lesc.imperial.ac.uk/
• e-mail lesc_at_imperial.ac.uk
• Funding
• PPARC e-Science Studentship (PPA/S/E/2001/03335)