some lessons scheduling

1
some lessonsscheduling queueing can teach us
about system design

• Adam Wierman
• (with some lessons stolen from Mor
  Harchol-Balter)

2
what I do
scheduling theory queueing theory stochastic
models (and now game theory)
aid system design by improving the tools
web servers routers databases
manufacturing call centers
3
the basic model

• Routers
• Supercomputing centers
• Database lock queues
• Web servers
• wireless access points

4
a warmup
QUESTION Suppose l ? 2l. We want to keep
ET unchanged. Should we
(a) Double service rate (b)
More than double service rate
(c) Less than double service rate
5
workload overview
6
impact of the workload
Response Time ET
Incoming jobs
m Avg. rate jobs served (server speed)
l Avg. rate jobs arrive (jobs/sec)
QUESTION Under exponential job sizes, which
policy has smaller ET?
or
PS
FCFS
7
impact of the workload
Response Time ET
Incoming jobs
m Avg. rate jobs served (server speed)
l Avg. rate jobs arrive (jobs/sec)
8
minimizing response time
QUESTION Under heavy-tailed job sizes, order
the following policies w.r.t. ET.
FCFS
PS
small ET
large ET
SRPT
SJF
RANDOM
9
minimizing response time
QUESTION Under heavy-tailed job sizes, order
the following policies w.r.t. ET.
greedy is optimal
SRPT lt PS lt SJF lt FCFS RANDOM
small ET
large ET
10
scheduling fights variability
deterministic
heavy tailed
exponential
Wierman, Harchol-Balter 05
11
real systems are very different
mean response time
12
3 research themes
13
you cant usually do SRPT
OPEN QUESTION How should small and large be
defined?
Wierman, Nuyens 07
14
you cant usually do SRPT
OPEN QUESTION When is it possible to be near
optimal without size information?
Wierman, Bansal 03
15
3 research themes
16
QoS
systems need to provide guarantees on Pr(Tgty)
study Pr(Tgty) for large y i.e. likelihood of a
large response time
17
tales of tails
QUESTION Under which policy are large response
times more likely?
or
FCFS
SRPT
OPEN QUESTION Can a policy be optimal in both
settings?
1) when job sizes are exponential
2) when job sizes are heavy-tailed
Wierman, Nuyens, Zwart 07
18
3 research themes
19
a growing trend towards multiserver designs
20
1 fast vs. many slow
cheap
FCFS
expensive
smart dispatcher
vs.
FCFS
QUESTION Which is better for ET under
?
random
21
1 fast vs. many slow
cheap
FCFS
expensive
smart dispatcher
vs.
FCFS
QUESTION Which is better for ET under
?
least-work-left
Wierman, Osogami, Harchol-Balter, Scheller-Wolf
2004
22
what is a better dispatcher?
FCFS
smart dispatcher
small jobs here
med jobs here
large jobs here
OPEN QUESTION How should the thresholds be
chosen?
size based dispatching
Harchol-Balter, Crovella, Murta 1999
23
a growing trend towards multiserver designs
24
now which is best?
QUESTION Order the following dispatchers
w.r.t. ET for each system?
random
size-based threshold
least-work-left
join the shortest queue
25
now which is best?
greedy
join the shortest queue
least-work-left
size-based threshold
random

OPEN QUESTION An analysis of JSQ in PS queues
26
a growing trend towards multiserver designs
27
multiserver SRPT
SRPT

QUESTION Why isnt SRPT optimal for ET in
multiserver systems?
long jobs are stuck at the end of the busy period
28
multiserver SRPT is still pretty good
SRPT

OPEN QUESTION How does multiserver SRPT perform
in a stochastic setting?
Leonardi, Raz 1997
29
cant use a central queue in practice
P
dispatcher
QUESTION Will the optimal pair do SRPT at the
hosts? Why or why not?
YES
30
optimality under immediate dispatch
SRPT
dispatcher
QUESTION What is a good dispatcher given SRPT
at the hosts?
maximize the variability of jobs sent to each host
OPEN QUESTION How does this policy perform in a
stochastic setting?
Avrahami, Azar 03
31
wrapup
32
some issues weve covered

• capacity planning
• heavy-tails scheduling
• QoS scheduling for the tail
• load unbalancing
• architectures 1 fast vs. many slow
• server farm routing
• optimal multiserver scheduling

33
fairness
QUESTION ABOUT DEFININING IT
OPEN QUESTION Can a policy be fair and efficient
under all workloads?
34
worrying about large jobs
QUESTION 10 Order these policies w.r.t. ET
for the largest jobs under exponential and
heavy-tailed job sizes.
FCFS
PS
small ET
large ET
SRPT
35
worrying about large jobs
QUESTION 11 Order these policies w.r.t. ET
for the largest jobs under exponential and
heavy-tailed job sizes.
exponential FCFS lt SRPT PS
heavy-tailed SRPT PS ltlt FCFS
small ET
large ET