Scheduling Theory

1
Scheduling Theory

• J. M. Akinpelu

2
Discussion

• For the M/G/1 queue that we have considered, what
  things do we know about the system? What dont we
  know?
• What are some service policies that are
  independent of service times?
• FCFS
• LCFS (Last Come First Served)
• Random

3
Service-Time Independent Policy Results

• All service policies that are independent of
  service times have the same limiting distribution
  for the number of customers in the system, and
  hence the same steady-state expected delay.
• If W is the queue delay and ? is the server
  utilization, then

4
Priority Queue Without Preemption

• Consider an M/G/1 queueing system with two types
  of customers, class 1 and class 2.
• The interarrival times for class i customers are
  exponentially distributed with rate ?i.
• The service time Si for class i customers has a
  general distribution.
• Class 1 customers have higher priority than class
  2 customers and are served first.
• If a class 2 customer is in service when a class
  1 customer arrives, the class 2 customer
  completes service before the class 1 customer is
  served. Preemption is not allowed.
• Customers within a class are served on a FCFS
  basis.
• What is the expected delay experienced by each
  class of customers?

5
Priority Queue Without Preemption

• Let WQ be the expected delay in the queue. By the
  Pollaczek-Khintchine formula, for any M/G/1 queue
  with arrival rate ? and service time S,
• Another expression for WQ is given by

6
Priority Queue Without Preemption

• Now consider an arbitrary class 1 customer.
• from which it follows that

7
Priority Queue Without Preemption

• For an arbitrary class 2 customer,
• so that

8
Priority Queue Without Preemption

• and finally we have

9
Priority Queue Without Preemption

• If we let ? ?1 ?2, then the expected queueing
  delay is
• One can show (with a little algebra) that to
  minimize expected queueing delay, we should
  assign priorities so that ES1 ES2 in
  other words, the higher priority class should be
  the one with the smaller expected service time.
  This expected delay is smaller than the expected
  delay in the corresponding M/G/1 system with no
  priorities.

10
Priority Queue Without Preemption

• Now consider an M/G/1 queue with n classes, where
  class j has higher priority than classes j1, ,
  n for j 1, , n?1. The expected delay in the
  queue experienced by class j customers is given
  by

11
Priority Queue Without Preemption

• The expected queueing delay
• is minimized by assigning priorities so that
• ES1 ES2 ESn .

12
Priority Queue Without Preemption

• Now assume that class j has a linear cost cj
  associated with the delay for customers in that
  class. To minimize the average delay cost per
  customer, order the priorities so that

13
Priority Queue Without Preemption

• Now consider an M/G/1 queue, and assume that
  there is some maximum service time Smax. Choose
  an integer n gt 1 and values t0, , tn such that
• 0 t0 lt t1 lt t2 lt lt tn Smax.
• Define class j to be all customers satisfying
• tj-1 lt S tj.

14
Priority Queue Without Preemption

• How should the customers be prioritized?
• What happens as n ? ??
• This leads to the Shortest Job First (SJF)
  scheduling policy. When preemption is not
  allowed, the SJF minimizes expected queueing
  delay.

15
M/G/1 Queue With Server Breakdown

• Consider an M/G/1 queueing system as follows
• When working, the server breaks down at an
  exponential rate ? repair begins immediately.
• The repair time R is a random variable that is
  independent of the number of times the server is
  repaired.
• When the server is repaired, the interrupted
  customer goes back into service, and service
  resumes where it left off.

16
M/G/1 Queue With Server Breakdown

• We want to determine
• the expected amount of time T from when a
  customer first enters service until it departs
  the system.
• the expected amount of time WQ that a customer
  spends waiting in queue before its service first
  commences.

17
M/G/1 Queue With Server Breakdown

• Let
• N be the number of times that the server breaks
  down while the customer is in service
• R1, R2, , RN be the amounts of time that the
  server spends in the repair facility on its
  successive visits repair times are independent
  of the customer service time.
• Then

18
M/G/1 Queue With Server Breakdown

• Conditioning on S, we get
• and, using iterated expectations,

19
M/G/1 Queue With Server Breakdown

• Similarly,

20
M/G/1 Queue With Server Breakdown

• Using the law of total variance,

21
M/G/1 Queue With Server Breakdown

• In summary,
• It follows that

22
M/G/1 Queue With Server Breakdown

• By the Pollaczek-Khintchine formula, for any
  M/G/1 queue with arrival rate ? and service
  time T,

23
Priority Queue With Preemption

• Again, consider an M/G/1 queue with n classes,
  where class j has higher priority than classes
  j1, , n for j 1, , n?1.
• A class j customer in service is interrupted if a
  higher priority customer arrives, i.e., the
  customer in service is preempted.
• A preempted customer resumes service when there
  are no longer any higher priority customers in
  the system.

24
Priority Queue With Preemption

• The rate at which a class j customer is
  interrupted is
• The preemption time Rj (aka repair time) for a
  class j customer is the time needed to serve all
  the higher priority customers.
• It can be shown that

25
Priority Queue With Preemption

• the mean amount of time T from when a class j
  customer first enters service until it departs
  the system is
• and the expected delay in queue before service is

26
Comparing No Preemption with Preemption Resume
27
Priority Queue With Preemption

• Some comments
• The mean queue delay for a customer in class j in
  the preemptive resume case is the same as it
  would be for the non-preemptive priority case if
  the lowest n ? j priority classes did not exist.
• When there are classes with priority lower than
  that of class j, the mean queue delay under
  preemptive resume is lower than that for
  non-preemptive priorities. However the time in
  the system after service starts is longer.

28
Priority Queue With Preemption

• Some comments
• A variation on preemption resume is to allow a
  customer that is near completion to finish
  service. The semipreemptive priority policy
  determines a customers priority based on its
  remaining service time, so that customers change
  priority as the system evolves. In the limit, as
  the class intervals are made arbitrarily small,
  this policy becomes the Shortest Remaining
  Processing Time (SRPT) policy.

29
Priority Queue With Preemption

• Some comments
• When the number of classes is small, the
  semipreemptive priority policy may not produce
  lower expected total delay compared with the
  preemptive resume policy operating with the same
  classes. However, when the number of classes
  increases, the semipreemptive rule becomes
  relatively better, and in the limit the SRPT
  policy becomes optimal with respect to minimizing
  expected total delay.

30
Class Exercise

• Give reasons why a manager of a non-preemptible
  queue may decide not to implement SJF.
• Give reasons why a manager of a preemptible queue
  may decide not to implement SRPT.
• Give an example in which the semipreemptive
  priority policy gives a worst mean delay than
  preemptive resume. (Note You may consider a
  finite set of customers.)

31
Priority Queue With Preemption

• Example in Which Semipreemptive Priority Policy
  Gives a Worst Mean Delay Than Preemptive Resume
• Consider a queueing system with two priorities
• Priority 1 0 lt S 2
• Priority 2 2 lt S lt ?
• Consider two customers
• Customer 1 with S 5 arrives at time 0
• Customer 2 with S 1 arrives at time 3

32
Priority Queue With Preemption

• Semipreemptive Priority
• Mean delay ½ (5 3) 4
• Preemptive Resume
• Mean delay ½ (6 1) 7/2

33
Other Scheduling Policies

• Processor Sharing (PS)
• the server is shared evenly among all customer
  classes with customers present (or individual
  customers) at every point in time.
• Least Attained Service (LAS)
• the customer with the least attained service
  (i.e., service already received) is served.

34
Multiserver Queues

• Assumptions
• There is a router which assigns customers to one
  of several servers.
• There is no delay at the router unless there is a
  central queue.
• Each server may have a dedicated queue in which
  customers routed to it wait for service
  alternatively, customers may wait for service in
  a central queue.

35
Multiserver Queues

• Routing Policies
• Round Robin
• each customer class with customers present (or
  each individual customer) is assigned service
  time slices in equal portions based on some
  predefined order.
• Join Shortest Queue
• Least Work Left
• Central Queue Shortest Job
• Size Interval Splitting
• Service times are divided into intervals that
  form customers classes (see chart 13) each
  server is assigned to serve one class of
  customers.

36
Squared Coefficient of Variation

• The squared coefficient of variation (SCV) for a
  distribution F is the ratio of its variance ? 2
  to the square of its mean ?
• For the exponential distribution, SCV 1.
• Distributions with SCV lt 1 are considered
  low-variance, while those with SCV gt 1 are
  considered high-variance.

37
Pareto Distribution

• A R.V. X has a Pareto distribution if
• where k gt 0 and xmin is the minimum value that X
  can take on.
• X has a bounded Pareto distribution if X also has
  some maximum value.
• Bounded Pareto distributions can have very large
  squared coefficients of variation.

38
Pareto Distribution
Pareto cumulative distribution functions for
various k  with xmin  1
39
Failure Rate

• For a continuous distribution F with density f,
  the failure (or hazard) rate function of F, ?(t),
  is given by
• In reliability theory, if the lifetime of some
  component has distribution function F, then ?(t)
  is the conditional probability that a component
  of age t will fail.
• If the service time of a customer has
  distribution function F, then ?(t) is the
  conditional probability that a service sojourn
  that has reached age t will end.

40
Failure Rate

• F is an increasing failure rate (IFR)
  distribution if ?(t) is an increasing function of
  t.
• This is analogous to wearing out.
• F is a decreasing failure rate (DFR) distribution
  if ?(t) is a decreasing function of t.
• This is analogous to burning in.
• The Pareto distribution is DFR, i.e., the longer
  a service time lasts, the more likely it is that
  it will continue.

41
Scheduling in Server Farms

• View slides at www.mascots-conference.org/Mascots-
  Keynote-Mor.ppt

42
A Word About Fairness

• So far our focus as been on expected delay, but
  decreasing overall delay may come at the expense
  of some customers.
• Harchol-Balter and Wierman define fairness for
  scheduling policies and classify scheduling
  policies as
• Always Fair (under all loads and service
  distributions)
• Sometimes Unfair (under some loads and service
  distributions)
• Always Unfair (under all loads and service
  distributions)

43
A Word About Fairness

• Always Fair
• PS
• Sometimes Unfair
• All service-based, non-preemptive policies (e.g.,
  SJF)
• SRPT
• Always Unfair
• All non-service based, non-preemptive policies
  (e.g., FCFS, LCFS, random)
• All service based, preemptive policies
• All age-based policies (e.g., LAS)

44
A Word About Fairness

• The above findings show that we are just
  beginning to understand unfairness in scheduling
  policies. This is a fertile area with many more
  properties yet to be uncovered.
• Quote from Adam Wierman, Mor Harchol-Balter,
  Classifying Scheduling Policies with Respect to
  Unfairness in an M/GI/1, SIGMETRICS 03, June
  10-12, 2003.

45
Homework

• Ross 8.41. 8.42, 8.44
• Consider two classes of customers with respective
  discrete service time distributions (in minutes)
• Class 1 arrivals and class 2 arrivals each have
  exponential interarrival times with rate 1/10
  customer/minute. Compute EWQ for both FCFS and
  SJF. (Assume that the jobs are not preemptible.)

46
References

• Richard W. Conway, William L. Maxwell, Louis W.
  Miller, Theory of Scheduling, Dover Publications,
  Inc., 1967.
• Michael L. Pinedo, Scheduling Theory, Algorithms,
  and Systems, Springer, 2008.
• Sheldon M. Ross, Introduction to Probability
  Models, Ninth Edition, Elsevier Inc., 2007.
• www.mascots-conference.org/Mascots-Keynote-Mor.ppt