CS352 Introduction to Queuing Theory

1
CS352 - Introduction to Queuing Theory
Rutgers University
2
Queuing theory definitions

• (Bose) the basic phenomenon of queueing arises
  whenever a shared facility needs to be accessed
  for service by a large number of jobs or
  customers.
• (Wolff) The primary tool for studying these
  problems of congestions is known as queueing
  theory.
• (Kleinrock) We study the phenomena of standing,
  waiting, and serving, and we call this study
  Queueing Theory." "Any system in which arrivals
  place demands upon a finite capacity resource may
  be termed a queueing system.
• (Mathworld) The study of the waiting times,
  lengths, and other properties of queues.

http//www2.uwindsor.ca/hlynka/queue.html
3
Applications of Queuing Theory

• Telecommunications
• Traffic control
• Determining the sequence of computer operations
• Predicting computer performance
• Health services (eg. control of hospital bed
  assignments)
• Airport traffic, airline ticket sales
• Layout of manufacturing systems.

http//www2.uwindsor.ca/hlynka/queue.html
4
Example application of queuing theory

• In many retail stores and banks
• multiple line/multiple checkout system ? a
  queuing system where customers wait for the next
  available cashier
• We can prove using queuing theory that
  throughput improves increases when queues are
  used instead of separate lines

http//www.andrews.edu/calkins/math/webtexts/prod
10.htmQT
5
Example application of queuing theory
http//www.bsbpa.umkc.edu/classes/ashley/Chaptr14/
sld006.htm
6
Queuing theory for studying networks

• View network as collections of queues
• FIFO data-structures
• Queuing theory provides probabilistic analysis of
  these queues
• Examples
• Average length
• Average waiting time
• Probability queue is at a certain length
• Probability a packet will be lost

7
Littles Law
System
Arrivals
Departures

• Littles Law Mean number tasks in system mean
  arrival rate x mean response time
• Observed before, Little was first to prove
• Applies to any system in equilibrium, as long as
  nothing in black box is creating or destroying
  tasks

8
Proving Littles Law
Arrivals
Packet
Departures
1 2 3 4 5 6 7 8
Time
J Shaded area 9 Same in all cases!
9
Definitions

• J Area from previous slide
• N Number of jobs (packets)
• T Total time
• l Average arrival rate
• N/T
• W Average time job is in the system
• J/N
• L Average number of jobs in the system
• J/T

10
Proof Method 1 Definition
in System (L)

1 2 3 4 5 6 7 8
Time (T)
11
Proof Method 2 Substitution
Tautology
12
Model Queuing System

• Use Queuing models to
• Describe the behavior of queuing systems
• Evaluate system performance

13
Characteristics of queuing systems

• Arrival Process
• The distribution that determines how the tasks
  arrives in the system.
• Service Process
• The distribution that determines the task
  processing time
• Number of Servers
• Total number of servers available to process the
  tasks

14
Kendall Notation 1/2/3(/4/5/6)

• Six parameters in shorthand
• First three typically used, unless specified
• Arrival Distribution
• Service Distribution
• Number of servers
• Total Capacity (infinite if not specified)
• Population Size (infinite)
• Service Discipline (FCFS/FIFO)

15
Distributions

• M stands for "Markovian", implying exponential
  distribution for service times or inter-arrival
  times.
• D Deterministic (e.g. fixed constant)
• Ek Erlang with parameter k
• Hk Hyperexponential with param. k
• G General (anything)

16
Kendall Notation Examples

• M/M/1
• Poisson arrivals and exponential service, 1
  server, infinite capacity and population, FCFS
  (FIFO)
• the simplest realistic queue
• M/M/m
• Same, but M servers
• G/G/3/20/1500/SPF
• General arrival and service distributions, 3
  servers, 17 queue slots (20-3), 1500 total jobs,
  Shortest Packet First

17
Poisson Process

• For a poisson process with average arrival rate
  , the probability of seeing n arrivals in time
  interval delta t

18
Poisson process exponential distribution

• Inter-arrival time t (time between arrivals) in a
  Poisson process follows exponential distribution
  with parameter

19
Analysis of M/M/1 queue

• Given
• l Arrival rate of jobs (packets on input link)
• m Service rate of the server (output link)
• Solve
• L average number in queuing system
• Lq average number in the queue
• W average waiting time in whole system
• Wq average waiting time in the queue

20
M/M/1 queue model
21
Solving queuing systems

• 4 unknowns L, Lq W, Wq
• Relationships
• LlW
• LqlWq (steady-state argument)
• W Wq (1/m)
• If we know any 1, can find the others
• Finding L is hard or easy depending on the type
  of system. In general

22
Analysis of M/M/1 queue

• Goal A closed form expression of the probability
  of the number of jobs in the queue (Pi) given
  only l and m

23
Equilibrium conditions
Define to be the probability of having
n tasks in the system at time t
24
Equilibrium conditions
l
l
l
l
n1
n
n-1
m
m
m
m
25
Solving for P0 and Pn

• Step 1
• Step 2

26
Solving for P0 and Pn

• Step 3
• Step 4

27
Solving for L
28
Solving W, Wq and Lq
29
Online M/M/1 animation

• http//www.dcs.ed.ac.uk/home/jeh/Simjava/queueing/
  mm1_q/mm1_q.html

30
Response Time vs. Arrivals
31
Stable Region
linear region
32
Example

• On a network gateway, measurements show that the
  packets arrive at a mean rate of 125 packets per
  second (pps) and the gateway takes about 2
  millisecs to forward them. Assuming an M/M/1
  model, what is the probability of buffer overflow
  if the gateway had only 13 buffers. How many
  buffers are needed to keep packet loss below one
  packet per million?

33
Example

• Measurement of a network gateway
• mean arrival rate (l) 125 Packets/s
• mean response time (m) 2 ms
• Assuming exponential arrivals
• What is the gateways utilization?
• What is the probability of n packets in the
  gateway?
• mean number of packets in the gateway?
• The number of buffers so P(overflow) is lt10-6?

34
Example

• Arrival rate ?
• Service rate µ
• Gateway utilization ? ?/µ
• Prob. of n packets in gateway
• Mean number of packets in gateway

35
Example

• Arrival rate ? 125 pps
• Service rate µ 1/0.002 500 pps
• Gateway utilization ? ?/µ 0.25
• Prob. of n packets in gateway
• Mean number of packets in gateway

36
Example

• Probability of buffer overflow
• To limit the probability of loss to less than
  10-6

37
Example

• Probability of buffer overflow P(more than
  13 packets in gateway)
• To limit the probability of loss to less than
  10-6

38
Example

• Probability of buffer overflow P(more than
  13 packets in gateway) ?13 0.2513
  1.49x10-8 15 packets per billion packets
• To limit the probability of loss to less than
  10-6

39
Example

• Probability of buffer overflow P(more than
  13 packets in gateway) ?13 0.2513
  1.49x10-8 15 packets per billion packets
• To limit the probability of loss to less than
  10-6

40
Example

• To limit the probability of loss to less than
  10-6
• or

41
Example

• To limit the probability of loss to less than
  10-6
• or 9.96

42
Empirical Example
M/M/m system