Introduction to Queuing Theory

1
Introduction to Queuing Theory
2
Queuing theory definitions

• (Bose) the basic phenomenon of queueing arises
  whenever a shared facility needs to be accessed
  for service by a large number of jobs or
  customers.
• (Wolff) The primary tool for studying these
  problems of congestions is known as queueing
  theory.
• (Kleinrock) We study the phenomena of standing,
  waiting, and serving, and we call this study
  Queueing Theory." "Any system in which arrivals
  place demands upon a finite capacity resource may
  be termed a queueing system.
• (Mathworld) The study of the waiting times,
  lengths, and other properties of queues.

http//www2.uwindsor.ca/hlynka/queue.html
3
Applications of Queuing Theory

• Telecommunications
• Traffic control
• Determining the sequence of computer operations
• Predicting computer performance
• Health services (eg. control of hospital bed
  assignments)
• Airport traffic, airline ticket sales
• Layout of manufacturing systems.

http//www2.uwindsor.ca/hlynka/queue.html
4
Example application of queuing theory

• In many retail stores and banks
• multiple line/multiple checkout system ? a
  queuing system where customers wait for the next
  available cashier
• We can prove using queuing theory that
  throughput improves increases when queues are
  used instead of separate lines

http//www.andrews.edu/calkins/math/webtexts/prod
10.htmQT
5
Example application of queuing theory
http//www.bsbpa.umkc.edu/classes/ashley/Chaptr14/
sld006.htm
6
Queuing theory for studying networks

• View network as collections of queues
• FIFO data-structures
• Queuing theory provides probabilistic analysis of
  these queues
• Examples
• Average length
• Average waiting time
• Probability queue is at a certain length
• Probability a packet will be lost

7
Model Queuing System

• Use Queuing models to
• Describe the behavior of queuing systems
• Evaluate system performance

8
Characteristics of queuing systems

• Arrival Process
• The distribution that determines how the tasks
  arrives in the system.
• Service Process
• The distribution that determines the task
  processing time
• Number of Servers
• Total number of servers available to process the
  tasks

9
Kendall Notation 1/2/3(/4/5/6)

• Six parameters in shorthand
• First three typically used, unless specified
• Arrival Distribution
• Service Distribution
• Number of servers
• Total Capacity (infinite if not specified)
• Population Size (infinite)
• Service Discipline (FCFS/FIFO)

10
Distributions

• M stands for "Markovian", implying exponential
  distribution for service times or inter-arrival
  times.
• D Deterministic (e.g. fixed constant)
• Ek Erlang with parameter k
• Hk Hyperexponential with param. k
• G General (anything)

11
Kendall Notation Examples

• M/M/1
• Poisson arrivals and exponential service, 1
  server, infinite capacity and population, FCFS
  (FIFO)
• the simplest realistic queue
• M/M/m
• Same, but M servers
• G/G/3/20/1500/SPF
• General arrival and service distributions, 3
  servers, 17 queue slots (20-3), 1500 total jobs,
  Shortest Packet First

12
Analysis of M/M/1 queue

• Given
• l Arrival rate of jobs (packets on input link)
• m Service rate of the server (output link)
• Solve
• L average number in queuing system
• Lq average number in the queue
• W average waiting time in whole system
• Wq average waiting time in the queue

13
M/M/1 queue model
L

Lq
l
m


Wq
W
14
Littles Law
System
Arrivals
Departures

• Littles Law Mean number tasks in system mean
  arrival rate x mean response time
• Observed before, Little was first to prove
• Applies to any system in equilibrium, as long as
  nothing in black box is creating or destroying
  tasks

15
Proving Littles Law
Arrivals
Packet
Departures
1 2 3 4 5 6 7 8
Time
J Shaded area 9 Same in all cases!
16
Definitions

• J Area from previous slide
• N Number of jobs (packets)
• T Total time
• l Average arrival rate
• N/T
• W Average time job is in the system
• J/N
• L Average number of jobs in the system
• J/T

17
Proof Method 1 Definition
in System (L)

1 2 3 4 5 6 7 8
Time (T)
18
Proof Method 2 Substitution
Tautology
19
M/M/1 queue model
L

Lq
l
m


Wq
W
L?W Lq?Wq
20
Poisson Process

• For a poisson process with average arrival rate
  , the probability of seeing n arrivals in time
  interval delta t

21
Poisson process exponential distribution

• Inter-arrival time t (time between arrivals) in a
  Poisson process follows exponential distribution
  with parameter

22
M/M/1 queue model
L?W Lq?Wq W Wq (1/µ)
23
Solving queuing systems

• 4 unknowns L, Lq W, Wq
• Relationships
• LlW
• LqlWq
• W Wq (1/m)
• If we know any 1, can find the others

24
Analysis of M/M/1 queue

• Goal A closed form expression of the probability
  of the number of jobs in the queue (Pi) given
  only l and m

25
Equilibrium conditions
Define to be the probability of having
n tasks in the system at time t
26
Equilibrium conditions
l
l
l
l
n1
n
n-1
m
m
m
m
27
Solving for P0 and Pn

• Step 1
• Step 2

28
Solving for P0 and Pn

• Step 3
• Step 4

29
Solving for L
30
Solving W, Wq and Lq
31
Online M/M/1 animation

• http//www.dcs.ed.ac.uk/home/jeh/Simjava/queueing/
  mm1_q/mm1_q.html

32
Response Time vs. Arrivals
33
Stable Region
linear region
34
Example

• On a network gateway, measurements show that the
  packets arrive at a mean rate of 125 packets per
  second (pps) and the gateway takes about 2
  millisecs to forward them. Assuming an M/M/1
  model, what is the probability of buffer overflow
  if the gateway had only 13 buffers. How many
  buffers are needed to keep packet loss below one
  packet per million?

35
Example

• Measurement of a network gateway
• mean arrival rate (l) 125 Packets/s
• mean response time (m) 2 ms
• Assuming exponential arrivals
• What is the gateways utilization?
• What is the probability of n packets in the
  gateway?
• mean number of packets in the gateway?
• The number of buffers so P(overflow) is lt10-6?

36
Example

• Arrival rate ?
• Service rate µ
• Gateway utilization ? ?/µ
• Prob. of n packets in gateway
• Mean number of packets in gateway

37
Example

• Arrival rate ? 125 pps
• Service rate µ 1/0.002 500 pps
• Gateway utilization ? ?/µ 0.25
• Prob. of n packets in gateway
• Mean number of packets in gateway

38
Example

• Probability of buffer overflow
• To limit the probability of loss to less than
  10-6

39
Example

• Probability of buffer overflow P(more than
  13 packets in gateway)
• To limit the probability of loss to less than
  10-6

40
Example

• Probability of buffer overflow P(more than
  13 packets in gateway) ?13 0.2513
  1.49x10-8 15 packets per billion packets
• To limit the probability of loss to less than
  10-6

41
Example

• Probability of buffer overflow P(more than
  13 packets in gateway) ?13 0.2513
  1.49x10-8 15 packets per billion packets
• To limit the probability of loss to less than
  10-6

42
Example

• To limit the probability of loss to less than
  10-6
• or

43
Example

• To limit the probability of loss to less than
  10-6
• or 9.96

44
Empirical Example
M/M/m system
45
Queue management
46
Simplified Network Queuing Model
Input Arrivals
System
Output Departures
Goal Move packets across the system from the
inputs to output System could be a single switch,
or entire network
47
Problem

• Too many packets in some part of the system
• Congestion

48
Problems with Congestion

• Performance
• Low Throughput
• Long Latency
• High Variability (jitter)
• Lost data

49
Causes of Congestion

• Many ultimate causes
• New applications, new users
• Bugs, faults, viruses, spam
• Stochastic (time based) variations, unknown
  randomness.

50
Types of Congestion Control Strategies

• Terminate existing resources
• Drop packets
• Drop circuits
• Limit entry into the system
• Packet level (layer 3)
• Flow/conversation level (layer 4)
• Resource reservation
• TCP backoff/reduce window
• Application level (layer 7)
• Limit types/kinds of applications

51
Packet level Traffic shaping

• Big Ideas
• Shape/Modify traffic at entrance points in the
  network
• makes the network more manageable and predictable
• Modify traffic in the routers
• Enforce policies on flows

• Policy enforcement
• Fair share
• User X pays more than user Y, X should get
  more bandwidth than Y.

52
Leaky Bucket

• Across a single link, only allow packets across
  at a constant rate

53
Leaky Bucket Analogy
Packets from input
Leaky Bucket
Output
54
Token Bucket

• Packets need token to enter network
• Tokens are generated and placed into the token
  bucket at a constant rate

55
Token Bucket
Packets from input
Token Generator (Generates a token once every T
seconds)
output
56
Token Bucket vs. Leaky Bucket
Case 1 Short burst arrivals
Arrival time at bucket
6
5
4
3
2
1
0
Departure time from a leaky bucket Leaky bucket
rate 1 packet / 2 time units Leaky bucket size
4 packets
6
5
4
3
2
1
0
Departure time from a token bucket Token bucket
rate 1 tokens / 2 time units Token bucket size
2 tokens
6
5
4
3
2
1
0
57
Token Bucket vs. Leaky Bucket
Case 2 Large burst arrivals
Arrival time at bucket
6
5
4
3
2
1
0
Departure time from a leaky bucket Leaky bucket
rate 1 packet / 2 time units Leaky bucket size
2 packets
6
5
4
3
2
1
0
Departure time from a token bucket Token bucket
rate 1 token / 2 time units Token bucket size
2 tokens
6
5
4
3
2
1
0
58
Question

• Data from an application on computer A transmits
  data at 20Mb/s for 20 msec every second. If the
  traffic is shaped using a leaky bucket running at
  2 Mb/s compute
• How long will each burst take to get sent over
  the network?
• What size buffer is needed such that no data will
  be lost?
• If the bursts from part (1) happen at a rate of 1
  every 100 ms, could the leaky bucket not lose any
  data?

59
Multi-link congestion?

• Multiple input queues/flows fight for a single
  output queue

60
Multi-queue management

• First cut round-robin
• Service input queues in round-robin order
• What if one flow/link has all large packets,
  another all small packets?
• Larger packets get more link bandwidth

61
Idealized flow model

• Goal - fair sharing
• Ideal model - fluid model
• Image we could squeeze factions of a bits on the
  link
• E.g. Interleave the bits on the link
• But we must work with packets
• How to approximate fluid model ?

62
Fluid model vs. Packet Model
63
Fair Queuing

• Keep track of each source queue
• Size of packets matter

64
Fair Queuing

• On packet arrival
• Compute finish number for the packet
• On packet completion
• Select packet with lowest finish number to be
  output

65
Computing Finish Numbers

• Finish of an arriving packet is computed as the
  size of the packet in bits
• Finish of previous packet in the same queue

66
Fair queuing

• What if not all the queues are active/busy all
  the time?

67
Virtual Time

• Used to keep track of service delivered
• Approximate bit-by-bit
• Virtual time is incremented each time a bit is
  transmitted for all flows
• If we have 3 active flows, and transmit 3 bits,
  we increment virtual time by 1.
• If we had 4 flows, and transmit 2 bits, increment
  Vt by 0.5.

68
Computing Finish Numbers

• Finish of an arriving packet is computed as the
  size of the packet in bits the greater of
• Finish of previous packet in the same queue
• Current virtual time

69
Finish Number

• Define
• - finish of packet k of flow i (in virtual
  time)
• - length of packet k of flow I (bits)
• - Real to virtual time function
• Then The finish of packet k of flow i is

70
A Fair Queuing Example

• 3 queues A, B, and C
• At real time 0, 3 packets
• Of size 1 on A, size 2 on B and C
• A packet of size 2 shows up at Real time 4 on
  queue A

71
FQ Example

• The finish s for queues A, B and C are set to
  1, 2 and 2
• Virtual time runs at 1/3 real time

72
FQ Example- cont.

• After 1 unit of service, each connection has
  received 10.33 0.33 units of service
• Packet from queue A departed at R(t)1
• Packet from queue C is transmitting (break tie
  randomly)

73
FQ Example- cont.

• Between T1,3 there are 2 connections virtual
  time V(t) function increases by 0.5 per unit of
  real time

74
FQ Example- cont.

• Between T3,4 only 1 connection virtual time
  increases by 1.0 per unit of real time

75
Final Schedule
76
Fair Queuing vs. Round Robin

• Advantages protection among flows
• Misbehaving flows will not affect the performance
  of well-behaving flows
• Misbehaving flow a flow that does not implement
  congestion control
• Disadvantages
• More complex must maintain a queue per flow per
  output instead of a single queue per output

77
Question
A router uses fair queuing to schedule packets.
Each output link runs at 8 Mbits/sec. Given the
above packet arrival schedule, what order are the
packets sent for all output links?
78
Weight on the queues

• Define
• - finish time of packet k of flow i (in virtual
  time)
• - length of packet k of flow I (bits)
• - Real to virtual time function
• - Weight of flow i
• The finishing time of packet k of flow i is

Weight
79
Weighted Fair Queuing

• Increasing the weight gives more service to a
  given queue
• Lowers finish time
• Service is proportional to the weights
• E.g. Weights of 1, 1 and 2 means one queue gets
  50 of the service, other 2 queues get 25.