Bridging Router Performance and Queuing Theory

1
Bridging Router Performance and Queuing Theory

• Dina Papagiannaki,
• Intel Research Cambridge
• with
• Nicolas Hohn, Darryl Veitch and Christophe Diot

2
Motivation

• End-to-end packet delay is an important metric
  for performance and SLAs
• Building block of end-to-end delay is through
  router delay
• We measure the delays incurred by all packets
  crossing a single router

3
Overview

• Full Router Monitoring
• Delay Analysis
• Modeling
• Delay Performance Understanding and Reporting
• Causes of microcongestion

4
Measurement Environment
5
Full Router Monitoring

• Gateway router
• 2 backbone links (OC-48), 2 domestic customer
  links (OC-3, OC-12), 2 Asian customer links
  (OC-3)
• 13 hours of trace collection on Aug. 14, 2003
• 7.3 billion packets 3 TeraBytes of IP traffic
• Monitor more than 99.9 of all through traffic
• µs timestamp precision

6
Packet matching
Set Link Matched pkts traffic C2-out
C4 In 215987 0.03
C1 In 70376 0.01
BB1 In 345796622 47.00
BB2 In 389153772 52.89
C2 out 735236757 99.93
7
Packet matching (cntd)
8
Overview

• Full Router Monitoring
• Delay Analysis
• Modeling
• Delay Performance Understanding and Reporting
• Causes of microcongestion

9
Store Forward Datapath

• Store storage in input linecards memory
• Forwarding decision
• Storage in dedicated Virtual Output Queue (VOQ)
• Decomposition into fixed-size cells
• Transmission through switch fabric cell by cell
• Packet reconstruction
• Forward Output link scheduler

10
Delays 1 minute summary
11
Minimum Transit Time
Packet size dependent minimum delay ?(L),
specific to router architecture and linecard
technology
12
Store Forward Datapath

• Store storage in input linecards memory
• Forwarding decision
• Storage in dedicated Virtual Output Queue (VOQ)
• Decomposition into fixed-size cells
• Transmission through switch fabric cell by cell
• Packet reconstruction
• Forward Output link scheduler

13
Overview

• Full Router Monitoring
• Delay Analysis
• Modeling
• Delay Performance Understanding and Reporting
• Causes of microcongestion

14
Modeling
15
Modeling
16
Model Validation
17
Model validation
18
Error as a function of time
19
Modeling results

• Our crude model performs well
• Use effective link bandwidth (account for
  encapsulation)
• The front end ? only matters when the output
  queue is empty
• The model defines Busy Periods time between the
  arrival of a packet to the empty system and the
  time when the system becomes empty again.

20
Overview

• Full Router Monitoring
• Delay Analysis
• Modeling
• Delay Performance Understanding and Reporting
• Causes of microcongestion

21
Delay Performance

• Packet delays cannot be inferred from output link
  utilization
• Source of large delays queue build-ups in output
  buffer
• Busy Period structures contain all delay
  information
• Busy Period durations and idle duration contain
  all utilization information

22
Reporting BP Amplitude
23
Reporting BP Duration
24
Report BP joint distribution
25
Busy periods have a common shape
26
Reporting Busy Periods

• Answer performance related questions directly
• How long will a given level of congestion last?
• Method
• Report partial busy period statistics A and D
• Use triangular shape

27
Understanding Busy Periods
28
Reporting Busy Periods
29
Summary of modeling part

• Results
• Full router empirical study
• Delay modeling
• Reporting performance metrics

30
Overview

• Full Router Monitoring
• Delay Analysis
• Modeling
• Delay Performance Understanding and Reporting
• Causes of microcongestion

31
Causes of microcongestion

1. Reduction in link bandwidth from core to the
   access.
2. Multiplexing of multiple input traffic streams
   toward a single output stream.
3. Degree and nature of burstiness of input traffic
   stream(s).

32
Stretching and merging
Queue Buildup!
33
Causes of microcongestion

1. Reduction in link bandwidth from core to the
   access.
2. Multiplexing of multiple input traffic streams
   toward a single output stream.
3. Degree and nature of burstiness of input traffic
   stream(s).

34
Multiplexing
35
Causes of microcongestion

1. Reduction in link bandwidth from core to the
   access.
2. Multiplexing of multiple input traffic streams
   toward a single output stream.
3. Degree and nature of burstiness of input traffic
   stream(s).

36
Traffic Burstiness

• Duration and amplitude of busy periods depends on
  the spacing of packets at the input.
• Highly clustered packets at the input are more
  likely to form busy periods.

37
Busy periods
Maximum amplitude 5 ms Maximum duration 15
ms 120,000 busy periods gt 1 ms
38
Methodology

• Run semi-experiments
• Simulate busy periods and measure their amplitude
  A(S, µ) under two different traffic scenarios,
  one that contains the effect studied and one that
  does not
• Define a metric to quantitatively capture the
  studied effect

39
Reduction in Bandwidth
40
Amplification factor

• Reference stream
• ST traffic from a single OC-48 link
• Output link rate µi
• Test stream
• Ss traffic from a single OC-48 link
• Output link rate µo

41
Amplification factor (2)
42
Link multiplexing
43
Link multiplexing

• Reference stream
• ST output link traffic
• Output link rate µo
• Test stream
• Si traffic from a single OC-48 link
• Output link rate µo

44
Link multiplexing (2)
45
Flow burstiness
Non-bursty flow
Bursty flow
46
Flow Burstiness

• Reference stream
• ST input traffic stream from a single OC-48 link
• Output link rate µo
• Test stream
• Sj top 5-tuple flow OR the set of ALL bursty
  flows
• Output link rate µo

47
Flow burstiness
48
Summary

• Methodology (and metrics) to investigate impact
  of different congestion mechanisms
• In todays access networks
• Reduction in link bandwidth plays a significant
  role
• Multiplexing has a definite impact since
  individual links would not have led to similar
  delays
• Flow burstiness does NOT significantly impact
  delay (bottleneck bandwidths too small to
  dominate the backbone)
• Congestion may be the outcome of network design!

49
Thank you!
50
References

• K. Papagiannaki, S. Moon, C. Fraleigh, P.Thiran,
  F. Tobagi, C. Diot.Analysis of Measured
  Single-Hop Delay from an Operational Backbone
  Network.In IEEE Infocom, New York, U.S.A., June,
  2002.
• N. Hohn, D. Veitch, K. Papagiannaki, C.
  Diot.Bridging router performance and queuing
  theory.To appear in ACM Sigmetrics, New York,
  U.S.A., June, 2004.
• K. Papagiannaki, D. Veitch, and N. Hohn.Origins
  of Microcongestion in an Access Router.In
  Passive Active Measurement Workshop, Antibes,
  France, April, 2004.

51
Busy Period Construction