Bandwidth and Latency: Their Changing Impact on Performance

1
Bandwidth and Latency Their Changing Impact on
Performance
Yiping Ding BMC Software 
2
Bandwidth and Latency

• Two dimensions of performance volume and time
• A third dimension cost

3
Dave Pattersons Observations (Oct. 04)

• Latency lags bandwidth
• Latency improved about 10X while bandwidth
  improved about 100X to 1000X

4
Dave Pattersons Observations (Oct. 04)

• Latency and bandwidth improvements from 1982 to
  2001

5
Latency and Bandwidth improvements from 93 to 01

• In the time that it takes for bandwidth to
  double, latency improves by no more than a factor
  of 1.2 to 1.4

6
Jeff Buzens Comments (Jan. 05)
Communications of the ACM Volume 48, Number 1
(2005), Pages 11-12
7
Jeff Buzens Comments (Jan. 05)

• There is a distinction between physical latency
  and congestion-based latency
• Physical Latency relates to hardware properties
• Congestion-based latency relates to queueing
  delays (waiting)
• We need to pay attention to congestion-based
  latency in the Internet era
• Web service
• Service-Oriented Architecture (SOA)
• Packet switching networks

8
High Bandwidth and High Speed

• They are related but different
• high bandwidth can reduce waiting
• high speed can reduce both service and wait
  times
• They are often used together to mean fast,
  e.g.,
• Verizon Business DSL Delivering powerful,
  high-speed broadband to your business. Starting
  as low as 19.95 per month. One month free when
  you order online
• High-speed Internet access is one of the few
  telecommunications issues embraced by the White
  House President Bush last year called for
  universal, affordable access to broadband by
  2007

9
A Queueing Model for Bandwidth and Latency
x

• X throughput (average data processed for a given
  time interval)
• B Bandwidth (maximum throughput when system
  saturates)
• when a gt x, X approaches B
• S service time (physical latency)
• W wait time (congestion-based latency)
• R Response time (overall latency)

10
Reducing Physical Latency
Reducing physical latency is equivalent to
reducing service time in a queueing model
Cutting the service time in half doubles the
bandwidth
11
Double Bandwidth (two interpretations)
gt
DBDQ
Doubling bandwidth does not cut the service time
in half
DBDS
12
Double Bandwidth Example M/M/1
gt
13
How to judge which one is better

• Judge by which one has larger a relative
  response time reduction

Response time of original system Response
time of changed system --------------------------
--------------------------------------------------
-------------- Response time of original system

• An example of cutting service time in half

14
Half service time vs. double bandwidth
Cutting service time in half is always better
than doubling bandwidth
High Utilization
Low Utilization
15
Implications of Pattersons observation
In the time that it takes for bandwidth to
double, latency improves by no more than a
factor of 1.2 to 1.4
There are two crossing points
Sometimes, improving latency by a factor of
1.2 or 1.4 is better than doubling bandwidth
16
Improving latency by a factor of 1.4
It is better than doubling bandwidth when U lt 80
17
Improving latency by a factor of 1.2
It is better than doubling bandwidth when U lt 40
18
Effectiveness of Increasing Bandwidth
As device bandwidth increases, the utilization of
the device will decrease the effectiveness of
reducing response time (congestion-based latency)
decreases
At 1 utilization, doubling bandwidth has little
effect on reducing response time
19
The Effect of Service Time Variation (C2)
Cut service time in half for an M/M/1 queue, C21
Double bandwidth as C2 increases
20
The Effect of Service Time Variation in Words

• When service time variation is huge, reducing
  the relative response time by doubling the
  bandwidth, in the DBDQ case, is the same as
  cutting the service time in half for an M/M/1
  queueing model
• It is always more effective to cut service time
  in half than to double the bandwidth regardless
  the service time variation
• When the service time variation increases, the
  relative response time reduction percentage by
  DBDQ of an M/G/1 system is bounded by that of an
  M/M/1 system with half the service time.

21
Cope with Performance Problems

• Cope with high latency
• Cope with low bandwidth
• Cope with bandwidth imbalance

22
Cope with High Latency

• Reducing latency is inherently more difficult
  than increasing bandwidth
• New hardware and software technology has to be
  introduced
• When approaching the speed of light, further
  reduction in physical latency will be more
  difficult and more costly
• We understand and will get used to the latency
• We see lightning before hearing thunder
• Controlling the Mars Rover is different from
  controlling an RC car

23
Coping with latency time is relative
                                     
24
Coping with Latency carry more
25
Coping with Limited Bandwidth
Current 146MB
130 MB limit
26
Coping with Latency Caching
27
Coping with Limited Bandwidth
Larger packet with lower overhead
28
Bandwidth Latency Impact on Protocol Performance

• Protocols and software were developed under
  certain assumptions about hardware
• Number of retries
• Wait time between retries
• Size of buffers
• New algorithms and protocols have to be
  introduced to match the latency and bandwidth
• High bandwidth (more processors) does not
  automatically guarantee better performance
• We often have to sacrifice performance for
  correctness
• Each and every technology has its lifetime and
  limit
• TCP cannot be successfully modified for space
  travel

29
The Effectiveness of Software Protocol
When utilization is low, certain algorithms may
not be as effective
30
Software will run slower over time

• Software protocol will age and its performance
  will decay
• Conditions change beyond design expectations

0 to 60 mph in 15 sec.
?
31
Bandwidth Imbalance
32
Bandwidth Imbalance
33
Bandwidth Imbalance (80s)
34
Bandwidth Imbalance (90s and 2000)
35
What does bandwidth imbalance mean?
36
What does bandwidth imbalance mean?
37
A link for simulation of lane-drop bottlenecks

• http//www.ce.berkeley.edu/jlaval/mltraffic/lane_
  drops.htm

38
What does bandwidth imbalance mean?
39
Coping with bandwidth imbalance
40
56 Kbps
41
100 Kbps
42
300 Kbps
43
I dont know
44
Summary

• Bandwidth and latency will continue to provide
  challenging issues, such as I dont know, for
  the CMG community to address.