Latency%20Lags%20Bandwidth%20(last%20~20%20years)

1
Latency Lags Bandwidth (last 20 years)

• Performance Milestones
• Disk 3600, 5400, 7200, 10000, 15000 RPM

2
Latency Lags Bandwidth (last 20 years)

• Performance Milestones
• Memory Module 16bit plain DRAM, 16b Page Mode
  DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM
• Disk 3600, 5400, 7200, 10000, 15000 RPM

3
Latency Lags Bandwidth (last 20 years)

• Performance Milestones
• Ethernet 10Mb, 100Mb, 1000Mb, 10000 Mbit/sec
• Memory Module 16bit plain DRAM, 16b Page Mode
  DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM
• Disk 3600, 5400, 7200, 10000, 15000 RPM

4
Latency Lags Bandwidth (last 20 years)

• Performance Milestones
• Processor 286, 386, 486, Pentium, Pentium
  Pro, Pentium 4
• Ethernet 10Mb, 100Mb, 1000Mb, 10000 Mbit/sec
• Memory Module 16bit plain DRAM, 16b Page Mode
  DRAM, 32b, 64b, 64b SDRAM, 64b DDR SDRAM
• Disk 3600, 5400, 7200, 10000, 15000 RPM

5
Rule of Thumb for Latency Lagging BW

• In the time that bandwidth doubles, latency
  improves only by factors of 1.2 to 1.4
• (and capacity approximately triples in the
  time that bandwidth doubles)

6
Improvement in Time for BW to Double
7
Recent Improvement in Time for BW to Double
(Last 3 Performance Milestones)
8
6 Reasons Latency Lags Bandwidth

• Latency helps BW, but not vice versa
• Spinning disk faster improves both bandwidth and
  rotational latency
• Higher linear density helps BW, not Latency
• Bandwidth hurts latency
• Longer queues help BW, hurt Latency
• Distance limits latency
• Speed of light and computers on network
• Size of DRAM block gt long bit and word lines gt
  most of DRAM access time

9
6 Reasons Latency Lags Bandwidth

• 4. Moores Law helps BW more than latency
• Smaller, faster transistors communicate over
  relatively longer lines limits latency
• Faster transistors, more transistors, more pins
  all help BW
• 5. Software overhead hurts latency
• Long messages amortize overhead overhead more
  for short messages
• Bandwidth is easier to sell
• E.g., 10 Gbit/s vs. 10 msec latency Ethernet

10
4 Ways to Cope with Latency Lags Bandwidth

• Caching (Leveraging Capacity)
• Processor caches, file cache, disk cache
• Replication (Leveraging Capacity)
• Read from nearest head in RAID, Distributed
  caches in SMP, Distributed ISPs
• Parallelism and Out-of-Order Processing
  (Leveraging Bandwidth)
• Processors, Disk controllers, Messages
• Prediction (Leveraging Bandwidth)
• Branches, Prefetching disk, caches

11
Conclusion

• If everything improves at the same rate, then
  nothing really changes
• When rates vary, we see real dislocation that in
  turn requires real innovation
• Hardware and software developers should plan that
  performance advances will emphasize BW over
  latency