Characterising Transport Protocol Performance AKA TCP Metrics

1
Characterising Transport Protocol PerformanceAKA
TCP Metrics

• Yee-Ting Li
• DataTAG Meeting _at_ CERN
• 29th October 2002

2
Background

• Lots of different TCP variants in development
• Grid TCP, HSTCP, FAST, TCP Westwood
• Lots of new non-tcp variants floating around too
• Tsunami, Equation Based Congestion Control, Video
  Conferencing, Video Streaming
• Network links speeds doubling every 8 months
• Even more with lamda switching
• Need to see which transport protocols perform
  best and why for future networks
• Find the next generation general choice for
  transport protocols
• Enable better design choices
• Make better choice of transport algorithms
• Determine conditions and applications most
  suitable for each protocol
• Pave the way for future transport protocols

3
High Performance Protocols

• High Performance Fat Link
• Typical Characteristics
• High capacity (1Gb/sec gt)
• Low Loss
• Quality of Service
• Must be adaptable to guaranteed network
  conditions

4
Problems of Current Protocols

• TCP
• Will have very large congestion windows
• Slow Start
• Will often exit at less than optimal value due to
  bit losses
• Congestion Avoidance
• Takes a long time to adapt sudden network changes
  takes 10s minutes to get to optimal value if
  there was loss
• UDP
• No form of congestion control does not scale!
• Will lead to network collapse if used a lot

5
Metrics

• Need way of characterising transport to enable
  comparisons of performance between them all
• Need to separate the mechanisms of the transport
  protocol from the actual performance
• Understand the macroscopic effect of microscopic
  changes
• Dont care about specifics of each transport
  protocol, e.g. how does it deal with dupacks
• Enables comparisons of different transport
  protocols
• What is performance?
• User level it is throughput
• How long does it take to transfer my file?
• How efficient is it
• Can I use the whole bandwidth available to me?
• How many retransmissions does it perform?
• How fair is it

6
Assumptions

• All protocols will be compared on both simulation
  (ns) and real-world conditions
• Protocol will be used for High Performance File
  Transfers
• Not limited by hard disk/system
• gt100mbit/sec
• Relatively low loss but analysis should include
  high loss rates for mass deployment reasons

7
Transport Protocols

• Transport protocols have two phases
• Taken very much from TCP
• An initial available bandwidth discovery phase
• UDP just send out as much as possible
• TCP probe to discover performance
• AKA Slow start
• A stabilisation phase
• A good transport mechanism should spend most of
  its time in this phase
• Tries to discover the optimal sending rate to
  suit network, i.e. measuring/reacting to loss,
  bandwidth estimation
• AKA Congestion Avoidance/Control

Stabilisation Phase
Available Bandwidth Discovery Phase
8
To Make Things Comparable

• Need fair ground for each transport protocol
• Need to compare each protocol with varying RTTs
• Metrics need to be a function of RTT
• Compare against varying Network Conditions
• Different loads/congestion
• Different number of competing streams
• All this really just means different loss rates
  per rtt
• Need to also consider different loss patterns
  (distributions of loss)
• Loss lengths.
• Only really interested in end-to-end
  throughput/path utilisation/bandwidth
• Only useful indicator for user
• Only comparable characteristic between different
  transport protocol implementations
• Can compare throughputs for different streams on
  same link as indication of fairness

9
Throughput

• Throughput is not a raw metric
• Derived from number of bytes/bits transferred
  over the duration of the transfer
• Count retransmissions? NO! however, may give an
  indication of efficiency of protocol
• Use Harmonic Means instead of averages?
• Bulk Transfer Capacity (IETF)
• Describes things too low level
• Only really care about what it gets, not how
  (find out how after we see what it best!)
• Need to know the optimal throughput
• Path/Link capacity (or that set by QoS)

10
Slow Start
Avail BW

• How close slow start exits to optimal stable
  value
• How fast (in rtts) it gets to this value
• If slow start doesnt apply, do not need to
  measure!

B
t0
t1
11
Stablisation
2B

• Given a sudden change of bandwidth how well does
  the protocol react?
• If there is more bandwidth
• how long (in rtts) does it take to take advantage
  of it
• If less bandwidth
• how long does it take to halve its throughput
• How close to the actual available bandwidth does
  the protocol throughput get?

B
t1
t2
2B
B
t1
t2
12
Predictability

• Important to guarantee stable transfer rates
• Do not want wild fluctuations in throughput (can
  cause network synchronisation)
• Makes it more robust???
• Need a metric to represent the degree of
  variation of throughput for the
• Entire transfer stdev/mean throughput
• Intervals every rtt, 2rtt

Frequency
Variation in Throughput
13
Fairness

• How well does a tcp implementation work if all
  the world was using that specific implementation?
• How well does it perform (throughput) against
  another tcp implementation
• specifically what side effects does that cause?
• Jains Fairness

14
Network Specific

• How well does the protocol perform with
• FIFO queues
• RED gateways
• Explicit Congestion Notification
• What performance penalty do we get if we conduct
  the test over a longer rtt path?
• Given certain loss probabilities, ranging from
  very low, to very high, how well does the
  protocol perform and how fair is it as a result.
  Will the protocol contribute to extra loss on the
  network?
• With varying loss patterns (constant, gaussian
  etc.), how does the protocol perform?

15
Compatibility

• How easy is it to deploy a new protocol?
• Eg. Does it require the tcp receiver to respond
  to tcp differently?
• If non-tcp, how cross compatible is the
  implementation?
• What is the performance penalty on the tcp
  implementation given that many hosts implement
  delayed acks?

16
Real-World Transfers

• Try to test on same source/sink hosts
• Same O/S 2.4.16/19 preferable
• Same performance hardware
• Need to be able to control loss
• Control in network constant loads/congestion
• Control in receiver do not ack acks!

17
Comparison Problems

• Shouldnt have problems fitting different
  transport protocols into framework
• Only need to know what the throughput at certain
  times given certain network conditions
• However, new protocols such as HSTCP does things
  differently depending on the throughput (cwnd)
• Similarly for Limited Slow Start

18
Summary

• Need to understand how well each new transport
  protocols perform (why reinvent the wheel)
• Use metrics that rely on throughput (common
  characteristic)
• Measure
• Bandwidth-discovery phase
• Stabilisation phase
• Fairness
• Consider over range of rtts
• Consider over different loss rates and loss
  patterns