Congestion Control on High-Speed Networks

1
Congestion Control on High-Speed Networks

• Injong Rhee, Lisong Xu
• Computer Science Department
• North Carolina State University

2
Outline

• Motivation
• Related Work
• BIC-TCP
• Future Work

3
High-Speed Networks

• Many high-speed networks are being developed.
• BW 10Gbps, and expected to be upgraded to
  higher speeds in the near future.

4
ESNET Energy Science Network
ESNET

• ESNet is funded by the Department of Energy to
  provide a high-speed network serving thousands of
  scientists worldwide

5
High-Speed Applications
Transport Protocols be able to transfer a large
amount of data over a long distance within a
short amount of time
High-Speed Networks
6
TCP Performance
Utilization of a link with 5 TCP connections
Cannot fully utilize the huge capacity of
high-speed networks!

• NS-2 Simulation (100 sec)
• Link Capacity 155Mbps, 622Mbps, 2.5Gbps,
  5Gbps, 10Gbps,
• Drop-Tail Routers, 0.1BDP Buffer
• 5 TCP Connections, 100ms RTT, 1000-Byte Packet
  Size

Ns-2 capacity 155Mbps, 622Mbps, 2.5Gbps,
5Gbps, 10Gbps 100 sources, 100 ms round trip
propagation delay, 1500-Byte Packet Size
7
TCP Congestion Control

• The instantaneous throughput of TCP is controlled
  by a variable cwnd,
• TCP transmits approximately a cwnd number of
  packets per RTT (Round-Trip Time).

8
TCP over High-Speed Networks

• A TCP connection with 1250-Byte packet size and
  100ms RTT is running over a 10Gbps link (assuming
  no other connections, and no buffers at routers)

slow increase
TCP
Packet loss
Packet loss
Packet loss
Packet loss
big decrease
cwnd
Time (RTT)
Congestion avoidance
Slow start
9
Response Function of TCP

• Response function of TCP is the average
  throughput of a TCP connection in terms of the
  packet loss probability, the packet size, and the
  round-trip time.

10
Response Function of TCP
10Gbps requires a packet loss rate of 10-10, or
correspondingly a link bit error rate of at most
10-14, which is an unrealistic (or at least hard)
requirement for current networks
Assuming 1250-Byte packet size, and 100ms RTT
11
Outline

• Motivation
• Related Work
• BIC-TCP
• Future Work

12
Proposed High-Speed Protocols

• Window-Based Protocols
• AIMD (Additive Increase Multiplicative Decrease)
• Jumbo Frame, GridFTP, PFTP, PSockets
• HSTCP (High-Speed TCP) by Sally Floyd at ICIR,
  Berkeley
• STCP (Scalable TCP) by Tom Kelly at Cambridge
  University
• FAST (Fast AQM Scalable TCP) by Steven Low at
  California Institute of Technology
• Rate-Based Protocols
• SABUL (Simple Available Bandwidth Utilization
  Library ) by Robert Grossman at University of
  Illinois at Chicago

window-based protocols are known for safer
incremental deployment. D. Bansal, H.
Balakrishnan, S. Floyd, and S. Shenker, "Dynamic
behavior of slowly responsive congestion
controls", In Proceedings of SIGCOMM 2001, San
Diego, California.
13
AIMD (Additive Increase Multiplicative Decrease)

• AIMD increases cwnd by a larger number, say 32,
  instead of 1 per RTT.
• After a packet loss, AIMD decreases cwnd by 1/8,
  instead of 1/2

TCP
Packet loss
Packet loss
cwnd
Time (RTT)
Slow start
Congestion avoidance
14
Response Function of AIMD

• TCP

• AIMD

The throughput of AIMD is always about 13 times
larger than that of TCP
15
Properties of AIMD
Bandwidth Scalable

• Bandwidth Scalability
• The ability to achieve 10Gbps with a reasonable
  packet loss probability

NOT TCP Friendly

• TCP-Friendliness
• The ability to share bandwidth with TCP
  connections on low-speed networks

16
STCP (Scalable TCP)

• STCP adaptively increases cwnd, and decreases
  cwnd by 1/8.

TCP
Packet loss
Packet loss
cwnd
Time (RTT)
Slow start
Congestion avoidance
17
HSTCP (High Speed TCP)

• HSTCP adaptively increases cwnd, and adaptively
  decreases cwnd.
• The larger the cwnd, the larger the increment,
  and the smaller the decrement.

TCP
Packet loss
Packet loss
cwnd
Time (RTT)
Slow start
Congestion avoidance
18
Response Functions of HSTCP and STCP
Bandwidth Scalable

• HSTCP

• STCP

HSTCP and STCP are both bandwidth scalable and
TCP friendly
TCP Friendly
19
Outline

• Motivation
• Related Work
• BIC-TCP
• Why another one?
• Protocol Design
• Parameter Setting
• Simulations Experiments
• Future Work

20
RTT Fairness

• Different connections may have quite different
  round-trip times, and a good protocol should
  allocate bandwidth fairly among those connections
• RTT fairness index throughout ratio of two
  flows with different RTTs

21
RTT Fairness on Low-Speed Networks

• For a protocol with the following response
  function, where c and d are protocol-related
  constants.

• The RTT Fairness Index (or the throughput ratio
  of two flows) on low-speed networks is

• On low speed networks, different protocols have
  the same RTT fairness

Lisong Xu, Khaled Harfoush, and Injong Rhee,
"Binary Increase Congestion Control for Fast
Long-Distance Networks", in Proceedings of IEEE
INFOCOM 2004, March, 2004, HongKong
22
RTT Fairness on High-Speed Networks

• For a protocol with the following response
  function, where c and d are protocol-related
  constants.

• The RTT Fairness Index (or the throughput ratio
  of two flows) on high-speed networks is

• On high speed networks, the RTT fairness of a
  protocol depends on the exponent d in the
  response function

Lisong Xu, Khaled Harfoush, and Injong Rhee,
"Binary Increase Congestion Control for Fast
Long-Distance Networks", in Proceedings of IEEE
INFOCOM 2004, March, 2004, HongKong
23
Slope Determines the RTT Fairness

• If the response function is
  then the RTT Fairness is

• The figure is shown in log-log scale, so exponent
  d in the response function is the slope of the
  line in the figure
• The slope of the line determines the RTT fairness
  of a protocol on high-speed networks

24
Simulation Results of RTT Fairness
Throughout ratio of two flows on a 2.5Gbps Link
Inverse RTT Ratio 1 3 6
AIMD 1 6 22
HSTCP 1 29 107
STCP 1 127 389
Simulation setup BDP Buffer, Drop Tail,
Reverse Traffic, Forward Background Traffic
(short-lived TCP, Web Traffic)
25
Simulation Result of 2 STCP Flows
Throughput of two STCP flows with different RTTs
on a 2.5Gbps link
Simulation setup 2.5Gbps, BDP Buffer, Drop
Tail, Reverse Traffic, Forward Background
Traffic (short-lived TCP, Web Traffic)
26
Design Goal
Scalability, RTT Fairness
TCP Fairness
27
Outline

• Motivation
• Related Work
• BIC-TCP
• Why another one?
• Protocol Design
• Parameter Setting
• Simulations Experiments
• Future Work

28
BIC (Binary Increase Congestion control)

• BIC adaptively increase cwnd, and decrease cwnd
  by 1/8

TCP
Packet loss
Packet loss
cwnd
Time (RTT)
Slow start
Congestion avoidance
29
A Search Problem

• A Search Problem
• We consider the increase part of congestion
  avoidance as a search problem, in which a
  connection looks for the available bandwidth by
  comparing its current throughput with the
  available bandwidth, and adjusting cwnd
  accordingly.

• How does TCP find the available bandwidth?
• Linear search
• while (no packet loss)
• cwnd

• Q How to compare R with A?
• R current throughput
• cwnd/RTT
• A available bandwidth
• A Check for packet losses
• No packet loss R lt A
• Packet losses R gt A

30
Linear Search
31
BIC Binary Search with Smax and Smin

• BIC - Binary search
• while (Wmin lt Wmax)
• inc (WminWmax)/2 - cwnd
• if (inc gt Smax)
• inc Smax
• else if (inc lt Smin)
• inc Smin
• cwnd cwnd inc
• if (no packet losses)
• Wmin cwnd
• else
• break

• Wmax Max Window
• Wmin Min Window
• Smax Max Increment
• Smin Min Increment

32
Binary Search with Smax and Smin
Available Bandwidth
Wmax
Wmin
33
Binary Increase Congestion Control (BIC)
Binary search increase
Additive increase
34
Outline

• Motivation
• Related Work
• BIC-TCP
• Why another one?
• Protocol Design
• Parameter Setting
• Simulations Experiments
• Future Work

35
Setting Smax

• Response Function of BIC on high-speed networks

Bandwidth scalability

• Bandwidth scalability of BIC depends only on Smax
• RTT Fairness of BIC on high-speed networks is the
  same as that of AIMD

36
Setting Smin

• Response Function of BIC on low-speed networks

• TCP-friendliness of BIC depends only on Smin

TCP friendliness
37
Response Functions
Bandwidth scalability
RTT Fairness
TCP-Friendliness
38
Outline

• Motivation
• Related Work
• BIC-TCP
• Why another one?
• Protocol Design
• Parameter Setup
• Simulations Experiments
• Future Work

39
Bandwidth Scalability

• NS-2 Simulation (100 sec)
• Link Capacity 155Mbps, 622Mbps, 2.5Gbps,
  5Gbps, 10Gbps,
• Drop-Tail Routers, 0.1BDP Buffer
• 5 Connections, 100ms RTT, 1000-Byte Packet Size

Ns-2 capacity 155Mbps, 622Mbps, 2.5Gbps,
5Gbps, 10Gbps 100 sources, 100 ms round trip
propagation delay, 1500-Byte Packet Size
40
TCP - Friendliness
Simulation setup 20Mbps, BDP Buffer, Drop Tail,
Reverse Traffic 2 TCP flows, 2 high-speed flows,
and some background traffic
41
RTT Fairness

• Throughput ratio of two flows with different
  RTTs on a 2.5Gbps link

Inverse RTT Ratio 1 3 6
BIC 1 12 38
AIMD 1 6 22
HSTCP 1 29 107
STCP 1 127 389
Simulation setup BDP Buffer, Drop Tail,
Reverse Traffic, Forward Background Traffic
(short-lived TCP, Web Traffic)
42
Buffer Size and Throughput
43
Queue Size Vs. Buffer Size
44
Packet loss vs. Buffer Size
45
Simulation Summary
AIMD HSTCP STCP BIC
Scalability ? ? ? ?
TCP-Friendliness ? ? ? ?
RTT Fairness ? ? ? ?
46
SLAC Experiments

• Evaluation of Advanced TCP Stacks on Fast
  Long-Distance Production Networks by SLAC
  (Stanford Linear Accelerator Center)
• Linux TCP
• Parallel TCP
• HighSpeed TCP
• Scalable TCP
• HighSpeed TCP Low Priority
• Hamilton TCP
• BIC TCP
• BIC TCP overall performs very well in our tests
• http//www-iepm.slac.stanford.edu/bw/tcp-eval/

47
Outline

• Motivation
• Related Work
• BIC-TCP
• Future Work

48
Conclusion and Future Work

• Conclusion
• Designed a congestion control protocol, which is
  scalable, TCP-friendly, and has a good RTT
  fairness.
• "Binary Increase Congestion Control for Fast
  Long-Distance Networks", in Proceedings of IEEE
  INFOCOM 2004, March, 2004, HongKong
• Our work has been reported by some newspapers.
• NBC (US)
• The Washington Times (US)
• The Times of India (India)
• Sina (China)
• Future Work
• Further improve the performance of BIC
• Solve other problems in high-speed networks

49
CUBIC
50
More information http//www.csc.ncsu.edu/facult
y/rhee/export/bitcp Thank you! Any questions?