Internet Congestion Control: Theory

1
Internet Congestion Control Theory Practice

• Steven Low
• Caltech, FastSoft
• netlab.CALTECH.edu

2
Acknowledgments

• Caltech Andrew, Chen, Choe, Doyle, Jin, Lee, Li,
  Mehyar, Newman, Ravot, Suchara, Tang (Cornell),
  Wang, Wei, Witt, Wydrowski, Yang
• CERN Martin
• Internet2 Almes
• KTH Hjalmarsson, Jacobson, Johansson
• Princeton Calderbank, Chiang
• SLAC Cottrell
• UCLA Paganini (ORT), Z. Wang

More collaborators StarLight, PSC,
Cisco, Level3, LANL
3
Outline

• From theory to deployment
• Theory
• Deployment
• Remarks

4
Caltech FAST Project
Control optimization of networks
5
Milestones
1999
2000
Caltech Doyle Paganini
Low Lapsley
Theory
6
Outline

• From theory to deployment
• Theory
• Overview
• Two open problems
• Deployment
• Remarks

7
Congestion control
pl(t)
xi(t)

• Example congestion measure pl(t)
• Loss (Reno)
• Queueing delay (Vegas)

8
TCP/AQM
pl(t)

• AQM
• DropTail
• RED
• REM/PI
• AVQ

• TCP
• Reno
• Vegas
• FAST

• Congestion control is a distributed asynchronous
  algorithm to share bandwidth
• It has two components
• TCP adapts sending rate (window) to congestion
• AQM adjusts feeds back congestion information
• They form a distributed feedback control system
• Equilibrium stability depends on both TCP and
  AQM
• And on delay, capacity, routing, connections

9
Network model
10
Network model example
for every RTT W 1 for every loss W
W/2
Reno
(AI)
Jacobson 1989
(MD)
11
Network model example
12
Duality model of TCP/AQM

• TCP/AQM
• Equilibrium (x,p) primal-dual optimal
• F determines utility function U
• G guarantees complementary slackness
• p are Lagrange multipliers

Kelly, Maloo, Tan 1998 Low, Lapsley 1999

• Uniqueness of equilibrium
• x is unique when U is strictly concave
• p is unique when R has full row rank

13
Duality model of TCP/AQM

• TCP/AQM
• Equilibrium (x,p) primal-dual optimal
• F determines utility function U
• G guarantees complementary slackness
• p are Lagrange multipliers

Kelly, Maloo, Tan 1998 Low, Lapsley 1999
The underlying convex program also leads to
simple dynamic behavior
14
Duality model of TCP/AQM

• Equilibrium (x,p) primal-dual optimal

Mo Walrand 2000
Low 2003
15
Duality model of TCP/AQM

• Equilibrium (x,p) primal-dual optimal

Mo Walrand 2000
Low 2003
16
Some implications

• Equilibrium
• Always exists, unique if R is full rank
• Bandwidth allocation independent of AQM or
  arrival
• Can predict macroscopic behavior of large scale
  networks
• Counter-intuitive throughput behavior
• Fair allocation is not always inefficient
• Increasing link capacities do not always raise
  aggregate throughput
• Tang, Wang, Low, ToN 2006
• FAST TCP
• Design, analysis, experiments
• Wei, Jin, Low, Hegde, ToN 2006

17
FAST in high speed networks
OC48
Harvey Newmans group, Caltech
I2LSR, SC Bandwidth Challenges
media coverage
18
Open problems

• Heterogeneous protocols
• Accurate dynamic models

Tang, Wang, Low, Chiang. ToN, 2007
Tang,Jacobsson, Andrew, Low. Infocom, 2007
Tang, Andrew, Jacobsson, Johansson, Low,
Hjalmarsson. Infocom 2008
19
The world is heterogeneous

• Linux 2.6.13 allows users to choose congestion
  control algorithms
• Many protocol proposals
• Loss-based Reno and a large number of variants
• Delay-based CARD (1989), DUAL (1992), Vegas
  (1995), FAST (2004),
• ECN RED (1993), REM (2001), PI (2002), AVQ
  (2003),
• Explicit feedback MaxNet (2002), XCP (2002), RCP
  (2005),

20

• Duality model

21

• Duality model

They use different prices!
22
Homogeneous protocol
same price for all sources
23
Heterogeneous protocol
heterogeneous prices for type j sources
24
Heterogeneous protocols

• Equilibrium p that satisfies

• Duality model no longer applies !
• pl can no longer serve as Lagrange multiplier

25
Heterogeneous protocols

• Equilibrium p that satisfies

• Need to re-examine all issues
• Equilibrium exists? unique? efficient? fair?
• Dynamics stable? limit cycle? chaotic?
• Practical networks typical behavior? design
  guidelines?

26
Some implications
homogeneous heterogeneous
equilibrium unique non-unique
bandwidth allocation on AQM independent dependent
bandwidth allocation on arrival independent dependent
Very interesting characterizations of equilibrium
But not much understanding on dynamics
27
Open problems

• Heterogeneous protocols
• Accurate dynamic models

Tang, Wang, Low, Chiang. ToN, 2007
Tang,Jacobsson, Andrew, Low. Infocom, 2007
Tang, Andrew, Jacobsson, Johansson, Low,
Hjalmarsson. Infocom 2008
28
Network model
29
FAST TCP
periodically

FAST
Jin, Wei, Low 2004
Single Link
30
Traditional integrator model
Aggregate FAST rate
Basic assumption
31
Traditional integrator model
Lags true link dynamics
32
New integrate micro effects
aggregate FAST rate
New model
Can recover all previous link models
33
New model tracks dynamics
34
Some implications

• Interaction of paced and un-paced flows
• FAST stability depends on heterogeneity of RTTs,
  not their particular values
• Important in applications where dynamics is
  persistent
• Single node analysis, FAST algorithm only

35
Outline

• From theory to deployment
• Theory
• Deployment commercialization
• Trends and opportunity
• Customer data
• Remarks

36
Trend 1

• Exploding need to deliver large content
• Video, software, games, business info

37
Trend 1 Online content is exploding

• CAGR 2005 2011 36
• Google worldwide 46PB/month
• Library of Congress 0.136PB

38
Trend 1 Video is exploding
39
Trend 1 Multi-year growth

• Only 10 of digital content is currently online

40
Trend 2

• Exploding need to deliver large content
• Video, software, games, business info
• Infrastructure growth to support delivery need

41
Trend 2 Broadband penetration

• Global broadband penetration is accelerating
• 11 million new subscribers/month globally
• 83 of US home Internet access is broadband by
  June 2007

42
Trend 3

• Exploding need to deliver large content
• Video, software, games, business info
• Infrastructure growth to support delivery need
• Centralize IT to reduce costs of management,
  space, power, cooling
• Exacerbated by virtualization of infrastructure
  and personalization of content

43
Trend 3 Centralization

• Power cooling cost is escalating, fueling
  centralization
• CAGR (2005-10) power cooling 11.2, new
  server spend 2.7
• In 2005, 1,000 servers cost 3.8M to power cool
  in 4yrs 2 increase in electricity cost raises
  cost by 200K

Worldwide Expense (US 4.5B in 2006 EPA)
Source IDC Sept 2006
44
Trend 3 Centralization
45
Content Volume Global Transfer Increasing

• Exploding need to deliver large content
• Video, software, games, business info
• Infrastructure growth to support delivery need
• Centralize IT to reduce costs of management,
  space, power, cooling
• Exacerbated by virtualization of infrastructure
  and personalization of content
• Implication
• More large contents over longer distance
• Served from centralized data centers

You could only get that sustained rate if you
are delivering within 100 miles, due to the way
current Internet protocols work. Tom Leighton,
MIT/Akamai, Oct 2007
http//sramanamitra.com/2007/10/21/speeding-up-the
-internet-algorithms-guru-and-akamai-founder-tom-l
eighton-part-5/
46
Solution Approaches

• Protocol problems degrade performance of
  long-distance transfers
• Current approach reduce distance
• FastSoft approach fix protocol problems

47
Content Delivery Networks (CDNs)
Protocol limitations inherent in Internet give
rise to Content Delivery Networks
CDNs circumvent protocol problems by placing
servers all around the world
48
Alternative Solution FastSoft
FastSoft helps companies deliver content from
centralized data centers
Distance is no longer a constraint in IT
infrastructure design
49
Long-tailed content

• Content popularity follows Pareto Principle
• 20 of most popular content accounts for 80 of
  accesses

Popularity
Content (rank order)
50
Long-tailed content

• Content popularity follows Pareto Principle
• 20 of most popular content accounts for 80 of
  accesses

Popularity
FastSoft is the right solution
Content (rank order)
51
Long-tailed content

• Content popularity follows Pareto Principle
• 20 of most popular content accounts for 80 of
  accesses

Popularity
There are a lot more businesses that need
FastSoft than those that need CDN
Content (rank order)
52
Basic deployment
Router
FastSoft AriaTM
Servers
53
Commercial Deployment FAST in a box
Internet
FastSoft
(FAST)
Throuput LA ? Tokyo
Throuput San Fran ? MIT
32.5x
28.1x
21.8x
17.6x
6.3x
3.8x
1.8x
FAST avg 233Mbps
Reno avg 35Mbps
50x delays are common over DSL speed links
with Aria
without Aria
54
Extreme loss resilience
Without Aria thruput 1Mbps
With Aria thruput 120Mbps
Heavy packet loss in Sprint network Aria
increased throughput by 120x !
San Francisco ? New York June 3, 2007
55
Outline

• From theory to deployment
• Theory
• Deployment
• Remarks

56
(No Transcript)
57
Engineering
58
Academia Govt
59
Academia Govt
60
New Products Services
Idea
Funding
Talent
61
New Problems Opportunities
Idea
Funding
Talent
62
Market
Academia Govt
VC
Industry
63
Remarks

• Integration of theory, prototyping, experiment,
  infrastructure
• Industrial partnership like ACCESS is key
• Idea Technology Product Company
• Entrepreneurship takes a lot more
• Closing the loop achieves more

Theory
Experiment
64
Remarks

• Integration of theory, prototyping, experiment,
  infrastructure
• Industrial partnership like ACCESS is key
• Idea Technology Product Company
• Entrepreneurship takes a lot more
• Closing the loop achieves more

Academic research
Technology transfer
Theory
Experiment
65

• Always looking for great engineers