Network Architecture (R02) IP Multipath

1
Network Architecture (R02)IP Multipath Path
SelectionCC

• Jon Crowcroft,
• http//www.cl.cam.ac.uk/jac22
• http//www.cl.cam.ac.uk/teaching/1213/R02/

2
Multipath

• Could be useful load balancing
• When Traffic Matrix deviates from expected
• How to assign rates to alternate paths
• IP or Application Layer
• CDN, especially P2P (Torrent or Storm) already
  effectively multipath at App
• Current IP routing mainly only corner cases

3
Multipath IP Routing

• Simplest case is equal-cost multipath
• Can be seen as simple bonding technique
• Combines with multihoming/resilience
• For any metric, in an interdomain protocol, can
  do k-shortest paths
• Problem 1 is path metric bottleneck link
  capacity and round trip time are both important
• Problem 2 is BGP

4
This paper concentrates on rate/path problem

• Sidesteps the question of route computation for
  now.
• Starts off from the BitTorrent example
• Looks at a MPTCP/MPIP model in contrast
• Builds an convex optimisation style framework (as
  per previous Frank Kelly et al) F. P. Kelly and
  T. Voice. Stability of end-to-end algorithms for
  joint routing and rate control. ACM SIGCOMM
  Computer Communication Review, 35(2)512,
  2005.see
• So max utility subject to path constraints

5
BitTorrent behaviour

• Currently, Swarms choose a number of neighbours
  to fetch blocks of a file from, monitor the TCP
  rate achieved, drop the slowest and pick a new
  neighbour at random
• SeeM. Mitzenmacher, A. Richa, and R.
  Sitaraman.The power of two random choices A
  survey of the techniques and results. In P.
  Pardalos, S. Rajasekaran, , and J. Rolim,
  editors, Handbook of Randomized Computing, pages
  255312. 2001

6
IP versus Torrent models
7
Load balancingc.f. Valiant/data centers
8
Two different rate assignments

• TCP is well known to have a 1/RTT dependence in
  the long term throughput of a given (unipath)
  flow.
• So do they allow for this or not in the multipath
  framework? Choice
• Coordinate rates, dont factor in rtt
• Uncoordinated rates, factor in rtt
• See also TCP Friendly rate controlled transport
  protocol work by Handley et al

9
Capacity regions
10
Note on this version of paper

• This is the shorter, CACM version theres a MSR
  tech report and an Infocom version.
• In Cisco manuals, you can do Multipath BGP, but
  be aware this is mainly just for multihomeing an
  ISP on another (same motive as OSPF-ECM).
• The general problem is very hard, see
• Loop-freeness in multipath BGP through
  propagating the longest path, Van Beijnum,
  Iljitsch (2008) Loop-freeness in multipath BGP
  through propagating the longest path. Masters
  thesis, University Carlos III of Madrid, Madrid,
  Spain

11
Other missing architectural pieces

• How to indicate at a sender a packet from a
  coordinated flow belongs on a particular
  sub-path, in general (if the end system isnt
  multihomed)?
• How to tell at a receiver which subpath a packet
  arrived over?
• What about short lived flows?

12
Obvious deployment scenarios

• Smart phone with wifi 3G
• Data center networks

13
Reference/credit for diagrams

• _at_articleKey2011PSM1866739.1866762, author
  Key, Peter and Massouli\'e, Laurent and
  Towsley, Don, title Path selection and
  multipath congestion control, journal Commun.
  ACM, issue_date January 2011, volume 54,
  number 1, month jan, year 2011, issn
  0001-0782, pages 109--116, numpages 8,
  url http//doi.acm.org/10.1145/1866739.1866762
  , doi 10.1145/1866739.1866762, acmid
  1866762, publisher ACM, address New
  York, NY, USA,

14
Optimization-based routing and congestion control

• Routing, congestion control as optimization
  problems
• how to route flows, set flow rates to optimize an
  objective (cost) function
• routing and congestion control protocols as
  distributed asynchronous implementations of
  optimization algorithms
• systematic approach towards protocol design
• e.g., TCP as distributed rate optimization

15
Optimization Framework

• W set of source-destination (sd) pairs
• rw rate of sd pair w
• Pw set of paths between sd pair w
• xp packet flow rate (fluid) on path p
• cij link capacity of link i,j, (assume same as
  cji)

path 3, rate x3
path 4, rate x4
path 2, rate x2
c12
c23
c13
path 1, rate x1
16
Optimization Framework

• routing problem rw (rate of sd pair w)
  typically given
• question what rate xp on each path p
• rate control problem rw (rate of sd pair w)
  variable
• question what rate xp on each given path p
• single path or multiple paths between sd pair w

path 3, rate x3
path 4, rate x4
path 2, rate x2
c12
c23
c13
path 1, rate x1
17
Optimization Framework
Key question how to set rates on paths?
Input rates may be fixed (routing) or variable
(rate control)

• routing problem rw (rate of sd pair w)
  typically fixed
• question what rate xp on each path p
• rate control problem rw (rate of sd pair w)
  variable
• question what rate xp on each given path p
• single path or multiple paths between sd pair w

path 3, rate x3
path 4, rate x4
path 2, rate x2
c12
c23
c13
path 1, rate x1
18
Optimization Framework

• lookout for where are path rates set
• centrally global computation
• at endpoints distributed algorithm with multiple
  endpoints at network edge
• at routers distributed algorithm with multplie
  routers within network

path 3, rate x3
path 4, rate x4
path 2, rate x2
c12
c23
c13
path 1, rate x1
19
Optimization Framework

• lookout for what cost function is being
  optimized? Typically
• minimize system-wide delay with variable routing
  given fixed sd traffic rates rw
• maximize system-wide utility, SwUw(rw), with
  variable traffic rates rw given fixed paths

path 3, rate x3
path 4, rate x4
path 2, rate x2
c12
c23
c13
path 1, rate x1
20
Optimization Framework

• lookout for how are capacity constraints taken
  into account

path 3, rate x3
path 4, rate x4
path 2, rate x2
c12
c23
c13
path 1, rate x1
21
Optimization-based routing

• read Gallagher 1992 sec 5.4 intro, 5.5, 5.6,
  Gallagher 1977.

W set of source-destination (sd) pairs rw fixed
rate of sd pair w (traffic to be routed) Pw set
of paths between sd pair w xp packet flow rate
(fluid) on path p cij link capacity of link
i,j, (assume same as cji)
22
Optimization-based routing
S
Dij(Fij)

minimize
all links ij
23
Optimization-based routing
equivalently
S
Dij
minimize
all links ij
This is the routing optimization problem how to
choose xp to minimize cost function above.
24
How to solve optimal routing problem?
Insight looking at properties of optimal routing
solution suggests algorithms (protocols) for
finding optimum!
Suppose move small amount flow d from p (with
non-zero xp at x) to path p between same OD
pair. Then
otherwise xp in x would not be optimal
gt
d
d
25
How to solve optimal routing problem?
Insight looking at properties of optimal routing
solution suggests algorithms (protocols) for
finding optimum!
At the optimum
for all non-zero xp1, xp2 in Pw, for all w

set flow rate on alternative path so partial
derivatives (marginal utilities) are equal
26
This suggests an algorithm!
For each sd pair, w

for all kw paths for sd pair w
evaluate
move small amount of flow to path with minimum
marginal increase from other kw-1 paths for sd
pair w in attempt to balance marginal utilities
on paths with non-zero flows
until for each sd pair, marginal utilities equal
for all paths with non-zero flow for that sd
pair for all sd pairs
27
A numerical example
c2 1 Mbps
c1 1 Mbps
1.6
1.1
28
Routing optimization hill descent

• key idea iteratively evaluate marginal path
  costs (gradient descent)
• various algorithms to determine which flows can
  accept a little more flow, which should give up
  a little flow , subject to
• maintaining flow conservation
• respecting capacity constraints
• maintaining loop freedom

29
Optimization-based approach towards congestion
control

• Resource allocation as optimization problem
• how to allocate resources (e.g., bandwidth) to
  optimize some objective function
• maybe not possible that optimality exactly
  obtained but
• optimization framework as means to explicitly
  steer network towards desirable operating point
• practical congestion control as distributed
  asynchronous implementations of optimization
  algorithm
• systematic approach towards protocol design

30
Model

• Network Links l each of capacity cl
• Sources s (L(s), Us(xs))
• L(s) - links used by source s
• Us(xs) - utility if source rate xs

example utility function for elastic application
Us(xs)
xs
x1
c1
c2
x2
x3
31
Optimization Problem
system problem

• maximize system utility (note all sources
  equal)
• constraint bandwidth used less than capacity
• centralized solution to optimization impractical
• must know all utility functions
• impractical for large number of sources
• well see congestion control as distributed
  asynchronous algorithms to solve this problem

32
The user view

• user can choose amount to pay per unit time, ws
• Would like allocated bandwidth, xs in proportion
  to ws

ps could be viewed as charge per unit flow for
user s
user problem
33
The network view

• suppose network knows vector ws, chosen by
  users
• network wants to maximize logarithmic utility
  function

network problem
34
Solution existence

• There exist prices, ps, source rates, xs, and
  amount-to-pay-per-unit-time, ws psxs such that
• Ws solves user problem
• Xs solves the network problem
• Xs is the unique solution to the system problem

35
Proportional Fairness

• Vector of rates, Xs, proportionally fair if
  feasible and for any other feasible vector Xs

result if wr1, then Xs solves the network
problem IFF it is proportionally fair Related
results exist for the case that wr not equal 1.
36
Solving the network problem

• Results so far existence - solution exists with
  given properties
• How to compute solution?
• ideally distributed solution easily embodied in
  protocol
• insight into existing protocol

37
Solving the network problem
linear increase
multiplicative decrease
change in bandwidth allocation at s
where

• congestion signal function of aggregate rate
  at link l, fed back to s.

38
Solving the network problem

• Results
• converges to solution of relaxation of
  network problem
• xs(t)Spl(t) converges to ws
• Interpretation TCP-like algorithm to iteratively
  solve optimal rate allocation!

39
Optimization-based congestion control summary

• bandwidth allocation as optimization problem
• practical congestion control (TCP) as distributed
  asynchronous implementations of optimization
  algorithm
• optimization framework as means to explicitly
  steer network towards desirable operating point
• systematic approach towards protocol design

40
Motivation
Congestion Control maximize user utility
Traffic Engineering minimize network congestion
Given routing Rli how to adapt end rate xi?
Given traffic xi how to perform routing Rli?
41
Congestion Control Model
Users are indexed by i
aggregate utility
Utility Ui(xi)
max. ? i Ui(xi) s.t. ?i Rlixi cl var. x
capacity constraints
Source rate xi
Congestion control provides fair rate allocation
amongst users
42
Traffic Engineering Model
Links are indexed by l
aggregate cost
Cost f(ul)
ul 1
min. ?l f(ul) s.t. ul ?i Rlixi/cl var. R
Link Utilization ul
Traffic engineering avoids bottlenecks in the
network
43
Model of Internet Reality
Congestion Control max ?i Ui(xi), s.t. ?i Rlixi
cl
xi
Rli
Traffic Engineering min ?l f(ul), s.t. ul ?i
Rlixi/cl
44
System Properties

• Convergence
• Does it achieve some objective?
• Benchmark
• Utility gap between the joint system and benchmark

max. ?i Ui(xi) s.t. Rx c Var. x, R
45
Numerical Experiments

• System converges
• Quantify the gap to optimal aggregate utility
• Capacity distribution truncated Gaussian with
  average 100
• 500 points per standard deviation

Access-Core
Abilene Internet2 (a research net)
46
Results for Abilene f eu_l
Aggregate utility gap
Gap exists
Standard deviation
47
Backward Compatible Design

• Simulation of the joint system suggests that it
  is stable, but suboptimal
• Gap reduced if we modify f

f(ul)
Cost f
f(ul)
ul 1
0
Link load ul
48
Abilene Continued f n(ul)n
Aggregate utility gap
n
Gap shrinks with larger n
49
Theoretical Results

• Modify congestion control to approximate the
  capacity constraint with a penalty function
• Theorem modified joint system model converges if
  Ui(xi) -Ui(xi) /xi

Master Problem min. g(x,R) - ?iUi(xi)
??lf(ul)
Congestion Control argminx g(x,R)
Traffic Engineering argminR g(x,R)