Interview talk at various universities and labs

1
TeXCP Protecting Providers Networks from
Unexpected Failures Traffic Spikes
Dina Katabi MIT - CSAIL dk_at_mit.edu nms.csail.mit
.edu/dina
2
Crash Course in XCP
TeXCP is TE with an XCP-Like Protocol
3
Problem Addressed by XCP
TCP has trouble providing a few Gb/s per-flow
throughput
4
Need to increase faster
Gigabit Links
5
? Need Explicit Feedback
Need to increase faster
How much faster?
Kb/s Link
6
? Need Explicit Feedback
Need to increase faster
How much faster?
Need to coordinate connections!
Scalability Constraint Routers should not
maintain per-connection state
7
XCP
8
How does XCP Work?
Feedback 1 packet/sec
9
How does XCP Work?
Feedback - 3 packet/sec
10
How does XCP Work?
Rate Rate Feedback
Explicit Feedback Make senders react according to
the amount of spare capacity
11
How Does an XCP Router Compute the Feedback?

• Efficiency Controller
• Fairness Controller

Router makes decisions every control interval D
12
How Does an XCP Router Compute the Feedback?
Efficiency Controller
Fairness Controller
13
Getting the devil out of the details
Efficiency Controller
Fairness Controller
No Per-Flow State
No Parameter Tuning
14
XCP Remains Efficient as Bandwidth or Delay
Increases
Utilization vs. Delay
Utilization vs. Capacity
Avg. Utilization
Avg. Utilization
Bottleneck Capacity (Mb/s)
Round Trip Delay (sec)
15
TeXCP Intra-Domain Traffic Engineering with XCP
with Srikanth Kandula, Asfandyar Qureshi, Shan
Sinha
16
What is the problem?
17
What is the problem?
Traffic Engineering routes the traffic demands of
IE pairs to achieve good network performance?
18
Good performance means

• Network is robust to unexpected events
• Link failures
• Traffic Spikes
• Support as much demands as possible given
  capacities

19
To minimize Max U ? Multi paths
Two issues - How much traffic to put on each
path? - How does TCP interact with multi-path
routing?
Two issues - How much traffic to put on each
path? - How does TCP interact with multipath
routing?
20
How much traffic to put on each path?

• Repeatedly
• Measure the utilization of all paths between L.A.
  and Boston
• Move some traffic from the highly utilized path
  to the underutilized path

Do it in the network!
21
How to divide traffic between available paths to
minimize max U?

• Simplistic solution give all information to a
  centralized computer and solve it as a linear
  programming problem
• But that wouldnt be realtime and wouldnt react
  to failures and attack traffic

Need a distributed, in network, realtime solution
22
Feedback Delays
Challenge 1
23
Feedback Delays
Challenge 1
Utilization?
Boston
L.A.
Utilization feedback might be obsolete
24
Coordination Without Global Knowledge
Challenge 2
25
Coordination Without Global Knowledge
Challenge 2
Boston
L.A.
NYC
SF
26
Coordination Without Global Knowledge
Challenge 2
Boston
L.A.
NYC
SF
Actions of uncoordinated ingress nodes might
cause undesirable effect
27
Use experience from congestion control
Solution Idea

• Congestion control TE are close
• CC single path want 100 utilization
• TE multi-paths want balanced utilization
• XCP tells us how fast each path can change its
  utilization so that system is stable despite
  delay
• XCP tells us how to coordinate multiple senders
  who share the same path

TeXCP traffic engineering with an XCP-like
protocol
28
TeXCP

• A TeXCP agent per IE, at ingress node

TeXCP Agent

• TeXCP agent knows the paths between IE, which are
  computed offline
• Paths are pinned

29
TeXCP Agent

• TeXCP divides the IE traffic between available
  paths to Minimize the Max U?
• A TeXCP agent 3 components
• Probe path utilization
• Load balancer
• Per-path XCP controller

30
Component 1
Probe Path Utilization

• Periodically, send probes on each path to learn
  Max utilization along the path

U1 0.4 U2 0.7
Egress 1
Ingress 1
x
Probes are sent along the slow path like ICMP
messages ? implementation is easy
31
Component 2
Load Balancer

• Objective Balance utilization across IE paths
• How? move traffic from overutilized paths to
  under-utilized paths
• Continuously estimate demands L(t)
• Assume xp is fraction of traffic that should be
  sent on path p
• Periodically, update xp
• Where rp is the sending rate on path p and up is
  its utilization

32
Component 3
Per-Path XCP Controller

• Load Balancer tells us how much to change
  utilization, but
• Cant increase/decrease immediately because of
  delays
• Need to coordinate the increase/decrease from all
  ingress nodes
• Run a lightweight XCP on each path
• Replace congestion header with probes along the
  slow path
• Change the slow path in core routers to answer
  probes

33
TeXCP Performance
34
Simple Topology

• Traffic demands per IE pair are modeled using
  100 Pareto on-off sources, or Poisson arrivals
• Also, 50 of the traffic is uncontrollable by
  TeXCP

35
TeXCP balances the load across available paths,
and reacts to demands change in realtime
Started at t 0 with Load1 1.3 Load20.8 Load3
0.8
Utilization
Decrease in Uncontrollable Traffic
Time (sec)
36
TeXCP is More Accurate than Previous Approaches
TeXCP
MATE
Utilization
Time (sec)
Time (sec)
37
Real Traffic and Provider Topology
38
TeXCP Improves Robustness Against Link Failures
39
TeXCP Improves Robustness Against Link Failures
Without TeXCP
Max Utilization
Max Utilization
TeXCP prevents congestion caused by link failures
40
When multiple solutions results in the same Max
U, TeXCP prefers solutions with shorter paths
41
TeXCP Prefers Shorter Paths
Ingress1
Ingress2
TeXCP prefers low delay paths while balancing
utilization
42
How about TCP?

• Multi-paths must NOT reorder TCP packets

Problem Can you take traffic at any backbone
router and accurately split it between multiple
paths without reordering TCP packets?
43
Simplistic Solution

• Assign TCP flows to each path proportionally to
  the desirable split
• Flows are not all equal Elephants Mice
• So, estimate the rate of each TCP
• But rates change over with time
• Too complex

44
Our Approach
1
2

• If pkt enters network after the previous pkt from
  the flow has left ? We can reassign the flow to a
  different path
• Use UDP traffic to correct for errors

45
Our Approach

• Each TeXCP agent keeps a Hash Table
• If (now-last_seen) gt Max delay, we can change the
  assigned path
• Assign a TCP flow to a new path with a
  probability equals to the fraction of traffic you
  want on this path

46

• OC12 traffic
• Split it between 2 paths, with desirable
  splitting changing with time as a sinusoidal wave

47

• OC12 traffic
• Split it between 2 paths, with desirable
  splitting changing with time as a sinusoidal wave

Path1 Fraction Path2 Fraction Path1 UDP Path2 UDP
48
Very, Very Cheap. Edge routers maintain a hash
table of 210 entries (10KB).
49
Conclusion

• TeXCP
• Adaptive multipath routing protocol
• Makes the network more robust against link
  failures and traffic spikes
• Note TeXCP does not assume XCP
• Multipath routing can be easily and effectively
  implemented without causing TCP packet reordering

50
More Information athttp//nms.lcs.mit.edu/dina/
texcp.html
Questions?
51
TeXCP Reacts in Real-Time to Traffic Spikes
52
Comparison with MATE

• Mate
• Minimize delay
• Conservative increase

53
Comparison with MATE
MATE
TeXCP
Link load / capacity
Link load / capacity
Time (sec)
Time (sec)
TeXCPs utilization is more balanced Avg. drop
rate in MATE is 2 while in TeXCP is 0
54
XCP Deals Well with Short Web-Like Flows
Average Utilization
Average Queue
Drops
Arrivals of Short Flows/sec
55
Efficiency Controller
Fairness Controller
No Per-Connection State
Stability Properties