Internet Routing COS 598A Today: MultiHoming

1
Internet Routing (COS 598A)Today Multi-Homing

• Jennifer Rexford
• http//www.cs.princeton.edu/jrex/teaching/spring2
  005
• Tuesdays/Thursdays 1100am-1220pm

2
Outline

• Multi-homing
• Motivations reliability, performance, and
  financial
• Do you really need to use a routing protocol?
• Controlling outbound traffic
• Shortest-path routing
• Primary and backup providers
• Load balancing over multiple links
• Controlling inbound traffic
• Primary and backup providers
• Selective advertising
• BGP communities
• State-of-the-art today

3
Why Connect to Multiple Providers?

• Reliability
• Reduced fate sharing
• Survive ISP failure
• Performance
• Multiple paths
• Select the best
• Financial
• Leverage through competition
• Game 95th-percentile billing model

Provider 2
Provider 1
4
The Stub AS Doesnt Need to Speak BGP

• Sending traffic
• Assume both providers can reach everyone
• Split traffic however you want (e.g., 50/50)
• But what if a provider cant reach someone?
• But what if one provider has a better path?

Provider 1
Provider 2
One static route
L1
L2
0.0.0.0/0 ? L1, L2
5
The Stub AS Doesnt Need to Speak BGP

• Receiving traffic
• Both providers can announce the prefix into BGP
• Ensures that everyone else can reach you
• But what if traffic load is very uneven?

Advertise 12.34.158.0/24
Provider 1
Provider 2
traffic
traffic
12.34.158.0/24
6
Controlling Outbound Traffic
7
Outbound Traffic Pick a BGP Route

• Easier to control than inbound traffic
• IP routing is destination based
• Sender determines where the packets go
• Control only by selecting the next hop
• Border router can pick the next-hop AS
• Cannot control selection of the entire path

Provider 1
Provider 2
(1, 3, 4)
(2, 7, 8, 4)
8
Outbound Traffic Shortest AS Path

• No import policy on border router
• Pick route with shortest AS path
• Arbitrary tie break (e.g., smallest router-id)
• Performance?
• Shortest AS path is not necessarily best
• Could have long propagation delay or congestion
• Load balancing?
• Could lead to uneven split in traffic
• E.g., one provider with shorter paths
• E.g., too many ties with skewed tie-break

9
Outbound Traffic Primary and Backup

• Single policy for all prefixes
• High local-pref for session to primary provider
• Low load-pref for session to backup provider
• Outcome of BGP decision process
• Choose the primary provider whenever possible
• Use the backup provider when necessary
• But
• What if you want to balance traffic load?
• What if you want to select better paths?

10
Outbound Traffic Load Balancing

• Selectively use each provider
• Assign local-pref across destination prefixes
• Change the local-pref assignments over time
• Useful inputs to load balancing
• End-to-end path performance data
• E.g., active measurements along each path
• Outbound traffic statistics per destination
  prefix
• E.g., packet monitors or router-level support
• Link capacity to each provider
• Billing model of each provider

11
Outbound Traffic Load, Performance, and Cost

• Balance traffic based on link capacity
• Measure outbound traffic per prefix
• Select provider per prefix for even load
  splitting
• But, might lead to poor performance and high bill
• Balance traffic based on performance
• Select provider with best performance per prefix
• But, might lead to congestion and a high bill
• Balance traffic based on financial cost
• Select provider per prefix over time to minimize
  the total financial cost
• But, might lead to bad performance

12
Outbound Traffic What Kind of Probing?

• Lots of options
• HTTP transfer
• UDP traffic
• TCP traffic
• Traceroute
• Ping
• Pros and cons for each
• Accuracy
• Overhead
• Dropped by routers
• Sets off intrusion detection systems

13
Outbound Traffic Getting Probes on Paths

• Problem
• Router selects one path per prefix
• How to measure the alternate paths?
• Solution 1 special sources (source routing)
• Special IP addresses for probe traffic
• Router configured to forward probe traffic
• Solution 2 special destinations
• Special destination servers in various locations
• At least one destination per provider AS
• Probe traffic sent to each destination

14
Outbound Traffic How Much Probing?

• How often?
• Continuously, at some rate
• In response to a perceived problem
• How diverse of destinations?
• Per destination prefix
• Just for popular/important prefixes
• Select servers throughout the Internet

15
Outbound Traffic How Often to Change Routes?

• ASes with downstream customers
• Each change leads to BGP updates
• If not, then no new BGP updates occur
• TCP flows that switch paths
• Out-of-order packets during transition
• Change in round-trip-time (RTT)
• Impact on the providers
• Uncertainty in the offered load
• Interaction with their own traffic engineering?
• Impact on other end users
• Good move traffic off of congested paths
• Bad potential oscillation as stub ASes adapt?

16
Controlling Inbound Traffic
17
Inbound Traffic Influencing What Others Do

• Harder to control than outbound traffic
• IP routing is destination based
• Sender determines where the packets go
• Control only by influencing others decisions
• Static configuration of the providers
• BGP route attributes sent by the stub
• Selective advertising of destination prefixes

Provider 1
Provider 2
18
Inbound Traffic Primary and Backup Providers

• Ask your provider to be a backup
• Provider violates prefer customer policy
• by assigning lower local-pref to customer
• Backup link is only used if the primary link fails

Provider 1
Provider 2
12.34.158.0/24
19
Inbound Traffic AS Prepending

• Make one path look longer
• Advertise short path one way
• and longer path another
• In the hope of influencing choices
• But, how much prepending to do?

Provider 1
Provider 2
12.34.158.024 (3, 3, 3)
12.34.158.024 (3)
20
Inbound Traffic Prepending and Prefer-Cust

• Example where prepending doesnt work
• Customer does prepending of AS path
• Provider has a prefer customer policy
• Provider 2 prefers the longer path

12.34.158.024 (1, 3)
Provider 1
Provider 2
12.34.158.024 (3, 3, 3)
12.34.158.024 (3)
21
Inbound Traffic Programming Your Provider

• Better to have selective control over provider
• Tell the provider whether to prefer your route
• on a per-prefix basis, with changes over time
• Enables adaptive load balancing
• without asking provider to reconfigure policy

Provider 1
Provider 2
12.34.158.0/24
22
Inbound Traffic RFC 1998 on BGP Communities

• Provider and customer agree on a tag
• One tag mean primary and the other backup
• Customer includes tags in BGP advertisements
• Provider sets local preference based on tags
• BGP community attribute
• Opaque attribute with no real meaning
• Two numbers usually AS number and arbitrary
  number
• Sprint example (http//www.sprint.net/policy/bgp.h
  tml)
• 123970 means assign local pref of 70
• 1239110 means assign local pref of 110

23
Example Tier-1 ISP Setting Local-Preference

• Customers
• 110 Primary path
• 100 Secondary path
• 80 Primary backup path
• 70 Secondary backup path
• Peers
• 81-99 In between
• Range for traffic engineering

Peer
Customer
24
Inbound Traffic Not Enough Prefixes

• Stub ASes usually have only a few prefixes
• E.g., one prefix, or at most a handful
• Not enough granularity to control traffic
• Solutions advertise smaller subnets of prefix
• Essentially, create a bunch of smaller prefixes
• And apply the load-balancing techniques
• Advertise selectively
• AS prepending
• Communities to set local-pref

25
Inbound Traffic Selective Advertising

• Divide the destination prefix
• Advertise one subnet to each provider
• Advertise the supernet to both providers
• Traffic splits due to the longest-prefix match
• Supernet ensures backup connectivity after
  failure

Provider 1
Provider 2
12.34.0.0/16 12.34.0.0/17
12.34.0.0/16 12.34.128.0/17
26
Inbound Traffic Small Subnets, Big Debate

• The players
• Stub ASes want more control
• Advertise smaller subnets
• ISPs want to limit table size
• Filter BGP advertisements for small blocks
• ARIN/RIPE/APNIC
• Publish guidelines for acceptable block sizes
• Problems
• ISPs not getting paid for their routing tables
• Risk of network crashes when memory is full
• Risk of black-holing a small subnet you filter

27
Project Ideas

• Intelligent route-control techniques
• Survey approaches to measuring performance
• Evaluation of different measurement approaches
• Techniques for controlling inbound traffic
• Negotiation scheme between ASes
• Economic approaches for balancing the tension
  between fine-grain control and table size
• Source routing
• Scalable techniques for stub AS to pick the
  end-to-end route (not just the next-hop AS)

28
Next Class Convergence Delay

• Two papers, intradomain and interdomain
• Analysis of Link Failures in an IP Backbone
• Delayed Internet Routing Convergence
• Reviews of both papers
• Summary
• Why accept?
• Why reject?
• New research directions
• Optional NANOG video
• Toward Millisecond IGP Convergence