Internet Routing (COS 598A) Today: Interdomain Traffic Engineering

1
Internet Routing (COS 598A)Today Interdomain
Traffic Engineering

• Jennifer Rexford
• http//www.cs.princeton.edu/jrex/teaching/spring2
  005
• Tuesdays/Thursdays 1100am-1220pm

2
Outline

• Border Gateway Protocol
• BGP protocol
• BGP policies
• BGP decision process
• BGP traffic engineering for outbound traffic
• Predicting effects of policy changes
• Identifying good policy changes
• What about inbound traffic?
• AS prepending and MEDs
• Inter-AS negotiation

3
Interdomain Routing Border Gateway Protocol

• ASes exchange info about who they can reach
• IP prefix block of destination IP addresses
• AS path sequence of ASes along the path
• Policies configured by the ASs operator
• Path selection which of the paths to use?
• Path export which neighbors to tell?

1
data traffic
data traffic
12.34.158.5
4
Components of BGP

• BGP protocol
• Definition of how two BGP neighbors communicate
• Message formats, state machine, route attributes,
  etc.
• Standardized by the IETF
• Policy specification
• Flexible language for filtering and manipulating
  routes
• Indirectly affects the selection of the best
  route
• Varies across vendors, though constructs are
  similar
• BGP decision process
• Complex sequence of rules for selecting the best
  route
• De facto standard applied by router vendors
• Being codified in a new RFC for BGP coming soon

5
BGP Protocol BGP Sessions
Establish session on TCP port 179
AS1
BGP session
Exchange all active routes
AS2
While connection is ALIVE exchange route UPDATE
messages
Exchange incremental updates
6
BGP Protocol Update Messages

• Update messages
• Advertisement
• New route for the prefix (e.g., 12.34.158.0/24)
• Attributes such as the AS path (e.g., 2 1)
• Withdrawal
• Announcing that the route is no longer available
• Numerous BGP attributes
• AS path
• Next-hop IP address
• Local preference
• Multiple-Exit Discriminator

7
BGP Policy Applying Policy to Routes

• Import policy
• Filter unwanted routes from neighbor
• E.g. prefix that your customer doesnt own
• Manipulate attributes to influence path selection
• E.g., assign local preference to favored routes
• Export policy
• Filter routes you dont want to tell your
  neighbor
• E.g., dont tell a peer a route learned from
  other peer
• Manipulate attributes to control what they see
• E.g., make a path look artificially longer than
  it is

8
BGP Policy Influencing Decisions
Open ended programming. Constrain
ed only by vendor configuration language
Apply Policy filter routes tweak attributes
Apply Policy filter routes tweak attributes
Receive BGP Updates
Best Routes
Transmit BGP Updates
Based on Attribute Values
Best Route Selection
Apply Import Policies
Best Route Table
Apply Export Policies
Install forwarding Entries for best Routes.
IP Forwarding Table
9
BGP Decision Process Path Selection on a Router

• Routing Information Base
• Store all BGP routes for each destination prefix
• Withdrawal message remove the route entry
• Advertisement message update the route entry
• Selecting the best route
• Consider all BGP routes for the prefix
• Apply rules for comparing the routes
• Select the one best route
• Use this route in the forwarding table
• Send this route to neighbors

10
BGP Decision Process Multiple Steps

• Highest local preference
• Set by import policies upon receiving
  advertisement
• Shortest AS path
• Included in the route advertisement
• Lowest origin type
• Included in advertisement or reset by import
  policy
• Smallest multiple exit discriminator
• Included in the advertisement or reset by import
  policy
• Smallest internal path cost to the next hop
• Based on intradomain routing protocol (e.g.,
  OSPF)
• Smallest next-hop router id
• Final tie-break

11
BGP Decision Process in Action
But, what if the path (3,4,1) would be better?
12
Manipulating Policy to Move the Traffic

• Assign local preference to
• Prefer one neighbor over another for a prefix
• Prefer certain AS paths over others
• Router configuration languages
• Specifying rules for setting local-pref attribute
• if path(3, , 1), then local-pref110
• else, local-pref100
• Allow policy to over-ride shortest AS path
• Indirect way of making one path look better or
  worse than another
• Main way to do BGP traffic engineering today

13
BGP Traffic Engineering
14
BGP Traffic Engineering

• Limitations of intradomain traffic engineering
• Alleviating congestion on edge links
• Making use of new or upgraded edge links
• Influencing choice of end-to-end path
• Extra flexibility by allowing changes to BGP
  policies
• Direct traffic toward/from certain edge links
• Change the set of egress links for a destination

2
4
1
3
15
BGP Modeling for Traffic Engineering

• Predict effects of changes to import policies
• Inputs routing, traffic, and configuration data
• Outputs flow of traffic through the network

BGP policy configuration
Topology
BGP routing model
eBGP routes
Offered traffic
Flow of traffic through the network
16
Steps in the Model

• Effects of import policy on the routes
• Inputs BGP routes, and import policies
• Output BGP routes modified by policies
• Set of best routes for the network
• Inputs BGP routes modified by policies
• Output set of best routes (max local-pref, min
  AS path,)
• Best route per router
• Inputs set of BGP routes, and intradomain costs
• Outputs best route (closest egress) per prefix
  per router
• Link-level paths
• Inputs best route per router, and intradomain
  costs
• Outputs link-level shortest path(s) from entry
  to egress

17
But, Hard to Select Good Import Policy Changes

• Cant enumerate all choices
• Many eBGP sessions
• Around 100K prefixes
• Highly programmable policies
• Risk of creating side-effects
• Overhead of BGP routing changes
• Unpredictable behavior of others
• Vulnerability to changes
• New eBGP routes from neighbors
• Shifts in the traffic volume per prefix

18
Traffic Engineering Guidelines

• Predictability
• Ensure the BGP decision process is deterministic
• Assume that BGP updates are (relatively) stable
• Outbound traffic (import policy, local
  preference)
• Easier to control how traffic leaves the network
• Cooperate with neighbor ASes for inbound traffic
• Limit overhead introduced by routing changes
• Minimize frequency of changes to routing policies
• Limit number of prefixes affected by changes
• Limit impact on how traffic enters the network
• Avoid new routes that might change neighbors
  mind
• Select route with same attributes, or at least
  path length

19
Measurement Data

• Analysis of peering links
• Links connecting ATT to other large providers
• Relatively small of high-end routers
• BGP routing tables
• Log daily output of show ip bgp command
• Extract BGP routes for each prefix
• Cisco Netflow
• Collect continuous flow-level measurement
• Aggregate the traffic by prefix
• Focus on outbound traffic to peers

20
Controlling the Scale

• Destination prefixes
• More than 90,000 destination prefixes
• Dont want to have per-prefix routing policies
• Small fraction of prefixes contribute most of the
  traffic
• Focus on the small number of heavy hitters
• Define routing policies for selected prefixes
• Routing choices
• About 27,000 unique routing choices
• Help in reducing the scale of the problem
• Small fraction of routing choices contribute
  most traffic
• Focus on the very small number of routing
  choices
• Define routing policies on common attributes

21
Avoid Impacting Downstream Neighbors
Will traffic volume change???
22
Predictable Routing Change

• Predictability
• Do not change the route sent to downstream
  neighbor
• Focus on prefixes where all best routes are
  identical
• Neighbors do not even receive a new BGP
  advertisement!
• Example application
• Multiple links to same peer, with one congested
  link
• Assign lower local-pref at that link for some
  prefixes
• Empirical results
• 83.5 of prefixes have shortest AS paths that are
  all identical (same next-hop AS, same AS path,
  etc.)
• These prefixes are responsible for 45 of the
  traffic
• Plenty of scope to move traffic in a predictable
  fashion

23
Semi-Predictable Routing Change

• Semi-predictability
• Do not change the length of the AS path sent to
  neighbor
• Neighbors receive new advertisement with same
  length
• (Hopefully) they still make the same routing
  decision
• Example application
• Need to move some traffic from one peer to
  another
• Find prefixes with best paths via both
  neighbors
• Assign lower local-pref at some links for some
  prefixes
• Empirical results
• 10-15 of prefixes have shortest paths with 2
  next hops
• These prefixes contribute 35-40 of the traffic
• Potential to move traffic in a semi-predictable
  fashion

24
Influence of AS Path Length

• AS path length
• Plays a significant role in the BGP decision
  process
• All best routes must have the same AS path
  length
• 10 of prefixes have choices with different path
  lengths
• An idea a more flexible approach
• Possible to disable consideration of AS path
  length, and incorporate AS path length in
  local-pref assignment
• E.g., treat paths of length 3 and 4 as equally
  good
• AS prepending by other ASes
• Inflating AS path length by adding fake hops
• E.g., 701 80 80 80 instead of 701 80
• 18 of routes had some form of AS prepending

25
Inbound Traffic
26
Why Inbound Traffic is Hard to Manage

• Other ASes decide how to send to you
• Destination-based routing
• Other ASes decide which path to take
• Based on their own policies

2
p
4
1
3
AS 2 doesnt know how AS 1 will send traffic
toward p
27
AS Prepending

• Artificial inflate AS path length
• Prepend your own AS in the path
• E.g., turn 3 4 5 into 3 3 3 4 5
• Hope to make the path less attractive

3 4 5
1
3
3 3 3 4 5
28
Multiple Exit Discriminator (MED)

• Tell your neighbor what you want
• MED attribute to indicate receiver preference
• Decision process picks route with smallest MED
• Can use MED for cold potato routing
• But, have to get your neighbor to accept MEDs

3 4 5 with MED1
1
3
3 4 5 with MED2
29
Inter-AS Negotiation

• Better to cooperate?
• Negotiate where to send
• Inbound and outbound
• Mutual benefits
• But, how to do it?
• What info to exchange?
• How to prioritize the many choices?
• How prevent cheating?
• Open research territory

Customer B
Provider B
multiple peering points
Early-exit routing
Provider A
Customer A
30
Next Time Multi-Homed Customers

• Two papers
• A Measurement-Based Analysis of Multi-homing
  (SIGCOMM03)
• Optimizing Cost and Performance for Multihoming
  (SIGCOMM04)
• Reviews of both papers
• Brief summary of the paper
• Reasons to accept the paper
• Reasons to reject the paper
• Suggestions for future research directions