Stable Internet Routing Without Global Coordination

1
Stable Internet Routing Without Global
Coordination

• Jennifer Rexford
• Princeton University
• http//www.cs.princeton.edu/jrex

Joint work with Lixin Gao, Michael Schapira, and
Yi Wang
2
What is an Internet?

• A network of networks
• Networks run by different institutions
• Autonomous System (AS)
• Collection of routers run by a single institution
• ASes have different goals
• Different views of which paths are good
• Interdomain routing is what reconciles those
  views
• To compute end-to-end paths through the Internet

Wonderful problem setting for game theory and
mechanism design
3
An Open Question
Evolvable Protocols (under-specified,
programmable)
?
Autonomy (autonomous parties, with different
economic objectives)
Global Properties (stability, scalability,
reliability, security, managability, )
Can we have all three? Under what conditions?
4
Autonomous Systems (ASes)
Path 6, 5, 4, 3, 2, 1
4
3
5
2
6
7
1
Web server
Client
5
Interdomain Routing Border Gateway Protocol

• ASes exchange info about who they can reach
• Destination block of IP addresses (an IP
  prefix)
• AS path sequence of ASes along the path
• Policies configured by the ASs network operator
• Path selection which of the paths to use?
• Path export which neighbors to tell?

I can reach d via AS 1
I can reach d
1
2
3
data traffic
data traffic
d
6
Interdomain Routing Convergence Challenges

• Must scale
• Address blocks 300,000 and growing
• Autonomous Systems around 35,000
• Must support flexible policy
• Path selection which path your AS wants to use
• Path export who can send packets through your AS
• Must converge, and quickly
• Routing convergence can take several minutes
• and the system doesnt necessarily converge at
  all!

Goal Guaranteed convergence of the global
routing system with purely local control.
7
Stable Paths Problem (SPP) Model

• Model of routing policy
• Each AS has a ranking of the permissible paths
• Model of path selection
• Pick the highest-ranked path consistent with
  neighbors
• Flexibility is not free
• Global system converges slowly, or not at all
• Depending on the way the ASes rank their paths

8
Conflicting Policies Cause Convergence Problems
1 2 0 1 0
1
0
2 3 0 2 0
3 1 0 3 0
3
2
Pick the highest-ranked path consistent with your
neighbors choices.
9
Global Control is Not Workable

• Create a global Internet routing registry
• Keeping the registry up-to-date would be
  difficult
• Require each AS to publish its routing policies
• ASes may be unwilling to reveal BGP policies
• Check for conflicting policies, and resolve
  conflicts
• Checking for convergence problems is NP-complete
• Link/router failure may result in an unstable
  system

Need a solution that does not require global
coordination.
10
Think Globally, Act Locally

• Key features of a good solution
• Flexibility allow diverse local policies for
  each AS
• Privacy do not force ASes to divulge their
  policies
• Backwards-compatibility no changes to BGP
• Guarantees convergence even when system changes
• Restrictions based on AS relationships
• Path selection rules which route you prefer
• Export policies who you tell about your route
• AS graph structure who is connected to who

11
Customer-Provider Relationship

• Customer pays provider for access to the
  Internet
• Provider exports its customers routes to
  everybody
• Customer exports providers routes only to
  downstream customers

Traffic to the customer
Traffic from the customer
d
provider
provider
customer
d
customer
12
Peer-Peer Relationship

• Peers exchange traffic between their customers
• AS exports only customer routes to a peer
• AS exports a peers routes only to its customers

Traffic to/from the peer and its customers
peer
peer
d
13
Hierarchical AS Relationships

• Provider-customer graph is a directed, acyclic
  graph
• If u is a customer of v and v is a customer of w
• then w is not a customer of u

w
v
u
14
Valid and Invalid Paths
Valid paths 1 2 d and 7 d Invalid path 5 8
d
Valid paths 6 4 3 d and 8 5 d Invalid paths
6 5 d and 1 4 3 d
1
3
4
2
d
5
6
Provider-Customer
7
8
Peer-Peer
15
Act Locally, Prove Globally

• Route export
• Do not export routes learned from a peer or
  provider
• to another peer or provider
• Global topology
• Provider-customer relationship graph is acyclic
• E.g., my customers customer is not my provider
• Route selection
• Prefer routes through customers
• over routes through peers and providers
• Guaranteed to converge to unique, stable solution

16
Our Local Path Selection Rules

• Classify routes based on next-hop AS
• Customer routes, peer routes, and provider routes
• Rank routes based on classification
• Prefer customer routes over peer and provider
  routes
• Allow any ranking of routes within a class
• E.g., can rank one customer route higher than
  another
• Gives network operators the flexibility they need
• Consistent with traffic engineering practices
• Customers pay for service, and providers are paid
• Peer relationship contingent on balanced traffic
  load

17
Solving the Convergence Problem

• Result
• Safety guaranteed convergence to unique stable
  solution
• Inherent safety holds under failures and policy
  changes
• Definitions
• System state current best route at each AS
• Activating AS re-do decision based on neighbor
  choices
• Sketch of (constructive) proof
• Find an activation sequence that leads to a
  stable state
• Any fair sequence (eventually) includes this
  sequence

18
Rough Sketch of the Proof

• Two phases
• Walking up the customer-provider hierarchy
• Walking down the provider-customer hierarchy

1
3
4
2
d
5
6
Provider-Customer
7
8
Peer-Peer
19
Economic Incentives Affect Protocol Behavior

• ASes already follow our rules, so system is
  stable
• High-level argument
• Export and topology assumptions are reasonable
• Path selection rule matches with financial
  incentives
• Empirical results
• BGP routes for popular destinations are stable
  for 10 days
• Most instability from failure/recovery of a few
  destinations
• ASes should follow our rules to make system
  stable
• Need to encourage operators to obey these
  guidelines
• and provide ways to verify the network
  configuration
• Need to consider more complex relationships and
  graphs

20
Playing One Condition Off Against Another

• All three conditions are important
• Path ranking, export policy, and graph structure
• Allowing more flexibility in ranking routes
• Allow same preference for peer and customer
  routes
• Never choose a peer route over a shorter customer
  route
• at the expense of stricter AS graph assumptions
• Hierarchical provider-customer relationship (as
  before)
• No private peering with (direct or indirect)
  providers

Peer-peer
21
Extension to Backup Relationships

• Backups more liberal export policies, and
  different ranking
• The motivation is increased reliability
• but ironically it may cause routing instability!
• Generalize rule prefer routes with fewest backup
  links
• Need to maintain a count of the of backup links
  in the path

Backup Provider
Peer-Peer Backup RFC 1998
provider
primary provider
backup path
backup path
failure
backup provider
failure
peer
22
Results Hold Under More Complex Scenarios

• Complex AS relationships
• AS pair with different relationship for different
  prefixes
• AS pair with both a backup and a peer
  relationships
• AS providing transit service between two peer ASes

• Stability under changing AS relationships
• Customer-provider to/from peer-peer
• Customer-provider to/from provider-customer

23
Extensions of the Work

• Influence of AS relationships on BGP convergence
• Algebraic framework and design principles for
  policy languages
• Fundamental limits on relaxing the assumptions
• Application of the idea to internal BGP inside an
  AS
• Sufficient conditions for iBGP convergence inside
  an AS
• What-if tool for traffic engineering inside an
  AS
• AS-level analysis of the Internet topology
• Inference of AS relationships and policies from
  routing data
• Characterization of AS-level topology and growth
• Practical applications of knowing AS
  relationships
• Analyzing your competitors business
  relationships
• Identifying BGP routes that violate export
  conditions

24
A Case For Customized Route Selection

• ISPs usually have multiple paths to the
  destination
• Different paths have different properties
• Different neighbors may prefer different routes

Shortest latency
Most secure
Bank
VoIP provider
School
Lowest cost
25
Neighbor-Specific Route Selection

• A node has a ranking function per neighbor

is node is ranking function for neighbor node j.
26
Stability Conditions for NS-BGP

• Surprisingly, NS-BGP improves stability!
• Neighbor-specific selection is more flexible
• Yet, requires less restrictive stability
  conditions
• Prefer customer assumption is not needed
• Choose any permissible route per neighbor
• That is, need just two assumptions
• No cycle of provider-customer relationships
• Do not export routes learned from one
  peer/provider to other peers/providers

27
Why Do Weaker Conditions Work?
1 2 0 1 0
1
0
2 3 0 2 0
3 1 0 3 0
3
2

• An AS always tells its neighbor a route
• If it has any route that is permissible for that
  neighbor

28
Deploying NS-BGP

• An AS can deploy NS-BGP alone
• Without upgrading their routers
• Without coordinating with all their neighbors
• Three aspects to the solution
• Disseminating extra BGP routes
• Customized route selection
• Directing traffic from ingress to egress
• Can be done exploiting existing mechanisms
• Designed for Virtual Private Networks (VPNs)

29
Disseminating Extra BGP Routes

• Advertising more than one BGP route
• Route distinguisher feature for VPNs
• Multiple internal BGP sessions
• ADD-PATHs extensions to internal BGP

30
Customized Route Selection

• Multiple virtual routing and forwarding tables
• Cisco Virtual Routing and Forwarding (VRF)
• Juniper Virtual Router

R3s forwarding table (FIB) entries
D (red path) R6 D (blue path) R7
31
Directing Traffic from Ingress to Egress

• Tunnels from ingress to egress
• IP-in-IP tunneling
• MPLS

?
32
Customized Route Selection

• Customized route selection as a service
• Select a different best route for different
  neighbors
• Different menu options
• Cheapest route (e.g., prefer customer)
• Best performing routes, or most secure routes
• Routes that avoid undesirable ASes (e.g.,
  censorship)
• Nice practical features of NS-BGP
• An individual AS can deploy NS-BGP alone
• without compromising global stability!

33
Conclusions

• Avoiding convergence problems
• Hierarchical of provider-customer relationships
• Export policies based on commercial relationships
• (Path ranking based on AS relationships)
• Salient features
• No global coordination (locally implementable)
• No changes to BGP protocol or decision process
• Guaranteed convergence, even under failures
• Guidelines consistent with financial incentives

34
References Related to This Talk

• The stable paths problem and interdomain
  routing
• Tim Griffin, Bruce Shepherd, and Gordon Wilfong
• http//portal.acm.org/citation.cfm?id508332
• Stable Internet routing without global
  coordination
• Lixin Gao and Jennifer Rexford
• http//www.cs.princeton.edu/jrex/papers/sigmetric
  s00.long.pdf
• Inherently Safe Backup Routing with BGP
• Lixin Gao, Tim Griffin, and Jennifer Rexford
• http//www.cs.princeton.edu/jrex/papers/infocom01
  .pdf
• Neighbor-Specific BGP More flexible routing
  policies while improving global stability
• Yi Wang, Michael Schapira, and Jennifer Rexford
• http//www.cs.princeton.edu/jrex/papers/nsbgp_sig
  metrics09.pdf

35
Other Related Research Papers

• Inherently Safe Backup Routing with BGP
• http//www.cs.princeton.edu/jrex/papers/infocom01
  .pdf
• Design Principles of Policy Languages for Path
  Vector Protocols
• http//conferences.sigcomm.org/sigcomm/2003/papers
  /p61-griffin.pdf
• Implications of Autonomy for the Expressiveness
  of Policy Routing
• http//conferences.sigcomm.org/sigcomm/2005/paper-
  FeaBal.pdf
• Meta-routing
• http//conferences.sigcomm.org/sigcomm/2005/paper-
  GriSob.pdf
• An Algebraic Theory of Interdomain Routing
• http//portal.acm.org/citation.cfm?id1103561
• Searching for Stability In Interdomain Routing
• http//www.cs.yale.edu/homes/schapira/PID808559.pd
  f

36
Related Papers With Game Theory

• Interdomain Routing and Games
• http//www.cs.huji.ac.il/mikesch/routing_games-fu
  ll.pdf
• Rationality and Traffic Attraction Incentives
  for Honest Path Announcements in BGP
• http//ccr.sigcomm.org/online/?qnode/395
• Incentive-Compatible Interdomain Routing
• http//cs-www.cs.yale.edu/homes/jf/FRS.pdf
• Mechanism Design for Policy Routing
• http//cs-www.cs.yale.edu/homes/jf/FSS.pdf
• The Complexity of Game Dynamics BGP
  Oscillations, Sink Equlibria, and Beyond
• http//www.cs.berkeley.edu/alexf/papers/fp08.pdf
• Specification Faithfulness in Networks with
  Rational Nodes
• http//www.eecs.harvard.edu/econcs/pubs/podc04.pdf
  
• Distributed Algorithmic Mechanism Design
• http//cs-www.cs.yale.edu/homes/jf/AGTchapter14.pd
  f
• Partially Optimal Routing
• http//www.stanford.edu/rjohari/pubs/por.pdf

37
Background on Interdomain Economics

• http//drpeering.net/a/Home.html
• http//www.fcc.gov/Bureaus/OPP/working_papers/oppw
  p32.pdf
• http//www.potaroo.net/papers/1999-6-peer/peering.
  pdf
• http//www.cisco.com/en/US/about/ac123/ac147/ac174
  /ac201/about_cisco_ipj_archive_article09186a00800c
  83a5.html
• http//www.cisco.com/en/US/about/ac123/ac147/ac174
  /ac200/about_cisco_ipj_archive_article09186a00800c
  8900.html
• http//www.vjolt.net/vol3/issue/vol3_art8.html