Stable Internet Routing Without Global Coordination

1
Stable Internet Routing Without Global
Coordination

• Jennifer Rexford
• ATT Labs--Research

http//www.research.att.com/jrex/papers/sigmetric
s00.long.pdf
2
Internet Architecture

• The Internet is
• Decentralized loose confederation of peers
• Self-configuring no global registry of topology
• Stateless limited information in the routers
• Connectionless no fixed connection between hosts
• These attributes contribute
• To the success of Internet
• To the rapid growth of the Internet
• and the difficulty of controlling the Internet!

sender
receiver
3
Research Overview Internet Control Plane

• Managing a single Autonomous System
• Measurement of traffic, routing, and
  configuration
• Traffic engineering techniques and tools
• Configuration checking and automation
• Interdomain routing between ASes
• Routing protocol convergence
• Inference of commercial relationships
• Characterization of routing dynamics
• End-to-end troubleshooting
• Root-cause analysis of routing changes
• Measuring the AS-level forwarding path
• DARPA Knowledge Plane seedling

This talk
4
Internet Routing Architecture

• Divided into Autonomous Systems (ASes)
• Distinct regions of administrative control
• Routers/links managed by a single institution
• Service provider, company, university,
• Hierarchy of Autonomous Systems
• Nation-wide tier-1 provider
• Medium-sized regional provider
• Small university or corporate network
• Interaction between Autonomous Systems
• Internal topology is not shared between ASes
• but, neighboring ASes interact to coordinate
  routing

5
Autonomous Systems (ASes)
Path 6, 5, 4, 3, 2, 1
4
3
5
2
6
7
1
Web server
Client
6
Interdomain Routing Border Gateway Protocol

• ASes exchange info about who they can reach
• IP prefix block of destination IP addresses
• AS path sequence of ASes along the path
• Policies configured by the ASs network operator
• Path selection which of the paths to use?
• Path export which neighbors to tell?

I can reach 12.34.158.0/24 via AS 1
I can reach 12.34.158.0/24
1
2
3
data traffic
data traffic
12.34.158.5
7
Interdomain Routing Challenges

• Scale
• Destination prefixes 150,000 and growing
• Autonomous Systems 17,000 visible ones, and
  growing
• AS paths and routers at least in the millions
• Policy
• Path selection selecting which path your AS
  wants to use
• Path export controlling who can send packets
  through your AS
• Convergence
• VoIP and video games need convergence in tens of
  milliseconds
• Routing protocol convergence can take several
  (tens of) minutes
• and the routing system doesnt necessarily
  converge at all!

8
Conflicting Policies Cause Convergence Problems
1 2 0 1 0
1
0
2 3 0 2 0
3 1 0 3 0
3
2
Pick the highest-ranked path consistent with your
neighbors choices.
9
Global Control is Not Workable

• Create a global Internet routing registry
• Keeping the registry up-to-date would be
  difficult
• Require each AS to publish its routing policies
• ASes may be unwilling to reveal BGP policies
• Check for conflicting policies, and resolve
  conflicts
• Checking for convergence problems is NP-complete
• Link/router failure may result in an unstable
  system

Need a solution that does not require global
coordination.
10
Think Globally, Act Locally

• Key features of a good solution
• Flexibility allow complex local policies for
  each AS
• Privacy do not force ASes to divulge their
  policies
• Backwards-compatibility no changes to BGP
• Guarantees convergence even when system changes
• Restrictions based on AS relationships
• Path selection rules which route you prefer
• Export policies who you tell about your route
• AS graph structure who is connected to who

11
Customer-Provider Relationship

• Customer pays provider for access to the
  Internet
• Provider exports its customers routes to
  everybody
• Customer exports providers routes only to
  downstream customers

Traffic to the customer
Traffic from the customer
d
provider
provider
customer
d
customer
12
Peer-Peer Relationship

• Peers exchange traffic between their customers
• AS exports only customer routes to a peer
• AS exports a peers routes only to its customers

Traffic to/from the peer and its customers
peer
peer
d
13
Hierarchical AS Relationships

• Provider-customer graph is a directed, acyclic
  graph
• If u is a customer of v and v is a customer of w
• then w is not a customer of u

w
v
u
14
Our Local Path Selection Rules

• Classify routes based on next-hop AS
• Customer routes, peer routes, and provider routes
• Rank routes based on classification
• Prefer customer routes over peer and provider
  routes
• Allow any ranking of routes within a class
• Do not impose ranking between customer routes, or
  between peer and provider routes
• Consistent with traffic engineering practices
• Customers pay for service, and providers are paid
• Peer relationship contingent on balanced traffic
  load

15
Solving the Convergence Problem

• Assumptions
• Export policies based on AS relationships
• Path selection rule that favors customer routes
• Acyclic provider-customer graph
• Result
• Guaranteed convergence of the routing protocol
• Holds under link/router failures and policy
  changes
• Sketch of (constructive) proof
• Activation sequence that leads to a stable state
• Any fair activation sequence includes this
  sequence

16
Proof, Phase 1 Selecting Customer Routes

• Activate ASes in customer-provider order
• AS picks a customer route if one exists
• Decision of one AS cannot cause an earlier AS to
  change its mind

3
2
1
An AS picks a customer route when one exists
d
0
17
Proof, Phase 2 Selecting Peer and Provider Routes

• Activate rest of ASes in provider-customer order
• Decision of one phase-2 AS cannot cause an
  earlier phase-2 AS to change its mind
• Decision of phase-2 AS cannot affect a phase 1 AS

3
AS picks a peer or provider route when no
customer route is available
1
2
4
d
0
6
5
8
7
18
Economic Incentives Affect Protocol Behavior

• ASes already follow our rules, so system is
  stable
• High-level argument
• Export and topology assumptions are reasonable
• Path selection rule matches with financial
  incentives
• Empirical results IMW02
• BGP routes for popular destinations are stable
  for 10 days
• Most churn due to small number of unpopular
  destinations
• ASes should follow our rules to make system
  stable
• Need to encourage operators to obey these
  guidelines
• and provide ways to verify the network
  configuration
• Need to consider more complex relationships and
  graphs

19
Different Rules More Flexible Import Policies

• Allowing more flexibility in ranking routes
• Allow the same rank for peer and customer routes
  with the same AS path length
• Never choose a peer route over a shorter customer
  route
• Stricter AS graph assumptions
• Hierarchical provider-customer relationship (as
  before)
• No private peering with (direct or indirect)
  providers

Peer-peer
20
Backup Relationships INFOCOM01

• Backups more liberal export policies
• Primary and a backup provider
• Peers giving backup service to each other
• Generalize rule prefer routes with fewest backup
  links

Backup Provider
Peer-Peer Backup
provider
primary provider
backup path
backup path
failure
backup provider
failure
peer
21
More Complex Scenarios

• Complex AS relationships
• AS pair with different relationship for different
  prefixes
• AS pair with both a backup and a peer
  relationships
• AS providing transit service between two peer ASes

• Stability under changing AS relationships
• Customer-provider to/from peer-peer
• Customer-provider to/from provider-customer

22
Conclusions

• Avoiding convergence problems
• Hierarchical AS relationships
• Export policies based on commercial relationships
• Guidelines for import policies based on
  relationships
• Salient features
• No global coordination (locally implementable)
• No changes to BGP protocol or decision process
• Guaranteed convergence, even under failures
• Guidelines consistent with financial incentives

23
Broader Influence of the Work

• AS relationships and BGP convergence
• Algebraic framework and design principles for
  policy languages
• Fundamental limits on relaxing the assumptions
• Internal BGP inside an AS
• Sufficient conditions for iBGP convergence inside
  an AS
• What-if tool for traffic engineering inside an
  AS
• AS-level analysis of the Internet
• Inference of AS relationships from routing data
• Characterization of AS-level topology and growth
• Network design and operations (e.g., at ATT)
• Analyzing competitors and changing BGP policies
• Setting protective route filters on BGP sessions

24
Longer-Term Research Plans Internet Control Plane

• Internet routing architecture
• Routing Control Point for moving intelligence out
  of the routers
• Distributed troubleshooting service based on
  measurement
• Router, protocol, and language extensions
• Extra routing information to help in
  troubleshooting
• Lightweight support for measurement in routers
• Better router and network configuration languages
• Campus, enterprise, and regional networks
• Fertile ground for new research problems
• New sources of measurement data and tech
  transfer