Lecture 14: Interdomain Routing Stability

1
Lecture 14 Inter-domain Routing Stability

• CS 268 class
• March 8th, 2004
• (slides from Timothy Griffins tutorial and Craig
  Labovitzs NANOG talk)

2
Outline of the Todays class

• An Introduction to BGP
• BGP and the Stable Paths problem
• Convergence of BGP in the real world
• Conclusions and Open Issues

3
Inter-domain Routing basics

• Internet is composed of over 16000 autonomous
  systems
• BGP Border Gateway Protocol
• Is a Policy-Based routing protocol
• Is the de facto inter-domain routing protocol of
  todays global Internet
• Relatively simple protocol, but configuration is
  complex and the entire world can see, and be
  impacted by, your mistakes.

4
BGP Operations (Simplified)
Establish session on TCP port 179
AS1
BGP session
Exchange all active routes
AS2
While connection is ALIVE exchange route UPDATE
messages
Exchange incremental updates
5
Four Types of BGP Messages

• Open Establish a peering session.
• Keep Alive Handshake at regular intervals.
• Notification Shuts down a peering session.
• Update Announcing new routes or withdrawing
  previously announced routes.

announcement
prefix attributes values
6
Two Types of BGP Neighbor Relationships

• External Neighbor (eBGP) in a different
  Autonomous Systems
• Internal Neighbor (iBGP) in the same Autonomous
  System

AS1
iBGP is routed (using IGP!)
eBGP
iBGP
AS2
7
iBGP Peers Must be Fully Meshed

• iBGP is needed to avoid routing loops within an
  AS
• Injecting external routes into IGP does not scale
  and causes BGP policy information to be lost
• BGP does not provide shortest path routing
• Is iBGP an IGP? NO!

iBGP neighbors do not announce routes received
via iBGP to other iBGP neighbors.
8
Important BGP attributes

• LocalPREF
• Local preference policy to choose most
  preferred route
• Multi-exit Discriminator
• Which peering point to choose?
• Import Rules
• What route advertisements do I accept?
• Export Rules
• Which routes do I forward to whom?

9
Route Selection Summary
Highest Local Preference
Enforce relationships
Shortest ASPATH
Lowest MED
traffic engineering
i-BGP lt e-BGP
Lowest IGP cost to BGP egress
Throw up hands and break ties
Lowest router ID
10
Implementing Customer/Provider and Peer/Peer
relationships
Two parts

• Enforce transit relationships
• Outbound route filtering
• Enforce order of route preference
• provider lt peer lt customer

11
Import Routes
From provider
From provider
From peer
From peer
From customer
From customer
12
Export Routes
provider route
customer route
peer route
ISP route
To provider
From provider
To peer
To peer
To customer
To customer
13
Outline of the Todays class

• An Introduction to BGP
• BGP and the Stable Paths problem
• Convergence of BGP in the real world
• Conclusions and Open Issues

14
What Problem is BGP solving?
Having an X can

• aid in the design of policy analysis algorithms
  and heuristics,
• aid in the analysis and design of BGP and
  extensions,
• help explain some BGP routing anomalies,
• provide a fun way of thinking about the protocol

This
talk
15
Q How simple can X get? A The Stable Paths
Problem (SPP)
2
An instance of the SPP

• A graph of nodes and edges,
• Node 0, called the origin,
• For each non-zero node, a set or permitted paths
  to the origin. This set always contains the
  null path.
• A ranking of permitted paths at each node. Null
  path is always least preferred. (Not shown in
  diagram)

1
most preferred least preferred (not null)
When modeling BGP nodes represent BGP speaking
border routers, and 0 represents a node
originating some address block
Yes, the translation gets messy!
16
A Solution to a Stable Paths Problem
2
2 1 0 2 0
A solution is an assignment of permitted paths
to each node such that
4 2 0 4 3 0

• node us assigned path is either the null path or
  is a path uwP, where wP is assigned to node w and
  u,w is an edge in the graph,
• each node is assigned the highest ranked path
  among those consistent with the paths assigned to
  its neighbors.

3 0
1 3 0 1 0
1
A Solution need not represent a shortest path
tree, or a spanning tree.
17
A Stable Paths Problem may have multiple
solutions
1 2 0 1 0
1 2 0 1 0
1 2 0 1 0
2 1 0 2 0
2 1 0 2 0
2 1 0 2 0
First solution
Second solution
DISAGREE
18
Multiple sets of BGP routing policies can map
down to the same Stable Paths Problem DISAGREE
in RPSL (Version I)
1 2 0 1 0
import from AS1 action pref 0 accept ANY
from AS0 action pref 10 accept ANY
export to AS2 announce ANY
export to AS1, AS2 announce AS0
import from AS2 action pref 0 accept ANY
from AS0 action pref 10 accept ANY
export to AS1 announce ANY
2 1 0 2 0
19
DISAGREE in RPSL (Version II)
import from AS-ANY action pref 0
accept community.contains(11) from
AS-ANY action pref 10 accept ANY export to
AS2 announce ANY
1 2 0 1 0
export to AS1 set community.append(21)
announce AS0 to AS2
set community.append(11) announce
AS0
2 1 0 2 0
import from AS-ANY action pref 0
accept community.contains(21) from
AS-ANY action pref 10 accept ANY export to
AS1 announce ANY
Assume AS1 and AS2 use neighbor send-community
command .
20
DISAGREE in RPSL (Version III)
1 2 0 1 0
import from AS-ANY accept ANY export to AS2
announce ANY
export to AS1 action
aspath.prepend(AS0, AS0, AS0) announce
AS0 to AS2 announce AS0
2 1 0 2 0
import from AS1 action pref 0 accept ANY
from AS0 action pref 10 accept ANY
export to AS1 announce ANY
The interaction of all BGP policies is directly
represented in SPP
21
Multiple solutions can result in Route
Triggering
1 0 1 2 3 0
1 0 1 2 3 0
primary link
2 3 0 2 1 0
2 3 0 3 1 0
backup link
3 2 1 0 3 0
3 2 1 0 3 0
Remove primary link
Restore primary link
22
SPP helps explain possibility of BGP divergence

• BGP is not guaranteed to converge to a stable
  routing. Policy inconsistencies can lead to
  livelock protocol oscillations.
  
• See Persistent Route Oscillations in
  Inter-domain Routing by K. Varadhan, R.
  Govindan, and D. Estrin. ISI report, 1996

23
BAD GADGET No Solution
With a BGP-like protocol, each node will do the
best it can, so at least one node will always
have the opportunity to improve its path.
Result persistent oscillation.
2 1 0 2 0
2
4 2 0 4 3 0
4
0
3
1
3 4 2 0 3 0
1 3 0 1 0
24
SURPRISE Beware of Backup Policies
2 1 0 2 0
Becomes BAD GADGET if link (4, 0) goes down.
2
4 0 4 2 0 4 3 0
4
BGP is not robust it is not guaranteed to
recover from network failures.
0
3
1
3 4 2 0 3 0
1 3 0 1 0
25
PRECARIOUS
Has a solution, but can get trapped
26
What is to be done?
Static Approach
Dynamic Approach
Extend BGP with a dynamic means of detecting
and suppressing policy-based oscillations?
Inter-AS coordination
Automated Analysis of Routing Policies (This is
very hard).
These approaches are complementary
27
Research papers on SPP
An Analysis of BGP Convergence
Properties Timothy G. Griffin, Gordon Wilfong
SIGCOMM99
Model BGP, show static analysis is hard
Policy Disputes in Path Vector
Protocols Timothy G. Griffin, F. Bruce
Shepherd, Gordon Wilfong
ICNP 99
Define Stable Paths Problem and develop
sufficient condition for sanity
A Safe Path Vector Protocol Timothy G. Griffin,
Gordon Wilfong
INFOCOM00
Dynamic solution based on histories
Stable Internet Routing without Global
Coordination Lixin Gao, Jennifer Rexford
SIGMETRICS00
Show that if certain guidelines are followed,
then all is well. Rule Do not forward route
advertisements from peers or Providers to other
peers or providers.
28
Outline of the Todays class

• An Introduction to BGP
• BGP and the Stable Paths problem
• Convergence of BGP in the real world
• Conclusions and Open Issues

29
Convergence in the real-world?

• Labovitz99 Experimental results from two year
  study which measured 150,000 BGP faults injected
  into peering sessions at several IXPs
• Found
• Internet averages 3 minutes to converge after
  failover
• Some multihomed failovers (short to long ASPath)
  require 15 minutes

30
Problems with Distance Vector

• Distance vector protocols (e.g. RIP) suffer
  routing table loops
• Counting-to-infinity
• Routing table loops
• Bouncing problem
• BGP uses path vector to solve problems seen
  with RIP and other Bellman-Ford derived protocols

31
Counting to Infinity
B
2
A
1
R
A 2 R 1
B 2 R 123
235
R 527
R 729
32
Taming Infinity

• Routing Information Protocol (RIP) solved
  counting to infinity problem by re-defining
  infinity.
• Added speedups poison reverse, split horizon,
  triggered updates.
• Strictly increasing O(N)
• ASPath limits infinity to the width of the
  Internet (an ASPath through all your neighbors)
• Monotonically increasing
• Upper bound?

33
BGP Convergence Example
34
N gt 4?
AS6453
AS2497
6453 1239 5696 237
AS6113
2497 5696 237
6113 2914 237
AS6461
6461 5696 237
AS1239
1239 5696 237
AS5696
5696 237
AS2914
2914 237
AS237
237
AS701
701 6461 5696 237
AS5000
5000 237
AS1
AS1673
1 5696 237
1673 5696 237
35
The Problem with BGP

• If we assume
• unbounded delay on BGP processing and propagation
• Full BGP mesh BGP peers
• Constrained shortest path first selection
  algorithm
• BGP is O(N!), where N number of default-free BGP
  speakers

There exists possible ordering of messages such
that BGP will explore all possible ASPaths of all
possible lengths
36
BGP and RIP

• RIP precisely monotonically increasing. Can
  explore metrics (1N)
• BGP monotonically increasing. Multiple (N!) ways
  to represent a path metric of N.
• BGP solved RIP routing table loop problem by
  making it exponentially worse

2117 5696 2129 2117 1 5696 2129 2117 2041 3508
3508 4540 7037 1239 5696 2129 2117 1 2041 3508
3508 4540 7037 1239 5696 2129 2117 2041 3508 3508
4540 7037 1239 6113 5696 2129 2117 1 2041 3508
3508 4540 7037 1239 6113 5696 2129
37
BGP Best Case

• What is the best we can expect from BGP?
• Implementation of MinRouteAdver timer leads to 30
  second rounds
• Time complexity is O(n-3)30 seconds
• State/Computational complexity O(n)
• At its best, BGP performs as well as RIP2 (but
  uses exponentially more memory in the process)

38
MinRouteAdver

• Minimum interval between successive updates sent
  to a peer for a given prefix
• Allow for greater efficiency/packing of updates
• Rate throttle
• Applied only to announcements (at least according
  to BGP RFC)
• Applied on (prefix destination, peer) basis, but
  implemented on (peer) basis

39
MinRouteAdver

• 30(N-3) delay due to creation mutual
  dependencies. Provide proof that N-3 rounds
  necessarily created during bounded BGP
  MinRouteAdver convergence
• Rounds due to
• Ambiguity in the BGP RFC and lack receiver loop
  detection
• Inclusion of BGP withdrawals with MinRouteAdver
  (in violation of RFC)

40
Conclusions

• Internet routing has serious convergence problems
• Result 1 Griffin et al. BGP does not satisfy
  the stable paths problem.
• Result 2 Rexford et al. If every AS follows a
  set of guidelines then Internet routing should
  not have convergence problems.
• Result 3 Labovitz et al. An extensive
  measurement study shows that Internet convergence
  can be in the order of several minutes.

41
Open issues?

• Convergence analysis (lower,upper) bounds are
  very weak are worst-worst case scenarios.
• Can we design a cleaner protocol that has
  provably good convergence properties?
• What about link-state routing?
• Should we really care about convergence?
• Routes to popular prefixes are stable IMC03