Internet Routing: Measurement, Modeling, and Analysis

1
Internet Routing Measurement, Modeling, and
Analysis
ACM Sigmetrics 2005 Tutorial
Dr. Jia Wang jiawang_at_research.att.com ATT Labs
Research Florham Park, NJ 07932,
USA http//www.research.att.com/jiawang/
Prof. Zhuoqing Morley Mao zmao_at_umich.edu Departmen
t of EECS University of Michigan Ann Arbor, MI
48109, USA http//www.eecs.umich.edu/zmao/
2
Outline

• Overview of Inter-domain routing
• Measuring inter-domain paths
• BGP Measurement
• BGP Modeling

Our opinions should not be taken to represent
ATT policies
3
Part I Overview of Inter-domain Routing
4
Internet

• Loose cooperative effort of Internet Service
  Providers (ISPs)
• E.g., ATT, Sprint, UUNet, AOL
• Best effort service
• Connectedness
• Anyone connected to the Internet can exchange
  traffic with anyone else connected to the
  Internet

5
Internet routing
Internet
Data plane forward traffic
IP traffic
rusty.cs.berkeley.edu IP169.229.62.116 Prefix169
.229.0.0/16
www.cnn.com IP64.236.16.52 Prefix64.236.16.0/20
6
Internet routing domain

• Autonomous routing domain
• Network devices under same technical and
  administrative control
• Common routing policy
• E.g., ISPs, enterprise networks
• Autonomous system
• Autonomous routing domain with an AS number (ASN)
• AS numbers 16 bits integer
• Public AS number 1 64511
• Private AS number 64512 65535
• Examples
• ATT 7018, 6431,
• Sprint 1239, 1240,
• MIT 3

7
More than 20,000 ASes today
Internet
Autonomous System
Qwest
Sprint
UUnet
ATT
Level3
Level3
Level3
GNN
Calren
Calren
Calren
IP traffic
Berkeley
Berkeley
Berkeley
CNN
company
University
8
Internet routing architecture
Inter-domain routing
Intra-domain routing
IP traffic
Internet
Berkeley
9
Intra-domain routing

• Run within a certain network infrastructure
• Optimize routes taken between points within a
  network
• Internal Gateway Protocols (IGPs)
• Metrics based
• OSPF (Open Shortest Path First)
• RIP (Routing Information Protocol)
• IS-IS (Intermediate System to Intermediate System)

10
Inter-domain routing

• Run between networks
• Provide full connectivity of entire Internet
• External Gateway Protocol (EGP)
• Policy based
• BGP (Border Gateway Protocol)

11
Link state protocols

• Examples OSPF, IS-IS
• Based on Dijkstras shortest path computation
• Each router periodically floods immediate
  reachability information to other routers
• Fast convergence
• High communication and computation overhead
• Not scalable for large networks
• Requires periodic refreshes

12
Vectoring protocols

• Distance vs. Path Vector
• Distance hop count (RIP)
• Path entire path (BGP)
• Helps identify loops
• Supports policy-based routing based on path
• Minimal communication overhead
• Takes longer to converge, i.e., in proportion to
  the maximum path length

13
Link state vs. vectoring
Link state
Vectoring
IGP
EGP
BGP is a path vector protocol
14
Classful addressing

• IPv4 32 bits
• Five classes of networks

Improve scaling factor of routing in the Internet
gt classless
15
CIDR Classless Inter-domain Routing (RFC1519)

• No implicit mask based on the class of the
  network
• Explicit masks passed in the routing protocol
• Allow aggregation and hierarchical routing

IP address 12.70.0.0
Mask 255.255.252.0
00001100 00100110 00000000 00000000
Address
00001100 00100110 00000000 00000000
Mask
11111111 11111111 11000000 00000000
11111111 11111111 11000000 00000000
Host identifier
Network prefix
CIDR representation 12.70.0.0/22
16
Address aggregation
Internet
12.70.3.0/24
12.70.0.0/24
ISP A
12.70.1.0/24
ISP B
12.71.0.0/16
12.70.2.0/24
12.70.0.0/22 12.71.0.0/16
17
Routing and forwarding

• Routing
• The decision process of choosing optimal path
  that is consistent with the administrative or
  technical policy
• Forwarding
• The act of receiving a packet, doing a lookup,
  and copying a packet to the next hop

18
Classless forwarding
Internet
12.70.0.20
10.20.128.10
10.20.128.1
10.20.0.1
IP traffic
10.20.1.1
135.120.0.1
Prefix Next hop 12.70.0.0/24 10.20.0.1 12.70.0.0/
16 10.20.1.1 12.0.0.0/8 10.20.128.1 0.0.0.0
10.20.128.10
19
Inter-domain routing with CIDR support

• BGP-4 RFC1771
• De facto EGP
• Carry routing information between ASes
• Path vector protocol
• Policy based routing
• Run on top of TCP for reliability
• Basic operations
• Set up BGP session
• Exchange all candidate routes
• Send incremental updates

20
Establish BGP session
Establish neighboring session between 12.10.0.1
and 12.10.0.2
TCP 179
12.10.0.1
12.10.0.2
Prefix Next hop 12.70.0.0/24 10.20.0.1 12.9.0.0/1
6 10.20.1.1
Prefix Next hop 135.120.0.0/24 10.128.0.1 68.35.0
.0/16 10.192.1.1
21
Exchange all candidate routes
12.70.0.0/24 10.20.0.1 12.9.0.0/16 10.20.1.1
12.10.0.1
12.10.0.2
135.120.0.0/24 10.128.0.1 68.35.0.0/16 10.192.1.1
Prefix Next hop 12.70.0.0/24 10.20.0.1 12.9.0.0/1
6 10.20.1.1 135.120.0.0/24 10.128.0.1 68.35.0.0/16
10.192.1.1
Prefix Next hop 135.120.0.0/24 10.128.0.1 68.35.0
.0/16 10.192.1.1 12.70.0.0/24 10.20.0.1 12.9.0.0/1
6 10.20.1.1
22
Send incremental updates
Withdraw 12.9.0.0/16
12.10.0.1
12.10.0.2
Prefix Next hop 12.70.0.0/24 10.20.0.1 12.9.0.0/1
6 10.20.1.1 135.120.0.0/24 10.128.0.1 68.35.0.0/16
10.192.1.1
Prefix Next hop 135.120.0.0/24 10.128.0.1 68.35.0
.0/16 10.192.1.1 12.70.0.0/24 10.20.0.1 12.9.0.0/1
6 10.20.1.1
23
BGP messages

• OPEN set up a peering session
• UPDATE announce new routes or withdraw
  previously announced routes
• NOTIFICATION shut down a peering session
• KEEPALIVE confirm active connection at regular
  interval

24
Internal vs. external BGP
I-BGP update
Internet
I-BGP
E-BGP update
I-BGP update
E-BGP
AS B
AS C
AS A
25
Scaling I-BGP for large AS

• Route reflectors

• Confederations

Only best paths being sent by RR
26
Establish connectivity
Prefix Next hop AS path 135.120.0.0/16 12.10.0.5
2 1
AS 3
Prefix Next hop AS path 135.120.0.0/16 12.10.0.1
1
12.10.0.6
IBGP
EBGP
12.10.0.5
AS 2
AS 1
EBGP
135.120.0.0/16
12.10.0.2
IBGP
12.10.0.1
IBGP
Prefix Next hop AS path 135.120.0.0/16 12.10.0.1
1
27
IGP and BGP working together
Prefix Next hop AS path 135.120.0.0/16 12.10.0.1
1
AS 3
Prefix Next hop 12.10.0.0/30 10.10.0.1 135.120.0.
0/16 10.10.0.1
12.10.0.6
IBGP
EBGP
12.10.0.5
AS 2
AS 1
12.10.0.1
EBGP
135.120.0.0/16
12.10.0.2
10.10.0.1
IBGP
12.10.0.0/30
IBGP
Prefix Next hop AS path 135.120.0.0/16 12.10.0.1
1
28
Policy routing
ISP2
ISP1
Connectivity DOES NOT imply reachability!
ISP4
ISP3
Cust1
Cust2
Policy determines how traffic can flow on the
Internet
29
BGP routing process
Apply input policy
Select best route
Apply output policy
Routes received from peers
Routes advised to peers
Best routes
Routing table
Forwarding table
BGP is not shortest path routing!
30
Best route selection

• Highest local preference
• Shortest AS path
• Lowest MED (Multi-Exit-Discriminator)
• I-BGP lt E-BGP
• Lowest I-BGP cost to E-BGP egress
• Tie breaking rules

31
Best route selection

• Highest local preference
• To enforce economical relationships between
  domains
• Shortest AS path
• Lowest MED (Multi-Exit-Discriminator)
• I-BGP lt E-BGP
• Lowest I-BGP cost to E-BGP egress
• Tie breaking rules

32
Best route selection

• Highest local preference
• Shortest AS path
• Compare the quality of routes, assuming shorter
  AS-path length is better
• Lowest MED (Multi-Exit-Discriminator)
• I-BGP lt E-BGP
• Lowest I-BGP cost to E-BGP egress
• Tie breaking rules

33
Best route selection

• Highest local preference
• Shortest AS path
• Lowest MED (Multi-Exit-Discriminator)
• To implement cold potato routing between
  neighboring domains
• I-BGP lt E-BGP
• Lowest I-BGP cost to E-BGP egress
• Tie breaking rules

34
Best route selection

• Highest local preference
• Shortest AS path
• Lowest MED (Multi-Exit-Discriminator)
• I-BGP lt E-BGP
• Prefer EBGP routes to IBGP routes
• Lowest I-BGP cost to E-BGP egress
• Tie breaking rules

35
Best route selection

• Highest local preference
• Shortest AS path
• Lowest MED (Multi-Exit-Discriminator)
• I-BGP lt E-BGP
• Lowest I-BGP cost to E-BGP egress
• Prefer routes via the nearest IGP neighbor
• To implement hot potato routing
• Tie breaking rules

36
Best route selection

• Highest local preference
• Shortest AS path
• Lowest MED (Multi-Exit-Discriminator)
• I-BGP lt E-BGP
• Lowest I-BGP cost to E-BGP egress
• Tie breaking rules
• Router ID based lowest router ID
• Age based oldest route

37
BGP route propagation

• Not all possible routes propagate
• Commercial relationships determine policies for
• Route import
• Route selection
• Route export

38
Typical AS relationships

• Provider-customer
• customer pay money for transit
• Peer-peer
• typically exchange respective customers traffic
  for free
• Siblings
• Mutual transit agreement
• Provide connectivity to the rest of the Internet
  for each other

39
AS relationships translate into BGP export rules

• Export to a provider or a peer
• Allowed its routes and routes of its customers
  and siblings
• Disallowed routes learned from other providers
  or peers
• Export to a customer or a sibling
• Allowed its routes, the routes of its customers
  and siblings, and routes learned from its
  providers and peers

40
Which AS paths are legal?

• Valley-free
• After traversing a provider-customer or peer-peer
  edge, cannot traverse a customer-provider or
  peer-peer edge
• Invalid path gt 2 peer links, downhill-uphill,
  downhill-peer, peer-uphill

41
Example of valley-free paths
1 2 3, 1 2 6 3 are valley-free
X
X
1 4 3, 1 4 5 3 are not valley free
42
Inferring AS relationships

• Identify the AS-level hierarchy of Internet
• Not shortest path routing
• Predict AS-level paths
• Traffic engineering
• Understand the Internet better
• Correlate with and interpret BGP update
• Identify BGP misconfigurations
• E.g., errors in BGP export rules

43
Existing approaches

• On inferring Autonomous Systems Relationships in
  the Internet, by L. Gao, IEEE Global Internet,
  2000.
• Characterizing the Internet hierarchy from
  multiple vantage points, by L. Subramanian, S.
  Agarwal, J. Rexford, and R. Katz, IEEE Infocom,
  2002.
• Computing the Types of the Relationships between
  Autonomous Systems, by G. Battista, M.
  Patrignani, and M. Pizzonia, IEEE Infocom, 2003.
• On AS-level Path Inference, by Z. Mao, L. Qiu, J.
  Wang, and Y. Zhang, ACM Sigmetrics, 2005.

44
Policy routing causes path inflation

• End-to-end paths are significantly longer than
  necessary
• Why?
• Topology and routing policy choices within an
  ISP, between pairs of ISPs, and across the global
  Internet
• Peering policies and interdomain routing lead to
  significant inflation
• Interdomain path inflation is due to lack of BGP
  policy to provide convenient engineering of good
  paths across ISPs

45
Path inflation

• Based on Mahajan03
• Comparing actual Internet paths with hypothetical
  direct link