BGP: Introduction and Issues

1
BGP Introduction and Issues

• Michalis Faloutsos
• (with the help of various contributions of slides)

2
What Is BGP?

• Border Gateway Protocol BGP-4
• The de-facto interdomain routing protocol
• BGP includes specifications
• Which information gets advertised and how
• BGP includes a routing protocol
• Establishes and uses a routing table
• Internal Gateway Protocol (I-BGP in the book)

3
Why Is There Such Fuss about BGP?

• BGP dictates routing at the AS level
• Absence of understanding poor performance
• BGP is complicated
• Designed to be flexible
• Involves multiple fields
• Understanding BGP behavior is not intuitive
• Implementation and business policies
• The routing of the Internet relies on BGP

4
Some Open Questions

• How well does BGP work now?
• How can I manage a BGP network?
• How secure and robust is BGP?
• Cyber-terrorism
• How would we re-design BGP now?
• How well will BGP scale for our future needs?

5
Roadmap

• Introduction to BGP
• Highlights of BGP issues
• Goal instigate interest in BGP

6
Some Basic Numbers

• 43,000 Autonomous Systems approx.
• Corporate Networks
• ISP Internal Networks
• National Service Providers
• Identified by ASN a 16 bit value
• Assigned by IANA
• Superlinear growth (Huston, Siganos et al.)

7
How A BGP graph Looks Like
AS 2
AS 5

• Each AS has designated BGP routers
• BGP routers of an AS communicate internally with
  another protocol (IGP)

AS 4
AS 3
AS 1
8
IP Addresses and Prefixes

• IP addresses have 32 bits 4 octets of bits
  (IPv4)
• A prefix is a group of IP addresses
• 128.32.101.5 is an IP address (32 bits)
• 128.32.0.0/16 is a prefix of the 16 first bits
• 128.32.0.0 128.32.255.255 (216 addresses)
• 128.32.4.0/24 is a prefix of the 24 first bits -
  longer

9
Routing is Based on Prefixes

• A BGP Routing table has prefixes for entries
• For a IP address of a packet, find longest match
• Example packet IP 128.32.101.1
• 128.1.1.4 matches the first 8 bits no match!
• 128.32.0.0/16 match for 16 bits
• 128.32.101.0/24 is a longer match

10
Prefix Matching in More Detail

• For a IP address of a packet, find longest match
• Example Compare
• packet IP 128.32.101.1
• With 128.32.0.0/16
• IP 01000000. 001000000. 01100101
  .00000001
• Mask 11111111. 111111111. 00000000 .00000000
• AND 01000000. 001000000. 00000000 .00000000
• Prefix 01000000. 001000000. 00000000. 00000000
• Equal? Yes

11
Advertising Routing Information

• Each AS advertises what it can reach from each
  BGP router
• Policies I filter what you advertise
• Policies II filter from what you hear advertised
  
• Build up a BGP routing table
• Remember which prefix you hear from which link

12
What Does a Routing Table Look Like?
Prefix Origin AS Path
128.32.0.0/16 123 14 56 123
123 34 101 203 123
128.32.101.0/24 15 50 15 15

• Origin AS owns the address
• Routing tables can have peculiarities

13
Route Advertising

• Distance Vector style protocol
• Hear advertisements IP prefix, AS-path
• Filter if desired (i.e. ignore)
• Append yourself IP prefix, myASAS-path
• Forward to appropriate ASs

14
Basic AS relationships

• Customer Provider
• Customer pays Provider for service
• The Customer is always right
• Peer to Peer mutual cooperation
• Ex. MCI and ATT
• Sibling-Sibling
• Ex. ATT research and ATT wireless

15
The Internet as a Directed Graph

• Every edge is bidirectional reg. traffic!
• Business relationships are represented

16
The Initial Idea

• Data flows between customers-providers
• Top level providers are peers
• They exchange information to ensure connectivity
• What can possibly go wrong?

17
And then came the rain

• Thousands of ASs
• Complicated relationships
• Multiple providers for one AS, and IP prefix!!
• Multihoming
• Traffic engineering
• I want to use multiple paths and load balance

18
AS Relationships
Provider
Customer
200
100
Peer
Peer
10
11
12
13
1
4
3
2

• Customer Provider customer pays and is always
  right
• Peer to Peer Exchange traffic only between their
  customers
• Sibling-Sibling Exchange traffic at will

19
The Rules of BGP Routing

• Transit traffic traffic that does not go to my
  customers (or their customers)
• A provider carries any traffic to, from customer
• Peers exchange traffic only if between their
  customers

20
How BGP Policy Restricts Routing
Provider
Customer
Peer
100
Peer
200
10
11
13
12
1
4
3
2

• Routing rules
• Provider accept everything
• Peer only if it is for its customers

• Path Properties
• Up then down
• No up-down-up, at most 1 peer-peer steps

21
What paths are allowed?
Provider
Customer
Peer
100
Peer
200
10
11
13
12
1
4
3
2
22
Implementing BGP Rules

• What do you do with an advertisement
• Through customer link
• Advertise to all (customers, peers, providers)
• Through sibling link
• Advertise to all
• Through provider link
• Advertise to customer only (and possibly
  siblings)
• Through peer link
• Advertise to customer only (and possibly
  siblings)

23
How Policies Affect Routing
Customer 1

• A Provider will get rid of traffic as soon as
  possible,
• But a Provider will carry the traffic for its
  customer
• Did anyone say traffic is asymmetric?

ISP1
ISP2
Customer 2
24
BGP Path-Length Asymmetry

• Consider number of AS traversed by a path
• Asymmetry 46 of pairs differ by at least one AS
  hop
• 
  Siganos 01

25
Determining The Logical Graph

• The business relationships are critical
• How can I find the relationships?
• Infer relationships from routing tables
• IRR database manually maintained error prone

26
Two Inference Algorithms

• Inference algorithm Gao 00
• Using 1 routing table
• Exploit the up-down path property
• in a routing path, assume highest degree node as
  peak
• Inference using multiple tables Subramanian02
• Use multiple points of observation to improve
  results

27
Things Become InterestingTraffic Engineering

• How can I pick a route?
• Local Preference path attribute
• AS2 wants to prefer fast thick link
• Advertisement from right router of AS2 has
  higher Local Preference
• Any BGP router in AS2 will prefer the thick link

AS 2
LP 100
LP 80
slow
208.1.1.0/24
AS 1
28
Load Balancing - Appetizer

• I want to share traffic between my two providers
• How can I do this?

29
Load Balancing Long Prefix Match Takes All!
ISP 3
138.39/16
138.39.1/24
ISP 2
ISP 1
138.39.1/24
138.39/16
138.39.1/24
Customer
138.39.1/24
30
So How Can I Balance the Load?

• Ask my provider to not aggregate my prefix
• Will this work?
• Split my prefix in two
• 138.39.1.0/24
• A 138.39.1.31/28
• B 138.39.1.32/28
• Advertise only one part to ISP2
• ISP2 traffic destined for prefixes in A
• ISP1 traffic destined for prefixes in B

31
Summary Up to Now

• BGP-4 is the de facto protocol for interdomain
  routing
• BGP was developed to achieve
• Flexible policy implementation
• Scalability via route aggregation given CIDR
• There are many open issues
• BGP is a hot research topic

32
The Growth of BGP Table
33
The BGP Growth The Truth

• Growth flattened out
• Why?
• Better management
• Dot-com crash?

34
Routing Table Variation
100k

• Larger ASes have significantly larger tables

35
Update Activity Per Prefix

• Measure rate of announcements withdrawals
  path updates
• Compare relative update rate per prefix
• length to the relative number of prefixes of
  that length
• gt1 implies higher than average update rate (less
  stable)
• lt1 implies lower than average update rate(more
  stable)

36
Measured Update Rate

• Bursty!

37
BGP robustness

• Measuring the BGP updates

38
Analyzing Messages By Content

• Aggregated per 30 seconds

39
Initial Observations

• Updates show daily and weekly periodicity
• There is no evidence of BGP disturbance
• The Baltimore tunnel train 18 July that destroyed
  Internet lines
• Sept 11 attack
• There are some spikes at
• 19 July
• 18-22 September

40
BGP Updates Correlations
41
BGP Under Attack
42
Router CPU Activity Correlates
43
The Attack of The Worm
44
Classification of Instabilities
45
Isolating Instability 1 unstable peer
46
Detecting abnormal BGP activity
47
The Worm Activity
48
The Worm Correlates Again
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
Summary of BGP Instability

• Globally correlated BGP instability is not
  uncommon
• Some causes are well understood
  (misconfiguration, bad path announcements)
• Some others are less well understood, and more
  worrisome
• worms

56
BGP Up Close and Personal

• Establishing a connection
• Messages
• Path Attributes

57
Establishing A BGP Session

• BGP uses TCP connections
• For reliability
• A BGP session is between two routers
• Typically directly connected (Ethernet, FDDI)
• Routers establish a BGP session
• Authentication and set-up
• Update and withdrawals
• If disconnected, all paths are invalidated

58
Messages

• First open TCP connection
• Identification and authentication
• BGP messages
• OPEN set-up, negotiate timer for keep-alive
• UPDATE routing changes
• NOTIFICATION termination, and error messages
• KEEPALIVE confirm that connection is active

59
UPDATE Message

• Advertise reachability information
• Withdraw paths to prefixes
• Update information prefixes
• Introduce new prefixes
• Modify important path attributes for new prefixes
  and the related paths

60
Standard Path Attributes

• Attribute Type, Length, Value
• Origin where did I hear this from
• BGP (external) or IGP (internal)
• AS Path sequence of ASs
• Flexible handling of loops
• Recovering from disconnected ASs!
• Next Hop set explicitly who the next router
  should be (possibly a non BGP speaker)

61
Multi-Exit Discriminator (MED)
MED 10
AS 2
AS 1
MED 50

• Indication to external peers of the preferred
  path into the AS
• Lowest Med Preferred

62
Note for MED

• Defines preference for incoming traffic
• One AS sets the values
• Another AS interprets and uses them
• Thus
• We need cooperative ASes
• Only between two ASes (1 hop scope)
• MED is meaningless in the next hop
• MED can be used only if both routes are
  advertised from the same AS

63
Local Preference
AS 2
AS 1

• Choosing paths internally
• Defining outgoing traffic
• Within an AS
• Set Local Preference to pick the path you want to
  send data to
• The higher Local Preference is preferred

L1
L2
AS3
Here MED can not be used since we have different
AS
64
Aggregation of updates

• AS want to aggregate as much as possible
• Reduce routing state
• Reduce information that needs to be exchanged
• Main idea send one update instead of two
• All other attributes are the same (path,
  preferences etc)
• The prefixes have to be subsets, or adjacent

adjacent
subset
65
Atomic Aggregate

• Sometimes we aggregate paths that are different
  (ie different AS sequence)
• Atomic aggregate shows that some of the
  destinations in this prefix are not necessarily
  following the same path
• This does not allow other routers to de-aggregate
  the path, thus creating entries that should not
  appear

66
Atomic Aggregate
138.39/16 3 1 4
138.39/16 1 4
AS 3
AS 1
AS 2
138.39/16
138.39.1/24
138.39.1/24
AS 4 138.39/16
AS 5

• AS 2 thinks that all 138.39/16 follow the 3 1 4
  path

67
BGP Route Selection Process

1. Maximum prefix length match
2. Highest Local Priority
3. Shortest AS Path
4. Lowest MED (if routes through same AS)
5. Min Cost Next hop router (consulting IGP)
6. Prefer external to internal routes
7. Pick lowest BGP identifier among many E-BGP
8. Pick lowest BGP identifier among many I-BGP

68
No Valley Prefer Customer Routing

• An abstraction of common sense policy
• No valley dont transit traffic for peer or
  provider
• Ie. If you dont get paid, dont do it
• ASes use the following rule, when selecting a
  path
• Prefer a path through a customer
• Prefer a path through a peer
• This policy can be implemented using BGP
  attributes for paths

69
(No Transcript)
70
Internal BGP (I-BGP)

• Communication between routers of an AS
• I-BGP very similar to E-BGP except
• Different advertising rules
• Do not re-advertise a path internally
• I-BGP 1 -gt I-BGP 2 -X- I-BGP3
• But readvertise (in -out, out -in)
• I-BGP 1 -gt I-BGP 2 -gt E-BGP1
• E-BGP 1 -gt I-BGP 2 -gt I-BGP1

71
I-BGP Re-Advertising
I-BGP
E-BGP

• B will not re-ad. to C what it hears from A
• But it will re-ad to D
• Why?
• Paths are identified by AS, and internally you
  have the same AS
• To avoid routing loops

A
B
D
C
72
I-BGP Mesh

• I-BGP routers form a fully connected mesh
• (clique)
• Scalability becomes an issue
• The full mesh is independent of physical
  connectivity

73
A Subtle Difference I-BGP vs E-BGP

• E-BGP routers communicate using the IP of the
  physical interface (link)
• Link based reliability
• I-BGP routers have virtual or loopback
  interface
• Even when link fails, routers may be reachable
• Node based reliability

74
Static vs Dynamic Configuration

• In practice, many customers do not speak BGP
• Do not have an AS number
• They are configured statically

75
One vs Many Providers

• Single homed vs multihomed
• Customers for reliability and performance connect
  to many providers
• Difference
• Single homed easy to manage
• Multihomed tricky
• Route aggregation
• Load balancing

76
Multihoming

• How can I use my multiple connections
  efficiently?
• Multihoming is quite widespread
• Users take it up to themselves to find
  reliability and good performance Huston
• Consequence non-aggregatable state

77
Two routers two links

• Multiplex traffic at link or IP layer
• Use Next Hop to point to virtual router IP
• Second look up how can I reach virtual IP?
• Pick one of the two links randomly or statically
• Furthermore, when one link is down the other one
  is chosen

ISP 1
B
C
A
78
Multihoming with One Routers 1 ISP
ISP 1

• Customer can advertise different prefixes on each
  link
• Reliability?
• Use Multi-Exit-Discriminator
• Cust. Sets MED, ISP uses it
• Use Local Preference
• ISP sets LP and picks link
• Customers IBGP to pick router

B
C
A
P1
P2
79
Issues

• The mechanisms are there, but they depend on many
  factors
• How much traffic each prefix generates?
• Balancing incoming and outgoing traffic
• Dynamically adapting to changing conditions
• Technical issues
• Some ISPs do not accept very long prefixes
• Longer than they would be in classfull routing
• Longer than 19 for new prefixes

80
Multihoming with Two Routers 1 ISP
Internet

• Similar case for Provider to Customer
• For Customer to Provider
• A could alternate paths
• Reordering of packets
• ISP 1 could advertise different addresses on each
  link
• I.e. 1 ISP customers
• 2 Default (everybody else)

ISP 1
B
C
ISP 1 Customers
A
P1
P2
81
Multihoming With Two Providers
ISP 3
138.39/16
138.39.1/24
ISP 2
ISP 1
138.39.1/24
138.39/16
138.39.1/24
Customer
138.39.1/24
82
Two Providers Multihoming Getting Address Space

• Given two ISP
• Get space from one (we saw before)
• Get space from both
• Advertise only related prefix to ISP
• Aggregation but not reliability
• Advertise both prefixes to ISPs
• No aggregation but reliability
• Get space independently of both
• Max flexibility, not reliability
• If too narrow of prefix, may not propagate, no
  connectivity

83
I-BGP Scalability

• Full mesh is not scalable O(N2) IBGP sessions
• Approaches to scalable I-BGP
• Hierarchical structure Route reflectors
• Divide and conquer Confederations

84
Route Reflection

• Explicitly allow some I-BGP routers to
  readvertise
• Route reflectors represent other routers
• Hierarchical structure avoids loops and problems

D
E
B
C
A
85
Confederations

• Decompose an AS to sub-AS
• Externally one AS
• Internally like E-BGP (E-I-BGP)
• Loop avoidance
• AS-CONFED-SET
• AS-CONFED-SEQUENCE

D
E
B
C
A
86
Confederation BGP Rules

• Differences of BGP between sub-ASes
• Local-Preference is transitive
• Recall in BGP it is not
• NEXT-HOP attribute is forwarded
• Path within AS is monitored with sub-AS no.
• AS-CONFED-SEQ
• AS-CONFED-SEQUENCE

87
Handling the Dynamic Nature
88
To Refresh or not to Refresh?

• BGP uses hard state
• BGP routers consider a path is usable
• until explicitly withdrawn
• the session fails
• How do I detect if a connection failed?
• Keep-alive messages

89
Path Updates Frequency

• Send updates of a path no sooner 30 sec
• Why?
• Stability
• Overhead reduction
• Side-effects
• Convergence can take longer
• What is the right interval?
• Recent studies say that 30s is too long

90
Route Flapping and Dampening

• Flapping constant alternating updates
• It can happen!
• Route update dampening
• Consider stability of path before using it
• How store penalty value for each path
• Issue router needs to remember withdrawn paths

91
Overview up to Now

• BGP is Distance Vector
• BGP uses TCP and hard-state
• Routing updates are delayed and batched
• Route dampening to alleviate instabilities

92
End
93
Current Research the AS Topology

• Characterize the topology using power-laws
  (Faloutsos 99, Siganos et al 01)
• Modeling the evolution of the topology
• (Barabasi, Siganos01)

94
Ongoing Research AS paths

• 107 Gb of AS paths over 3 years
• Exponential growth, but distances remain the same
  
• Inflation due to policy
• 20 of paths are larger than they could
• Significant Routing Asymmetry
• 40 paths by at least one hop

95
Near Future Research Plans

• Mine the collected paths for patterns
• How stable were the paths?
• Can we identify illegal paths?
• Identify pathologies (ie. loops)

96
Overview of Plans for Research

• How well does BGP work now?
• How secure and robust is BGP?
• Cyber-terrorism how much damage can be done?
• How would we design BGP now?
• People are asking this question
• How well will BGP scale?
• How can we manage BGP (avoid human errors)?
• Approach
• Analytical and simulations with SSFNET

97
Conclusions

• BGP is an open and exciting topic
• The community knows very little
• Big ticket items
• Measurements and modeling
• Robustness, security
• Network Management traffic engineering
• Scalability

98
Multihoming With Two Providers
ISP 3
138.39/16
138.39.1/24
ISP 2
ISP 1
138.39.1/24
138.39/16
138.39.1/24
Customer
138.39.1/24
99
Policies on Transitivity
Isp 1
Isp 2
Nontransit A customer should not Be transit for
its Providers
Not allowed
AS X
AS 1
Transit
AS 3
AS 4
AS 2
100
More BGP attributes

• Communities
• Confederations
• Route Reflectors

101
Convergence
102
Multi-Exit Discriminator(MED)

• Indication to external peers of the preferred
  path into the AS
• Lowest Med Preferred

103
Note in MED

• One AS sets the values
• Another AS interpets and uses them
• Thus
• Cooperative ASes
• Only between two ASes (1 hop scope)
• MED is meaningless in the next hop
• MED can be used only if both routes are
  advertised from the same AS

104
Local Preference

• Choosing paths internally
• Within an AS
• Set Local Preference to pick the path you want to
  send data to
• The higher Local Preference is preferred

105
Atomic Aggregate

• Sometimes we aggregate paths that are different
  (ie different AS sequence)
• Atomic aggregate shows that some of the
  destinations in this prefix are not necessarily
  following the same path

106
Aggregator

• Notify that an aggregation took place
• Which AS
• Which router
• For management and traceback purposes

107
Example The Intended Use
108
BGP Graph and Routing Policies
200
100
10
11
12
13
1
4
3
2

• Up then down 1, 10, 100, 200, 13, 4
• No valleys, no up-down-up, no more than 1
  peer-peer