Experiences in BGP Simulation:

1
Experiences in BGP Simulation
An In-Depth Look at the SSFNet BGPSimulator and
Its Uses

• BJ Premore
• Dartmouth College

2
Acknowledgments

• David Nicol
• Andy Ogielski Jim Cowie
• Tim Griffin

3
Overview

• Background
• summary of the SSFNet simulation package
• summary of BGP
• SSFNet BGP
• design goals
• implementation issues encountered
• example
• convergence studies

4
How I Spent My Summer Vacation
clique size 15 after withdrawal
5
What is SSFNet?

• software for modeling and simulation of computer
  networks
• Java-based
• IP packet-level granularity

6
Features of SSFNet

• fully integrated network environment
• many detailed network components included
• components all inter-operable
• scalability
• designed to handle large, complex simulations
• achievable simulation sizes vary by model and
  hardware
• configurability
• all components have multiple configurable
  attributes
• sometimes above and beyond actual implementations
• and more ...
• repeatability (random number package (CERN Colt))
• monitoring
• plotting
• modular
• open source

7
SSFNet Architecture
DML Domain Modeling Language - model
configuration
SSFNet SSF Network Models - not independent
SSF Scalable Simulation Framework - a standard
for discrete-event simulation of large,
complex systems
8
Simulation Layers
DML
Model Instances
configures
Network Components
SSFNet
enhances
Simulator Implementations
CSSF
Raceway
DaSSF
implements
implements
implements
C
C
Java
SSF
Simulator API
9
Network Components
- each is a Java class or Java package - includes
state, behavior, config info
protocols
IP
TCP
Sockets
FTP client
BGP
HTTP client
OSPF
10
Building a Model

• think hierarchically
• understand NHI addressing

11
DML Layer
DML
Model Instances
configures
Network Components
SSFNet
enhances
Simulator Implementations
CSSF
Raceway
DaSSF
implements
implements
implements
C
C
Java
SSF
Simulator API
12
Basic DML Properties

• goal simplicity
• attribute/value pairs
• hierarchical
• extensibility
• substitution

13
DML Example
Net
host
id 1
interface
id 1
Net
router
id 2
interface

link
attach 1(1)
attach 2(1)
2
1
1
14
DML The Protocol Stack
router graph ProtocolSession
name bgp use SSF.OS.BGP4.BGPSession
ProtocolSession name ospf use
SSF.OS.OSPF.sOSPF ProtocolSession
name tcp use SSF.OS.TCP.tcpSessionMaster
ProtocolSession name ip
use SSF.OS.IP
BGP
Sockets
TCP
IP
protocol graph
15
Routing Basics

• forwarding vs. routing

Direction
Destination
2
129.17.29.32
1
1
192.14.155.13
2
3
3
129.44.100.1
2
128.42.0.1
16
Routing Basics
National ISP 2

• two-level routing hierarchy

National ISP 1
Regional ISP 2
Regional ISP 1
Dartmouth
17
Routing Basics

• BGP inter-domain routing

BGP
BGP
BGP
BGP
BGP
BGP
BGP
BGP
BGP
BGP
BGP
Dartmouth
18
Routing Basics

• a simplified view

19
What is BGP?

• BGP is a distributed all-points preferred path
  algorithm, essentially
• no universal metric
• the glue that holds the Internet together

20
BGP Basics

• routing algorithm
• 1. Learn neighbors
• 2. Share reachability information with neighbors
• 3. Continue sharing updated reachability
  information
• incremental updates
• advertisements
• withdrawals
• decisions
• neighbors paths policy
• loop detection
• rate limiting
• Minimum Route Advertisement Interval (MRAI)

may serve as implicit withdrawals
21
SSFNet BGP Design Goals

• compliance with latest specifications
• core RFC 1771 and latest drafts
• policy recommendations RFC 1772
• common extensions iBGP, route reflection,
  communities
• not yet route flap dampening, confederations
• simplicity
• modular, easily extensible readable code
• allow exploration of BGP variants and extensions
• at the possible expense of efficiency (CPU,
  memory)
• allow the study of BGP itself
• extensive monitoring functionality
• allow the study of implemention decisions
• RFC targets interoperability, not exact
  implementation

22
Simplified Implementation Diagram
timer expired event
in buffer
message or event
event handler
set
TIMERS
various
route filter
update message
degree of preference
expiration event
reset timer
MRAI timer expiration
choose best routes
send message
delayed update
disseminate to peers
send update
send update
23
Validation Methodology

• no standards, created our own suite
• basic behavior in simple topologies
• Peering session maintenance (Hold KeepAlive
  timer operation)
• Route advertisement and withdrawal
• Route selection
• Reflection
• Internal BGP
• general behavior in complex topologies
• End-to-end data delivery
• Exercises basic behaviors as well
• policy testing
• Converging and non-converging gadgets
• Griffin Wilfong 1999

24
Unspecified Behavior

• allow advertisement withdrawal together?
• recent issue for routing working group
• Should notice-only update events be used?
• if not, long queues could time out a session
• use split horizon?
• Labovitz et al. did not, I did

25
A Quote from the RFC
Any technique which ensures that the
interval between two UPDATE messages sent from a
single BGP speaker that advertise feasible routes
to some common set of destinations received
from BGP speakers in neighboring autonomous
systems will be at least MinRouteAdvertisementInte
rval, and will also ensure a constant upper bound
on the interval is acceptable.
26
Unspecified Rate-Limiting Behavior

• per-peer vs. per-destination timers
• Cisco vs. SSFNet vs. Juniper
• What if MRAI prevents implicit withdrawal?
• Send explicit withdrawal, then advertisement
  later?
• Or, send only advertisement later? (implicit
  withdrawal)
• priority of MRAI expiration events?
• pre-empt ongoing update handling, or not?

27
Effects of Implementation Choices

• How much does a slight change matter?
• luckily, we have the right tool for a comparison!
• implementing if there's any doubt, make it an
  option
• experience says minimal impact, usually
• by interpretation of a "reasonable"
  implementation, and the intentions of the BGP
  specifications
• others have implemented more radical, but
  compliant versions of BGP, such as the stateless
  version
• split horizon comparison

28
Split Horizon or Not?
clique size 15 after withdrawal
29
Parameter Space Choices
convergence after a withdrawal, MRAI 30
30
The Memory Problem

• proportional to V2E
• long path length is also a factor
• don't reuse, copy
• due to emphasis on simplicity
• separate class for each message type
• rather than byte arrays, for example
• multiple prefixes per AS
• reuse must be done with care

31
Alleviating the Memory Problem

• at the cost of some functionality
• hash table instead of radix tree
• aggregation no longer possible
• 40 memory reduction
• reuse of IP address objects
• no reduction
• reuse of path attribute objects (not AS path)
• lt 5 reduction
• AS paths become linked list of integers
• segment types no longer used
• 20 memory reduction

32
Other Problems

• CPU delay model
• when to charge the time?
• BGP crashing and restarting
• must restore original state

33
A Routing Study

• a general inquiry into routing dynamics
• not as well-understood as other protocols
• distributed behavior more complex than end-to-end
• some parameters pulled "out of the blue sky"
• ubiquitously used in Internet
• using the BGP model in SSFNet
• full-fledged routing models not previously
  available

34
BGP Convergence

• Given a change in the network, how long does it
  take for all BGP speakers to return to a stable
  state?

35
Previous Work

• no convergence bound
• persistent oscillations possible
• Varadhan, Govindan, Estrin 1997
• empirical measurements
• lots of updates!
• convergence not so good ...
• Labovitz et al, 1997-2000

36
Goals

• overall
• better understand dynamic behavior of BGP
• how does rate limiting impact convergence?
• precise analytical model?
• seems unlikely ...
• so we use simulation
• and start small

37
Experiments

• simple topologies, simple policies

38
Experiments

• UP phase
• advertise a single destination
• DOWN phase
• withdraw a single destination

39
Model Parameters

• size
• rate-limiting interval
• min max processing times
• link delay
• sender-side loop detection
• withdrawal rate limiting
• jitter
• continuous rate-limiting
• random number seed index

40
Experiment DML
Net the all-encompassing Net frequency
1000000000 nanosecond simulation frequency
randomstream generator MersenneTwister
stream 165123420046345823 reproducibility_leve
l timeline Net id 1 AS_status boundary
router ... Net id 2 AS_status boundary
router ... ... link attach 11(1)
attach 21(7) delay 0.01 link attach 11(2)
attach 31(7) delay 0.01 ... bgpoptions
... define global BGP options end of the
all-encompassing Net
41
Experiment DML
router id 1 graph ProtocolSession name
test use SSF.OS.BGP4.Widgets.Advertiser
workload_file /home/bj/blah start_time
50 ProtocolSession name bgp use
SSF.OS.BGP4.BGPSession
autoconfig true ProtocolSession name socket
use SSF.OS.Socket.socketMaster
ProtocolSession name tcp use
SSF.OS.TCP.tcpSessionMaster ProtocolSession
name ip use SSF.OS.IP ProtocolSession name
probe use SSF.OS.ProbeSession
file "out.data" stream "bgpstream"
42
Experiment DML
ProtocolSession name bgp use
SSF.OS.BGP4.BGPSession autoconfig false
connretry_time 120 min_as_orig_time 15
reflector false neighbor as 2 address
1(7) use_return_address 1(1) hold_time
90 keep_alive_time 30 mrai 10
infilter _extends .filters.permit_all
outfilter _extends .filters.permit_all
neighbor as 3 address 1(1) ...
43
Experiment DML
bgpoptions define global BGP options
show_conn_estab true show connection
establishment show_snd_update true show
when updates are sent ssld false
no sender-side loop detection auto_advertise
false show_fwd_table_add true
show_rcv_notif true show_socket_events
false show_state_changes false
global_ebgp_mrai 20 startup_jitter_bound 0.1
about 50 more ...
44
Average Total Updates
clique size 15
45
Average Total Updates
clique size 15
46
Average Convergence Time
clique size 15
47
Average Convergence Time
clique size 15
48
Generalized Results forConvergence Time
Observed optimal values much lower than values
used in practice!
penalty
convergence time
0
30
rate-limiting interval (sec)
49
Generalized Results
Observed optimal values much lower than values
used in practice!
50
Continuing Work

• more realistic topologies and policies
• route flap dampening
• long-term oscillations
• internal AS topologies
• multiple destinations
• per-route vs. per-peer MRAI
• accurate processing time models

51
SOS

• Scripts for Organizing 'Speriments
• Create families and groups of experiments
• Specify DML template, parameter values, and
  extractors
• Automatically generates DML, runs sets of
  experiments, extracts desired measurements
• Stores results in database

52
Documentation References
SSFNet DML (info tutorials) http//www.ssfnet.
org/
DML (tutorial) http//www.cs.dartmouth.edu/beej/t
alks/
SSFNet BGP http//www.cs.dartmouth.edu/beej/bgp/
53
DML Example
Net host id 1 interface id 1 host
id 2 interface id 1 link attach 1(1)
attach 2(1)
54
SSF.OS.BGP4 Functionality

• Finite state machine, timers, RIB
• TCP transport (via Sockets)
• Peering exterior and interior
• Route reflection
• Messages and path attributes
• Policy
• filter based on path attribute
• attribute modification
• Monitoring of protocol operation
• gather stats on practically any event of interest

55
IDE Package
DML
Model Instances
configures
Network Components
SSFNet
enhances
Simulator Implementations
CSSF
DaSSF
Raceway
implements
implements
implements
C
C
Java
SSF
Simulator API
56
SSFNet Layer
DML
Model Instances
configures
Network Components
SSFNet
enhances
Simulator Implementations
CSSF
Raceway
DaSSF
implements
implements
implements
C
C
Java
SSF
Simulator API