New Interdomain Routing Architectures

1
New Interdomain Routing Architectures

• Jennifer Rexford

2
Well, BGP Has Some Problems

• Instability
• Not guaranteed to converge to a unique, stable
  solution (e.g., oscillation and bi-stable
  solutions)
• Slow convergence
• Path exploration can take a very long time
• Non-linearity
• Small changes can have large effects (e.g.,
  intradomain changes leading to BGP changes)
• Feature interaction
• Unexpected side effects (e.g., the interaction of
  route-flap damping and path exploration)

3
Well, BGP Has Some More Problems

• Scalability challenges
• Large memory, processing, and message-passing
  overhead, dependent on behavior in other ASes
• Security vulnerabilities
• BGP update messages are relatively easy to forge
  with bogus prefix, AS path, or other attributes
• Difficult to configure
• Operators must express their (many) goals in an
  indirect and contorted manner
• Difficult to troubleshoot
• Hard to infer the root cause of a routing change,
  or even the AS(es) responsible

4
But Wait, Theres More!

• No performance guarantees
• BGP announcements do not include any information
  about the performance of the path
• Limited flexibility for path selection
• Only single-path routing on destination prefix
• Limited control over path selection
• Chosen path is a byproduct of the composition of
  local routing policies in many different ASes
• Forwarding path may not match AS path
• Routers may deflect packets to other paths (e.g.,
  route aggregation, misconfiguration, and malice)

5
And, Perhaps the Biggest Problem of All

• Hard to change
• BGP is the glue that holds the disparate parts of
  the Internet together
• So, hard to change BGP in a fundamental ways
• or is it?

6
Why is BGP Such a Mess?

• Absence of underlying models
• Protocol designed without a design for the
  decision process or policy language
• BGP models (e.g., SPP) came much later
• Incremental evolution of the protocol
• Many additions to BGP over the years
• New route attributes and decision-process steps
• Rapid growth of the Internet
• Many ASes, and much more complex topology
• Commercialization of the Internet
• More complex routing policies and competition
• Heightened importance of security concerns

7
Whats a Networking Researcher to Do?

• Characterize the existing routing system
• Measurement, modeling, simulation
• Improved operational practices
• Best common practices for configuration
• Timer settings, route filtering, features to use,
  
• Methods and tools for complicated tasks
• Traffic engineering, config checking, root-cause
  analysis
• Incremental changes to BGP
• Better implementations of the protocol
• New route attributes for greater flexibility
• Graceful handling of failures, config changes,

8
Whats a Networking Researcher to Do?

• Build on top of the existing routing system
• Overlay networks
• Propose new architectures anyway
• Identify and evaluate new and better solutions
• even if we cant readily deploy them today
• Explore incremental-deployment approaches
• Meta solutions that enable new architectures
• With the goal of leading to a better architecture
• Create models of the fundamental limits
• Maybe the problem BGP is solving is really hard

9
Key Ingredients of Architectural Proposals

• More control at the edge
• Source routing and multi-path routing
• Negotiation between ASes
• Explicit coordination for path selection
• Design for the common case
• Handle common routing policies well (e.g., HLP)
• Servers computing the routes
• Greater visibility and control than any one
  router
• Multiple simultaneous architectures
• Virtualization to support customized solutions

10
Source Routing

• The ultimate in flexibility
• Sender determines path for each packet
• At the router level, or at the AS level
• At the cost of
• Overhead of propagating topology information
• Need for fast path changes after a failure
• Lost control for intermediate routers/ASes

B
C
A
ABEF
F
D
E
ADECF
11
Source Routing Deployment Challenges

• ASes have little incentive to cooperate
• No business relationship with ASes far away
• Dont want to relinquish control over routing
• So, source routing is typically disabled
• Policy-compliant source routing
• Limiting what sources routes can be used
• Progress on limited forms of source routing
• Overlay networks
• Source routing on an overlay topology
• Multi-homing route control
• One hop source routing

12
Multipath Routing

• Expose more paths to ASes
• Multiple paths through same next-hop AS
• Giving ASes greater control over path selection
• Extreme case export all paths
• High protocol overhead
• Lost control for intermediate ASes
• Compromise export selective paths
• Under control of intermediate ASes
• Based on their routing policies, pricing, etc.

13
Exposing Extra Paths
BEFBCF
CFCEFCBEF
B
C
ABEFADEF
A
F
F
D
E
EF ECF
DEFDABEF

• Example AS A doesnt like AS E
• Default routes both go through E
• Good to learn alternate route that avoids E

14
Encapsulation to Use Non-Default Paths
e
e
d
d
B
C
A
F
D
E

• Direct packet along alternate route
• Destination-based forwarding not enough
• Encapsulate the packet to egress point

15
Inter-AS Negotiation

• Directives
• Tell other AS what to do
• E.g., which link to use
• Suggestions
• Tell other AS your wishes
• Which link you prefer used
• Cooperation
• Negotiate where to send
• Across flows and time
• Inbound and outbound
• Trade small losses for larger gain (win win)

Customer B
Provider B
multiple peering points
Early-exit routing
Provider A
Customer A
16
Inter-AS Negotiation

• But, how to do it?
• What info to exchange?
• How to prioritize the many choices?
• How prevent cheating?
• Open research territory
• Some initial work on exchanging preferences
• E.g., ASes exchange preferences, and iterates

Customer B
Provider B
multiple peering points
Early-exit routing
Provider A
Customer A
17
Revisiting the Division of Labor

• Routers
• Forward packets yes
• Compute paths maybe not
• End users or ASes
• Dictate path properties yes
• Control path selection maybe not
• Intermediate ASes (e.g., ISPs)
• Forward packets yes
• Business relationships yes
• Compute end-to-end paths maybe not

18
Removing Routing From Routers

• Should routers forward packets yes
• Must be done at high speed
• in line-card hardware on fast routers
• So, needs to be done on the routers
• Should routers compute routes no
• Hard to do in a distribution fashion
• Difficult to make load-sensitive routing stable
• Lacking complete information for good decisions
• Not flexible enough for end users
• Difficult to extend over time

19
Routing As a Service (RAS)

• Goal third parties pick end-to-end paths for
  clients to satisfy diverse user objectives
• Forwarding infrastructure
• Basic routing (e.g., default routing)
• Primitives for inserting forwarding-table entries
• Routing Service Provider (RSP)
• Contracts ISPs for service (e.g., virtual links)
• Selects end-to-end routes on behalf of clients
• Competes with other selectors for customers
• End host
• Queries RSP to pick path install forwarding
  state

20
Routing As a Service (RAS)
RSP
ISP
ISP
user
ISP
21
Routing Control Platform (RCP)

• Goal Move beyond todays artifacts, while
  remaining compatible with the legacy routers
• Incentive compatibility phased evolution
• Intelligent route reflector in a single AS
• Learning BGP routes directly from neighbor ASes
• Interdomain routing between RCPs
• Backwards compatibility BGP to the routers
• Using BGP to push answers to the routers
• No need to change the legacy routers at all
• Keep message format and router implementation

22
How Does This Differ From Overlays

• Overlays circumventing the underlay
• Host nodes throughout the network
• Logical links between the host nodes
• Active probes to observe the performance
• Direct packets through good intermediate nodes
• Routing services controlling the underlay
• Servers collect data directly from the routers
• Servers compute forwarding tables for the routers
• Data packets do not go through the servers
• Like an overlay for managing the underlay

Maybe some combination of the two makes sense?
23
Concurrent Architectures Better than One

• Multiple customized architectures in parallel
• Multiple virtual routers on a single platform
• Resource isolation in CPU, forwarding table,
  bandwidth
• Programmability for different protocols and
  mechanisms (routing, forwarding, addressing, )

24
Cabo Applications Within an Single ISP

• Customized virtual networks
• Security for online banking
• Fast-convergence for VoIP and gaming
• Specialized handling of suspicious traffic
• Testing and deploying new protocols
• Evaluate on a separate virtual network
• Rather than in a dedicated test lab
• Large scale and early-adopter traffic
• Leasing virtual components to others
• ISPs have unused node and link capacity
• Can allow others to construct services on top

25
Cabo Enabling Economic Refactoring
Infrastructure Providers
Service Providers

• Infrastructure providers Maintain routers,
  links, data centers, other physical
  infrastructure
• Service providers Offer end-to-end services
  (e.g., layer 3 VPNs, SLAs, etc.) to users

Today ISPs try to play both roles, and cannot
offer end-to-end services
26
Key Ingredients of Incremental Deployability

• Backwards compatibility
• Technically possible to deploy the solution
• E.g., change anything but the message format
• Incentive compatibility
• Offer concrete benefits to early adopters
• Generate incentives for others to follow
• Do not rely on complete participation
• QoS, multicast, secure routing, IPv6,
• Need creativity on incremental deployment

27
Lessons on Incremental Deployment

• Adding on top tunneling in the data plane
• Circumvent destination-based forwarded
• Traverse routers
• Adding on top servers in the control plan
• Tricking the routers into doing the servers
  bidding
• Implementing new functionality on the servers
• Sneaking in on the side virtualization
• Running additional virtual networks in parallel
• Offering new functionality for special
  applications
• while continuing to support legacy Internet

28
Discussion

• Tussles between stake-holders
• Users and ISPs
• Division of function
• Data, control, and management planes
• End host, edge routers, and core routers
• How much better could BGP be?
• While still being an interdomain protocol with
  control distributed across Autonomous Systems?
• With an even cleaner slate than that?
• Where should the economics be???