Peering Exchange Architectures

1
Peering Exchange Architectures

• Jeff Bartig
• University of Wisconsin
• WiscNet Engineering

2
Overview

• OmniPOP - new CIC RE exchange in Chicago
• Way too many hours of conference calls about
  hardware and architecture of OmniPOP
• I returned Paul Schopis phone call.

3
Layer 2 Shared Exchange

• Layer 2 switch
• Each peer connects to switch
• Single broadcast domain
• Generally a single subnet allocated - each peer
  gets an address out of the block
• Used at many commercial exchanges

4
Layer 2 shared - Pros/Cons

• Cons
• Single broadcast domain
• Peer abuse possible
• Broadcast traffic
• Multicast difficulties
• Per-peer statistics more difficult to collect
  (per-MAC address stats necessary)
• Single MTU size

• Pros
• Easy/Simple
• No exchange provider involvement needed to
  establish a new peer
• Many peers on a single port - lower cost

5
Layer 2 VLAN Exchange

• Each peer has a trunk interface on layer 2
  exchange switch
• Each peer may have a VLAN to every other peer

6
Layer 2 VLAN Exchange

• Each peer has a trunk interface on layer 2
  exchange switch
• Each peer may have a VLAN to every other peer
• (n (n-1)/2 VLANs for a full mesh

7
Layer 2 VLAN - Pros/Cons

• Pros
• Separation
• MTU per VLAN
• Multicast easier
• Many peers on a single port - lower cost
• Logical interface stats

• Cons
• Many VLANs
• VLAN conflicts possible
• Exchange provider involvement if VLANs are not
  preallocated

8
Layer 1

• No central exchange hardware
• Each peer establishes cross connects to parties
  they want to peer with

9
Layer 1 Pros/Cons

• Requires an interface for each peer

• No exchange provider hardware needed - may be a
  cost savings
• Capacity/protocol flexibility

10
Layer 3 Exchange

• Each peer gets a port on the exchange router
• Each peer establishes a BGP session with exchange
  router

11
Layer 3 - Pros/Cons

• Cons
• Less control
• Routing policy determined by exchange provider
• Extra AS hop in path, possible impact on routing
  decisions

• Pros
• Simpler
• Single BGP session
• No need for a peering coordinator - outsourcing
  to exchange provider

12
Layer 3 Exchange - AS-path Length Concern Example

• Peer to WN AS Paths
• Layer 2 Exchange
• 2381
• 701 2381
• WN direct peer shorter
• Layer 3 Exchange
• 54321 2381
• 701 2381
• Equal paths. Which one will be chosen by peer?

13
AS-Path Length Experiment

• WiscNet peers with 31 networks at Equinix-Chicago
• What difference would it make if we instead did
  this peering via a layer 3 exchange? Depends
  upon the routing policies of the peers (local
  pref wins over AS-path length)
• Prepended extra AS hops into the advertisements
  to see what would happen.

14
AS-Path Length Experiment Results

• 1350 - prepended extra hop
• 450 to 350Mb/s drop
• 22 loss
• 1426 - prepended 2nd extra hop
• 410 to 350Mb/s drop
• additional 15 loss
• 1445 - removed all prepending
• 310 to 430Mb/s increase
• 38 increase

15

• End