Interesting Peering Activities at the Exchange Points

1
Interesting Peering Activities at the Exchange
Points
Naiming Shen Cisco Systems
1
2
Peering Activities at NAPs

• During the Summer of 1997
• Pointing default
• Rewrite eBGP nexthop
• Passing third party nexthop
• Misconfiguration

3
Case1 Rewrite eBGP Nexthop
ACLs
ISP 3
cpe2
Mae-East NAP
ISP 2
iMCI
ISP 1
Private Peering
4
Case1 Continue...

• Netflow shown 15 extra traffic from a single
  subnet
• traceroute -g shown the traffic coming to us
• Install a static route of 212.x.x.x pointing to
  this router and traceroute stopped at ISP1
• Install the route in BGP, traceroute shown it
  coming back to us
• Thus this router of ISP3 had to rewrite the eBGP
  nexthop base on the AS numbers
• This could not be misconfiguration or a simple
  pointing default. Also this was not just used
  towards iMCI.

5
Case1 Continue...

• Install a packet filter on one of the links
• Install the packet filter on both links, which
  forced the traffic going to ISP2
• After the filter was removed, it came back
• A New packet filter was applied

6
Case 1 Continue...

• ACL 123access-list 123 permit icmp x.x.x.0
  0.0.31.255 anyaccess-list 123 permit udp x.x.x.0
  0.0.31.255 any gt 32000access-list 123 permit
  udp x.x.x.0 0.0.31.255 any eq 53access-list 123
  deny ip x.x.x.0 0.0.31.255 anyaccess-list permit
  ip any any
• The new filter was there for four days

7
Case2 Passing 3rd Party Nexthop
NAP LAN
ISP 5
traffic
iMCI
Peering/customer
Peering
ISP 4
8
Case2 Continue...

• Netflow did not find this case
• Even you can rewrite the nexthop to your peers
  address, you cant stop your peer passing your
  nexthop to the 3rd party
• route-map commandset ip next-hop peer-address
• Use next-hop-self

9
Case3 Pointing Default
ISP 6
iMCI
internetMCI.net
ISP 7
10
Case3 Continue...

• It first pointing to ISP6, then to iMCI
• reverse DNS lookup was xxx.internetmci.net
• SNMP query had default route MIB
  valueip.ipRouteTable.ipRouteEntry.ipRouteNextho
  p.0.0.0.0 IpAddress192.41.177.180
• After we exchanged some email, they pointed to
  someone else

11
Case4 Tunneling
GRE
ISP 9
NAP1
ISP 8
NAP2
ISP 9
12
Case4 Continue...
ISP 10 Upstream Provider
NAP3
E1
E3
ISP 11
13
Other Activities

• Run IGP at the NAPs
• Run Native Multicast
• Inconsistent route announcement at different
  peering points
• Run CDP

14
Detection

• Netflow stats for reverse route lookup and
  traffic matrix
• traceroute -g
• If LSR is disabled, use Ping-Pong trace
• MAC address accounting

15
Filtering

• Packet level filtering
• MAC address filtering/rate-limit, sometimes
  combined with WRED
• Null out offenders routes within your domain

16
Preventive Measures

• NAP GIGAswitch L2 filtering
• NAP ATM PVCs
• Use next-hop-self and reset peer-address
• Remove non-customer routes from NAP routers
• Do not carry NAP subnets in the backbone
• Enforce consistent route announcements