Implementing and Maintaining an ISP Backbone

1
Implementing and Maintaining an ISP Backbone

• Kevin Butler

2
(No Transcript)
3
(No Transcript)
4
Tier 1 ISP Backbones

• Comprise some of the worlds largest IP networks
• Tier 1 companies include Sprint, ATT, PSINet
• UUNET has the worlds largest IP data network,
  presence on four continents and future expansion
  into Latin and South America

5
Service Level Agreements

• SLAs are an important and prestigious tool in
  attracting and maintaining customers
• Comprised of uptime guarantees and bounds on
  latency through various geographic regions
• most ISPs currently have latency the US

6
Supporting the Customer

• Quality and expertise of first-line customer
  support varies wildly between companies
• depending on size, geographic location and
  company focus, some front-line support teams
  outsourced to third parties
• some in-house high level support teams have
  skills equivalent or superior to NOCs

7
Network Operations Centres

• Generally the teams concerned with backbone
  maintenance and support
• trend towards consolidation into Super-NOCs
  (eg. one for Americas, one for Europe)
• specialisation within NOC for product support
  (eg. dial, VPN, backbone NOCs)

8
(No Transcript)
9
NOC Tools

• NOCOL - Network Operations Centre On Line
  (freeware UNIX)
• Mediahouse monitoring (mainly web)
• Micromuse Netcool (now owned by Lucent) - used by
  MCI WorldCom, PSINet, BT

10
Dial Access

• Dial is a major selling point, especially with
  customers who travel a lot or are their own ISPs
• connections made through an Ascend MAX TNT, which
  can support up to 720 concurrent callers
• back-end is a DS-3 into a backbone router,
  routers advertised by an IGP (eg. RIP)

11
Dial-Related Technologies

• COBRA (Central Office Based Remote Access) allow
  building of virtual POPs by backhauling PRIs
• RADIUS (Remote Authentication Dial In User
  Service)

12
Integrated Services Digital Network

• ISDN customers authenticate by RADIUS similar to
  dial users
• underlying architecture similar but dial
  equipment often administrated differently
• ISDN maintained within same AS as backbone
  whereas dial often in its own AS

13
DS-1 and high-speed access

• Customer connections usually multiplexed, come
  into DSU as a channelised DS-3
• gateway routers on ISP side usually Cisco 7500
  series, increasingly using Cisco 12000
• customers connect using Cisco 1604, 2621, some
  3600 series, very large customers use 7500 series
  routers

14
Gateway Routers

• obtain routes from customers usually statically,
  but sometimes by BGP
• usually run link-state IGP within AS (eg. OSPF,
  IS-IS)
• Cisco 7513 backplanes 1.8 Gbps while 12008 does
  40 Gbps

15
Where does traffic go from here?

• Most ISPs have two levels of networks above the
  access router
• Metropolitan networks aggregate gateway traffic,
  generally city-wide (if multiple POPs in city)
• transit networks aggregate metro networks
  traffic, responsible for inter-city transport

16
ATM Switches

• Terminate long-haul OC-12, OC-48 circuits and
  metro rings
• Choice of vendor contingent on ISP, commonly
  Newbridge, Fore Systems (ASX-1000 and ASX-4000)

17
Example of an ATM interface
TR1.EG1 interface ATM2/0 description To
HA13.BLAH1 3C1 atm vc-per-vp 512 atm pvc 16 0 16
ilmi ! interface ATM2/0.195 point-to-point descrip
tion To XR1.BLAH1 ATM6/0 ip address
146.188.200.98 255.255.255.252 ip router isis
Net-Backbone atm pvc 195 0 195 aal5snap clns
router isis Net-Backbone
18
Implementation of BGP

• BGP run between autonomous systems and peers, as
  well as multi-homed customers
• monolithic AS broken up into BGP confederations
  for ease of work
• routes controlled using access lists and route
  maps

19
BGP

• Communities are destinations that share common
  attributes (eg. through access-list filters)

BGP table version is 23718690, local router ID is
205.150.242.2 Status codes s suppressed, d
damped, h history, valid, best, i -
internal Origin codes i - IGP, e - EGP, ? -
incomplete Network Next Hop
Metric LocPrf Weight Path i24.64.0.0/19
198.133.49.7 100 0 6327
6172 i i24.64.0.0/14 198.133.49.7
100 0 6327 i i24.64.32.0/19
198.133.49.7 100 0 6327
6172 i i24.64.64.0/19 198.133.49.7
100 0 6327 6172 i i24.64.96.0/19
198.133.49.7 100 0 6327
6172 i i24.64.192.0/19 198.133.49.7
100 0 6327 6172 i i24.64.224.0/19
198.133.49.7 100 0 6327
6172 i i24.65.0.0/19 198.133.49.7
100 0 6327 6172 i i24.65.96.0/19
198.133.49.7 100 0 6327
6172 i i24.65.128.0/19 198.133.49.7
100 0 6327 6172 i
20
Advantages of BGP for User

• Allows for load-sharing and redundancy
• routes can be biased through AS path prepending
• requirement is high-quality router with close to
  100 uptime to avoid connection flaps and
  subsequent route dampening

21
Common Customer Issues

• Static routes on backbone - often difficult to
  spot, can cause very strange routing results
• pull-up routes for netblocks smaller than /24,
  required to avoid BGP dampening
• BGP recalculations - if done on a transit router,
  entire backbone segments can experience outages

22
Customer Requirements of the Backbone

• Redundancy - networks are redundant but card
  failures can take down whole routers
• physical connection to POP from customer is SPF
• low latency - massive increases in demand on
  backbone makes this difficult
• over 2 million a day spent on global backbone
  upgrades

23
DSL low cost, high speed

• DSL might phase out ISDN connections
• difficult to troubleshoot from network standpoint
• connections pass through telcos frame or ATM
  cloud between DSLAM and VR
• RedBack SMS (Subscriber Management System) 1000
  commonly used as VR

24
RedBack SMS 1000

• Supports up to 4000 sessions
• OC-3 out to metro network
• traffic-shaping accomplished with profiles

atm profile samplecust counters shaping vbr-nrt
pcr 1000 cdvt 100 scr 100 bt 10
25
Increasing Capacity

• Backbone capacity increasing at a huge rate
• Traffic engineering combined with high backplane
  becoming increasingly important
• many ISPs turning to Juniper routers
• UUNET rolled out production OC-192c with Juniper
  M160 running MPLS

26
Juniper Routers

• JUNOS supports MPLS and RSVP

isis interface all ospf area
0.0.0.0 interface so-0/0/0
metric 15 retransmit-interval 10
hello-interval 5
edit
27
Distributed DOS attacks

• Can be very detrimental to backbone (even causing
  switch crashes)
• Combated by rate-limiting ICMP on routers
• Most effective defense is community-wide egress
  filtering requires co-operation throughout the
  Internet

28
Canadian Network Challenges

• Geographically, population resides in virtually a
  straight line across the south
• major focus is on southbound capacity to the US
• CRTC regulations on telcos create different
  arrangements
• heterogeneous network to the US, integration a
  big issue

29
Questions?

• Anything I can clarify or expand on...
• Thank you!