Olivier H. Martin (1)

1
A Quick Internet Technology tourwith special
emphasis on NGI,the next generation Internet

• Lecture 2 presented at the 26th International
  Nathiagali Summer College on Physics and
  Contemporary Needs, 25th June 14th July,
  Nathiagali, Pakistan
• Olivier H. Martin
• CERN - IT Division
• June 2001
• Olivier.Martin_at_cern.ch

2
Presentation Outline

• Internet, what is it?
• Internet Backbone Technologies (ATM, POS, PoWDM,
  MPLS)
• Internet Routers
• Internet circuits
• Wave Division Multiplex (DWDM, CWDM)
• IAB Workshop State of the Internet
• Next Generation Internet
• Challenges ahead
• QoS
• Gigabit/second file transfer
• Security architecture
• IPv4 to IPv6 transition coexistence

3
Internet, what is it?

• A network of networks with IP as the thin
  Inter-network layer, also serving as the
  insulation layer between layer2 and layer 4 and
  above.
• There is a wealth of layer 2 access technolgies
  ranging from
• Ethernet (10/100/1000 BaseT)
• FDDI
• ATM
• Packet over Sonet
• HDLC
• Wireless
• Analog/Digital telephones
• GSM
• Satellite
• ADSL, Cable TV
• .

4
(No Transcript)
5
Internet Backbone Technologies (ATM)

• ATM still ubiquitous in many large Internet
  backbones, especially in Europe
• Back in 1996, the fastest router and switch
  interfaces available were ATM based
• ATM switch based core versus IP router based
  core.
• There is nothing wrong with cell based
  switching, however there is a problem with the
  availability of very high speed ATM router
  interfaces (Segmentation And Re-assembly (SAR)).
• Will stay as an access technology and for
  building Virtual Private Networks (VPN).
• Offers unparalleled granularity and class of
  services.
• MPLS (see later) can be seen as a form of frame
  ATM.

6
(No Transcript)
7
Internet Backbone Technologies (POS)

• Packet over SONET (POS) is definitely well ahead
  of ATM.
• Very high speed interfaces available (i.e.
  2.5Gbps (OC-48c) and more recently 10Gbps
  (OC-192c))
• Packet over WDM is becoming the norm (the idea is
  to bypass the Sonet/SDH layer, which is generally
  felt to be too heavy and expensive, and to
  perform the SONET APS (Automatic Protection
  Switching) function at layer 3, using MPLS).
• Note that SONET frame format is still used.

8
High Speed IP Network Transport
Multiplexing, protection and management at every
layer
IP
Signalling
ATM
SONET/SDH
Optical
B-ISDN
Higher Speed, Lower cost, complexity and overhead
9
(No Transcript)
10
Internet Backbone Technologies (MPLS/1)

• MPLS (Multi-Protocol Label Switching) is an
  emerging IETF standard that is gaining impressive
  acceptance, especially with the traditional
  Telecom Operators and the large Internet Tier 1.
• Recursive encapsulation mechanism that can be
  mapped over any layer 2 technology (e.g. ATM, but
  also POS).
• Departure from destination based routing that has
  been plaguing the Internet since the beginning.
• Fast packet switching performed on source,
  destination labels, as well as ToS. Like ATM
  VP/VC, MPLS labels only have local significance.
• Better integration of layer 2 and 3 than in an IP
  over ATM network through the use of RSVP or LDP
  (Label Distribution Protocol).
• Ideal for traffic engineering, QoS routing, VPN,
  IPv6 even.

11
Internet Backbone Technologies (MPLS/2)

• MPLS provides 2 levels of VPNs
• Layer 3 (i.e.conventional VPNs)
• Layer 2 (i.e encapsulation of various layer2
  frame formats), e.g.
• Ethernet
• ATM
• PPP
• MPLS can also be used for circuit and/or
  wavelength channel restoration.
• MPlS (MPLambdaS), GMPLS (Generalized MPLS)

12
Emerging Terabit Internet routers (1)

• A number of startups are successfully challenging
  Ciscos dominant position, e.g.
• Juniper (M160), Avici (TSR), Nexabit/Lucent(NX6400
  0), Pluris (TNR20000), Unisphere/Argon
• CW, Qwest,/KPNQwest, UUnet (MCI/Wordlcom) are
  using Juniper M160.
• Some layer 2/3 switch vendors are also trying to
  enter the WAN market but with mixed success, so
  far (e.g. Foundry, Cabletron/Interasys)
• less functionality
• less performance

13
Emerging Terabit Internet routers (2)

• Fastest routers are still relatively slow (i.e.
  less than 300Gbps),but things improved very
  recently Juniper M160, Cisco GSR 12416
  (15OC-192c).
• Density (space) still a problem, e.g. GSR12016
  can scale to 5Tbps (i.e.22.5 Tbps, but 17 racks
  needed!
• ASICs are problematic
• Juniper is said to have packet re-ordering
  problems at very high speed
• Every bug entails 6 month delay, this is
  reportedly what caused Cisco to be 12 months
  behind Juniper for 10Gbps interfaces.
• New Ciscos 7600 OSR family with PXF (Parallel
  Express Forwarding) technology.
• OC-768c (40Gbps) coming, but.
• recent Avici annoucement at SuperComm 2001, BUT
  over composite link (i.e. 162.5 Gbps)!

14
Internet circuits

• Internet Backbone Circuits
• 622Mbps (OC-12c) common, 2.5Gbps (OC-48c)
  (almost) standard in large backbones,
• 10Gbps (OC-192c) coming very fast, even in Europe
  (GEANT)!
• But, partly because of the way the Internet is
  now organized (e.g. CDNs), reported usage is
  still very low, i.e. many backbones are grossly
  over-dimensioned.
• Internet Access circuits
• 34/45Mbps (common), 155Mbps (rare), 622Mbps
  (exceptional)
• Unlike backbone circuits, the cost of local loops
  can be quite high.

15
Internet Backbone Speeds
MBPS
IP/?
OC12c
OC3c
ATM-VCs
T3 lines
T1 Lines
16
DWDM, CWDM

• Dense Wave Division Multiplex (DWDM)
• Fiber optic technology has been making gigantic
  progress
• This was needed in order to support the explosive
  growth of the Internet
• and to remove bottlenecks on trans-oceanic
  routes, in particular.
• The technology is evolving very fast in terms of
• number of channels,
• capacity per channel,
• distance without repeaters.
• Coarse Wave Division Multiplex (CWDM)
• Cheap form of WDM suitable for Metropolitan Area
  Networks (MAN) and/or substitute for local
  SONET/SDH local loops.

17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Transmission Systems of The Recent Past
Low-rate Data
Low-rate Data
30-50 km
E D M U X
XMTR
Regen. Repeater
Regen. Repeater
RCVR
Regenerative Receiver
Transmitter (DFB Laser)
Opto-Electronic Regenerative Repeaters
Electronic Multiplexer
Electronic Demuliplexer

• Single channel operation
• Opto-electronic regenerative repeaters - one per
  50 km per fiber
• 30-50 km repeater spacing
• Capacity upgrades increased speed

Still Found In Legacy Network Systems
24
Todays Transmission System
l1
80-140 km
XMTR
RCVR
l1
O M U X
O D M U X
XMTR
RCVR
l2
Regen. Repeater
l2
ln
XMTR
RCVR
ln
Optical Demultiplexer
Optical Amplifiers
Optical Multiplexer

• Multi-channel WDM operation
• One amplifier supports many channels
• 80-140km amplifier (repeater) spacing
  regeneration required every 200-300 km
• Capacity upgrades adding wavelengths (channels)
  increasing speeds

However, regeneration is still very expensive and
fixes the optical line rate
25
Next GenerationThe Now Generation
l1
80-140 km
XMTR
l1
O M U X
O D M U X
RCVR
XMTR
l2
RCVR
l2
ln
XMTR
RCVR
ln
1600 km
Optical Demultiplexer
Optical Multiplexer

• Multi-channel WDM operation
• One amplifier supports many channels
• 80-140km amplifier (repeater) spacing
  regeneration required only every 1600 km
• Capacity upgrades adding wavelengths (channels)
  increasing speeds

Over 1000 Km optically transparent research
network tested on the Qwest network
26
Dare to extrapolate for the next 5 years(Yves
Poppe/TeleGlobe)?

• Will Moores law and related laws for growth of
  fiber transmission capacity and internet growth
  continue to apply?
• Probably
• The laws of gravity still apply, even in the New
  Economy. Progress alternates between periods of
  exponential growth and plateaus were the progress
  is absorbed.
• Progress continues unabated
• Alcatel tested 10Tb over single fiber with 256
  channels at 40Gb and demonstrated 3TB over 7300km
  using wide band EDFA
• Intel announced chipsets for OC192 and 10GbE
• Ciena announces 160 channels at 25GHz spacing
• Although traditionally spacing in GHz2.5x
  channel capacity in Gb Ciena claims to have
  10Gbps using 12.5GHz spacing in lab

27
(No Transcript)
28
(No Transcript)
29
IAB Workshop

• The Internet Architecture Board (IAB) held a
  workshop on the state of the Internet Network
  Layer in July 1999, a number of problem areas and
  possible solutions were identified
• Network/Port Address Translators (NAT/PAT),
• Application Level Gateways (ALG) and their impact
  on existing and future Internet applications.
• End to end transport security requirements
  (IPSEC)
• Transparency (e.g. H.323)
• Realm Specific IP (RSIP).
• Mobility (completely different set of protocol
  requirements)
• IPv6
• Routing (growth of routing table, route
  convergence)
• DNS (renumbering)

30
(No Transcript)
31
Recent evolution of the Internet

• The original Internet was
• unregulated, flat charge, simple protocols, few
  but open protocols, end to end transparency.
• Todays Internet is
• trends toward more regulations, flat charge,
  sometimes no charge even, but increasing number
  of paying services!
• no longer simple, large number of plug-ins
  proprietary protocols in use, end to end
  principle seem to have more or less disappeared.
• Restricted client server model.
• What about the peer to peer model?
• Increasingly fragmented,
• In such a restricted environment, what about the
  next killer application?
• the threat is that everything may be layered on
  top of the Web (HTTP).
• Napster, Gnutella, distributed games,...

32
Client/Server Architecture is breaking down
Private Address Realm

• For web-based transactions
• Sufficient to allow clients in private address
  spaces to access servers in global address space

Global Addressing Realm

• For telephones and I-Msg
• You need to use an address when you call them,
  and are therefore servers in private realm

Private Address Realm
33
Loss of End to end transparency

• Loss of end to end transparency due to
• proliferation of Firewalls, NATs, PATs
• Web caches, Content Engines, Content Distribution
  Networks (CDN),
• Application Level gateways, Proxies, etc.
• Cons
• violation of end to end transport principle,
• possible alteration of the data,
• only partially fits the client-server model (i.e.
  server must be outside)
• Pros
• better performance, service differentiation, SLA,
  
• cheaper to deliver services to large number of
  recipients, etc.

34
Growth in BGP Route Table
Source http//www.telstra.net/ops/bgptable.html
35
Routing and Addressing inthe Billion Node Network

• Address Efficiency and Route Aggregation
• Using addresses more efficiently
• Adopt hierarchies within addresses allow for
  remote abstraction of routing information
• Private Addressing .. Maybe!
• Using less public addresses when we can
• Network Address Translation (NAT) and
  Realm-Specific IP (RSIP)
• Address extension
• Getting more addresses by changing protocol
  platforms
• IPv6 and the next address pool

36
Next generation Internet, what is it?

• A natural evolution from what the Internet is
  today (or rather was yesterday)?
• A completely new model following some
  technological revolution (e.g. all optical
  networks), or increased regulations, new
  economic/charging model (e.g. portals).
• Being part of a Global community, we need to make
  sure that new technological developments properly
  take into consideration the constraints of each
  region, e.g.
• limitations of transoceanic cables (i.e. lmited
  number of fiber pairs)
• bandwidth in less networked advanced countries

37
Several major issues

• Quality of Service (QoS)
• High performance (i.e. wire speed) file transfer
   end to end 
• Will CDN technology help?
• Is the evolution towards edge services likely to
  affect global GRID services?
• Impact of security
• Internet Fragmentation, one vs several Internets
• e.g. GPRS top level domain
• Transition to IPv6 and long term coexistence
  between IPv4 IPv6

38
Quality of Service (QoS)

• Two approaches proposed by the IETF
• integrated services (intserv),
• intserv is an end-to-end architecture based on
  RSVP that has poor scaling properties.
• differentiated services (diffserv).
• diffserv is a newer and simpler proposal that has
  much better chances to get deployed in some real
  Internet Service Providers environments, at
  least.
• even though diffserv has good scaling properties
  and takes the right approach that most of the
  complexity must be pushed at the edges of the
  network, there are considerable problems with
  large diffserv deployments.
• ATM is far from dead, but has serious scaling
  difficulties (e.g. TEN-155, Qwest/ATM).
• MPLS is extremely promising, today it looks like
  it is where the future lies (including ATM AAL5
  emulation!)

39
Quality of Service (QoS)

• QoS is an increasing nightmare as the
  understanding of the implications are growing
• Delivering QoS at the edge and only at the edge
  is not sufficient to guarantee low jitter, delay
  bound communications,
• Therefore complex functionality must also be
  introduced in Internet core routers,
• is it compatible with ASICs,
• is it worthwhile?
• Is MPLS an adequate and scalable answer?
• Is circuit oriented technology (e.g. dynamic
  wavelength) appropriate?
• If so, for which scenarios?

40
Gigabit/second networking

• The start of a new era
• Very rapid progress towards 10Gbps networking in
  both the Local (LAN) and Wide area (WAN)
  networking environments are being made.
• 40Gbps is in sight on WANs, but what after?
• The success of the LHC computing Grid critically
  depends on the availability of Gbps links between
  CERN and LHC regional centers.
• What does it mean?
• In theory
• 1GB file transferred in 11 seconds over a 1Gbps
  circuit ()
• 1TB file transfer would still require 3 hours
• and 1PB file transfer would require 4 months
• In practice
• major transmission protocol issues will need to
  be addressed
• () according to the 75 empirical rule

41
(No Transcript)
42
Very high speed file transfer (1)

• High performance switched LAN assumed
• requires time money.
• High performance WAN also assumed
• also requires money but is becoming possible.
• very careful engineering mandatory.
• Will remain very problematic especially over high
  bandwidthdelay paths
• Might force the use Jumbo Frames because of
  interactions between TCP/IP and link error rates.
• Could possibly conflict with strong security
  requirements

43
Very high speed file transfer (2)

• Following formula proposed by Matt Mathis/PSC
  (The Macroscopic Behavior of the TCP Congestion
  Avoidance Algorithm) to approximate the maximum
  TCP throughput under periodic packet loss
• (MSS/RTT)(1/sqrt(p))
• where MSS is the maximum segment size, 1460
  bytes, in practice,and p is the packet loss
  rate.
• Are TCP's "congestion avoidance" algorithms
  compatible with high speed, long distance
  networks.
• The "cut transmit rate in half on single packet
  loss and then increase the rate additively (1 MSS
  by RTT)" algorithm may simply not work.
• New TCP/IP adaptations may be needed in order to
  better cope with lfn, e.g. TCP Vegas

44
Very high speed file transfer (3)

• The Mathis formula shows the extreme variability
  of achievable TCP throughputs in the presence of,
  
• even small, packet loss rates (i.e. less than
  1),
• Small packets vs large packets (e.g. Jumbo
  frames),
• Delay (RTT), also called long fat networks
  (lfn), i.e. with large bandwidthdelay products,
  hence the need for very large windows
• 3.3MB over 155Mbps link to Caltech and 170ms RTT.
• and 53MB over 2.5Gbps to Caltech!
• Consider a 10Gbs link with a RTT of 100ms and a
  TCP connection operating at 10Gbps
• the effect of a packet drop (due to link error)
  will drop the rate to 5Gbs. It will take 4
  MINUTES for TCP to ramp back up to 10Gbps.
• In order to stay in the regime of the TCP
  equation, 10 Gbit/s for a single stream of 1460
  byte segments, a packet loss rate of about 1E-10
  is required
• i.e. you should lose packets about once every
  five hours.

45
Acceptable link error rates
46
Very high speed file transfer (tentative
conclusions)

• Tcp/ip fairness only exist between similar flows,
  i.e.
• similar duration,
• similar RTTs.
• Tcp/ip congestion avoidance algorithms need to be
  revisited (e.g. Vegas rather then Reno/NewReno).
• Current ways of circumventing the problem, e.g.
• Multi-stream parallel socket
• just bandages or the practical solution to the
  problem?
• Web100, a 3MUSD NSF project, might help
  enormously!
• better TCP/IP instrumentation (MIB)
• self-tuning
• tools for measuring performance
• improved FTP implementation
• Non-Tcp/ip based transport solution, use of
  Forward Error Corrections (FEC), Early Congestion
  Notifications (ECN) rather than active queue
  management techniques (RED/WRED)?

47
CERNs new firewall technology and topology
Gbit Ethernet
Cabletron SSR
Gbit Ethernet
Fast Ethernet
FastEthernet
DxmonFE and FDDIbridge
CiscoPIX
Cisco RSP7000
FastEthernet
100/1000 Ethernet
FastEthernet
Cabletron SSR
Securitymonitor
Gbit Ethernet
48
CERNs New firewall routing/recovery
Cernh3
Cernh6
Cernh9
Fullrouting
TEN-155
Static
Backupdefault
Default
Cernh2
iBGP
PIX
Cernh8
Policyrouting
iBGP
Rca80 (BGP RR)
OSPF(RIP2)
CERN Gb backbone