CHEETAH: Circuitswitched Highspeed EndtoEnd Transport ArcHitecture

1
CHEETAH Circuit-switched High-speed End-to-End
Transport ArcHitecture

• Concept/analysis
• Opticomm 2003 paper
• CHEETAH a scalable solution for wide usage
• Focus File transfers
• Practical implementation
• NSF EIN project UVA, NCSU, ORNL, CUNY
• Focussed on eScience project - Supernova
• NSF ANIR hardware signaling project
• NSF ITR project
• Commerical applications financial and retail
  institutions transfer large files

2
Outline

• CHEETAH concept file transfers
• Soap box
• A pitch for considering scalability while
  creating these optical testbeds
• NSF EIN project implementation
• Connection-oriented internet

3
CHEETAH concept

• What is it?
• Dynamic end-to-end circuits
• Circuit Ethernet EoS Ethernet
• Leverages
• Ethernet King of LANs
• SONET King of MANs/WANs
• Why this solution?
• Ethernet and SONET already deployed
• Add necessary upgrades to switches and software
  to end hosts to enable the use of these circuits

4
Add-on solution to Internet
5
Leverage presence of Internet path

• Run circuit-switched network in call blocking
  mode
• If call is blocked, fall back to Internet path
• Engineer network for high utilization at the cost
  of blocking
• File transfers Use Internet path for reverse
  direction error control and flow control messages
  and some retransmissions
• Same approach for other applications

6
Signaling protocol

• Since in CHEETAH we propose holding circuits only
  for duration of file transfer
• A 10MB file takes 800ms over a 100Mbps circuit
• Reduce end-to-end call setup delay to r.t.
  propagation delay
• We need fast signaling engines high call
  handling capacity at MSPPs/switches
• Solution Hardware implementation
• NSF funded project We have completed
  implementing an RSVP-TE subset on an FPGA
• Can handle 400,000 calls/sec
• Each call setup takes 4?s
• Research work hard for a university to actually
  build an operational switch. Nevertheless we are
  building a prototype SONET switch with a 40Gbps
  fabric and our signaling FPGA

7
Transport and routing

• Transport protocol for file transfers on
  combination circuit TCP/IP path
• Fixed-Rate Transport Protocol (FRTP) on circuit
  TCP on IP path
• Routing decision module
• Expected delays
• Utilization consideration

8
Crossover file sizes from delay perspective
When Tprop 50ms, always attempt a circuit
9
Plot of utilization u withrc 100Mbps, k20
For 50ms paths, set a crossover file size
When load is low, operate at a high blocking rate
Pb0.3
Pb0.01
10
To implementthe CHEETAH concept

• At circuit switches
• Ability for dynamic fast provisioning of circuits
• At end hosts
• Applications upgrade to use circuits
• A transport protocol, e.g., FRTP
• Signaling protocol client end
• Routing decision module choose between two
  network choices available to privileged CHEETAH
  end hosts

11
Outline

• CHEETAH concept file transfers
• Soap box
• A pitch for considering scalability while
  creating these optical testbeds
• NSF EIN project implementation
• Connection-oriented internet

12
Provisioning research community

• Example Canaries UCLP
• Created network to map Ethernet EoS Ethernet
  lightpaths with equipment like 15454s
• 15454s do not implement routing/signaling/LMP
  protocols
• For provisioning lightpaths
• Created external databases to manage routing
  information to reach a destination host, what is
  the set of switches to traverse?
• Create external databases to manage inventory
  information - which 454 has what cards?
• Inter-domain issues (multiple carriers) policy
  manager
• Then set up circuit with TL1 commands
• Another example Elematics software company
  that created NMSs for routing, signaling
  functionality
• key not integrated with network elements

13
Fine solution

• if we just want to build small-scale networks
• Our goal
• large-scale connection-oriented network
• why? reduce costs
• the very concept of eScience is possible because
  of the cheap wide-scale Internet
• the scientists we strive to serve today will be
  served better if we create large-scale
  connection-oriented networks capable of providing
  rate-guaranteed connectivity!
• Value of the network grows exponentially with
  number of scientists connected
• Example the huge-scale 64kbps telephone network
  would not have been possible without signaling
  integrated into switches

14
Problems with provisioning with NMSs

• Scalability
• Call setup delays
• Need to rethink the tens of seconds numbers if we
  need to support non-eScience apps. for
  scalability
• Using network management systems outside the
  network elements to manage
• routing data
• inventory data
• takes significant effort Operations costs
• Other problems
• Interoperability of different vendors equipment
• Inter-carrier issues

15
Scalability charges

• Levied against Optical/TDM networks
• Widely recognized that the current GLIF
  optical/TDM networking model does not scale
  beyond a limited number of sites Internet 2 talk
  dated 10/15/2003
• While such circuit-switched networks may not
  necessarily be suitable for deployment of the
  scale of the Internet, they are still viable
  candidates for specialized deployments for
  connecting a small number of DOE large-scale
  science nodes Report of DOE Workshop on Ultra
  High-Speed Transport Protocols and Dynamic
  Provisioning for Large-Scale Science
  Applications, April 10-11, 2003, Argonne, IL,
  dated Oct. 27, 2003.
• Unfair charge because optical/TDM networks
  could have distributed signaling/routing/LMP to
  make large-scale nets

GLIF Global Lambda Integration Facility
16
Inventory problem

• TL1 command to set up a crossconnect through a
  15454
• Command
• ENT-CRS ,CT
• Example
• ENT-CRS-STS1BODEGASTS-5-1,STS-12-51162WAY
• TID unique name for the system
• From and To Access Identifiers to identify
  timeslots on interfaces
• STS-1 on the card in Slot 5
• STS-5 on the card in Slot 12
• CTAG unique identifier used to match response
  with request
• CCT Crossconnection type e.g., 1WAY or 2WAY

Shouldnt need to hold this info outside the
switch itself!
17
Plug-and-play CL networks

• Network switches should be equipped with all the
  functionality needed for plug-and-play operation
• just as seamless as booting up a PC these days
  with a network card - because of DHCP
• DHCP not only gets IP address
• It discovers a gateway address too
• What is all the functionality for
  plug-and-play?
• back-stage
• address acquisitions
• automatic neighbor discovery (manages its own
  inventory)
• automatic topology discovery through routing
  protocol
• routing table computation
• front-stage
• packets show up forward them as per routing
  table
• Amazingly most of the above achieved with
  connectionless packet switches
• Ethernet switches
• Even IP routers?

18
To create a connection-oriented (CO) network in
the same plug-and-play vein

• Additional back-stage functions
• handle call setup requests/releases signaling
  protocol
• perform authorization and authentication before
  call setup
• gather call-level data for billing
• CO networks offer us the opportunity to build
  capitalistic networks to complement todays CL
  socialistic Internet
• Front-stage
• circuit-switched networks
• data shows up on interfaces switch based on
  position space, time, wavelength
• CO PS networks
• switch packets based on header data but hold and
  serve packets based to resource allocation for
  connection

19
Network management

• NM functions
• Fault management
• Performance monitoring
• Accounting management
• Security management
• Configuration management
• overall planning of topology
• All these are essential to the running of a
  network but these are back-office operations!
• Hence fine to offload these to NMSs.
• But leave front-office operations at the NEs
• dynamic provisioning is a front-office operation
  in CO nets

20
Signaling approach to connection setup
distributed

• Call setup request carries destination IP address
  D bandwidth B incoming timeslot/l
• Lookup routing data table (same function as in an
  IP router)
• find outgoing interface O to reach destination D
• Resource allocation
• Allocate bandwidth B on interface O
• Select outgoing timeslot/l
• Program switch fabric
• Map incoming timeslot/l to outgoing timeslot/l
• Send call setup request to neighbor connected by
  interface O

21
Industry answer tosupport distributed signaling
approach

• IETF GMPLS
• Routing OSPF-TE
• Routing built into network switches
• Signaling RSVP-TE
• Link Management Protocol (LMP)
• Inventory data stored in network switches
• Auto-discovery of neighbors
• OIFs UNI-C, UNI-N, NNI
• Addresses carriers inter-domain issues

22
Actually implemented!

• Not just idle specifications
• Implemented by many switch vendors
• And interoperability-tested by an OIF-sponsored
  effort led by UNH
• Demoed at OFC2003
• Demoed at Isocore in March 2004
• Another demo planned for Supercomm June 2004

23
From keynote at Opticomm, Dallas, Oct. 03

• Rajiv Ramaswami, CTO , Optical Systems Group,
  Cisco, Keynote
• UCP (Unified Control Plane) Benefits
• Superfast Provisioning
• Enables E2E circuit setup without SP intervention
  while reducing provisioning times
• Enables future bandwidth on demand applications
  as policy billing standards mature
• Enhanced Scalability
• Network level Support for thousands of nodes,
  links and circuits per inter-connected network
• Lightweight EMS Move from EMS based
  (centralized) provisioning to node level
  (distributed) provisioning using signaling

24
UCP Benefits contd.

• Interoperable vendor implementations
• Reduces EMS/NMS integration / interoperability
  issues
• UCP/GMPLS A Driver for Evolution
• Build Network as a Database
• Simplify provisioning by driving intelligence
  (topology, circuit inventory and link
  characteristics) into the NEs with updates to EMS
  (CTC/CTM)
• Enable migration from an NMS based network
  database to NEs based network database,
  retrievable on demand by NMS
• Deliver Advanced Benefits
• New services features (Ethernet,OVPN Storage)
  not possible today
• Reduce costs, increase revenues, address scale of
  growing networks
• Enable multi network/vendor/SP interoperability

25
Our research community

• It was hard to wait for network equipment
  manufacturers to integrate these
  routing/signaling/LMP into their NEs
• It was much easier for us to go off on our own
  and build NMS software external to switches for
  routing, inventory, provisioning
• But now, we are there. The equipment vendors do
  have such NEs. Lets use them.

26
Outline

• CHEETAH concept file transfers
• Soap box
• A pitch for considering scalability while
  creating these optical testbeds
• NSF EIN project implementation
• Connection-oriented internet

27
CHEETAH implementation NSF EIN project

• Buy circuit switches that come close to
  plug-and-play
• distributed signaling solution
• distributed routing solution
• auto-discovery of neighbors
• Currently, SONET and some WDM switches integrated
  with GMPLS capability
• RSVP-TE
• OSPF-TE
• LMP

28
A lab demo with one circuit switch emulating a
SONET network

• Implement FRTP, signaling client, routing
  decision module at end hosts (Linux)

29
Wide-area deployment

• Deploy SONET switches in Raleigh and Atlanta
• Our scientist co-PIs are in NCSU and ORNL
• Networking co-PIs in ORNL already have OC192 link
  from ORNL to Atlanta
• Purchase lambda from NLR (?) for research tests
  terminate on SONET OC48 cards
• Test end host CHEETAH software for file transfers
  from ORNL Cray to NCSU (TB files created by TSI
  scientists)
• Find other scientists who connect to ORNL in
  NC/DC area to test sharing of OC48
  Raleigh-Atlanta link to test signaling

30
Revision to CHEETAH thinking

• Inspite of wide-spread deployed SONET network,
  upgrade of these switches with GMPLS software is
  eons away!
• Now, we have the possibility of MPLS LSPs through
  Abilene
• And dedicated circuits using VLANs through
  Ethernet switches
• Hence, we should design a ....

31
Connection-oriented internet

• To complement todays Connectionless Internet
• Whats a CO internet?
• an internetwork of various CO networks
• whats CO a network that can be asked for
  bandwidth before data is sent
• VLANs, MPLS, SONET, optical lightpaths
• User-plane interworking
• Control-plane interworking

32
Conclusions

• Lets build such a connection-oriented internet!
• To make this large-scale, need to revisit
  assumptions about applications
• not just TB files, but perhaps even 100MB files
• immersive apps graphics apps
• AccessGrid video-telephony, video-conferencing

Questions? Malathi Veeraraghavan, Univ. of
Virginia mv_at_cs.virginia.edu