U'S' Optical Network Testbeds Status

1
U.S. Optical Network Testbeds Status

• Grant Miller
• National Coordination Office
• July 3, 2004

2
U.S. Optical Network Testbeds

• UltraSciencenet DOE
• CHEETAH NSF
• DRAGON NSF
• STARLight NSF
• HOPI Internet2
• OMNInet Nortel
• National LambdaRail (NLR) Consortium
• Also CALREN, Colorado, Connecticut, Florida,
  Indiana (I-Light), Illinois (I-Wire),
  Maryland/DC/ Virginia (MAX), Michigan, Minnesota,
  NY(NEREN), North Carolina, Ohio, Oregon, Rhode
  Island, SURA, Texas, Utah, Wisconsin

3
Applications Network Performance

• Applications requirements are drivers for
  bandwidth needs
• DWDM delivers up to 100 Gbps
• SONET framing delivers up to 40 Gbps
• TCP/IP delivers about 15 Gbps
• Site firewalls deliver about 7 Gbps to an
  application
• But high-end applications require 40 Gbps, e.g.
• Terascale Supernova Initiative Terabyte in days
• High Energy Physics Terabyte data transfers
• Therefore, consider optical networking options
  to
• Bypass firewalls
• Carry non-IP frames (e.g., Fiber Channel over
  SONET)

4
UltraScienceNet Planned Capabilities

• Sparse, lambda-switched, dedicated,
  channel-provisioned testbed
• Connects hubs close to DOEs largest science
  users Users pay last-mile costs
• Provide an evolving matrix of switching
  capabilities
• Separately fund research projects to support
  applications
• High-performance protocols
• Control
• Visualization

5
UltraScienceNet Resources

• Off-hours capacity from DOEs ESnet Expected 2 x
  OC48 between Sunnyvale and Chicago
• Dedicated lambdas on NLR
• 2 x 10G lambdas between Chicago and Sunnyvale
• Possibly two more in year 2 or 3
• Two dedicated lambdas on the Oak Ridge National
  Laboratory Chicago-Atlanta Connector
• Switching technologies
• Ciena, Cisco or Sycamore (SONET) migrating to
• Calient Glimmerglass all-optical or a hybrid
• Progression of point-to-point (P2P) transport
  technologies (Fiber channel, Infiniband)
• Migrate to the production ESnet environment

6
UltraScienceNet Engineering Approach

• Network engineering
• Connect Atlanta-Chicago via ORNL
• 16 P2P circuits OC192, 10 Gig-E
• Provide the NLR alternate route to close its ring
• Buy IRUs from Qwest and TVA
• Light with equipment from Ciena
• Please see http//www.csm.ornl.gov/ultranet

7
CHEETAH Sponsored by the NSFJanuary
2004-December 2007

• Goal Develop the infrastructure and networking
  technologies to support a broad class of
  escience, and specifically the Terascale
  Supernova Initiative
• Concept Create a network to provide on-demand
  end-to-end dedicated bandwidth channels to
  applications as well as an IP path to support
• High throughput file transfers
• Interactive remote visualization
• Remote computational steering
• Multipoint collaborative computation
• Participation by
• Oak Ridge National Laboratories
• University of Virginia
• North Carolina State University
• City University of New York

8
CHEETAH Technology

• Dedicated channel High-speed Ethernet mapped to
  Ethernet-over-SONET circuit
• Leverage existing technologies
• 100 Mbps/1Gbps Ethernet in LANs
• SONET in MANs/WANs
• Availability of Multi-Service Provisioning
  Platforms (MSPP) class devices that can
• Map Ethernet to Ethernet over SONET
• Cross-connect dynamically
• Rate-control Ethernet ports
• Provide a 1 Gbps ORNL-Atlanta Channel

9
CHEETAH Implementation

• Application tools
• File tansfer
• Visualization Ensight or Aspect/Paraview, Custom
  open GL codes
• Computational steering
• Transport protocols
• File transfers
• Control channels small portion of channel
  bandwidth
• Rate-based flow control 2 x Dell 2.4 Ghz PCs
  with 100 Mhz 64-bit PCI busses
• Make it wide-area e.g., use NLR, MPLS tunnels
  through Abilene, or collocated switches at
  Abilene PoPs

10
DRAGON Funded by NSF

• Provide Cyberinfrastructure application support
  and advanced network services on an experimental
  infrastructure using emerging standards and
  technology
• Advanced services
• Dynamic provisioning of deterministic end-to-end
  paths
• Rapid provisioning of application-specific net
  topologies
• Reserve resources and topology in advance,
  instantiate as needed
• Provide AAA
• Protocol, format, framing agnostic direct
  transmission of any optical signal

11
DRAGON Design

• All optical transport in the metro core
  Edge-to-edge wavelength switching. Push OEO
  demarc to the edge
• Standardized GMPLS protocols for dynamic
  provisioning intra-domain connections
• Develop inter-domain protocols to distribute
  Transport Layer Capability Sets (TLCS) across
  multiple domains

12
DRAGON Research Areas

• Inter-domain routing to advertise the TLCS
  Network Aware Resource Broker (NARB)
• Ability to request deterministic network
  resources
• Virtual label switched routers Translate GMPLS
  requests into configuration commands to switches
  via the SNMP protocol
• Minimize OEO requirements for light-paths
• Formalized definition language to instantiate
  complex application topologies

13
DRAGON Network Points

• Un. Of Maryland
• NASA GSFC
• DC, Northeast
• NCSA
• ISI-East
• Connection to Bossnet, MIT/Haystack
• Note Commercial partner is Movaz

14
StarLight

• Exchange point 1 GigE and 10 GigE for national
  and international research networks (over 30
  networks)
• NSF Teragrid (10 x 10 Gb over I-wire), Extensible
  Teragrid Facility (ETF) NLR
• UltraScienceNet (DOE)
• Global Lambda Integrated Facility (GLIF)
• GEANT
• WIDE
• APAN
• SURFnet
• Many others
• Calient Diamondwave Switches at StarLight and
  NetherLight facilities

15
Hybrid Optical Packet Infrastructure (HOPI)
Project

• Architecture based on availability of optical
  infrastructure dark fiber acquisitions at
  national, regional, local level
• Implement a hybrid of shared IP packet switching
  and dynamically provisioned optical lambdas.
• Infrastructure
• MPLS tunnels on Abilene
• Internet2 Wave on the NLR footprint
• Regional Optical Networks (RONs)
• Model waves using deterministic paths
• Provide basic service of 1 GigE or 10 GigE
  unidirectional point-to-point path
• Access through Abilene through direct or MPLS
  L2VPN tunnel
• Support 15-20 experiments, e.g. dynamic
  provisioning

16
HOPI Status

• Deterministic path CERN to LA
• - Internet2
• GEANT
• CANARIE
• Others StarLight, SURFnet
• Address issues
• Different technologies
• Cross administrative domains
• Dynamic provisioning
• http//hopi.internet2.edu

17
Optical Metro Network Initiative (OMNInet)

• Metropolitan 10 Gbps DWDM WAN and LAN photonic
  switched network trial
• Partnership of Nortel Networks, SBC
  Communications, International Corporation of
  Advanced Internet Research (iCAIR)/Northwestern
  Un.
• Services O-VPNs, dial-a-lambda service, router
  bypass
• Emerging applications Optical Grids, storage
  on-demand, data mining, 3D teleconferencing,
  large-science apps, visualization

18
OMNInet Architecture

• 4 sites in Chicago
• 6 fiber spans
• 4 wavelength planes switching without wavelength
  translation
• DWDM Lightpaths

19
National LambdaRail (NLR)

• National-scale member-owned/managed optical
  networking and research facility
• NLR Objective Bridge the gap between optical
  networking research and state-of-the-art
  applications research
• NLR is a set of facilities, capabilities, and
  services supporting multiple experimental and
  production networks for the U.S. research
  community
• Networks exist side-by-side on the same fiber but
  are physically and operationally distinct
• Virtuous Circles Participants dedicated optical
  capability from campus labs to the NLR network.
  NLR works with RONs to deliver NLR capabilities
  to campuses.

20
NLR Characteristics

• Experimental platform for research
• Optical switching and network layers
• 50 of capacity is reserved for research
• Experimental Support Center
• Use high-speed Ethernet for WAN Transport First
  national-scale Ethernet deployment
• Sparse backbone technology Members develop local
  optical networking and performance in their areas
• Acceptable Use Policy Free

21
NLR Planned Capabilities

• Point-to-point waves 10 GigE LAN PHY, OC-192
  Cisco systems
• Switched Ethernet network using Cisco switches
• Experimental IP network using Cisco routers
• Dark fiber for optical layer research
• Traditional NOC services
• Dense Wave Division Multiplexing national optical
  footprint Capacity of 40 wavelengths per fiber
  pair deployed on 10,000 miles of dark fiber

22
NLR Deployment

• Initial 4 lambdas
• One lambda for national switched Ethernet
  experimental network
• One lambda for national 10 Gbps IP network
• One lambda for quick start facility for new
  research projects
• One lambda for Internet2 HOPI testbed
• Additional lambdas provisioned as needed
• National deployment (California to DC to Florida)
  by August 2004
• http//www.nationallambdarail.org