Internet2 Network Design

1
Internet2 Network Design

• Rick Summerhill, Director
• Network Research, Architecture, and Technologies
• Internet2
• Community Design Workshop
• 15, 16 June 2006
• Indianapolis, IN

2
Contents

• Strategic Objectives
• Background
• Requirements
• Architecture
• Network Usage Policy
• Engineering
• Network Research
• Advisory Groups
• Partnerships
• Agreement Summary

3
Strategic Objectives
Strategic Objectives

• Ensure community control of the underlying
  network infrastructure
• Control those elements that allow development of
  new network capabilities
• Leverage the capabilities of a global
  telecommunications leader
• Providing carrier class reliability and expanded
  breadth of services, along with a broad set of
  partnership options
• Capitalize on the latest technological
  advancements in networking
• Create an asset that benefits the entire
  community - researchers, universities, regional
  optical networks, industry, government, K-12, and
  the international community

4
Background - I
Background

• A defining goal of Internet2 is to support the US
  research universities need for scalable,
  sustainable, high-performance networking
• In 1998, the Abilene network was created using
  2.5 Gbps SONET circuits as part of a partnership
  with Qwest Communications, Cisco Systems, and
  Nortel Networks
• The original agreement with Qwest was later
  extended to October of 2007 and the Abilene
  network was upgraded to a 10 Gbps network
• A further extension for one year was possible if
  notification was made by March 31, 2006, an
  extension that was recently declined

5
Background - II
Background

• Abilene is an IP packet based network similar in
  design to the original NSFnet, but with much
  greater capabilities and bandwidths
• Today there is great Interest in Hybrid networks
• Greater availability of circuit based
  capabilities
• Greater interest world wide to bring circuit
  based services closer to the edge
• Most RONs now have developed this capability
  already
• The HOPI project has experimented with the basic
  ideas for the last two years

6
Process I
Background

• With pending end of MoU with Qwest, Internet2
  began complete examination of all the
  possibilities
• Think out of the box - look at new ideas
• Consider different network options - IP, hybrid
  networks, etc.
• Considered a variety of different carrier and
  existing fiber options. For example,
• How does one share wavelengths on an existing
  system?
• Several viable options were available

7
Process II
Background

• In the end, one option stood out - the Dedicated
  Wave System (DWS) from Level3
• A Hybrid Network, not just an IP network and
  waves
• Supports a wide variety of services and
  capabilities, from highly experimental to
  production level
• Control at layer 1
• Uses equipment and fiber dedicated to Internet2
• Internet2 board has approved the DWS for the
  Internet2 network and the contract has been
  executed.

8
Requirements
Requirements

• Throughout the process, the requirements of the
  community were considered - the process did not
  occur in a vacuum
• Requirements documents produced during the spring
  of 2005 were considered
• The Group A report, produced by a joint
  engineering committee from NLR and describing a
  wide ranging set of network attributes
• The Abilene Technical Advisory Committee (TAC)
  Report outlining a need to provide for new
  experimental services along with production
  services like IP
• The MORPHnet document (1997) - Multi-Modal
  Organizational Research and Production
  Heterogeneous Network

9
Requirements II
Requirements

• Input from the Quilt Optical workshops and
  discussions with the RONs
• Research examining a wide variety of different
  types of capabilities
• DRAGON and CHEETAH - NSF funded projects
  examining dynamic provisioning
• Science related projects such as UltraLight and
  eVLBI
• OptIPuter and TeraGrid
• HOPI Testbed
• Input from the International community
• GEANT2 providing hybrid types of capabilities
• GLIF and lightpath capabilities for scientific
  work

10
Process Revisited
Requirements

• The actual process of negotiating with Level3 was
  done by a relatively small number of Internet2
  staff
• The negotiations could not have been done with a
  large community group
• The process created a framework for the Internet2
  network and enabling community involvement
• The process used the knowledge gained from the
  above requirements documents and discussions, and
  was done in the spirit of developing a network to
  serve the community by providing a wide range of
  capabilities for advancing the state of the art
  of networking
• Within the above framework, there are many
  details yet to be understood and ideas to be
  developed
• That is the purpose of this meeting, starting the
  process and
• Leading to network deployment, future planning,
  and continuing community involvement

11
Architecture Design Goals I
Architecture

• Develop an innovative optical system on a
  national footprint to serve the broad research
  and education community
• The goal is to support all Internet2 members and
  to enhance the ability to serve a wider community
• Develop a hybrid network capable of providing
  point-to-point services together with an IP
  network
• Every connector (RON) connects a NewNet backbone
  ring across a metro location not requiring
  extensive backhaul
• The community retains complete control of the
  layer 1 optical system including provisioning and
  switching of wavelengths

12
Architecture Design Goals II
Architecture

• Internet2 does not have to concentrate on
  reliability and sparing - Level 3 is responsible
  for an SLA.
• The community focus is on networking, not on
  managing devices like amplifiers
• The system is capable of supporting network
  research in wide variety ways
• Minimal Conditions of Use (CoU), allowing full
  participation from the entire community in
  providing new services and capabilities

13
The System - I
Architecture

• System uses dedicated fiber from the carrier
• Significant financial advantage if owned (via
  IRU) footprint is used
• Built on Infinera platform providing innovative
  optical technology
• Simple and convenient add/drop technology
• Simple and convenient wave setup
• Demonstrated high reliability in initial period
  of operation on the Level3 network
• Economics of Infinera system are disruptive in
  the market place

14
The System - II
Architecture

• Architecture has maximum flexibility. Every RON
  can access every wave on the system if needed
• System includes grooming capabilities -
  lightpaths can be built over Ethernet or SONET
• Can take advantage of advanced SONET cpabilities
  like GFP, VCAT, and LCAS
• Capable of lightpath provisioning to the campus

15
Lightpath Capabilities
Architecture
Rick Summerhill
16
System - III
Architecture

• RON connectivity expected to evolve to 2 x 10
  Gbps connections
• 10 Gbps IP connection
• 10 Gbps point-to-point connection (capable of
  STS-1 granularity lightpaths provisioned in
  seconds), most likely provision using Ethernet
  (GFP based)
• Hybrid capabilities
• Expect 20 - 24 connectors
• Simple and consistent connection scheme
• Promoting aggregation
• Need input and discussion on exceptional cases

17
System - IV
Architecture

• The following diagrams represent preliminary
  thoughts on topology
• Exact locations may vary, but fiber is expected
  to be on the Level3 footprint
• One of the goals of this meeting is to clarify
  and optimize these locations

18
Layer 1 Topology
Architecture
PROVISIONAL TOPOLOGY SUBJECT TO DISCUSSION
Rick Summerhill
19
IP Network
Architecture

• IP network built on top of optical system
• High reliability - architecture provides a
  variety of protection options
• Commodity service offering - standard connection
  will include commodity services
• Current plan is to continue to use highly
  reliable Juniper routers, but open to new
  technologies
• Will likely use fewer routers, emphasizing
  point-to-point capabilities and hybrid networking
• Potential near term option of 40 Gbps
• Community input is important these issues

20
Layer 1 Topology with IP Network
Architecture
PROVISIONAL TOPOLOGY SUBJECT TO DISCUSSION
Rick Summerhill
21
Architecture Summary
Architecture

• Initially provisioned with ten 10-Gbps
  wavelengths (100 Gbps)
• Capacity up to 80 wavelengths (0.8 Tbps)
• Scalability Potential for 40 100 Gbps
  wavelength support
• Reliability Carrier provides standard service
  assurances for these wavelengths
• Provide a variety of different types of
  reliability
• Flexibility Support for dynamic provisioning and
  wavelength switching

22
Node Architecture
Architecture

• Advanced optical DWDM equipment
• Grooming capabilities to provide sub channels
• Support for other networks having a national
  footprint
• Simplified and standardized interface to
  connectors, exchange points, and other global
  research and education networks
• 2 x 10 Gbps interfaces
• Example displayed is the Denver optical node,
  showing connections required by the collocated IP
  router node and an additional network

PROVISIONAL ARCHITECTURE SUBJECT TO DISCUSSION
Rick Summerhill
23
Node Architecture
Architecture

• Node without IP router
• IP connection to remote router
• Same basic architecture

PROVISIONAL ARCHITECTURE SUBJECT TO DISCUSSION
Rick Summerhill
24
Node Architecture
Architecture

• Node without network connector
• Support for Other Network
• Same basic architecture

PROVISIONAL ARCHITECTURE SUBJECT TO DISCUSSION
Rick Summerhill
25
Example Circuit Services - I
Architecture

• Services on NewNet
• Short Term Dynamically Configured STS-1
  Granularity Deterministic Circuits using two
  waves on the full footprint, with framing either
  SONET or GFP mapped Ethernet. There is no
  additional cost to use these capabilities. They
  are included in the base connectivity Costs. The
  may be call blocking in the use of these
  capabilities.
• Long Term Circuits configured as above. There is
  an extra cost for these circuits, proportional to
  bandwidth, and charged on a day-by-day basis.
  The shortest term is one day, and there is no
  call blocking for these capabilities.
• Long Term full waves configured and provisioned
  for a minimum of a year.

26
Example Circuit Services - II
Architecture

• Additional Carrier Services
• WaveCo services provisioned by Level3 as part of
  the agreement.

27
Example Projects on NewNet - I
Architecture

• Dynamic services
• Creating lightpaths across multiple
  administrative domains in seconds
• Development of Domain Controllers
• Support for applications - setup of Application
  SpecificTopologies
• Dynamic inclusion of optical paths into IP
  networks
• TCP data transfers
• File transfers with long segments not having to
  require congestion control

28
Example Projects on NewNet - II
Architecture

• Network research projects deploying programmable
  Ethernet capable chipsets at each optical node
  using NetFPGA cards from Stanford
• Service trial with GEANT2 on dynamic provisioning
  of 1 GigE circuits across Internet2 and GEANT2

29
Internet2/GEANT2 Service Trial
Architecture

• Develop inter-domain 1 GigE services between
  hosts or clusters of hosts in Europe and the US
• Develop the technology to provide real services
• Domain controllers and interoperability
• Develop policies and cost models for providing
  such services
• Interested in enlisting a couple of RONs and
  campuses willing to participate
• The following diagram illustrates the problems

30
Internet2/GEANT2 Service Trial
31
Network Usage Policy I
Network Usage Policy

• The use of the network facility shall be
  consistent with the broad set of non-profit
  objectives of the U.S. research and education
  community and its affiliates
• Internet2 will seek to maximize the ability of
  the network to support the broad collaboration
  requirements of the Internet2 membership and
  their affiliates to interact with the broadest
  scope of domestic and international networks and
  collaborating organizations
• Each organization utilizing the facility will be
  responsible for complying with all technical and
  operational standards developed for the network,
  and will avoid actions that adversely impact the
  performance or stability of the facility itself
  or those experienced by other users

32
Network Usage Policy II
Network Usage Policy

• The facility shall not be used for illegal,
  classified, life-safety, or unrelated commercial
  activities
• Additionally, Internet2 will seek to maximize to
  its members the value of IP transit by expanding
  connectivity through peering with commercial
  networks including content providers (e.g.,
  Google, Akamai) and other regional and national
  service providers (i.e., tier 2 ISPs)

33
Engineering - I
Engineering

• Building on the experience from projects such as
  Abilene, HOPI, and NLR, operational support is
  expected to fall into three broad categories
• Control plane development, deployment, and
  support. The Mid-Atlantic crossroads through the
  DRAGON project has focused on dynamic resource
  allocation.
• Application support, enabled by targeting key
  applications for the research community,
  applications such as eVLBI. The North Carolina
  Research and Education Network has focused on
  this area for the past year through the HOPI
  project
• Engineering, monitoring, and management has been
  provided by the Global Network Operations Center
  at Indiana University

34
Engineering - II
Engineering

• These groups were originally selected as the HOPI
  testbed support center
• Internet2 expects to continue these relationships
  for the new network

35
Network Research - I
Network Research

• The Internet2 network is an ideal platform for
  network research - the ability to support both
  highly experimental projects along with
  production based services is a key objective
• The Abilene observatory will be expanded to
  include the new capabilities of the network
• Data collection at all layers of the network,
  with datasets made available to network
  researchers
• Support for collocation of equipment in optical
  nodes

36
Network Research - II
Network Research

• Internet2 has supported individual network
  research projects whenever possible. Examples
  include
• The router clamp
• MPLS tunnels for researchers - these can now be
  supported at layer 1
• Support for the 100x100 project
• Flexibility is key feature of the new network
• Smaller projects at lower bandwidths can be
  supported on variable footprints
• Projects can also be supported for variable
  lengths of time
• Projects like VINI and GENI can be supported by
  the new network

37
Advisory Groups - I
Advisory Groups

• Internet2 currently has four councils Network
  Planning and Policy Advisory Council (NPPAC), the
  Applications Strategy Council (ASC), the Industry
  Strategy Council (ISC), and the Network Research
  Liaison Council (NRLC). The latter in particular
  will play a major role because of the
  developmental nature of the network (Note that
  the Internet2 board is in the process of
  reviewing the Internet2 governance)
• A network executive committee consisting of
  senior executives from Internet2, Level3, and
  others including Infinera

38
Advisory Groups - II
Advisory Groups

• A technical advisory committee that reports on a
  regular basis to NPPAC
• The advisory committee could have detailed
  working groups or other advisory groups (e.g.,
  the HOPI design team) organized to provide advice
  on specific subjects
• It could commission working groups to work on
  particular aspects of the network
• Input from the community is crucial - a later
  session on this will seek further input

39
Partnerships
Partnerships

• The agreement with Level3 and the many
  capabilities of the network provide an
  opportunity for strong partnerships to form. For
  example,
• The partnership with Level3 to examine new
  service capabilities
• Joint work on control plane investigations
• Examination of new optical capabilities with
  Infinera and others
• International partnerships, for example related
  to service trials
• Federal government partnerships, continuing and
  expanding on collaborations with other networks
  such as Esnet
• Continued work with the original Internet2
  partners, Qwest communications, Cisco Systems,
  and Nortel Networks

40
Additional Agreement Information
Agreement Summary

• Continuation of FiberCo
• Introduction of WaveCo
• Essentially, a circuit version of FiberCo
• Provides a platform for collaboration with the
  corporate and academic research communities to
  examine new services and ideas
• More information will be presented later

41
(No Transcript)