CAnet II

1
Internet2 DANTE CANARIE areas of focus
future collaboration Ann Arbour, MI 18-19
December 2003
René Hatem Chief Engineer, CANARIE rene.hatem_at_cana
rie.ca http//www.canarie.ca/
2
proposed agenda

• 0930 - 1100 - network infrastructure
  plans/services/operations
• 1100 - 1230 - development/implementation of new
  capabilities/services (bandwidth on
  demand/lightpath/etc.)
• 1330 - 1500  - end to end performance/performanc
  e measurement and monitoring architecture/infrastr
  ucture
• 1500 - 1630 - security (network and host
  efforts, network authentication/mobility)
• 1630 - 1730 p.m. - wrap up, identification of
  issues for further discussion on Friday

3
Network Infrastructure plans/services/operations

• CAnet 3 turned down August 2002
• CAnet 4 completed July 2002
• expected end of life August 2007
• 2 main TDM providers there can be more
• 19 pt-pt OC-192 TDM emulating WDM
• 15 CAnet 4 PoPs located in carrier colo space or
  University facility
• layer 1 gear Cisco ONS 15454

4
(No Transcript)
5
CAnet 4 is NOT an optical network

• CAnet 4 is made up of many parallel application
  empowered or customer empowered specific networks
  eg
• Computer back planes (Westgrid)
• High energy physics network
• It extends the Internet 2 architecture of
  GigaPOPs connecting a small number of RE
  institutions to a much finer scale with many
  parallel application empowered Internet 2 like
  networks connecting individual researchers and/or
  applications
• With added feature that the application or user
  can dynamically manage their own IP network
  topology
• Application empowered networks peer with each
  other at GigaPOPs and at optical switches which
  provides for greater reliability
• The CAnet 4 wavelengths and switches are
  partitioned such that application empowered
  networks can control their own partition and
  incorporate alarms, topology and discovery into
  their IP network
• New ITU draft standard Y.1312 - Layer 1 VPNs

6
Drivers for application empowered networks -1

• Distributed back planes between HPC Grid centers
• Westgrid 1 GbE moving to 10 GbE
• SHARCnet 1 GbE
• Distributed Single Mount file systems Yotta,
  Yotta - SGI
• Needs very consistent performance and throughput
  to truly act as a back plane
• Frequent topology changes to meet needs of
  specific applications
• Canada ATLAS 980 Gbytes FCAL data once a month
  from CERN to Carleton U, UoAlberta, UoArizona,
  etc
• Will significantly increase to Terabytes when
  production runs start
• Would take over 80 days on IP RE network

7
Drivers for application empowered networks - 2

• CERN Low level trigger data to UoAlberta with
  GARDEN
• Initially streaming data rates 1 Gbps moving to
  10Gbps later in the year
• Canadian virtual observatory
• .5 Tbyte per day to UoToronto and UoHawaii
• 250 Mbps continuous streaming from CCD devices
• Neptune Canada (and US?) under sea laboratory
  multiple HDTV cameras and sensors on sea floor
• Canada Light Source Synchrotron remote
  streaming of data acquisition to UoAlberta
• 2 to 5 Gbps continuously
• Canadian remote Nano and micro electronics
  laboratories

8
UCLP - A VPN alternative to GMPLSLayer 1 VPNs

• Allows customer to create customer owned and
  managed networks with resource heterogeneity
• Integration of wavelengths and dark fiber from
  different carriers
• Customers can manage their own restoral and
  protection schemes
• Customer can create daughter VPNs and offer to
  other users
• Customer can autonomously connect VPNs with other
  third parties
• Customer managed traffic engineering
• Dynamic BGP re-routing

9
Enables new network architectures

• Eliminate expensive high end routers and replace
  them with partial mesh of lightpaths between edge
  routers and servers
• But circuits are NOT intended to replace packet
  networks
• Extend the Internet end to end principle to the
  topology layer
• The success of the Internet is largely
  attributable to the classic e2e principle where
  control is at the edge
• Users can now control topology as well as
  applications
• Allowed development of exciting new applications
  or services
• Many exciting new overlay networks
• Knowledgeplane
• Oceanstore, Chord
• PlanetLab
• Application empowered networks allow overlay
  network to optimized underlay topology
• Customer controlled traffic engineering

10
The GigaPOP concept
University
University
Commodity Internet
GigaPOP
GigaPOP
University
University
vBNS
University
University
11
CAnet 4 Internet 3?
University
Dept
High Energy Physics
CERN
Commodity Internet
University
CAnet 4 IP
University
Direct Peering
GigaPOP
GigaPOP
University
University
eVBLI
12
High Energy Physics
Alberta Event Trigger
CERN
Alberta control switch directly using UCLP
software
User controlled topology
Alberta
High Energy Physics
GigaPOP
GigaPOP
STAR LIGHT
13
Network Infrastructure plans/services/operations

• CAnet 4 IP aggregation backbone
• One of many parallel networks across CAnet 4
• uses a small subset of lightpaths
• physically diverse paths (7 internal, 6 external)
• standardized on STS-24c Bw and GbE interfaces
• 5 routers for 15 GigaPoPs spread over 8000 km
• IPv4 and IPv6 are true protocol peers
• routing policy objectives
• enforce the CAnet 4 Acceptable Use Policy (IPv4
  only)
• minimize path latency
• accommodate the requirement for route diversity
• enforce symmetric routing

14
Lightpath Capabilities/services

• Dedicated lightpath services
• for end-end applications or for NRN use
• short term (8 weeks)
• STS-1, STS-3c, STS-6c, STS-9c, STS-12c, STS-24c,
  STS-48c, STS192c
• user-controlled (scheduled by end of 2004)

15
end-end performance/performance measurement

• tools
• iperf (host based)
• jaalaM apparenet (IP backbone only)
• netflow / cflowd / flowcan (IP backbone only)
• mrtg GbE traffic rates
• QVision
• traffic reports
• traffic map for display

16
unknown traffic

• Unknown network application traffic break down
• Cflowd/Flowscan rely on by port number to
  identify app. Unkown usually accounts for more
  than 40 of total.
• QVISION uses applications signature to capture
  application traffic on port as well as port
  hopping. Deployed on CAnet 4 as part of pilot
  project.
• Using Qvision we have discovered that greater
  than 70 of traffic is either NNTP plus music
  file sharing e.g. Kazaa, Bit Torrent, etc

17

• Cflowd/Flowscan network application traffic
  graphic for c4-RISQ

35 of CAnet 4 bound traffic and 64 of RISQ
bound traffic is Other
18
QVISION application graphic for RISQ peer
RISQ bound traffic
CAnet 4 bound traffic
Green is web application Yellow is P2P Gray is
Management Pink is known_to_client_or_server Blue
is data transfer Black is unknown
19
security

• network infrastructure
• private IP subnet for access
• ssh from known subnet only
• route and packet filtering
• prefix filtering based on IRR
• uRPF packet filtering on domestic routes
• looking at S-BGP and soBGP
• With UNB exploring prescriptive rather than
  descriptive wire speed security solutions
• Using UCLP lightpaths to create private VPNs
• Challenge and clean PCs before connecting to
  campus networks

20
international lightpath grid infrastructure

• TransLight
• Providing transit lightpaths to Taiwan and
  Ireland
• Soon will also provide lightpath to Korea
• GLIF
• HOPI ?