SINET3 An Academic Network Bridges Now and the Future

1
APII workshop 2008
SINET3 - An Academic Network Bridges Now and the
Future
Yusheng Ji National Institute of Informatics
(NII)
2
Outline

• Introduction
• Network Structure of SINET3
• Network Service Features in SINET3
• Architecture to Accommodate Diversified Services
• Summary

3
Evolution of Science Information Network in Japan

• SINET3 is integrated successor network to two
  academic networks, SINET and Super-SINET,
  economically and flexibly providing rich variety
  of services.
• SINET3 started its operations in April 2007 and
  completed its migration in May 2007.

Packet Switching Network
?1987.1
Internet backbone for more than 700 universities
and research institutions
SINET
?1992.4
Super-high-speed environment for cutting-edge
research
Super-SINET
?2002.1
SINET3
- Growing traffic and diversified user
requirements - Limited abilities of existing IP
routers - New trend of end-to-end circuit services
?2007.4
4
Network Structure of SINET3

• SINET3 has two-layer structure with edge and core
  nodes.
• Edge nodes are edge layer-1 switches with layer-2
  multiplexing, which are located in universities
  or research institutions and accommodate user
  equipment.
• Core nodes are composed of high-end IP routers
  and core layer-1 switches located in public data
  centers.

SINET/Super-SINET
SINET3
IP Router
Backbone
Core Node
Backbone Router
Core L1 Switch

Super-SINET/SINET Router
Edge L1 Switch With L2 Mux
Edge Node
SINET Router
10GE/GE/FE STM-16
L1 (Dedicated/On-demand)
L3 (IP)
L2 (Ethernet)
L3 (IP)
5
Network Topology of SINET3

• Has 63 edge nodes and 12 core nodes (75 layer-1
  switches and 12 IP routers).
• Deploys Japans first 40 Gbps lines between
  Tokyo, Nagoya, and Osaka.
• Links form three loops in backbone to enable
  quick service recovery against link and node
  failures and for efficient use of network
  bandwidth.

40 Gbps package
L1 Switch(NEC UN5000)
IP Router (Juniper T640)
Hong Kong
Los Angeles
2.4 Gbps
622 Mbps
Singapore
10 Gbps
622 Mbps
New York
40 Gbps 10 to 20 Gbps 1 to 20 Gbps Core
Node (L1 Switch IP Router) Edge Node (L1
Switch)
Japans first 40 Gbps (STM256) lines
6
High-level Network Architecture

• High-level network architecture is composed of
  transport network, adaptive network control
  platform, and user-oriented service control
  platform to accommodate a variety of services,
  effectively control the network, and attain an
  easy-to-use network.

• User-oriented Service Control Platform

• Bandwidth on demand
• Enhanced network security
• Middleware/application coordination

• Adaptive Network Control Platform

SINET3
BoD
Middleware
Security

• Dynamic resource control
• Resilient network control
• Performance monitoring

UNI, API, GUI
Service Control Platform
Dynamic Control
User side
Network Control Platform

• Hybrid Optical and IP/MPLS Network

IPv6, Multicast, VPN, QoS

• Multi-layer accommodation
• Enriched VPN
• Enhanced QoS
• High availability
• Flexible resource assignment
• 40 Gbps (STM-256) lines

Layer 3 (IP)
Layer 2 (Ethernet/MPLS)
Layer 1 (TDM/Lambda)
7
Service Features in SINET3

• SINET3 is the new Japanese academic backbone
  network for more than 700 universities and
  research institutions, providing a rich variety
  of services.
• SINET3 emphasizes four service aspects transfer
  layer, virtual private network (VPN),
  quality-of-service (QoS), and bandwidth on
  demand.

8
Multiple Layer Services

• SINET3 provides all transfer layer services in a
  single platform.
• Users can freely choose the best transfer layer
  for their applications.
• It flexibly assigns network resource for
  ever-changing and unpredictable service demands.

Past Networks
SINET3
User Equipment
User Equipment
IP network (Layer 3)
Innovative Integration
IP Router
IP Router

• Provides all transfer layer services
• Integrated network

Ethernet network (Layer 2)
Ethernet Switch
Ethernet Switch
Dedicated line network (Layer 1)
Cutting-edge Device
Cutting-edge Device
9
Multiple VPN Services

• For collaborative research activity closed user
  group environment (virtual private network VPN)
  is essential for security reasons.
• Users can choose from L3VPN (IP), L2VPN/VPLS
  (Ethernet), and L1VPN services.

Virtual Private Network (VPN) Virtual Private
LAN Service (VPLS)
Previous network (Super-SINET)
SINET3
e.g. Nuclear fusion research
IP Router
IP-based VPN (L3VPN)
Expansion of Services Sites
L3VPN
IP Router
Secure Closed User Group
VPLS
L1VPN
Ethernet Switch
e.g. eVLBI
e.g. Earthquake research, Grid computing
research
Analysis device
10
Multiple QoS Services

• SINET3 provides two types of QoS services
  packet-based and circuit-based.
• Packet-based QoS services use four forwarding
  queues expedited forwarding (EF), network
  control (NC), assured forwarding (AF), best
  effort (BE).
• Circuit-based QoS services assign end-to-end
  bandwidth on demand and provide smallest packet
  delay, no delay variance, no packet loss.

SINET3
SINET/Super-SINET
e.g. HDTV for remote education and medicine
HDTV
QoS- aware
Node
Congestion
Expedited
NW control
End-to-end on-demand path
Assured
Best effort
e.g. OSPF, BGP, RSVP-TE, TRAP
Best Effort
Uncompressed HDTV

• Network congestion affects all services

• Packet-based and circuit- based QoS services

11
Bandwidth on Demand (BoD) Services

• SINET3 provides bandwidth-on-demand (BoD)
  services as part of layer-1 services.
• Users can specify the destinations, duration,
  bandwidth with granularity of about 150Mbps, and
  route option.
• BoD server receives path setup requests for
  users, schedules accepted reservations, and
  triggers layer-1 path setup.

Hokkaido
Web-based Interface (Destination, Duration,
Bandwidth, Route option)
On-demand Server
User
1 Gbps (1300-1400)
Layer-1 path setup trigger
Osaka
Fukuoka
2 Gbps (1700-1800)
On-demand layer-1 path
1 Gbps (1500-1600)
SINET3
Tokyo
12
Accommodation of Multi-layer Services

• L3 and L2 traffic are accommodated in shared
  bandwidth by L2 multiplexing and transferred to
  IP router, where each traffic is encapsulated
  with MPLS labels as needed.
• L1 service is assigned dedicated bandwidth and
  separated from L2/3 traffic.
• L2/3 traffic bandwidth can be hitlessly changed
  by LCAS for flexible accommodation of multi-layer
  services. For adjusting L2/3 traffic to assigned
  bandwidth, we use a flow control using PAUSE
  frames between layer-1 switch and IP router.

Shared Layer-2/3 traffic

Multi-protocol Label Switching (MPLS) Link
Capacity Adjustment Scheme (LCAS)
MAC
data
preamble /SFD
MAC
data
GFP header
VLAN
VLAN
MAC
data
preamble /SFD
VLAN
FE/GE/10GE
Flow Control (IEEE802.3x)
Hitless bandwidth change by LCAS
10GE
data
IP
Ether
data
IP
Ether
VLAN
IP Router
L2 Mux
L3
data
Ether
VLAN
data
IP
IP Router
IP/MPLS
IP/MPLS traffic
data
Ether
MPLS
data
Ether
VLAN
MPLS

Layer-1 service
L2
Ethernet Switch
STM64/STM16
Core L1 Switch
Edge L1 Switch
L1
Cutting-edge device
GE/10GE/ STM16
STM256/STM64
SINET3
13
Accommodation of Multi-VPN Services

• L3VPN, L2VPN, and VPLS as well as IPv4/IPv6 dual
  stack are logically separated by internal VLAN
  tags and logical routers. Neighboring logical
  routers of each service are connected to each
  other with logical interfaces (i.e. VLANs).
• L1VPN and on-demand services need GMPLS protocols
  to set up layer-1 paths and have separate control
  planes from that of IP routers.

Generalized MPLS (GMPLS)
or
data
IP
data
IP
MPLS
10GE
data
Ether
VLAN
MPLS
Logical Router
IP Router
IPv4/IPv6 (L3)
Virtual routing/forwarding table
Aggregation
L3VPN (L3)
data
IP
Ether
L2 MUX
data
IP
Ether
VLAN
IPv4/IPv6
L2VPN (L2)
L3
L3VPN
data
Ether
VLAN
VPLS (L2)
data
Ether
Shared Layer-2/3 traffic
L2VPN
L2
Layer-1 traffic
VPLS
IP/MPLS traffic
L1 VPN
L1 VPN
L1
L1VPN
Edge L1SW
Core L1SW
GMPLS Control Plane
14
Accommodation of QoS Control Services

• Packet-based QoS
• User Priority bits of internal VLAN tags are
  marked at edge L2 MUX.
• User Priority bits are mapped into DSCP (IP) or
  EXP (MPLS) bits at IP router.
• There are four forwarding queues EF, NC, AF,
  BE.
• Circuit-based QoS
• End-to-end bandwidth with VC-4 (about 150Mbps)
  granularity is assigned on demand.

Marking User Priority bits by identifying
IP/Ethernet header
Mapping User Priority bits into IP DSCP or MPLS
EXP bits
IP
data
IP
IP Router
L2 MUX
IP
VLAN
IP (L3)
Priority mapping for IP
Prioritizing for IP
IP
MPLS
EF
EF
DSCP EXP based classifier
User Priority based classifier
Internal VLAN tag Identifier
NC
NC
data
Ether
Priority mapping for Ether
Shared Layer-2/3 traffic
AF
Prioritizing for Ether
AF
Ethernet (L2)
BE
BE
VLAN
MPLS
Ether
Ether
Dedicated (L1)
Layer-1 traffic
Edge L1SW
Core L1SW

• Smallest packet delay
• No delay variance
• No packet loss

QoS class identifier
15
Accommodation of Bandwidth-on-Demand Services

• For circuit-based QoS services, we have
  bandwidth-on-demand (BoD) capabilities.
• BoD server receives reservation requests,
  schedules accepted requests, and triggers
  layer-1 path setup to source layer-1 switch via
  L1-OPS.
• Source layer-1 switch sets up layer-1 path toward
  destination using GMPLS.
• BoD server changes L2/L3 traffic bandwidth by
  LCAS via L1-OPS as needed.

Destinations, Duration, Bandwidth, Route Option
Layer-1 BoD Server
Scheduling
Route calculation
User
Front-end
Path control
Resource management
Path setup trigger
L1-OPS
Path setup request
GMPLS control and management plane
GMPLS
L1SW
L1SW
L1SW
L1SW
On-demand
L2 MUX
L2 MUX
Ethernet
IP
Hitless bandwidth change by LCAS
IP Router
IP Router
16
High-availability Networking Functions

• Multiple loops easily enable multi-layer traffic
  to be detoured in different directions.
• Layer-1 switches detect link failures very
  quickly and inform them to neighboring layer-1
  switches and IP routers.

Fukuoka
Hiroshima
Kyoto
Kanazawa
Hokkaido
TDM
IP
MPLS
MPLS Protection Fast Reroute
Tokyo2
IP route recalculation
(option) GMPLS LSP Rerouting
MPLS
TDM
IP
Sendai
Tsukuba
Tokyo1
Nagoya
Osaka
Matsuyama
17
Service Parameters of L1 BoD Services

• BoD server allows users to specify connection
  style destinations, duration, bandwidth,
  route option via Web-based interface.

Connection Style Destinations
VPN-A
Duration
VPN-B
Non-VPN
Pre-configured interfaces
- Start Time - Finishing Time (by 15 minutes)
VPN
Extranet
Public
Route Option
Bandwidth
VC-4 Granularity (about 150 Mbps)
- Minimum Delay or - Unspecified
GE
GE
VC-4-7v
VC-4-Av
STM-64
STM-64
1 A 7 1 B 64
VC-4-17v
STM-16
10GE
VC-4-Bv
Lambda
Bandwidth-specified
18
Demonstration of L1 BoD Service on Feb. 1st

• Two layer-1 paths (0.9 Gbps x 2) were established
  with no negative effect for other services
  between Hokkaido University and NII. After
  hitlessly reducing bandwidths of L2/L3 paths by
  1.8Gbps by LCAS, two L1 paths were established on
  demand via client PC at Hokkaido Univ.
• Non-compressed HDTV was transmitted between the
  sites very stably.

Hokkaido
BoD Server
IP Router
NII (Tokyo)
Tokyo
IP Router
IP Router
IP Router
L1-OPS
Hokkaido
Sapporo
Sendai
Tsukuba
NII
Tokyo
L2 Mux
L2 Mux
Client PC
L2/L3 path
L1 path (0.9Gbps)
L1 path (0.9Gbps)
19
Main Features of SINET3 (Summary)
20
Schedule and Plan

• SINET3 is providing IPv4/IPv6, L3VPN, L2VPN, and
  VPLS services and is using packet-based QoS
  capabilities to transfer control packets,
  starting QoS services soon.
• We successfully demonstrated layer-1 BoD
  capabilities and are ready to provide on-demand
  circuit-based QoS services.

Overlay construction
In operation (01/04/2007)
Migration
Complete (31/05/2007)
IPv4/IPv6 dual stack L3VPN
IP-QoS
L2VPN
VPLS
L2-QoS
On-demand (GMPLS-based)
L1VPN (static)
Enhanced GMPLS
Today
21
Thank you very much!