The UCSD OptIPuter Deployment

1
The UCSD OptIPuter Deployment
UCSD is Prototyping a Campus-Scale OptIPuter
To CENIC
Dedicated Fibers Between Sites Link Linux
Clusters
SDSC
SDSC
SDSCAnnex
SDSC Annex
Preuss
High School
JSOE
Engineering
CRCA
SOM
6th College
Medicine
Phys. Sci -Keck
Collocation
Node M
Earth Sciences
SIO
Source Phil Papadopoulos, SDSC Greg Hidley,
Cal-(IT)2
2
To Build a Campus Dark Fiber NetworkFirst, Find
Out Where All the Campus Conduit Is!
3
UCSD Packet Test BedOptIPuter 2003
4
SoCal CalREN-XD OptIPuter Build-Out 2003
5
UCSD Campus-Scale Routed OptIPuter with Nodes for
Storage, Computation and Visualization-Feb 2005
6
2007
7
CAMERA Network 2007
8
Calit2 CAMERA ProductionCompute and Storage
Complex is On-Line
512 Processors 5 Teraflops 200 Terabytes
Storage
9
CAMERA Networking Core
Calit2 networking equipment includes Force 10
E1200 router Cisco Infiniband switch Edge and
distribution switches
10
Sunlight
11
The New Optical Core of the UCSD Campus-Scale
TestbedMoving to Parallel Lambdas in 2007

• Goals by 2007
• gt 50 endpoints at 10 GigE
• gt 32 Packet switched
• gt 32 Switched wavelengths
• gt 300 Connected endpoints

Funded by NSF MRI Grant
Lucent
Glimmerglass
Approximately 0.5 TBit/s Arrive at the Optical
Center of Campus Switching will be a Hybrid
Combination of Packet, Lambda, Circuit -- OOO
and Packet Switches Already in Place
Force10
Source Phil Papadopoulos, SDSC, Calit2
12
OptIPuter_at_UCI is Up and Working
ONS 15540 WDM at UCI campus MPOE (CPL)
10 GE DWDM Network Line
Wave-2 layer-2 GE. 67.58.33.0/25 using 11-126 at
UCI. GTWY is .1
Floor 4 Catalyst 6500
Engineering Gateway Building,
SPDS
Kim Jitter Measurements Lab E1127
Floor 3 Catalyst 6500
Catalyst 3750 in 1st floor IDF
Wave-1 layer-2 GE 67.58.21.128/25 UCI using
141-254. GTWY .128
Floor 2 Catalyst 6500
Catalyst 3750 in NACS Machine Room (Optiputer)
ESMF
Catalyst 6500, 1st floor MDF
Beckman Laser Institute Bldg.
Berns Lab-- Remote Microscopy
Catalyst 3750 in CSI
10 GE
Wave 1 1GE
Created 09-27-2005 by Garrett Hildebrand Modified
02-28-2009 by Smarr/Hildebrand
Wave 2 1GE
13
Optiputer at UC Irvine

• Initial Connection
• via Tustin POP to Cisco 15540
• Used WDM on Campus
• Via New Wave on CENIC/UCI Last Mile Fiber
• Layer-2 PropagationNo Routing Here
• UCSD Address Space (Now Calit2 Space)
• Propagated Using Layer-2 Gigabit Switches
• Main Distribution from Central Plant

14
A Parallel Network at Start-up

• Separate Fiber from UCInet
• Used Existing Fiber from Previous Campus Projects
• First End-Point
• Engineering GatewayVisualization Classroom
• Falko Kuester was the Early Adopter

15
First Major Rollout Calit2

• When Completed, the Calit2 Building Used One
  Cisco 6500 on Each Floor.
• Floors were Connected via 10 GE
• A Separate Optiputer VLan was Created for
  Optiputer
• When Optiputer is Required, a Port is Assigned
  the Optiputer VLan

16
UCI Optiputer Endpoints

• Engineering Gateway Academic Data Center
• Testing Point for Both Waves There are Two and
• Special Room for Special Experiments (Kane Kim,
  et al).
• Calit2 Building
• Server Rooms, Suite 4100 VTC, and Viz Lab
  (Hyperwall)
• Dr. Bernss remote microscopy
• Beckman Laser Lab

17
Optiputer Upkeep at UCI

• Requires Close Coordination with Calit2 Network
  Personnel at UCSD
• Few Actual Problems at UCI
• Most Have Been with Off-Campus Configs
• Little Maintenance Required
• Lots of Testing when Problems Arise to Determine
  Locality
• Tuning of End-Points Important!