SNOLAB Computing and Networking

1
SNOLAB Computing and Networking

• SNOLAB Infrastructure
• Computing
• On-site Networking
• Off-site Networking
• Redundancy / Autonomy

2

• Computing
• build on SNO experience
• provisioning for growth
• infrastructure UPS, A/C, electrical, monitoring
• use SNO equipment as a base as SNOLAB
• comes into existence
• servers
• compute farm
• gt 10 TB disk farm
• SNOLAB datagrid initiative exists
• develop management policies appropriate for
• multi-experiment environment

3
New Surface Building IT Server Room
4

• IT Server Room Attributes
• 600 square feet
• Space for 24 racks
• Clean fire suppression system (Novec 1230)
• Racks logically grouped by 8
• Each group on own UPS system
• Matched with A/C capacity
• Initially install SNO computing into first 8
  racks
• First block of 8 plus matching A/C on diesel
  backup
• Planned growth in two steps of 8 racks plus UPS
• and A/C
• VOIP

5
Provision for 3kW per rack
6

• Onsite networking experience
• significant part of the experiments downtime
• associated with loss of UG ?? surface
• communications
• carried on 12 optical fibers (two bundles of 6)
• ? regularly broken in shaft (x 5) and drift (x
  2)
• ? safe fail-over / shutdown procedures in the
• event of loss of connectivity / control
• from surface

7
Optical fibers
8
Off-site Networking
9
Networking (Offsite) Present SNOLAB 100Mbps
(GSTi) LU 1Gbps (GSTi) GSTi 10Gbps (Orion)
Canarie
100Mbps
Commodity Future SNOLAB
1Gbps (GSTi) GSTi 10Gbps
(Orion) Canarie

1Gbps
LU 100Mbps
Commodity
Peering from Canarie to ESnet, VBNS, rest of
research world
10

• initial bandwidth 1 Gbps
• possibility for dedicated wavelength on demand
• capability to provide 10Gbps point to point
• on dark fiber
• no contention with other traffic
• absolute security
• guaranteed QoS
• Enabling .
• alternative data distribution / storage schemes
• remote control rooms
• IP video / audio conferencing
• new lab could perhaps supply IP video-
• conferencing bridge?

11
Redundancy / Autonomy Actions to Date

• data to tape UG, copied to surface before UG
  tape recycled
• redundant network (2 x 100Mbps) bonded channel
  distributed
• over separate fiber bundles
• redundancy elsewhere as possible
• ? FMS (phones, remote shutdown, safety)
• decouple UG operations from surface computing
  resources
• ? self-sufficient (name/font/boot servers, etc.)
• redundant run of fibers (only in shaft at
  moment)
• increased UPS protection
• added fail-over UG oscillator to GPS time system

12
Redundancy / Autonomy Plans for New Lab

• complete redundant run of fibers to lab and
  surface building,
• including SM
• ? physically separate wherever and to extent
  possible
• eliminate / replace obsolete FMS system
• move functionality to IP network
• move phone function to VoIP
• UG VoIP autonomous
• copper (INCO) backup
• move video to network
• upgrade bandwidth to 2 x 1Gbps (bonded,
  redundant)
• redundant GPS receivers, new distribution scheme
  to all expts

13
End
14
SNOLAB Computing and Networking