Clock Distribution for IceCube

1
Clock Distribution for IceCube
http//www.phys-astro.sonoma.edu/people/students/b
aker/SouthPoleFoucault.html

• Gerald. Przybylski
• Lawrence Berkeley National Laboratory,
• Design Review, September 16, 2005

2
History

• string 18 implementation o Rubidium module
  slaved to GPS Clocko fan-out port for each
  domcom card.
• string 21 implementation OCXO GPS clock,
  Symmetricom ET-6000 Passive fan-out to each DOM
  Hub (DSB Card) Sub-nanosecond skew and jitter
  demonstrated Simple and Reliable, but not
  scalable

3
MCU Requirements

• Straightforward conceptually very simple
• 5ns absolute accuracy (skew and jitter), within
  the IceCube counting house Across all DOM
  Hubs (at DOR cards) Fixed and stable offset
  from Universal Time, Coordinated (UTC) Based
  on Scattering Length in the ice
• Distribute 10 MHz, 1Hz and Time Value String
• Free from Metastable states/events no glitches
• Measurable and Verifiable
• Single driver per output-port no shared drivers
• Robustness requirements in ERD x Mainly
  dealing with satellite drop-out and loss of
  availability x Also dealing with tracking
  multiple GPS clocks
• Phase accuracy 0.4ns at fan-out, 0.7ns at
  DSB, and 1.0 ns at DOR

4
IceCube IceTop AMANDA
NOT COVERED IN DETAIL IN THIS TALK

• IceTop same as IceCube
• IceCube to AMANDA Zeuthen/Wuppertal
  Install 05/06 Holger et. al. Fiberoptic
  transmitter driven by GPS clockFiberoptic
  receiver drives TWR (GPS4TWR)Autonomous from
  the MCU Clock Fan-Out subsystem 10 MHz BNC,
  1Hz BNC, IRIG-B BNC10 ns precision with respect
  to IceCube time
• AMANDA to IceCube Same playersTrigger system
  signals over Fiberoptic link to IceCube counting
  houseDepends on DOM Main Board(s) on a String
  81

Justification We realized for Amanda that
any artifial jitter below 15 ns in the MC has no
influence on reco accuracy or background
rejection.We then said 10 ns just for safety.
Then others came and said that for nearly
vertical tracks close to a string scattering may
be negligible and a 5 nsec request makes sense.
-- Christian
5
Our GPS clock

• ET-6000
• Time to first fix
• Outputs operational
• Timing accuracy better than 2 µS frequency
  accuracy better than 1E-8
• Full system accuracy (100nS) within one hour.
• 10 MHz output, 1Hz output, Time burst output
• GPS from US Naval Observatory clock 2x10-15
  Accuracy

6
MCU Brick-Walls

• ET-6000 Specifications
• 1Hz output is positive (rising) edge on
  time, within 100 nanoseconds relative to
  either UTC or GPS with six or more satellite
  averaging with 95 confidence. ( 150ns peak)
• 40ns RMS accuracy (jitter not specified)
• Cannot Vote multiple clocks Neighboring clocks
  dont track!x Tracking Algorithms in GPS clock
  PLLsx Variations in path, and multipathx
  Constellations and satellite switch-over

7
Unlock Behavior

• Power Up Sync to Satellites within an hour
  Elapsed Time Format until Tracking- Hic-ups
  while searching every 2 hours
• Potential Loss of lockx Clock Firmware (a-la
  TrueTime 2000 problem)x Power Outagex
  Misadventure (Murphy)x Wind/Weather damage to
  Antenna (speculative)x National Security outage
  (speculative)

8
Flywheeling/Freewheeling

• TCXO Clock continues for hours
• Optional OCXO continues 1 week Aging
  5 x 10-10 per day, 5 x 10-8 per year
  Phase Noise -115 dbc/Hz _at_ 10Hz
  -94 dBc/Hz _at_1Hz
• Optional Rubidium Oscillator clock 100ns per
  day slew WRT Aging over 20 years, Phase Noise -90
  dBc/Hz _at_10Hz -80 dBc/Hz _at_ 1 Hz Ru
  Good short term stability, best hang timeOCXO
  best short term stability, good hang time
• Based on Symmetricom ET6000 series product
  specifications/experiences11

9
2004-2005 Implementation

• One GPS clock, an ET6000-OCXO
• Simple Passive fan-out (resistive splitters)
• All 9 DOM Hubs driven All clock BNCs used!
• 0.35ns Jitter and Skew measured in situ at NPX
• DOR Firmware Improvements fixed NPX GPS
  glitches
• Not scalable to 90 Hubs

10
2005-2006 Implementation

• 2U chassis with 24 port fan-out (2 cards)o
  Modulates 1Hz signalo RJ-45 distribution cables
  carry 10 MHz , modulated 1Hz, TVSo All
  balanced signaling
• Passed MOAT sps-ichub04, sps-ichub05,
  domhubjr, domhub51
• Good Noise Immunity
• Stepping stone scalable to needs of IceCube

11
Design Goals/Drivers

• 90 ports plus Spares
• Meet accuracy, jitter and skew requirements
• Convenience Single distribution cable per DOM
  Hub
• Measurability/Verifiability Easy to confirm
  phase across all ports
• Reliability/Robustness Quality components.
  No electrolytics.
• Noise Immunity Balanced signals to DSB cards
• Minimize hard connections between racks
  Magnetic coupling
• Modularity/Extensibility/Maintainability
• Hot-Swappable Port Cards
• Independent Port Drivers for each signal
• Low power
• Off the Shelf Components
• No heroic solutions
• Simplicity! (no programmable logic in clock
  distribution)

12
Signaling Details

• Balanced 10 MHz 500mV P-P through ethernet
  magnetics- High common-mode immunity- Suppress
  EMI emission RFI pick-up- Avoid Ground
  Bounce pick-up in the counting house- Commodity
  components compact, inexpensive
• Modulate the 1Hz signal (180deg Phase
  Modulation) required to pass through ethernet
  magnetics
• RS-422/RS-485 differential serial12/-7V CM range

10 MHz
1Hz/1PPS
Registered 1Hz
U47-4
Modulated
1Hz
13
Rev 0 Fan-Out Card

• 12 Port
• 0.6ns port to port skew, worst pair
• Symmetry matters!
• Passes MOAT
• Revised DOR Firmware now supports Modulated 1Hz
• Inputs directly from GPS clock in Stand-Alone
  configuration
• Passes Fluorescent Lamp noise immunity test
• Status LEDs Test Header

14
2006 Implementation and Beyond

• VME form factor conditioner card (1) in
  2006
• VME backplane fanout mezzanine card (1) in
  2006
• VME form factor 12 channel fan-out cards (9)
  proto now
• VME form factor Monitor card (1) 2006 goal
  (Least well defined)

DOM Hub
Coax Cable (2)
GPS Clock

DSB
RJ-45 Cable (90)
Serial Cable
15
Port Card Features, Rev 1

• 2-wideVME form factor 12 RJ-45 port
• Common 10 MHz from Backplane
• Common modulated 1Hz from Backplane
• Common Serial from Backplane
• LVDS inputs, instead of direct GPS clock signals
• One Point Signal conditioning
• 2 minor schematic corrections
• Skew tweaks
• Additional Monitoring points P2 and Header
  (e.g. demodulated 1Hz)

16
Backplane Fan-Out

• Active Piggy-back card mated to Back-of-Crate
  style VME backplane over Socket 2
• Independently drive each EVEN socket
• Match phase at each driven socket
• 1-to-10 LVDS-to-LVDS fan-out chips
• Match skew to each driven socket

Piggy-back Card
Industry Standard Back of Crate style Backplane
17
Conditioner card

• Driven by GPS clock 10 MHz, 1Hz, and Serial
• Modulate 1Hz signal
• Control skew by design
• Ensure symmetry of port signals
• Drive LVDS backplane fan-out
• Status indicators/LEDs
• Prototyped on the Fan-Out card

18
Monitor Card

• Occupies an Odd Slot in VME card cage (e.g. 1)
• Implement GPS clock Monitoring specified by PDR
  Document
• Could contain FPGA, SOPC, or SBC in a DIMM form
  factor
• Could report via ethernet
• Scope TBD

19
Watch List/Wish list

• Lock status from GPS clocks
• Tracking status from GPS clock
• Power status from GPS clock
• Parse time strings for error conditions
• Parse GPS Clock console port output Rich set
  of Status bits/words Satellite constellation
• Monitor phase offset between multiple GPS clocks

20
Verification

• Reference signals on header10 pin header on
  DSB10 pin header on Fan-Out card4 pin header on
  DOR card (inside DOM Hub)

21
The Bottom Line

• Fan-Out card comfortably meets the 5ns
  Requirement
• Our GPS clock is a good choice, for the money
• No speed bumps this year
• On track for the final assaultSingle unit,
  flexible, modular designBuilds on previous
  successesAvoids heroics and death marches.

Fin
22
Verifications in situ

• IceCube AMANDA
• Opt 1 Measure Fiber Round-Trip time (2 fibers)
• Opt 2 Use Portable Atomic Clock - packaged
  PRS-10- battery power
• Steps 1. Measure/calibrate WRT IceCube
  clock 2. Transport clock to MAPO 3. Measure
  fiber distribution signals in MAPO against Atomic
  Clock 4. Transport clock back to IceCube
  Clock 5. Check CalibrationRepeat 1 to 5
  until satisfied
• Measure/Verify within 1ns should be achievable.

4