The Compact Array Broadband Backend Upgrade

1
The Compact ArrayBroadband Backend Upgrade

• Andrew Brown
• ATNF Engineer
• 5 February 2007

2
CABB User Requirements

• CABB minimum requirements
• Do what the old correlator did but now with 2GHz
  of BW instead of 128MHz!
• ADC 9-bit sampling
• 2GHz of output bandwidth
• Dual polarisation
• Dual frequency
• Up to 8 antennas
• Full BW and Zoom Modes
• Delay and fringe processing
• Correlation and Beamforming
• Multiple phase centres
• Pulsar imaging

3
CABB System Overview

• New IF conversion system to have 2GHz BW
• 9-bit ADC with 4GSPS - represents 8x2x4Gx8bx2, or
  1Tbps
• Fibre transmission system with WDM technology
  (40Gbps per fibre)
• Coarse delay (using DDR2 memory can have VLBI
  delays)
• Digital Filter banks in Virtex-4 FPGAs, with
  multiple zooms
• Fringe stopping (also up to VLBI rates)
• Data distribution using AdvancedTCA mesh
  backplane
• Correlator with large number of modes

Fibre Tx WDM
Coarse Delay
ADC 9-bit 4GSPS
Digital Filter Bank Wide/Zooms
Fringe Stopping
Data Distr.
Correlator
IF Conv.
4
CABB ADC Board

• ADC interleaving sampling scheme phase, offset,
  gainadjusted real-time
• 38dB SNR39dB SFDR
• 2-4GHz zone

Intel 10Gbps Serialiser
Atmel ADC and Dmux
Power
IFinput
Four10Gbpsstreams
splitter
ADC with Synthesizer in enclosures
Synthesizer Inputs(2048, 640, 32 MHz)
Xilinx SX35 FPGA
5
CABB Optical Transmission

• The ADC output goes to a XC4VSX35 Virtex-4 FPGA
  where it is framed and sent out to 4 x 10.24 GB/s
  Serialisers.
• The 4x10.24 Gb/s outputs go to the 4 individual
  wavelength laser transmitters which are then WDM
  muxed onto a single fibre to give the 40.96 Gb/s
  link from the antenna to control room.
• At the receiving end is the corresponding WDM
  demux, photodiodes, the deserialiser and FPGA,
  before being sent to the DSP processing board.
• This setup shows a 4km fibre spool as the link

6
CABB Digital Signal Processing Engine

• Connects to RTM which connects to ADC board
• 1 Xilinx LX80-11 FPGA with 2-DDR2 modules for
  delay
• 2 - Xilinx SX55-11 FPGAsfor the DFB wide/zoom
• 4 Xilinx Pro50 FPGAs for mesh connection (64
  rockets)
• 2 - Xilinx SX55-11 FPGAs with 2-DDR2 modules
  for the correlator
• 1 Xilinx Pro50 for high speed IO (10GBe, VLBI,
  Nasa, Second Frequency Rack)
• SOM with PCI 32-bit/66MHz for control/data
  acquisition
• AdvancedTCA form factor

7
CABB DSP Board History

• 24 February 2006 DSP board functionality and
  proposed implementation
• 22 May 2006 Protel Schematic finished
• 20 July 2006 Cadence Layout completed by Siart
  Design Systems (India)
• 26-layers 3mm thick, track width/space 0.004,
  via size .012
• 4188 components, 22876 holes, 19217 nets, total
  trace length 576metres!
• 24 August 2006 prototype PCB arrives at ATNF
• 25 September 2006 Prototype assembled with all
  components
• 5 February 2007 Pulsar project nearly
  completing

8
AdvancedTCA Backplane
4 LVDS pairsto each hub 8 LVDS pairsto every
other node/hub slot 6 LVDS pairs (Multi-drop) 10
LVDS pairs Between two slots
9
Example Data Mesh Connections
Transmit Slot
Receive Slot
Each lineis 4 differentialpairs Transmit slot
is a single Polarisation with 2.5GHz of BW
Receive slot processes all sixteen polarisations
for one- sixteenth thefrequency Channels (160MHz
)
10
Serial Data Transport

• There are many important considerations for
  serial data transport
• Cost per link
• Cost per Gbps
• Cost of physical PCB area
• In February 2006 there were several choices
• Virtex-2 Pro FPGA various sizes, speed grades
• Virtex-4 FX60 had been burnt badly by this in
  the past
• Virtex-4 LX with external SerDes
• The design was space limited so it had to be the
  first two options
• AdvancedTCA mesh had nominal capacity of 3Gbps
• Bad experience with FX60resulted in Pro50-6
  choice
• Virtex-5 FPGAs now available uh, ah, um?
• Data connectivity in array processing is critical!

11
Conclusions

• CABB is a very large upgrade!
• Installation is planned for later this year
• Although the new filterbank/correlator is much
  smaller- it is much more complicated!
• Can be used for other systems (Pulsar Digital
  Filter Bank)
• Still lots of work to do
• Full BW filterbank completed
• MOPS zoom filterbanks already demonstrated to be
  highly effective
• See parts of the hardware in the demonstration!

12
Thank You

• ATNF / Electronics Group
• Name Andrew Brown
• Title Research Engineer
• Phone (eg. 61 3 9372 4283)
• Email andrew.brown_at_csiro.au
• Web www.atnf.csiro.au

Contact CSIRO Phone [phone] 9545
2176 Email enquiries_at_csiro.au Web www.csiro.au