CMS: GCT2: Concentrator Card PreManufacture Review

1
CMS GCT2 Concentrator CardPre-Manufacture
Review

• 11th August 2006
• version 4

2
What is the concentrator ?

• Last processing stage before calorimetry data
  sent to Global Trigger
• 9U VME64x card
• 2 DPMCs mounts for electron-leaf cards
• 2x3 Samtec cables to interface to jet-wheel cards
• 1 DPMC mount for Global Trigger interface

3
Trigger
All paths 40 MHz DDR -gt 80MHz
Electron ? data from Leaf DPMC
Electron data 2 x 160 Single Ended Via J11, J12,
J21, J22
1) Iso Elec 2) Non-Iso Elec 3) Energy Sum 4) Jet
Counts
Electron ?- data from Leaf DPMC
Electron V4 FPGA
Sorted Et and jet count 2 x 50 Diff Pairs Via
Samtec J2 J3
Global Trigger DPMC 7 x dual channel Serdes links
80 Single Ended
Jet ? data from Wheel Card
Leaf
Leaf
2 x 180 Single Ended Via fully populated 1/2 DPMC
Jet V4 FPGA
Leaf
Jet ?- data from Wheel Card
Leaf
Sorted unfinished jets 2 x 150 Diff Pairs Via
Samtec J1 J3
5) Forward Jet 6) Central Jet 7) Tau Jet
Leaf
Leaf
4
Control Readout
40 MHz DDR -gt 80MHz
Slink VME TTCrx Clock Control FMM USB Ethernet
1) Iso Elec 2) Non-Iso Elec 3) Energy Sum 4) Jet
Counts
Electron ? data from Leaf DPMC
2 x 40 Single Ended Via J23
Electron ?- data from Leaf DPMC
Electron V4 FPGA
Comm V2 FPGA
2 x 32 Single Ended
Jet ? data from Wheel Card
Leaf
Jet V4 FPGA
Leaf
2 x 34 Diff Pairs Via Samtec J2 V2 driving
LVDSEXT
Leaf
5) Forward Jet 6) Central Jet 7) Tau Jet
Jet ?- data from Wheel Card
Leaf
V4
V2
Leaf
100
100
DCI LVDS requires 62.5mW per pair
Leaf
5
Implementation

• Processing
• Two Xilinx Virtex4 FPGAs
• XC4VLX100-FF1513
• Must concentrate large amount of data
• Choose package with most I/O
• Integrated differential termination makes layout
  simpler
• High speed I/O provide reserve capability
• Communication
• Xilinx Virtex2 FPGA
• XC2V3000-BF957
• Robust in 3.3V enviroment
• VME 64x interface
• Slink
• TTCrx
• Ethernet PHY USB for future

• Elec FPGA
• Isolated Electrons
• Non-Isolated Electrons
• Energy Sums
• Jet Counts
• Jet FPGA
• Forward Jets
• Central Jets
• Tau Jets

6
Top Side
Switching power supplies
J1 VME
DPMC ElecLeaf (and one on other side)
Comm JTAG header
5V
Elec
Comm
J2 VME Slink
Jet
Clock Distribution
DPMC GlobalTigger
CPLD, System Misc JTAG headers
DPMC 3.3V power
Trigger data flow
Both sides of board Top side only
RJ45-LVDS Out (e.g. for TTS)
7
Bottom Side
DPMC ElecLeaf (and one on other side)
Proms
Readout path
Duplicated Single
DPMC GlobalTigger
USB
RJ45-LVDS Out (e.g. for TTS)
Clock distribution
RJ45-LVDS In (e.g. for TTS)
Duplicated Single
Ethernet
8
Board Specifications (I)

• Layers 14
• 7 signal
• 50 ohm single ended
• track width
• 100 ohm differential
• 2 ground
• 5 power

9
Board Specifications (II)

• Vias Specs
• Min drill diameter 0.3mm
• Through via
• layers 1 to 14
• depth 2.45mm
• aspect ratio 8.21
• Blind Top via
• layers 1 to 8
• depth 1.3mm
• Blind Bottom via
• layers 9 to 14
• depth 0.8mm

• Thickness 2.45 mm
• Recommended not to exceed this otherwise
• Through hole component lead length too short
• Aspect ratio on vias too large
• Additional layers
• May need at least one extra power ground layer
• ExceptionPCB in UK
• 20 layers possible
• Need to look at stack up to verify.

10
Power Supply (I)

• Input power
• From ECAL backplane,
• but with 5V.
• 3 connectors
• 3MMP2-SP10-51M1
• Each connector has 5 pins power, 5 pins ground
• 6.5A per pin gt 32.5A per connector
• At present 2 of the 3 are used to supply 5V
• 65A or 325W
• 1 connector is spare.

• Power Consumption (very approx)
• 2x40W, Leaf DPMC
• 20W, Global Trigger DPMC
• 10W, 3 FPGAs core voltage
• 30W, 3 FPGAs I/O voltage
• 140W Total

11
Power Supply (II)

• Clock System
• Driven by linear regulators
• 2.5V, 0.5A from LT1763CD
• QPLL
• 3.3V, 3A from LT1764E
• TTCrx, SN65LVDS125, SN65LVDS125
• Dual PMC sites
• Leaf cards GlobalTrigger card
• Supplied by either
• Datel LSM-10A switcher
• TPS75933 linear, 3.3V, 7.5A
• Both situated under PMC site. Not ideal because
• Height of switcher is 9mm
• Power dissipation of linear 5A from 5V 8.5W
• Could easily add temp sensor to exiting I2C
  chain.

• FPGAs and rest of board
• Main power from 6 Datel LSM-10A switchers
• 3 of the switchers provide core volatges for each
  FPGA
• 2 x 1.2V
• 1 x 1.5V
• 3 of the switchers provide I/O power to the FPGAs
  
• 2 x 2.5V
• 1 x 3.3V
• FPGA Proms are powerwed from LT1763CD, 1.8V, 0.5A
  

12
JTAG Chain
Header Misc
Header System
TTCrx Slink Ethernet
Header CPLD
V4 PROM Jet
V4 PROM Elec
Voltage Error
VME
CPLD
DPMC ElecPos
Header Comm
DPMC ElecNeg
Control
WheelPos
DPMC GlobalTrig
WheelNeg
Red 3.3V Blue 2.5V
PMC Spare
x8
13
Clock Distribution (I)
Destinations 3 x FPGAs 3 x DPMCs 2 x Wheel Cards
Destinations 1 x PMC 1 x two 50 ohm SMB 1 x
header 0.1 dual
SN65LVDS125 Cross-Point Switch
Comm FPGA
CLK_DIFF_A (CLK40) CLK_DIFF_B
(CLK80) CLK_DIFF_C (CLK80) CLK_DIFF_D (CLK40)
CLK160
TTCrx
QPLL
SN65LVDS108
Out 0
In 0
CLK40DES1
CLK80
SN65LVDS108
Out 1
In 1
Header Dual 0.1
CLK40
SN65LVDS108
Out 2
In 2
Header Dual 0.1
Out 3
In 3
SN65LVDS108
14
Clock Distribution (II)

• At present everything driven by 40MHz 80MHz
  QPLL clock, including VME.
• Comm FPGA cannot configure the clock without
  leaving itself vunerable.
• The QPLL has searche mode, which may force DCMs
  to unlock.
• Solution
• Plan to hard code QPLL and Cross-Point Switch
  configuration.
• No DCMs
• Backup
• Use 40MHz utility clock for VME. Then bridge to
  QPLL clock domain.

• Clock traces are at present not length matched
• Suggest that at least the 6 traces to the Comm,
  Jet and Elec FPGA are length matched to within an
  inch.
• Would need an extra power/gnd layer
• Extra layer would also help routing around Samtec
  bolt holes.
• An extra layer would allow length matching up
  onto the leaf cards.

15
Interfaces (not VME)

• RJ45
• Require an RJ45 with LVDS outputs for Trigger
  Throttle System
• Added extra capacity
• 2 x RJ45-LVDS outputs
• 2 x RJ45-LVDS inputs
• USB Ethernet
• USB uses Cypress CY7C68001
• Wrong connector at present (Type A rather than
  Type B)
• Ethernet Uses Intel LXT971A
• Both used on Source/IDAQ cards
• Can benefit from experience at Imperial College
• Both request careful layout. Not yet checked
• Slink
• Will use ECAL transition card and DCI

16
Possible improvements
Replace sense resistor with fuses on
switchers. What about linear and leaf supplies?
Can 30A flow through antipads
Replace caps with fuses for 5V supply
Match length clock traces
Move clock outputs (simple)
Change USB type A to type B
Mounting holes for Samtec
Decoupling caps at connectors p3v3, p2v5a, p2v5b,
p5v, p1v2b and 3.3V for all DPMC sites
17
No change recommended
Comm FPGA JTAG Hdr in awkward location
Switcher 9mm high. Caps on leaf at least 1mm.
Max clearance 15mm. Depnds on leaf
Linear dissipating 8.5W in confined space. May
have to opt for SMD device because T220 package
not avaiable
18
Global Trigger Card (I)
XC2C256 FT256
DS92LV16
DPMC
Tx or Rx
42
Cable 0
DS92LV16
Tx or Rx
Self contained power from linear
regulators LT1764E / 3.3V LT1763CD /
2.5V LT1763CD / 1.8V
Instantiate 8 times - 7 for transmit - 1 for
receive
Provides 2.5V and 3.3V logic level
conversion. Also allows us to test DS92LV16
serdes units in loop back mode, albeit one at a
time, should we encounter problems.
19
Global Trigger Card (II)

• Susceptible to power supply noise
• SerDes power will be provided by local linear
  regulators
• Mounted on DPMC
• If design revision necessary cost and turnaround
  time should be substantially less.

• Clock distribution similar to that of
  Concentrator Card.
• Cross-point switches receives differential clk40,
  clk80 from concentrator.
• Inverted clk80 from clk80
• External source
• Distributed by four ICS83948I_147 ICs
• May add option to drive 2 clocks from Jet or Elec
  FPGA
• Single ended clk faily easy
• Differential clk more difficult

20
Testing

• Connectivity test
• Can test connectivity of 80 of board with
  either JTAG or custom firmware.
• Samtec connections
• Loopback with production cables
• PMC sites
• DPMC test board (Matt Stettler)
• FPGA-FPGA connnections
• Insitu tests
• VME Slink etc are probably best tested by final
  or test firmware
• E.g. for VME by writing/reading register many
  times
• Alternative is a dedicated JTAG loopback system.
• Time consuming to construct

21
Schedule Status

• PCB manufacture assembly in September
• Manufacture 3 PCBs
• Assemble 1
• At present 1 month behind schedule
• Also need GT DPMC card layout and manufacture

22
Appendix

• Following slides list the signal counts and how
  they were obtained

23
Incoming data

• Jet interface
• Arrives from 2 x wheel cards via high speed
  Samtec cable assemblies
• 240 LVDS signals from each Wheel card (40 MHz DDR
  -gt 80Mhz)
• 200 for trigger path
• 34 for control readout
• 2 for clk
• 4 for jtag
• Electron interface
• Arrives from 2 x leaf dual PMC cards mounted on
  concentrator card
• Only 206 out of 360 I/O used)
• 160 for trigger path
• 40 for control readout
• 2 for clk
• 4 for jtag

24
Outgoing data

• Global Trigger interface
• Tranmit to Global Trigger on 7 cables
• 2 unidirectional SerDes channels per cable
• Each channel driven by NatSemi DS92LV16
• Takes 16 bit parallel data at 80MHz. Transmits
  at 1.44 Gb/s
• Loopback testing possible with 1 extra cable
• Mounted on Dual PMC
• All 360 I/O connected
• Allows relatively fast cheap modifications if
  problems exist with high speed serial links
• Slink to DAQ
• Signals connect to VME J2
• Uses ECAL transition card to host SLINK
  transmitter card

25
Jet Trigger Guide
How to understand the next slide
6 x Leaf (3 per Wheel)
2 x Wheel
Jet
U2 1 x JetFinder Cntrl
6 clustered jets (12) Ht(13) H 3x160, R 3x85
What does this mean ?
All numbers in bits assuming 80 MHz data
transmission on single-ended differential
pairs 6 clustered jets (12) Ht(13) 6
clustered jets of 12 bits each and 13bits for Ht
per leaf card H 3x160 Have aprrox 160 bits
from each of the 3 leaf cards R 3x85 Require
85 bits from each of the 3 leaf cards (6x1213)
Spare capacity on bus
Bus at limit, although running at double capacity
(160MHz data) should be possible in future
26
Jet Trigger
6 x Leaf (3 per Wheel)
2 x Wheel
1 x Concentrator
Et(13), Exy(34) Ht(13) H 2x80, R 2x60
Ctnrl (64), Et(13), Exy(34), Ht(13) H 3x160,
R 3x124
1) Iso Elec 2) Non-Iso Elec 3) Energy Sum 4)
LoopBack cable
Energy
U2 1 x JetFinder Cntrl
Electron
SerDes pair (40) H 180, R 160
6 clustered jets (122 spare) JetCount(60) H
3x160, R 3x144
H 160, R 0
Et(13), Exy(34), Ht(13) Double Compare 3x3
requires 2x3 pre clusters (18) H 400, R 168
GT
H 160, R 0
H 2x20, R 2x0
SerDes pair (40) H 180, R 160
Jet
U1 2 x JetFinder
Jet
5) Forward Jet 6) Central Jet 7) Tau Jet 8)
JetCounts
12 sorted jets (14 2 spare) JetCnt(60) H
2x280, R 2x192
12 clustered jets (122 spare) JetCount(60) H
3x320, R 3x228
Next Leaf
Leave at least 2bits per bus for BC0 (equivalent
to 1 signal at 80MHz)
Double Compare 3x3 requires 2x3 pre clusters
(18) H 120, R 108
27
Signal From single Jet-Wheel

• Jet data sent to Jet FPGA
• Top 4 rank of central, forward and tau jets.
  Hence 12 objects
• 12 sorted jets (min 14 bits each)
• 5 bits phi
• 3 bits eta (no need for sign)
• 6 bits rank
• 9 unsorted jets (min 14 bits each)
• 18 phi regions -gt max 9 jets
• 1 bit phi (each jet covers 2 phi)
• 0 bits eta (events in the middle)
• 10 bits Et
• 1 bit tau veto
• 2 bits spare
• Total 147 signals _at_ 80MHz
• 294 bits
• Available 150 signals _at_ 80MHz

• Jet data sent to Elec FPGA
• Et-total (13 bits)
• 121 bits mag overflow
• Et-missing (26 bits)
• x y components
• 121 bits mag overflow
• Jet counts (36 bits)
• 6 jet count regions each 5 bit
• Alternative more flexible system of 12 jet count
  regions of 3 bits each (not TDR)
• Ht (13 bits)
• 121 bits mag overflow
• Total 38 7 signals _at_ 80MHz
• 75 13 bits
• Available 40 10 signals _at_ 80MHz

28
Signal From single Elec-Leaf

• Electron data sent to Elec FPGA
• Top 4 rank of isolated and non-isolated electrons
  (min 14 bits each)
• 8 electron objects (14 bits)
• 5 bits phi
• 3 bits eta (no need for sign)
• 6 bits rank
• To reduce the latency the FPGAs on the electron
  leaf card will not share data. Hence each FPGA
  will send 8 electron objects to the concentrator
  (i.e. concentrator receives 16 electron objects
  from each leaf card)
• Total 112 signals _at_ 80MHz
• 224 bits
• Available 160 signals _at_ 80MHz

29
Signal Between Jet/Elec FPGAs

• Data transmitted between V4 FPGAs
• The 9 unsorted jets on the boundary between the
  two wheels are turned into clusters in the Jet
  FPGA
• These jets will contribute to Ht and the
  jet-counts being summed in the Elec FPGA
• Ht (13 bits)
• 121 bits mag overflow
• Jet counts (60)
• 12 types each 5 bits
• Total 37 signals _at_ 80MHz
• 73 bits
• Available 80 signals _at_ 80MHz

30
Signal Control Readout

• Readout
• Max slink sustained rate 200 MB/s
• Assume no source generates more than 100MB/s
• 10bits _at_ 80MHz
• Control
• Serial VME
• 2bits
• L1A BC0
• 2bits
• Serial Fast Commands from TTC B channel (e.g.
  resync)
• 1bit
• AsyncReset
• 1bit
• Serial FastFeedback
• 1bit
• Total
• Required 17 signals
• Minimum available 32 signals

31
Signal GT interface

• Global Trigger interface
• GT receives 7 cables (2 unidirectional SerDes
  channels per cable)
• Each channel driven by NatSemi DS92LV16
• Takes 16 bit parallel data at 80MHz. Adds 2
  bits. Transmits at 1.44 Gb/s
• Require 2 bits for powerdown/sync
• 252 signals (7 x 2 x 18)
• Require 1 cable for loopback testing
• Generates 16 bits parallel data
• Require 4 bits for lock, refclk, powerdown and
  recovered clk
• 40 signals (1 x 2 x 20)
• NatSemi chips do not have JTAG
• Could use local loopback to test data lines only.
• Require 2 bits for outenable and local loopback
  on all chips
• 44 signals ((14 x 2) 16
• Mounted on dual PMC
• All 380 I/O connected,
• Require at least 292 signals, perhaps 336
• Slink