The Front-End Driver Card for the CMS Silicon Strip Tracker Readout 8th Workshop on Electronics for LHC Experiments Colmar S.A.Baird, K.W.Bell, J.A.Coughlan, C.P.Day, E.J.Freeman, W.J.F.Gannon, R.N.J. Halsall, J.Salisbury, A.A.Shah, S.Taghavirad,

1
The Front-End Driver Card for the CMS Silicon
Strip Tracker Readout8th Workshop on
Electronics for LHC Experiments
ColmarS.A.Baird, K.W.Bell, J.A.Coughlan,
C.P.Day, E.J.Freeman, W.J.F.Gannon, R.N.J.
Halsall, J.Salisbury, A.A.Shah, S.Taghavirad,
I.R.TomalinCLRC Rutherford Appleton
LaboratoryE. Corrin, C.Foudas, G.HallImperial
College LondonPresented by John
Coughlanj.coughlan_at_rl.ac.uk
2
CMS Silicon Strip Tracker FEDSilicon Strip
Tracker Readout Overview
L1-Trig
Buffer
9 million Silicon Strip channels ON Detector
73K APV25 pipeline chips _at_ L1 Trigger MUX APV
Frame output clocked at 40 MHz Analogue Data
readout via Optical links (APV Frame Header
Strip Data) Max L1-Trigger rate 100 kHz
25
Hybrid
Front-End Hybrid
Silicon Strips
On Detector
Pipeline Location Address
2 x APV25 Data Frame
Header
FPGA
256 x Analogue Strip Data _at_ 40 MHz
DAQ
Counting Room
Back to Back Frames
FEDs
in absence of Triggers APV25 outputs Tick marks
7 micro-secs
3
CMS Silicon Strip Tracker FEDSilicon Strip
Tracker Readout Overview
9 million Silicon Strip channels ON Detector
73K APV25 pipeline chips _at_ L1 Trigger MUX APV
Frame output Analogue Data readout via Optical
links (APV Frame Header Strip Data) OFF
Detector Front-End Drivers (FED) Digitise /
Zero Suppress / DAQ readout 440 x 9U VME64x
boards 96 ADC channel boards Front-End Hybrid
Ulrich Goerlach Tracker System Test Nancy
Marinelli FED Project Status 50 x PMC 8 ADC
Prototypes used for module tests First Full Scale
96 ADC Prototypes FF1 about to be manufactured
25
Hybrid
Front-End Hybrid
Silicon Strips
On Detector
FPGA
DAQ
Counting Room
VME 9U FEDs
4
CMS Silicon Strip Tracker FED FED Layout
96 Tracker Opto Fibres
CERN Opto- Rx
Modularity 9U VME64x Form Factor Modularity
matches Opto Links 8 x Front-End
modules OptoRx/Digitisation/Cluster
Finding Back-End module / Event Builder VME
module / Configuration Power module Other
Interfaces TTC Clk / L1 / BX DAQ Fast
Readout Link TCS Busy Throttle VME Control
Monitoring JTAG Test Configuration
9U VME64x
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
5
CMS Silicon Strip Tracker FED FED Layout
96 Tracker Opto Fibres
CERN Opto- Rx
9U VME64x
Data Rates 9U VME64x Form Factor Modularity
matched Opto Links Analogue 96 ADC channels
(10-bit _at_ 40 MHz ) _at_ L1 Trigger processes 25K
MUXed silicon strips / FED Raw Input 3
Gbytes/sec after Zero Suppression... DAQ Output
200 MBytes/sec 440 FEDs required for entire
SST Readout System (_at_ L1 max rate 100 kHz)
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
6
CMS Silicon Strip Tracker FED FED Layout
96 Tracker Opto Fibres
CERN Opto- Rx
9U VME64x
Digital Processing Flexible Digital Logic Xilinx
Virtex-II FPGAs 40K-gt3M gates some in pin
compatible packages Features Dual Ported Block
Rams Digital Clock Managers DCM Double Data Rate
I/O DDR Digitally Controlled Impedance
I/O Various I/O signal standards Debugging Logic
Analyser cores FPGAs programmed in VHDL
VERILOG
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
7
CMS Silicon Strip Tracker FED FED Layout
96 Tracker Opto Fibres
CERN Opto- Rx
Modularity 8 x Front-End modules
9U VME64x
Analogue/Digital
FE-FPGA Cluster Finder
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
8
CMS Silicon Strip Tracker FED Front-End module
Dual ADC 10-bits 40 MHz
RL
OpAmp
CERN Opto Rx
Digital Processing
1
1
1
10
1 MIP 80 ADC counts
ASIC
2
Double-Sided Board
2
3
4
3
5
12 Fibre Ribbon
6
PD Array
4
7
8
5
9
10
6
11
DCM
CLOCK
12
DATA
CONTROL
12x trim DAC
AD9218
EL2140
9
CMS Silicon Strip Tracker FED Front-End module
Dual ADC 10-bits 40 MHz
OpAmp
CERN Opto Rx
Digital Processing
1
1
1
1
5
10
ASIC
Data Control
N
3
2
2
3
CLK
Each individual ADC clock skew is adjustable in
steps 1nsec
Delay FPGA
4
LVDS
CLK40 from TTC
3
5
2
12 Fibre Ribbon
6
Double Data Rate I/O
PD Array
DATA OUT _at_ 160 MHz
4
4
7
CLK
Delay FPGA
8
Full
Partially Full
5
9
3
RESET
10
6
11
CLK
DCM
Cluster Finding FPGA
CLOCK
Delay FPGA
12
DATA
Temp Sensor LM82
6
CONTROL
12x trim DAC
XC2V1500
XC2V40
AD9218
EL2140
10
CMS Silicon Strip Tracker FED Front-End FPGA
Logic
1x
per adc channel phase compensation required to
bring data into step
Cluster Finding FPGA VERILOG Firmware
Clock 40 MHz
2x
DLL
4x
Synch in
Synch out
2 x 256 cycles
256 cycles
nx256x16
Synch
emulator in
trig2
trig3
trig4
trig1
Synch error
10
10
11
11
Re-order cm sub
s-data
16
10
8
Hit finding
16
Phase Registers
sync
ADC 1
d
Ped sub
d
DPM
Global reset
s-addr
8
hit
Sub resets
8
8
Sequencer-mux
a
a
No hits
Control
Full flags
8
averages
header
status
control
mux
4x
data
4
Packetiser
256 cycles
256 cycles
nx256x16
160 MHz
trig1
trig2
trig3
10
10
11
11
Re-order cm sub
s-data
16
10
Phase Registers
8
16
sync
Hit finding
d
Ped sub
d
DPM
ADC 12
s-addr
8
Serial I/O
hit
Sequencer-mux
8
8
a
a
Serial Int
No hits
8
header
status
averages
Temp Sensor
Local IO
Opto Rx
Delay Line
Raw Data mode, Scope mode, Test modes...
BScan
Config
11
CMS Silicon Strip Tracker FED Front-End module
Dual ADC 10-bits 40 MHz
Each individual ADC clock skew is adjustable in
steps 1nsec
OpAmp
CERN Opto Rx
Digital Processing
1
1
1
1
5
10
ASIC
Data Control
N
3
2
2
3
CLK
Delay FPGA
4
LVDS
CLK40 from TTC
3
5
2
12 Fibre Ribbon
6
Double Data Rate I/O
PD Array
DATA OUT _at_ 160 MHz
4
4
7
CLK
Delay FPGA
8
Full
Partially Full
5
9
3
RESET
10
FE module Repeated x 8
6
11
CLK
DCM
Cluster Finding FPGA
CLOCK
Delay FPGA
12
DATA
Temp Sensor LM82
6
CONTROL
12x trim DAC
XC2V1500
XC2V40
AD9218
EL2140
12
CMS Silicon Strip Tracker FED FED Layout
96 Tracker Opto Fibres
CERN Opto- Rx
Modularity 9U VME64x Form Factor Modularity
matches Opto Links 8 x Front-End
modules OptoRx/Digitisation/Cluster
Finding Back-End module / Event Builder VME
module / Configuration Power module Other
Interfaces TTC Clk / L1 / BX DAQ Fast
Readout Link TCS Busy Throttle VME Control
Monitoring JTAG Test Configuration
9U VME64x
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
13
CMS Silicon Strip Tracker FED Full-Scale 9U
Layout
96 Tracker Opto Fibres
CERN Opto- Rx
Back-End module For each L1-Trigger Collects 8
x FE variable length data fragments Formats FED
event for DAQ Appends TTC synch
information Buffers in External QDR SRAM Sends
data via DAQ Front-end Readout Link FRL Signals
to TCS Busy/Throttle
9U VME64x
Analogue/Digital
FE-FPGA Cluster Finder
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
PinDiode /Amp
ASIC
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
14
CMS Silicon Strip Tracker FED Back-End FPGA Logic
x2
BSCAN
1
Clock40
x4
DCM
x1
diagnostics
Circular Buffers
1
VME
8
Control
Header
64
64
Data
FRL to DAQ SLINK64
Reset
1
80 MHz
x8
APV hdrs
Frame_Syncs
FIFO
x8
DECODE CONTROL MONITOR
Readout_Syncs
Lengths
x8
18
Data Out
Monitor_Syncs
FIFO
160 MHz
2
FF/PF Flags
Bx,Ex
2 x QDR SRAM x2 burst
Lengths
FIFO
40 Mhz
R/W Address Generator
TTC Rx
20
Address
Em Hdr
TCS
160 MHz
FIFO
2 MBytes
L1 100 kHz
4
Data_stream 0
64
18
Data In
80 Mhz
4
160 MHz
Data_stream 7
160 MHz
15
CMS Silicon Strip Tracker FED FED-DAQ Interface
Due to mechanical constraints place DAQ link card
on Transition card
Transition Card
TCS
Busy Throttle
BE-FPGA Event Builder
DAQ Mezzanine Card
TTC
TTCrx
DAQ Front-end Readout Link FRL
S-LINK64
Buffers
FRL DAQ links use S-LINK64 standard Implementation
Channel Link 800 MBytes/sec max Average DAQ
rate 200 MBytes/sec
See talk CMS Data to surface transportation
architecture Attila Racz
16
CMS Silicon Strip Tracker FED FED Layout
96 Tracker Opto Fibres
CERN Opto- Rx
Modularity 9U VME64x Form Factor Modularity
matches Opto Links 8 x Front-End
modules OptoRx/Digitisation/Cluster
Finding Back-End module / Event Builder VME
module / Configuration Power module Other
Interfaces TTC Clk / L1 / BX DAQ Fast
Readout Link TCS Busy Throttle VME Control
Monitoring JTAG Test Configuration
9U VME64x
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
17
CMS Silicon Strip Tracker FED VME module
VME64x Interface A32/D64 Slave Master DMA
engine Interrupter Plug Play geographic
addressing Live Insertion Transceivers FPGA
Configuration Xilinx System ACE Compact
Flash MPU-VME interface for in-situ reprogramming
via Crate Controller JTAG for Configuration
Test
9U VME64x
JTAG
System ACE CF
VME Interface
VME-FPGA
FPGA Configuration
Xilinx Virtex-II FPGA
18
CMS Silicon Strip Tracker FED Power module
CERN Opto- Rx
Standard LHC spec crate supplies 3.3V, 5V,
12V Derive -5V, 1.5V, 2.5V on board Board
Estimate 80 W Hot Swap Controllers Sequence
power Protection against out of range voltage
current Over temperature shutdown
9U VME64x
Analogue/Digital
Power DC-DC
Temp Monitor
Warning
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
19
CMS Silicon Strip Tracker FED Testing
96 Tracker Opto Fibres
CERN Opto- Rx
JTAG Boundary Scan Digital connections Chip
Scope Integrated Logic Analyser Cores Capture
raw ADC data (without VME) Opto-Tests or Inject
electrical signals post-OptoRx Special FPGA
loads e.g. Pattern Generators Additional Test
Features Internal/External Clocks External
Triggers
9U VME64x
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
20
CMS Silicon Strip Tracker FED Monitoring
96 Tracker Opto Fibres
CERN Opto- Rx
Performance Status VME slow monitor
data Cluster Finder Logic SPY channel in Delay
FPGA keeps selected copy of Raw
data Synchronisation Tracker Front-End
Synchronous Compare APV25 Pipeline Location APV
Emulator cards broadcast Global Pipeline
Location Buffering FE and BE-FPGAs monitor in
real-time status of buffers. Throttle to TCS.
9U VME64x
Analogue/Digital
JTAG
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
21
CMS Silicon Strip Tracker FED Status
96 Tracker Opto Fibres
CERN Opto- Rx
9U VME64x
Analogue/Digital
JTAG
Board Status
FPGA Configuration
FE-FPGA Cluster Finder
VME Interface
VME-FPGA
BE-FPGA Event Builder
TCS
TTC
TTCrx
DAQ Interface
Buffers
Power DC-DC
Temp Monitor
Front-End Modules x 8 Double-sided board
Xilinx Virtex-II FPGA
TCS Trigger Control System
22
CMS Silicon Strip Tracker FED Summary Schedule
FF1 Full scale Prototype FF2
Pre-production FF3 Final production
Summary Presented a 9U VME64x board for
off-detector readout of Silicon Strip
Tracker Digitisation Zero Suppression Event
Readout Meets SST data rate requirements Analogue
96 channel 10-bit ADC _at_ 40 MHz Digital
Virtex-II FPGAs flexible logic Test and Monitor
features aid debugging
Schedule 2002/Q4 2 x FF1 _at_ RAL for
test 2003/Q4 10 x FF1 _at_ CERN 2004/Q4 10 x
FF2 manufacture 2005/Q2 500 x FF3
manufacture (funds permitting)
http//www.te.rl.ac.uk/esdg/cms-fed/index.html
23
CMS Silicon Strip Tracker FED Counting Room
Layout (illustration)
FED
DAQ

• 40 K ADC Channels 10 Bit_at_40MHz
• Max Trigger Rate 100 kHz
• Input Rate 1.5 T Byte/s
• Output rate 25 Gbyte/s/

• 440 Boards 96 ADC/Board
• 24 Crates
• 8 Racks

4 TTC Partitions
24
CMS Silicon Strip Tracker FED Control
Monitoring
1
24
50K ADC Channels
VME SBC RTOS
FED CRATES
E-Net Switch
Tracker Control WS
FEC CRATES
TTC CRATES
D-BASE
DCS
DAQ
R/C
25
CMS Silicon Strip Tracker FED Software
Architecture
GUI
Supervising WS
Calibration Setup
USER
Tracker System Kernel
Network
Network
Network
CRATE SBCs Real Time OS
Fast Monitoring Exception Handling
FED API Library
USER
Hardware Driver
VME
Memory Map
Hardware FED, FEC, TTC
Hit Finding Logic Analyser
FPGA

• Software Architecture
• CMS XDAQ HAL

26
CMS Silicon Strip Tracker FED PMC Prototype

• 8 x 10 Bit 40 MHz ADC
• 64K Memory/per ADC
• 40 K Gate FPGA Control
• PCI Interface
• Mounts on Commercial VME CPU Board (or with an
  adapter in a PC slot)

• 60 in service
• Present Generation of ADC PMC

Small scale module testing System Test beams e.g.
LHC 25 nsec
27
CMS Silicon Strip Tracker FED Crate Layout
1
21
2
FE 1
LAN
FE 2
100MBit/s
FE 3
FE 4
DAQ
FE 5
TTC
100 KHz
FE 6
Throttle
FE 7
NN Synch
F-Bus
FE 8
B-Scan

• Crate Input Data Rate 50 Gbyte/s
• Crate Output Data Rate 1 GByte/s per percent
  hit occupancy

28
CMS Silicon Strip Tracker FED FED-DAQ Interface
(alternative)
TCS
BE-FPGA Event Builder
TTC
Alternative scheme Channel Link in Virtex-II
BE-FPGA
TTCrx
Buffers
DAQ