Introduction and Performance of COPPER

1
Introduction and Performance of COPPER
SuperBELLE
2
Introduction

• Global data flow
• At higher luminosity, L 1035cm-1s-1
• L1 trigger rate 3kHz (?).
• Total data size 100kB.

e-
DAQ Platform
e
Event Builder
Current design15 MB/s
Belle
Online Computer Farm
Storage
Trigger Module
Data flow at L1 trigger will be 200 - 400 MB/s
3
Consideration in DAQ Upgrade

• Less bandwidth out of detector.
• Data compression by online CPU.
• Pipelined DAQ for less deadtime.
• Put data buffer in data stream.
• Smaller DAQ unit From crate size to board size
• Data size will be tripled. Smaller unit is
  preferred to fit in electronics hut.
• Limited manpower encourages use of commercial
  products.
• Follow computer-market trend.

• Our conclusion toward L 1035cm-1s-1
• Development of new DAQ platform equipped with CPU
  and data buffer along with market trend.

4
Conceptual Design

• FINESSE (ADC/TDC) Detector Interface
• FINESSE is a mezzanine module.
• An actual implementation of FINESSE is not
  defined by the platform, but can be defined by
  each sub-detector group so that best ADC/TDC can
  be designed.
• Event Buffer
• FIFOs are implemented to minimize DAQ dead-time.
  FIFOs are to separate asynchronous data-read by
  CPU from event trigger.

ADC/TDC internal design depends on each
sub-detector
Analog signal from Belle
Buffer (FIFO) to minimize dead-time
Sampling Clock
Trigger Timing
5
Conceptual Design - Contd

• PCI bus based DAQ
• Up to 133 MB/s can be marked by PCI that is third
  of VME.
• PCI is used commonly in PCs. New technologies
  are expected to be provided on it, and we can aim
  long lifetime of the base system while using
  newest products.
• Utilization of PMC (PCI Mezzanine Card) Module

Data compression
ADC
CPU
Ethernet, etc
Belle
EB
Bridge
PCI
Local Bus
TriggerInterrupter
6
COPPER Board
VME-9U size board
FINESSE
PMC
FINESSE
PMC
Front
Rear
FINESSE
PMC for CPU
FINESSE
SPIGOT
7
SPIGOT Board
VME-6U size board
PMCfor Ethernet,IEEE1394, etc
PMC
Front
Rear
TTD-RX
COPPER
8
Schematic Drawing of the COPPER
Local Bus
PCI Bus
Mezzanine Cards
FIFO
FINESSE
Bridge
Detector Signals
FIFO
FINESSE
Bridge
PCI Mezzanine Cards
FIFO
Memory
FINESSE
FIFO
CPU
FINESSE
Control
Bridge
Trigger interrupt
Trigger input
9
Components of the COPPER

• COPPER board comprises most of current DAQ
  modules
• Four slots for FINESSEs.
• FIFOs for less dead-time pipelined DAQ, which are
  connected to local bus.
• Programmable logic for local bus and FIFO
  control.
• PCI-9054 (PLX) to connect local bus and PCI bus.
• PMC-CPU on PCI bus for data formatting and
  compression.
• Interface to accept triggers from TTD-RX.
• UniverseII PCI-VME bridge.

Design of COPPER is finished. First version will
be delivered from company (Densan Co.Ltd. and
Design Tech Co.Ltd.) during November or so.
10
FINESSE ADC/TDC

• Design
• Sampling timing system-clock (43MHz) or reduced
  clock (arbitrary). They are delivered by Trigger
  module.
• Resolution Up to 16 bit.
• Eight 8-bit read/write registers for status
  reading or firmware downloading.
• L1 pipelined storage see next slide.
• Trigger-busy handshaking should be implemented.
• Test FINESSE for COPPER debugging will be ready
  during December.

• Local Bus

Setup Register
Detector Signals
FIFO
ADC/TDC
L1 pipelined storage

• FIFO Write Busy

• System CLK
• Reduced CLK

• L1 Trig
• Trigger Info

• Trigger Busy

11
FINESSE ADC/TDC - Contd

• Prototype of L1 pipelined storage

Data in
L1 trig
Ready
switch
Data READ
EMPTY
Data READ
Data READ
Trigger HOLD
Always WRITE
Always WRITE
switch
Busy
Data out (to COPPER)

• Trigger Busy

12
FINESSE ADC/TDC - Contd

• Dimensions
• Depth Width 168.0(d) 71.0(w) mm2.
• Layout example

component side
COPPER side
13
FINESSE ADC/TDC - Contd

• Connectors
• COPPER pin headers andFINESSE receptacles.
• 2.54 mm height normal pitchconnector (Hirose).
• 23 pins 2 rows 2 connectors.
• Maximum component height onFINESSE will be 1 mm.
• Design of actual FINESSE
• Will start soon by Tauchi-san.
• Design of TDC COPPER I/F willbe finished by
  the end of this year.
• Design of ADC COPPER I/F willbe finished by
  the end of next March.

Connector pin assignment
14
Local Bus
PCI bus
FINESSE
FINESSE
FINESSE
FINESSE
Init ROM
IDT72V2103L15
PCI-9054 (with DMA facility)
ALTERA FPGA
FIFO (128kB/18bit) ? 2
FIFO (128kB/18bit) ? 2
FIFO (128kB/18bit) ? 2
FIFO (128kB/18bit) ? 2
FIFO Controller
of words in FIFO 3
of words in FIFO 2
of words in FIFO 1
of words in FIFO 0
33MHz, A32/D32 Local bus
15
Event FIFO and Local Bus Controller

• CPU
• Read of words register.
• DMA for stored data.
• Repeat this for 4 FIFOs.

FINESSE

• FIFO Write Busy

• FIFO Write CLK

writein trun
CPU
Counter of of words in FIFO
Address Decoder for FIFO/Regs.
Status Registers
ALTERA FPGA
FIFO
FIFO
Memory

• FIFO Write Busy

• FIFO Write CLK

4 words
16
Event FIFO and Local Bus Controller - Contd

• Local bus controller knows of words in each
  FIFO
• Read concatenated data from single FIFO
  register.
• Unfragmemted DMA works faster.

FIFO 0
Single DMA
FIFO 1
CPU
FIFO 2
Memory
FIFO 3
17
PCI Bus
VME bus
Detailed talk is given by Yamagata-san.
UniverseII (TUNDRA)
Local bus
Memory
21152 (Intel)
PCI-9054
PCI (S)
CPU
Trigger interrupt
to EB
TTD-RX
Network module
PCI (P) 32MHz A32/D32
Memory
21152 (Intel)
PCI (C)
Belle trigger
18
PMC-CPU (RadiSys EPC-6315)
The fastest CPU among commercial products.
RJ-45 Ethernet port
CompactFlash Socket
IntelPentiumIII 800MHz
Memory
RJ-45
Serial, USB, Keyboard, Mouse
LocalEthernet
Host bus
RadiSys82600
COPPER
Local PCI bus
Backplane PCI bus
IDE
CompactFlashSocket
HD
19
PMC-CPU (RadiSys EPC-6315) - Contd

• Test of CPU gets started
• The CPU is booed from IDE HD,and Linux 2.4.19 is
  running.
• Boot from CompactFlash socket is also OK.
• Local Ethernet port is available.
• slogin to/from bdaq.fbdaq.kek.jp works.
• In future, we boot CPU from local Ethernetport
  (? Nagasaka-san talk).
• IRQs are correctly assignment to PMC modules.
• Module over PCI-PCI bridge (Texas Instruments
  PCI2050) is also correctly identified. BIOS is
  working well.
• Expected CPU performance estimated by test
  compression program is marked.

CPU
Test setup with test mother-board
20
PMC-CPU (RadiSys EPC-6315) - Contd

• Items to be tested
• Test another CPU (Motorola PrPMC700).
• We use PCI-PCI bridge 21552 (Intel). According
  to company, BIOS configuration of 21152 and of
  modules over 21152 works (though identified bug
  already exists, they said). Need confirmation.
• Also check configuration of VME-PCI bridge
  (TUNDRA UniverseII).
• Check data transfer speed over 21152.
• Ethernet card on backplane PCI bus does not work.
  Why?
• Performance of data transfer via PMC/PCI with PIO
  and DMA after PMC memory board purchased.
• Identify the necessity to develop
  device-driver/BIOS for 82600.
• And many other things .

PrPMC700
21
Reset of COPPER Board
22
TTD-RX
Detailed talk is given by Nakao-san.

• TTD-RX trigger receiver from upper-stream
  transmitter
• PCI module resides on SPIGOT.
• Issues event ready interrupt on PCI bus to
  initiate data transfer by CPU.
• Manages busy handshake to ADC/TDC modules.
• Throttles trigger delivery when event FIFO is
  getting full.
• Trigger FIFO to hold event-by-event trigger
  information.

• TriggerBusy

• FIFO Full

• System CLK
• Reduced CLK
• L1 Trig

TTD-RX

• PCI Bus

Programmable Logic
Trigger FIFO

• L1 Trig

• Trigger Throttle

23
Data Transfer to Event Builder
Detailed talks are given by Yasu-san and
Igarashi-san.

• Ethernet, Gigabit-Ethernet, IEEE1394, etc.
• At the very early stage of new DAQ system, we
  might use VME bus for data transfer.
• We will not use local Ethernet port of CPU,
  because it might cause an interference of data
  compression by CPU.

Network module
EB
24
Summary

• Considering the market trends, we start to
  develop a new DAQ platform based on PCI bus.
• Many persons are already working on this project,
  and the project is proceeding
• Design of the COPPER board has finished and the
  first version will be delivered from company
  during November or so. Then debug procedure
  starts.
• Design of prototype FINESSE will be finished
  soon. FINESSE with ADC/TDC will be ready by the
  end of next March.
• TTD-RX is being developed.
• The study of the PCI bus and CPU gets started.
• Network performance to event builder is being
  tested.

25
Future Prospects

• COPPER
• Establish initialization procedure of bus
  bridges.
• Debug of local-bus-controller FPGA.
• FINESSE
• Design ADC and TDC.
• Debug interface part to COPPER.
• PCI
• Study protocol in detail.
• Utilize DMA facilities.
• Test data transfer performance.
• CPU
• Develop BIOS software (if necessary) and device
  drivers.
• Network boot.
• Write data processing software.
• Study CPU Performance.

26
Future Prospects - Contd

• TTD-RX
• Finalize FPGA programming.
• Communication with TTD-TX.
• Develop PCI interface.
• Network
• Study performance of available buses.
• Develop device driver if necessary.

If everything goes smoothly, we can have full
test setup by the end of next February or so.