Data%20Acquisition%20Mark%20Bowden,%20Margaret%20Votava

1
Data AcquisitionMark Bowden, Margaret Votava
2
Parameters

• 250,000 channels
• 4 Bytes/chan
• 2 MHz sample rate
• Front-end data rate 2 TBytes/sec

3
Front-end (ICARUS)
Caen V789 (Digital Board) V791 (32 channel A/D
Board)
Rack 18 X 32 channels
4
FLARE Front-end Assumptions

• Newer components allow greater packaging density
• 128 or 256 channels (if new amplifier/shaper
  ASIC)
• Digital board can be reduced to single FPGA and
• combined with Analog board
• FE Board count drops from 16,000 to 2,000
• Eliminate VME
• VME is expensive and provides no added value
• Direct Ethernet connections to FE boards
• 6U Eurocard or 1U (pizza box) packaging

5
FLARE Front-end
48V Power
40 X 128 channels (6U packaging)
20 X 256 channels (1U packaging)
6
Data Rates
Without FE Data Compression
With FE Data Compression
1 MByte/sec 2 GByte/sec
1 KByte/sec 2 MByte/sec
- per FEB - system total
Spill Only
1 MByte/sec 2 GByte/sec
1 GByte/sec 2 TByte/sec
Continuous
2 msec per 2 sec cycle assuming 128
channels/FEB
Note BTeV is 0.5 TByte/sec
7

Data Network


















FE switches (100) 15K GBE switches
(50) 25K Cables 10K Total 50K
5 GByte/sec
8
Switch-based DAQ

• Any size/bandwidth system can be built from low
  cost
• switch components.
• Cost scales as NlogN.
• Up to 100m (copper) between switches.

9

Processing Data Storage
1U dual processor 2K
DVD writer 7K
(Delay purchase of computers to maximize
performance/cost.)
10

Event Processing

• An Event is an arbitrary time slice of data
  from all channels
• 2 msec 4 MBytes
• 20 msec 40 MBytes
• 200 msec 400 MBytes
• 2 sec 4 GBytes
• Choose largest time slice that fits in processor
  memory
• For this application, 100 msec

11

Event Processing
Front-end (2000 data sources)
1
2
2000
Ethernet Network
Processors
1
2
200
Destination
Source 1 Source 2 Source 3
1
2
3
200
1
2
3
1
2
3
200
1
2
3
1
2
3
200
1
2
3
12

Event Processing
Front-end
1
2
2000
Ethernet Network
Processors
1
2
200
Data transfer time 40 MBytes/sec
time_slice 4 sec Processing time
Nprocessors time_slice 20 sec
13
Data Storage
Tape 300K/PByte
Hard Disk 800K/PByte
Blu Ray 900K/PByte lt100K/PByte
DVD-R 100K/PByte DVD-RW 300K/PByte
14
Data Storage

• continuous readout, without online tracking
• - 2 GBytes/sec X 107 sec 20 PBytes/yr
• - 2-6M/yr media cost
• continuous readout, with online tracking
• - 2 MBytes/sec X 107 sec 20 TBytes/yr
• - 10K/yr media cost
• Note BTeV includes 2 PBytes of hard disk (_at_
  600K/PByte) in base
• DAQ construction cost. Offline storage is part
  of operating cost.

15
DAQ Costs
Ethernet Network (50K)
Processing Farm (500K)
Data Storage (50K _at_ 2MB/sec, 1M _at_ 2GB/sec
media cost)
Note DAQ based on all commercial hardware
16
DAQ Costs
Additional hardware costs Slow
Controls 200K Timing System 50K Development
System 100K Infrastructure 50K
17
DAQ Cost Summary

• with online tracking
• DAQ hardware 1.0M
• DAQ software 2.0M
• DAQ management 0.5M
• DAQ operating cost 100K/yr
• without online tracking
• (fewer online processors, but more data
  recorders)
• DAQ hardware 1.7M
• DAQ software 1.5M
• DAQ management 0.5M
• DAQ operating cost 4M/yr

18
Conclusions

• data compression at front-end is preferred
• (necessary for continuous readout)
• online tracking will significantly reduce
  operating cost by
• reducing storage media expense
• DAQ base cost estimate is 3.5M
• DAQ contingency and overhead estimate is 1M