Trigger

1
Trigger Data Acquisitionfor LHC
UpgradesInternational Workshop on Future Hadron
CollidersFermilabOctober 16-18, 2003

• Andrew J. Lankford
• University of California, Irvine

2
Acknowledgements

• My presentation draws heavily upon
• Work performed by contributors to the preliminary
  studies reported in
• Physics Potential and Experimental Challenges of
  the LHC Luminosity Upgrade (hep-ph/020487).
• A recent presentation by Nick Ellis at Erice.

3
SLHC Implications of Higher Luminosity

• Higher Luminosity gt
• Increased detector occupancy
• Increased trigger rates
• Increased radiation effects
• Assumed SLHC luminosity 1035 cm-2s-1

4
SLHC Implications of Higher Luminosity - 2

• Higher Luminosity gt Increased detector occupancy
• Assuming 12.5 ns bunch spacing,
• 125 interactions/crossing
• Occupancy of tracks increases 5-fold (10-fold for
  some sub-detectors)
• Radiation-induced hits increase 10-fold.
• Pile-up noise increases 2.2-fold (3-fold for some
  sub-detectors)
• Pile-up degrades performance of trigger
  algorithms
• Reduction in efficiency of e/gamma isolation cuts
• Increased muon candidate rates due to
  radiation-induced accidentals
• Larger event size to read out
• Increased by factor between 5 and 10
• Demands reduced trigger rate or increased data
  bandwidth

5
SLHC Implications of Higher Luminosity - 3

• Higher Luminosity gt Increased trigger rates
• Arising from
• Increased interaction rates
• Occupancy/pile-up induced trigger degradation
• Less rejection at fixed efficiency
• Need more selective triggers for same trigger
  rate
• Increased thresholds
• More exclusive selection (less inclusive trigger)
• Fortunately, more selective triggers are okay
• (see later)
• Need even more data acquisition bandwidth if
  higher trigger rate.

6
SLHC Implications of Higher Luminosity - 4

• Higher Luminosity gt Increased radiation effects
• Directly affects on-detector trigger logic
• Mechanisms
• Permanent damage
• Single-event-upset effects
• (Note that radiation effects upon detectors and
  their electronics must be handled by trigger and
  by data acquisition.)

7
Enabling Technologies

• Enabling technologies for readout and trigger
• Integrated circuits
• Custom ICs, FPGAs, memories, etc.
• Commodity computing
• processors, networking, memory, storage,
  fiberoptics
• These technologies enabled current generation of
  expts.
• Including the nearly-deadtimeless, integrated
  trigger and data acquisition systems that are now
  standard.
• These technologies will also be the technologies
  that enable successful upgrades future
  experiments.
• RD programs to develop these technologies for
  HEP applications were crucial to current
  generation of expts.
• A vital RD program will be crucial to SLHC
  upgrades.

8
Data Transfer Data Processing

• Trigger Data Acquisition systems provide
• Data transfer
• Data processing
• Limitations or costs of these functions define
  limitations and costs of Trigger/DAQ systems.
• e.g. First Level Triggers (FLT)
• Data input to FLT limits density of FLT
  electronics.
• Causes extensive interconnections betw. modules
  and complex backplanes.
• Needed to connect steps in FLT selection.
• Needed to seamlessly find tracks and clusters in
  different sections of detector (to avoid loss of
  efficiency

9
SLHC Trigger Menu

• Need for 3 types of triggers foreseen
• Discovery physics
• Very high-pT (thresholds as high as hundreds of
  GeV)
• Completion of LHC physics program
• e.g. precise measurements of Higgs sector
• Lepton/photon/jet thresholds as low as for LHC
• Final states known gt exclusive selection
  possible
• Control / Calibration triggers
• e.g. Ws, Zs, top
• Low thresholds needed, but can be pre-scaled
• None of the above pose rate problems.

10
Inclusive Triggers samples rates
LHC LHC SLHC SLHC
Selection Threshold Rate Threshold Rate
  (GeV) (kHz) (GeV) (kHz)
inclusive single muon 20 4 30 25
inclusive, isolated e/gamma 30 22 55 20
muon pair 6 1 20 few
isolated e/gamma pair 20 5 30 5
         
inclusive jet 290 0.2 35 1
jet missing ET 100100 0.5 15080 1-2
inclusive ET     150 lt1
multi-jet triggers various 0.4 various low
Note that inclusive e/? trigger dominates rate.
(Added degradation from pile-up not included
above)
11
FLT Importance of FLT upgrades

• Upgrades to First Level Trigger (FLT) are of
  central importance.
• They can reduce new demands on
• Front-end electronics (FEE)
• Retaining 100 kHz FLT rate avoids changes to FEE
  systems that do not require upgrade for other
  reasons.
• Data Acquisition (DAQ)
• By reducing new demands for data bandwidth.
• High Level Triggers (HLT)
• By reducing new demands for algorithms and
  processing
• FLT upgrades deserve early consideration.

12
FLT New Demands on Processing

• New demands on FLT processing posed by
• Increased event pile-up occupancy
• New sub-detectors or readout
• 12.5 ns crossing period
• Some (at least) new processing will be needed.

13
FLT Impact of 12.5 ns crossings

• LHC FLTs
• pipelined processors
• driven at 40 MHz LHC crossing rate
• identify crossing of interest
• Should SLHC FLTs run at 80 MHz Xing rate?
• Can FLTs remain at 40 MHz ?
• Can FLTs work at both 40 80 MHz ?

14
FLT Can SHLC FLTs operate at 40 MHz ?

• Concept of readout time frames
• FLT identifies pair of crossings (or more) for
  read out
• e.g. PEP II / BABAR
• PEP II runs at 250 MHz (4 ns).
• BABAR system clock 60 MHz (16 ns).
• L1T runs at multiples of 16 ns.
• DAQ reads out time frames as large as few
  microsec, as appropriate to subdetector
  technology.
• May not be possible for many FEE systems
• Increases data volume on FEE links through DAQ
• Yes, SLHC FLTs could operate at 40 MHz,
• But, 40MHz FLTs will generally suffer more from
  pile-up.

15
FLT Should SLHC FLTs operate at 80 MHz ?

• Advantages
• Reduced pile-up effects
• more effective algorithms
• less data volume (for detectors that identify
  Xing during r/o)
• Ability to identify crossing that caused trigger
• Allows more processing steps within fixed latency
• 80 MHz electronics feasible (Portions already at
  80MHz.)
• Disadvantages
• Requires FLT upgrades
• Increased data bandwidth into within FLT
• FEE may not be able to deliver data to FLT at
  80MHz
• Study (cost/benefit) needed.

16
FLT Can SLHC FLTs operate at both 40 80 MHz ?

• Portions of FLT could operate at 40 MHz while
  other portions operate at 80 MHz.
• Note Identification of 12.5 ns crossing can be
  derived from 40 MHz samples for calorimeters.
• Time resolution of calorimeter pulses is much
  better than 25 ns.
• Timing of pulses is derived by digital filtering
  of multiple samples.
• Such a scheme already used in ATLAS CSCs w/
  sampling at 20 MHz
• Some such hybrid likely to be a good solution.

17
High-Level Triggers Data Acquisition

• Commercial computing networking technologies
  will provide the advances necessary for High
  Level Triggers (HLTs) and Data Acquisition (DAQ)
  systems to perform at SHLC rates.
• e.g. Moores Law will provide x10 improvement in
  priceperformance ratio wrt LHC start-up 5 years
  after start-up.
• e.g. Appetite for high-bandwidth graphical
  computing applications will drive networking
  capabilities up and costs down.
• RD is necessary to develop technology advances
  for HEP applications.
• Data transfer
• Data links
• HLT/DAQ networks
• Data Sources / Readout Buffers
• Data processing
• New HLT algorithms to maintain HLT selectivity
  with increased pile-up
• Complexity handling

18
Data Links

• New challenges at SLHC
• for data links from Front-end Electronics to
  Trigger DAQ
• Higher bandwidths due to higher occupancies (
  FLT rates?)
• Radiation effects at transmitting end
• for some subdetectors, e.g. systems optimized for
  LHC
• Also to increase input capabilities of FLTs
• Limitations on data input often limit FLT
  capability.
• RD needed
• Applications of commercial developments
• Possible custom developments

19
HLT/DAQ Networks

• Networks connect HLT processors to data sources.

CMS/Cittolin

• Every processor connects to every source.
• Data moves from sources to processors in parallel
  (parallel event building) to handle high
  trigger rates.

20
HLT/DAQ Networks

• Commercial networking equipment provides the
  infrastructure for interconnection (switches) and
  data transfer (links).
• At present, the cost of this equipment determines
  how much data experiments can afford to move to
  HLT processors.
• Thus, cost can limit trigger rate capability.
• ATLAS has adopted an RoI-based Level 2 trigger
  in order to reduce overall data bandwidth
  requirements.
• CMS has developed a scaleable event building
  architecture.
• SLHC network bandwidth at least 5-10 times LHC
  b/w.
• Even if SLHC FLTs can provide same rate as at LHC
• Network bandwidth requirements grow with
  occupancy.

21
HLT/DAQ Networks
Complete HLT/DAQ systems require large networks.
Individual network switches not yet of size
required for full interconnectivity.
22
HLT/DAQ Networks RD

• RD should track evolution of network technology
• Seek switches with requisite number of ports to
  avoid multiple switches extra ports
• Technical challenge to manufacture of switches
  with many high-speed ports is bandwidth
  capability of switch fabric (backplane) that
  interconnects ports.
• Switches from different vendors behave
  differently within HEP systems
• Due to internal differences (e.g. buffer sizes)
• RD will sometimes require very large-scale
  testbeds, which could be provided by large farms
  foreseen for LHC computing and Grid projects.
• If we desire to use commodity technologies, then
  anticipate use of 10 Gbit Ethernet, as well as
  Gigabit Ethernet in SLHC systems.

23
Data Sources / Readout Buffers

• This is electronics that buffers detector data at
  the input of HLT/DAQ systems and that sources
  data into HLT/DAQ networks.
• They tend to operate at highest rates
• (relative to other HLT/DAQ components).
• Cannot benefit from event parallelism,
• as exploited by other components
• Each data source must function at full FLT rate.
• Increased occupancy (and increased FLT rate)
  increase data source internal bandwidth
  requirements.
• These elements likely to need upgrade for SLHC
  rates.
• Compare performance with SLHC requirements

24
Data Sources / Readout Buffers RD

• Upgrade directions
• Internally, these elements tend to employ busses
  for data transfer.
• Reasonable for O(80Mword/sec) on circuit board
• Challenging for higher bandwidths between groups
  of modules (e.g. on backplanes)
• High-speed serial connections may provide better
  data transfer in upgrades.
• Technology trends in this direction
• FEE inputs to these elements already on serial
  links.
• Serial output links to HLT/DAQ network with
  bandwidth comparable to bandwidth of input links
  will remove bottleneck.
• I.e., push network technology closer to FEE, to
  exploit
• High-speed serial links
• Parallelism afforded by networking

25
HLT Algorithms

• New HLT algorithms needed
• To maintain selectivity in face of increased
  occupancy and pile-up
• e.g. Electron triggers
• Dominate FLT output rate, unless thresholds
  raised high
• FLT rate 22 kHz _at_ 30 GEV threshold at LHC
• FLT selectivity degraded because pile-up blurs
  isolation
• HLT algorithms must recovery selectivity despite
  degraded isolation.
• How can this be accomplished? Will refined
  tracking information need to brought to bear
  early in selection sequence?
• e.g. Muon triggers
• Degraded by occupancy
• Degraded by loss of momentum-resolution at higher
  threshold
• HLT algorithms must recover selectivity.
• HLT algorithms must accommodate upgraded muon
  detectors.
• Moores Law will provide data processing power
  for new algorithms.

26
Complexity Handling

• HLT/DAQ systems are extremely complex.
• Large numbers (thousands) of processors
• Even larger numbers of software processes tasks
• Highly distributed, heterogeneous system
• Real-time demands not present in offline
  systems
• Complex control (e.g. startup shutdown)
  procedures
• Remote access required for monitoring
  troubleshooting
• Very high reliability required
• Robustness, redundancy, fault tolerance
• Including robustness of complex selection
  algorithms
• Note Technology evolution may mean that SLHC
  HLT/DAQ systems are no larger than for LHC. In
  this case, SLHC complexity stays same as LHC

27
Complexity Handling RD

• RD can develop solutions to manage complexity.
• RD can track development of tools for
  complexity handling in very large commercial
  and other applications.
• E.g. web tools from e-commerce for high-level
  controls and user interfaces
• Similar remote access, security, database issues

28
Summary

• Higher Luminosity gt
• Increased event pile-up detector occupancy
• Increased trigger rates data volume
• Increased radiation effects
• Enabling technologies
• Integrated circuits custom, FPGAs, memories,
  etc.
• Commodity computing networking
• Challenges arise in data transfer in data
  processing
• Processing challenges felt mainly in First Level
  Triggers
• Data transfer challenges felt in both FLTs and
  HLT/DAQ
• Trigger rates manageable by
• Increasing thresholds on inclusive triggers
• Using exclusive triggers where low thresholds
  needed
• Technical solutions exist, but RD is required.

29
RD Summary

• First Level Triggers
• Operation with 80 MHz crossing rate
• Coping with increased occupancy (muons) pile-up
  (calorimeter)
• Adapting to new sub-detectors or readout
• Radiation-tolerant on-detector FLT electronics
• Input data links from Front-end Electronics
• High Level Triggers Data Acquisition
• Coping with increased data bandwidth
• Input data links, HLT/DAQ networks, data sources
  / readout buffers
• Coping with increased occupancy pile-up ( new
  detectors or r/o)
• New HLT algorithms
• Complexity handling