Title: Architecture of the
1 Architecture of the ATLAS High Level Trigger
Event Selection Software
Monika Grothe, CERN EP, on behalf of the ATLAS
Trigger/DAQ High Level Trigger group
- Outline
- General strategy of the HLT event selection
- The components of the HLT event selection
software - Using offline code in the HLT online context
2The stage
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
3ATLAS data flow diagram
Level 1 trigger
Region of Interest RoI
Level 2 trigger
High Level Trigger
Event Filter
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
4ATLAS trigger overview
- Level1 hardware trigger, 40 MHz ? 75 kHz, 2.5
ms maximum hardware latency - Looks for regions of potentially interesting
activity, with high pT objects - Region of Interest (RoI) muon, electromagnetic,
tau/hadronic, jet clusters - Looks in calorimeters, muon spectrometer
- Does not combine information of more than one
detector
- Level2 software trigger, 75 kHz ? 1 kHz, 10 ms
average latency - Data are held in readout buffers (ROB) during
Level2 processing - Selection software run by Processing Application
on one node of Level2 farm - (PCs, Linux)
- Input (seed) is Level1 RoI (its type, position,
pt threshold passed) so that typically - only few of full event information in the
ROBs need to be transferred to Level2 - Feature extraction in RoI region by specialized
algorithms that are optimized for - speed and cover all sub-detectors
sequentially - For events accepted by Level2 EventBuilder
builds full event
- EventFilter software trigger, 1 kHz ? 100 Hz,
1 sec average latency - Full event from EventBuilder passed to
EventFilter farm (PCs, Linux) - Independent Processing Applications run
selection algorithms on farm nodes - Selection software consists of offline-type
algorithms that have access - to latest calibration and alignment data
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
5How the HLT works
Signature ?
- Level1 finds 2 isolated
- em clusters with each pTgt20 GeV
- possible signature for Z ? ee-
- Method
- Validate step-by-step
- Check intermediate signatures
- Reject at earliest possible moment
- Managed by
- HLT Steering package
Iso lation
Iso lation
STEP 4
Signature ?
pTgt 30GeV
pTgt 30GeV
STEP 3
Signature ?
t i m e
track finding
track finding
STEP 2
Signature ?
Alg
Alg
Cluster shape
Cluster shape
Alg
STEP 1
Alg
Alg
Alg
Alg
Alg
Alg
Level1 seed ?
Alg
Alg
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
6Main components of HLT selection software
HLT DataFlow Software
Event Filter
HLTSSW
HLT Selection Software
Processing
HLT Core Software
Application
Level2
Steering
HLT Algorithms
Processing
Application
HLT Algorithms
Data Manager
ROBData Collector
Event DataModel
Event selection how ? Steering
what ? - HLT ALgorithms Data access
how ? - DataManager what ? -
EventDataModel
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
7Event selection - how ? HLT steering
- For each step there exists a
- TriggerMenu
- List of signatures for which
- an event can be accepted,
- e.g.
- SequenceTable
- Specifies which algorithms are
- to be run given the input objects
- to the step, e.g. run on each
- input object algo
StepController
StepHandler
ecand
ecand
StepSequencer
StepDecision
STEP 1
ecand
track finding
STEP i
Different from the typical find-all-electrons/trac
ks/ in event Localized reconstruction, guided
by seed Steering calls relevant algorithm once
per seed, in same event
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
8Event selection - how ? HLT steering (2)
Steering hands over seed to algorithm via
TriggerElements Algorithm returns result of
processing as another TriggerElement
- TriggerElement
- has a label
- doesnt hold any data content
- is related to DataObjects and other
TriggerElements in a navigable way - Signatures in a TriggerMenu consist of logical
AND/OR of TriggerElements
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
9Event selection - how ? HLT steering (3)
Steering realm Hypotheses
HLT algorithm reconstruction realm Supporting
information
etc.
etc.
etc.
uses
TriggerElement e
DataObject track
DataObject cluster
uses
1 ..
seeded by
TriggerElement ecand
uses
DataObject CaloCells
uses
DataObject CaloCluster
1 ..
seeded by
uses
TriggerElement EM20i
DataObject Level1 EMRoI
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
10Data Access What ? Raw Event Data Model
- DataFlow system ships raw event data to HLT in
bytestream format, - for Level2 fragments on demand, for
EventFilter complete event - Raw data objects (RDO) object representation
of raw event data (bytestream) - Algorithms access raw event data by means of
RDOs and transient event store - Importance of raw Event Data Model (EDM)
- Different granularity of data as provided by
DataFlow system ( min. granularity ROB) - and data as deployed by algorithms (per geometric
unit DetectorElement)
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
11Raw EDM (2)
Example ATLAS Muon Spectrometer
DetectorElement
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
12Data Access How ? ByteStreamConverters
Transient EventStore
Region Selector
Algorithm
Trans. Event Store
HLT Algorithm
Region Selector
region
list DetElem IDs
list DetElem IDs
list DetElem IDs
ROB ID
raw event data
DetElems
DetElems
ByteStreamConverter maps DetectorElements onto
minimum readout granularity Geometry ? ROBs
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
13ByteStreamConverters (2)
- HLT steering calls an HLT algorithm by passing
- to it a TriggerElement with a navigable link
to a - RoI DataObject that contains the RoI
position - The HLT algorithm typically needs to access the
- raw event data in the DetectorElements
- in a certain region around the RoI position
- The HLT algorithm requests from the
- RegionSelector the identifiers of all
- DetectorElements within that region
- The HLT algorithm passes the list of
- identifiers to the transient event store
- At this stage 2 cases can occur
- 1. The requested DetectorElements are already
available and are just returned - 2. The information first needs to be retrieved
from the DataFlow system - The transient event store invokes a
ByteStreamConverter which requests - the bytestream data from the correct ROB
- The ByteStreamConverter extracts from the
succession of 32-bit words in - the ROB bytestream the relevant event data and
fills the collections of RDOs - The thus obtained collections of RDOs are stored
in the transient event store
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
14Offline code in online context
- Design aim Use same environment for development
maintenance (offline) - and deployment (online) of HLT selection
algorithms - Advantage
- Ease of developing HLT algorithms and studying
their performance on - simulated and real data in well-tested
user-friendly offline environment - Ease of code migration offline ? EventFilter ?
Level2 - ATLAS offline framework ATHENA, based on a GAUDI
(developed by LHCb) core - Basic principles
- Separation of data and algorithms which produce
and consume data - Separation of transient and persistent
representation of data - Converters convert one data representation into
another - Consequences for HLT design e.g. Algorithms
forced to communicate via transient - event store, persistent data made available by
converting into transient - representation stored in transient event store,
common database access tools, etc - Importance of these offline-imposed elements to
meet stringent online - performance requirements
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
15HLT selection software offline dependencies
HLT DataFlow software
Event Filter
HLT selection software
HLTSSW
Processing
HLT Core Software
Application
1..
Level2
Steering
Monitoring
Service
HLT Algorithms
Processing
Application
Data Manager
HLT Algorithms
ROBData Collector
1..
EventData Model
MetaData
Service
ltltimportgtgt
ltltimportgtgt
ltltimportgtgt
ltltimportgtgt
Offline
Package
Offline
Athena/ Gaudi
StoreGate
Reconstruction
EventDataModel
Algorithms
Interface
Offline Reconstruction
Offline Architecture Core Software
Dependency
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
16Complications from special Level2 environment
- Stringent requirements on framework in Level2
- Speed
- Robustness
- Multi-threading
- Example multi-threading Used to keep idle time
of CPUs in Level2 farm to minimum - Implications
- Control over event loop needs to be with the HLT
DataFlow software - All algorithms/services have to be
configured/initialized outside event loop - ATHENA controls event loop in the offline
environment - How to reconcile with common online/offline
environment for HLT select. software ?
Athena Level2
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
17Complications from special Level2 environment (2)
- Solution
- SteeringController as interface layer between HLT
selection HLT DataFlow software - 1. Uses GAUDI light, i.e. SteeringController
uses only minimum set of GAUDI features - 2. Uses Modification to GAUDI base libraries
such that for every thread - EventLoopMgr_(threadID)-gtexecuteEvent()
- AnyServiceNeededByAlgo_(threadID)
- AnyHLTAlgorithm_(threadID)
- Modified GAUDI can be used in offline environment
as ATHENA is based - on GAUDI (suffix collapses to blank)
- Modifications since included in GAUDI as standard
features - Unexpected pitfall of using off-the-shelf
software in Level2 STL locks - Thread-safe but thread-inefficient implementation
of STL containers - Use different implementation of allocator
- Due to different way DataFlow and HLT selection
software interact in EventFilter - ATHENA can be used as framework for running HLT
algorithms - Timing issue 30 of processing time spent in the
raw EDM available at that time
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
18Conclusions and outlook
- For HLT Technical Design Report in summer 2003
- Architecture of the HLT selection software is
currently in validation stage - Integration of HLT selection software with other
components of the HLT - software on-going
- Setting-up of full trigger vertical slices
(electron/g and m) in testbeds - Detailed measurements of timing and physics
performance on-going - Re-use of offline code in online context
currently experimental - News from testbeds encouraging, though fair
amount of code hand-tuning - by online required
- But also very fruitful dialogue offline-online
with concrete results
Why dont you program your trigger in assembler
? (offline to online) Improvement of CPU speed
allows ever higher level of abstraction
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
19More details in the ATLAS talks _at_ parallel
sessions
- The StoreGate a Data Model for the Atlas
Software Architecture - P. Calafiura, Monday StoreGate
- The Algorithm Steering Trigger Decision
mechanism of the Atlas High Level Trigger - G. Commune, Tuesday HLT Steering
package - The Region of Interest Strategy for the ATLAS
Second Level Trigger - V. Boisvert, Tuesday RegionSelector
- Configuration of event-selection criteria in the
ATLAS trigger system - M.Elsing, Thursday Use of
TriggerMenus - Use of a generic identification scheme
connecting events and detector description - in the ATLAS experiment
DetectorElement Collection identifiers - R.D. Schaffer, Thursday
Further ATLAS Trigger/DAQ talks The DataFlow
system of the ATLAS Trigger DAQ, G. Lehmann,
Monday The use of Ethernet in the DataFlow of the
ATLAS Trigger DAQ, S. Stancu, Monday
Experience with multi-threaded C applications
in the ATLAS DAQ system, S. Gadomski,
Monday Supervision of the Atlas High Level
Trigger System, S. Wheeler, Tuesday Online
Monitoring software framework in the ATLAS
experiment, S. Kolos, Thursday
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03
20Level2 testbed
Purpose Integration of all components of the HLT
selection software Only minimum of components of
the DataFlow system necessary to run the HLT
selection software
Switch with 8 fast Ethernet ports (for single
processor PCs) and 2 Gigabit Ethernet ports
(for dual processor PCs)
Monika Grothe Architecture of the
ATLAS High Level Trigger Event Selection Software
CHEP03