Twelve Ways to Build CMS Crossings from Root Files

1
Twelve Ways to Build CMS Crossings from Root Files

• Benefits and deficiencies of Root trees and
  clones when - NOT dealing with TObjects,-
  reading the trees entries NOT sequentially,-
  processing them NOT one by one.

2
Outline

• Goal scope.
• Main use-case.
• 4 kinds of containers for the crossing data
  model.
• 3 kinds of persistency managers.
• Results.
• Conclusions.

3
Goal Scope

• Evaluate the benefits of TTree and TClonesArray
  for the persistency of CMS event data (whose
  classes heavily rely on templates and external
  packages).
• Focus on the generation of crossings (pile-up of
  about 160 simulated events chosen pseudo
  randomly).
• Not covered yet meta-data, associations between
  persistent objects, schema evolution .

4
Main Use-Case
Digitizer
signal event (hits)
minbias event (hits)
digis
Persistency Manager
Persistency Manager
Persistency Manager
minbias events file
digis file
signal events file
minbias events file
5
Crossing Data Model

• The folders //root/crossing/ represent the
  events composing the current crossing.
• Each event folder contains a container for each
  kind of event objects TrackHit, CaloHit,
• The kind of container is chosen among four
• stdvectorltgt (by value).
• dynamic C array (each event object is wrapped
  inside a class instrumented with classdef).
• TObjArray (each event object is wrapped inside a
  class derived from TObject).
• TClonesArray (each event object is wrapped inside
  a class derived from TObject).

6
Persistency Managers

• The task of a persistency manager is to transfer
  an event from memory (TFolder) to disk (TFile)
  and vice-versa.
• Three flavors have been implemented
• RtbPomKeys directly write the TFolder in the
  TFile, each time with a different meaningful
  name.
• RtbPomTreeMatrix for each container in the
  folder, creates a TMatrixD and attach it to a
  branch of a TTree.
• RtbPomTreeDirect attach directly each container
  to a branch of a TTree.

7
Implementation issues

• Recent progress
• can now use -ansi -pedantic.
• nice support of foreign classes.
• better and better support of templates and std
  containers.
• Recurrent problems with Root I/O
• must explicitly ask to parse namespaces,
  components types and template instances.
• multiple containers sizes and misleading
  operator,
• tuning of chain branchs,
• it is unclear which subset of C is supported by
  Root I/O, and which in TTree, and which in
  TClonesArray.

8
Configuration

• Pentium 4, 1.8 GHz.
• 512 Mo of RAM.
• IDE disk.
• RedHat Linux 7.3
• Gcc 3.2
• Root 3.05/03

9
Parameters of the Testbed

• compression level.
• size of buffers.
• split level.
• randomness burst and jump.
• size of the containers within the events.
• number of crossings.
• direct inheritance or not from TObject, direct
  instrumentation or not with ClassDef.
• resetting or not the values in the empty
  constructors.

10
Best Results
File size (Kb/event)Cpu time (s/crossing) Stl C ObjArray Clones
Keys 1523.16 1754.82 1559.65 1554.43
Matrix 1492.44 1492.85 1493.15 1492.72
Tree 1532.63 1764.05 1567.27 541.87
11
Remove compression
File size (Kb/event)Cpu time (s/crossing) Stl C ObjArray Clones
Keys 3411.76 5683.00 4278.27 3842.95
Matrix 4001.01 400 1.45 400 1.71 400 1.23
Tree 3431.53 5702.70 4296.17 2141.16
12
Then increase random
File size (Kb/event)Cpu time (s/crossing) Stl C ObjArray Clones
Keys 3412.20 5683.42 4278.63 3843.40
Matrix 4001.31 400 1.78 400 2.03 400 1.59
Tree 3432.45 5703.71 4296.98 2142.71
13
Then reduce containers /10
File size (Kb/event)Cpu time (s/crossing) Stl C ObjArray Clones
Keys 34.90.82 54.20.96 43.61.51 39.51.05
Matrix 40.60.48 40.60.51 40.60.59 40.60.52
Tree 35.41.09 55.01.40 44.21.59 22.71.36
14
Then remove random
File size (Kb/event)Cpu time (s/crossing) Stl C ObjArray Clones
Keys 34.90.32 54.20.44 43.60.95 39.50.52
Matrix 40.60.15 40.60.18 40.60.24 40.60.18
Tree 35.40.22 55.00.32 44.20.66 22.70.14
15
Other Results

• Resetting the attributes to 0 in the empty
  constructors of the event data does not help
  compression.
• Write cpu time of the different strategies
  compares rather similarly than the read time,
  with the Tree strategies a little slower.

16
Conclusions

• We succeeded to read pseudo-random entries from a
  chain and dispatch them to few hundred folders
  (despite tuning of TChain branches has not been
  straightforward).
• Support for foreign classes, templates and C
  standard library has greatly improved.
• The magic couple TTree/TClonesArray has proved
  very efficient, yet it requires top level
  TObjects and the benefits can become losses with
  less data or random access pattern.
• One can simply use std vectors and store them
  directly into root files. Their integration in a
  TTree is not as worth as a TClonesArray. This
  could change in the next release of ROOT.
• It would be interesting to retry with direct
  associations between objects, and to apply the
  testbed to POOL.