Simulation production summer

1
Simulation production summer 02

• LHCb week
• 19th September 2002

2
Goals for the summer production

• In May 2002, the goal for the summer production
  presented by Olivier Schneider was
• 20 different channels for the signal
• Minimum Bias
• BB inclusive
• CC inclusive
• 27.4 M events to be produced in total

3
(No Transcript)
4
Results of the summer production

• In September 2002, we have produced
• 3.3 M events produced
• 37 different channels
• 7 weeks - 48 days
• 9 production centres
• 3.6 TB of data stored at CERN

5
Some explanations

• If you just look at the number, we failed BUT
• We have started with some delays
• During the LHCb LIGHT meeting in August , it has
  been decided to produce 20 K for each channel and
  wait for feedback.
• The production has been done by several centers
• SO we successfully produced 3.3 M useable events
  and we learnt a lot about the production schema
  for the next round.
• It has been decided to stopped the production
  because we found that the efficiency and/or the
  robustness of the algorithms was not as expected.

6
Data samples

• We have produced 37 different channels
• For each channel we have produced around 20K
• For BB inclusive, gt 1 M events
• For Minimum Bias, gt 600 K events
• For cc inclusive, gt 400 K events
• These samples represent 2.9 TB of RAWH and 0.7 TB
  of OODST
• We decided to keep only RAWH produced at CERN and
  all DST produced everywhere (impossible to keep
  all datasets 60 TB according to the expectations)

7
Channel Nb events produced
Min bias 676 850
CC incl 476 000
360000 24 500
360200 60 000
BB incl 1 070 500
411300 149 000
411400 24 500
411410 29 000
411500 25 000
411800 21 500
412000 21 500
412100 65 000
412300 20 000
Channel Nb events produced
412400 65 000
412500 47 500
412510 28 500
412700 21 000
412800 20 500
412810 58 000
414500 19 500
414700 21 500
415000 25 500
415100 24 500
416100 55 500
512200 19 000
516100 20 500
Channel Nb events produced
611000 19 500
611300 22 000
611400 20 000
612200 23 000
612400 20 000
612500 20 000
613300 25 500
613400 22 500
615000 19 500
615100 17 500
615200 21 000


8
Production centres

• Contributions 2 main centres and lot of others
• CERN (1452 k) and Bologna (1009 k)
• Lyon (595 k)
• RAL (130 k)
• Cambridge (37 k)
• Moscow (27 k)
• Amsterdam/VU (25 k)
• Rio (24 k)
• Oxford (24 k)
• Thanks to Vincenzo, Andrei, Gennady, Alexander,
  Karl, Miriam, Ivan, Sander for the baby sitting
  of their center.

9
New production centres

• Some new centres are installing the tools to run
  the production to be part of the next round
• Bristol
• Edinburgh
• Liverpool
• Imperial College

10
Data produced outside CERN

• Productions are launched via the central web page
• Jobs are created and launched in the batch system
  in the production centre
• (1 job 3 SICBMC 1 Brunel)
• Jobs are monitored by each production manager
• OODST are transferred at CERN (only the RAWH
  produced at CERN are kept)
• For centre with Mass Storage System, a copy could
  have been kept locally
• Bookkeeping updated

11
Performance

• In 48 days we produced 3.3 M events
• We have shown we can produced 70 K per day
• We can expect to produce 100 K per day when all
  centres are operational.

SICBMC (s/evt) Brunel (s/evt) RAWH (KB/evt) OODST (KB/evt)
Min bias 39 28 400 94
BB incl. 188 / 100 125 / 70 900 232
In red, timing on fast machine (1 GHz Pentium III)
12
Problems

• Failure rate of job 16 20 it is not
  acceptable
• Some explanations
• 2/3 program crash
• Few crash in SICBMC (ZFATAL error)
• In Brunel (segmentation fault or violation,
  floating exception)
• Typical errors follow a bad pointer in the
  DST, arithmetic operations,
• 1/3 system problems and others
• afs problems (volumes full, afs server down)
• Castor problems
• Bad manipulation

13
Problems (contd)

• Problem with production tools
• Authentication for the servlet is not reliable
• Modification of the main servlet is difficult
• Mixing of the production and development
  environment
• Management of new centre
• Each centre is different and the setting of the
  configuration is difficult to put in place
• Bookkeeping update
• Incorrect information
• Information not updated
• Static page ( http//cern.ch/lhcb-comp/SICB/pcsf/h
  tml/prod-cards.htm )

14
Analysis

• Now we need to analyse all these datasets, but
  WHERE ?
• As all OODST are sitting at CERN, it is
  convenient to run at CERN, but in the future,
  physicist will run in their home institute.
• This implies that you are able to transfer file
  where you run or you are able to use dataset
  which are already in the in the local Mass
  Storage.
• Tools are ready and a new version of the
  bookkeeping is ready to handle this case
• We need to test them, now.

15
Conclusions

• This summer production was a good exercise to
  show that
• We can produce lot of useable events
• Half of the production has been done outside CERN
• Production tools should be improved to handle
  properly several centers and trap most of the
  errors
• We are ready for the next round !!!