Data Quality Today

1
Data Quality Today
Physics Convenors Meeting February 13th, 2004

• Whats the story?
• Solutions in New and Old Data
• Places where help is needed to recover data.

Its unfair to pick on only the calorimeter.
2
Whats the story?

• We wanna make the data as good as it can be for
  physics. This is what I came up with. ?
• Calorimeter Noise (Quasi-stable, Not even)
• Hot cells, hot towers, warm regions, coherent
  noise, Ring-of-welding.
• These are pedestal shifts
• They cause bad runs, worse resolution, bad
  luminosity blocks
• Above is not the whole the story but
• Its a big piece and we dont know how much but
  we have guesses
• We can see possibilities for solutions

Its unfair to pick on only the calorimeter.
3
Outline of a Plan

• Experts R.Z., P.P., S.S., D.S., G.B., U.B.,
  S.N., J.S., H.M. and others.
• It turned out that most of the outline of a plan
  was in place in pieces
• Parts of a plan are even due to occur nowish
• A good plan for this has to
• Integrate online and offline operations
• Accommodate old, fresh, and future data
• Fit in within the wider offline schedules

4
Outline of a Plan Shifter Procedure

• Ensure that valid pedestals exist for all data.
  Maintain accurate shifter/database documentation
  of hot cells, hot towers, warm regions.
• Fix stuff and understand what causes failures
• Seems obvious but can be difficult in practice.
  For instance, pedestal shifts dont occur on
  schedule and arent always immediately apparent.
• R.Z. puts it this way human intervention in the
  gears of the mill is to be avoided.
• Thats not enough.

5
Outline of a Plan Online Procedure

• For New and Fresh Data DQ_Calor (running)
• Measure pedestals in unzero-suppressed zero-bias
  monitor stream events
• Need sufficient events or a procedure to
  determine when to use a different runs
  information (human intervention)
• Need to define threshold for hot cell, hot tower,
  warm region
• Write these delta pedestals into hardware
  database (widths come from the regular pedestal
  runs) for Reco to use when it goes 24 hours
  later.
• A key part of the goal is automation
• Qs does DQ_Calor find warm regions?

6
Outline of a Plan Online Procedure

• For Old Data the procedure has a lot of
  expert-based intervention
• Look at the data and list all of the
• Hot cells (done)
• Hot towers (in progress)
• Warm regions (dont know how to do yet)
• Experts identified this as a place they need
  help maybe that means some sleuthing work in
  the logbook
• Put in the hardware database

7
Outline of a Plan (Offline Procedure)

• Hardware Database
• At present its an old flat file from which the
  2.5 s noise zero-suppression is determined
• Kind of like the online database
• It is or is almost ready to be tried out (without
  the delta-peds part, which is the next step.
• P17 (on the farms June 14th ) for the New Data
• Implication is suggestion that p17 runs on all
  postshutdown data
• Thumbnail pass for Old Data
• Not clear we can get all info in time for
  upcoming pass but I am not sure when that is and
  readiness may depends on whether help can be
  found.

8
Summary

• Outlined a plan that seems to have already been
  there, more-or-less.
• The reason I describe this as an outline of a
  plan is that I dont know the detail that gets us
  
• from here to TMB pass
• from here to P17 June 14th.
• H.M. may have some intermediate steps in mind.
• Corrections

Its unfair to pick on only the calorimeter.
9
Data Quality Status

• Data Quality for Cal, Jet/MET, Muon, partial on
  trackers is done thru Sept-2003

• There is improvement in 2003 compared to 2002.
• 2002 41 Bad
• 2003 13 Bad

10
Whats coming in Data Quality Scoring?

• Testing is done by Run, Lumy block, Event
• By Run
• So far Muon, CTT, Cal, Jet/MET, some SMT
• New CFT (includes preshowers) SMT
• (CFT) MF, SJL, MC nearly finished evaluating
  number of good/bad/hot channels in all runs
  through the shutdown. The group is deciding on
  quality scoring definitions.
• (SMT) Michele Weber is evaluating the runs.
  (Really New).
• By Lumy Block
• New Jet/MET based on average MET will allow
  recovery of ? 10 ? of total lumy.
• New Lumy system to handle SES run-pausing
  alarms
• By Event
• ? Only ? muon has data integrity checks in the
  TMBs.