Proposal for a general-purpose unfolding framework in ROOT

1
Proposal for a general-purpose unfolding
framework in ROOT

• Tim Adye
• Rutherford Appleton Laboratory
• BaBar Statistics Working Group
• BaBar Collaboration Meeting
• 8th December 2004

2
Outline

• What is Unfolding?
• and why might you want to do it?
• Overview of a few techniques
• Regularised unfolding
• Iterative method
• Idea for a ROOT package
• but not much code yet!
• References

3
Unfolding

• In other fields known as deconvolution or
  unsmearing
• Given a true PDF, µ, that is corrupted by
  detector effects, described by a response
  function, R, we measure a distribution ?. In
  terms of histograms
• This may involve
• inefficiencies lost events
• smearing events moving between bins
  (off-diagonal Rij)
• With infinite statistics, it would be possible to
  recover the original PDF by inverting the
  response matrix

4
Not so simple

• Unfortunately, if there are statistical
  fluctuations between bins this information is
  destroyed
• Since R washes out statistical fluctuations, R-1
  cannot distinguish between wildly fluctuating and
  smooth PDFs
• Obtain large negative correlations between
  adjacent bins
• Large fluctuations in reconstructed bin contents
• Need some procedure to remove wildly fluctuating
  solutions
• Give added weight to smoother solutions
• Solve for µ iteratively, starting with a
  reasonable guess and truncate iteration before it
  gets out of hand

5
What happens if you dont smooth
6
(No Transcript)
7
So why dont we always do this?

• If the true PDF and resolution function can be
  parameterised, then a ML fit is usually more
  convenient
• Directly returns parameters of interest
• Does not require binning
• If the response function doesnt include smearing
  (ie. its diagonal), then apply bin-by-bin
  efficiency correction directly
• If result is just needed for comparison (eg. with
  MC), could apply response function to MC
• simpler than un-applying response to data

8
When to use unfolding

• Use unfolding to recover theoretical distribution
  where
• there is no a-priori parameterisation
• this is needed for the result and not just
  comparison with MC
• there is significant bin-to-bin migration of
  events

9
Where could we use unfolding?

• Traditionally used to extract structure functions
• Widely used outside PP for image reconstruction
• Dalitz plots
• Cross-feed between bins due to misreconstruction
• True decay momentum distributions
• Theory at parton level, we measure hadrons
• Correct for hadronisation as well as detector
  effects
• Maybe could use smoothing for standard ML fits?

10
1. Regularised Unfolding

• Use ML to fit smeared bin contents to measured
  data, but include regularisation function
• where the regularisation parameter, a, controls
  the degree of smoothness.
• various criteria are used to select a
• eg. minimise mean squared error
• Various choices of regularisation function, S,
  are used
• Tikhonov regularisation minimise curvature
• for some definition of curvature, eg.
• RUN by Volker Blobel
• GURU by Andreas Höcker and Vakhtang
  Kartvelishvili
• using Singular Value Decomposition
• Maximum entropy

11
2. Iterative method

• Uses Bayes theorem to invert
• and using an initial set of probabilities, pi
  (eg. flat) obtain an improved estimate
• Repeating with new pi from these new bin contents
  converges quite rapidly
• Truncating the iteration prevents us seeing the
  bad effects of statistical fluctuations
• Fergus Wilson has implemented this method in
  ROOT/C

12
A ROOT framework

• It would be nice if these different methods could
  be made available as ROOT/C classes
• a la RooFit
• Could have a common interface to specify
• unfolding method and parameters
• response matrix
• or it could calculate it from MC samples
• measured histogram
• and return reconstructed truth histogram
• Handle 1D and 2D at least

13
A ROOT framework (2)

• Should also handle simple cases
• correction factors
• direct inversion of response matrix
• allowing to easily see whether full unfolding
  required
• Could also be useful outside BaBar!

14
Progress so far

• I have played around with Fergus code
• Can now be used in interactive ROOT
• Having a few problems right now extending beyond
  simple example
• Need to understand more options before designing
  an interface
• Also started to look at Andreas GURU package
• Is this a good idea?
• Or would I better spend my time with yet another
  tweak to the bookkeeping system ?
• I am still just learning, so pointers,
  suggestions, and ideas are most welcome

15
References

• G. Cowan, A Survey of Unfolding Methods for
  Particle Physics, Proc. Advanced Statistical
  Techniques in Particle Physics, Durham (2002)
• http//www.ippp.dur.ac.uk/Workshops/02/statistics/
  
• G. Cowan, Statistical Data Analysis, Oxford
  University Press (1998)
• R. Barlow, SLUO Lectures on Numerical Methods in
  HEP (2000),Lecture 9 Unfolding
• www-group.slac.stanford.edu/sluo/Lectures/Stat_Lec
  tures.html
• V. Blobel, Unfolding Methods in High Energy
  Physics,DESY 84-118 (1984) also CERN 85-02
• A. Höcker and V. Kartvelishvili, SVD Approach to
  Data Unfolding,NIM A 372 (1996) 469
• www.lancs.ac.uk/depts/physics/staff/kartvelishvili
  .html