mclimits.C The LogLikelihood Ratio

1
mclimits.C The Log-Likelihood Ratio

• Catherine Wright
• 12/02/08

2
Inputs

• Un-normalised histograms of signal and background
  discriminating variable, SEPARATELY (Required to
  correctly deal with MC statistical fluctuations
  in mclimits).
• Scale Factors for a given luminosity e.g.
  10fb-1, calculable using-
• e.g. H??? m?? spectrum _at_ 130GeV/c2

3
The Test and Null Hypotheses

• We postulate two hypotheses
• - the Null Hypothesis, H0 of Background Only.
  And
• - the Test Hypothesis, H1 of Signal (at a given
  mass) Background.
• These hypotheses are then tested by generating
  pseudo data and using the Log-Likelihood Ratio as
  a Test Statistic.

4
Generate Pseudo data

• Choose a template for the toy MC to follow. e.g.
  signal background experiment or background only
  experiment?
• ? For this work choose sb experiment
• (see evidence that in results there is no
  difference)
• Model real data on selected template sample
  from histogram using the function-
• toyMC?FillRandom(sb_template,Nsb)
• Now have-
• Expected b-only distribution of the
  discriminating variable.
• Expected sb distribution of the
  discriminating variable.
• Set of toy MC modelled on the sb
  distribution and representing the observed
  data.
• See previous slide for example H???
  distributions of invariant mass.

5
The Test Statistic

• Assume poissonian probabilities for the arrival
  rate of signal and background events and write
  the Likelihoods as
• Such that the Log-Likelihood Ratio is written as
• Where
• - si and bi are the number of expected signal
  and background events in the bin, i.
• - ni is the number of observed data (toy MC)
  events in the bin, i.
• - N is the total number of bins in the
  distribution of the discriminating variable.

6
Producing the LLR Distributions

• With the definition of test statistic, we then
  calculate a value of the LLR for each of the
  hypotheses, H0 and H1, setting ni equal to the
  number of toy MC events in each bin, i.
• This produces a single value of the ts for H0 and
  for H1.
• In order to produce a distribution we generate
  another set of toy MC and recalculate LLRH0 and
  LLRH1.
• As many pseudo-experiments as is required to
  produce smooth Probability Density Functions
  (PDFs) for both hypotheses should be carried out.
  
• For the examples shown throughout, 10000
  pseudo-experiments have been completed. Need 1E8
  to really calculate significance effectively.

7
The PDFs

• The PDFs should look like this (H??? example)

8
Interpreting the Results
Some Definitions 1-CLsb Discovery
potential. CLsb False Exclusion Rate CLb
Exclusion potential 1-CLb False
Discovery Rate (Power) ? Significance Level
9
Interpreting the Results (2)
To exclude at Confidence Level (CL) require
So, for example, to Exclude at 95 CL
i.e. the False Exclusion Rate (CLsb) cannot be
any more than 5 of the Exclusion Potential
(CLb). ? Protects against excluding when you are
insensitive to the result.
10
Expected Results
An example from H???

• Two PDFs one for each hypotheses.
• Fit PDFs i.e. in the case where uncertainties
  are symmetric, with a gaussian.
• Use fits to calculate 1-CLb in a median sb
  experiment.
• Use fits to estimate separation.
• - Should be comparable.
• Use fits to calculate CLs in a median b-only
  experiment.
• Finally, produce integrated Luminosity values
  the minimum required to achieve Exclusion given
  there is no Higgs, Evidence and Discovery given
  that there is a Higgs Boson. See over for process.

Significance (for 10fb-1) 3.3?
Example of Integrated Luminosity Results
11
Estimating Integrated L.dt

• Function in mclimits, lumipaux() calculates the
  scale factors needed to produce the Luminosity
  limits.
• Scales sb and b-only distributions by factor x.
• Calculates 1-CLb and CLs
• Determines if discovery, evidence or exclusion
  limits have been reached.
• IF YES
• - Outputs scale factor for input luminosity
  that determines the required luminosity for each
  result.
• e.g. if input histograms scaled to 10fb-1 and
  exclusion luminosity scale factor is 0.34, then
  integrated luminosity required for exclusion is
  3.4fb-1.

12
Estimating Integrated L.dt (contd)

• IF NO
• - Original distributions are scaled by 1/2x
• - Calculates 1-CLb and CLs
• - Determines if discovery, evidence or
  exclusion limits have been reached.
• IF YES
• - Outputs scale factor
• IF NO
• - sb and b scaled by 2x
• and so on until a limit is found. Repeated
  232 times occasionally no limit is found.