LIMITS

1
LIMITS

• Why limits?
• Methods for upper limits
• Desirable properties
• Dealing with systematics
• Feldman-Cousins
• Recommendations

2
WHY LIMITS?

• Michelson-Morley experiment ? death of aether
• HEP experiments
• CERN CLW (Jan 2000)
• FNAL CLW (March 2000)
• Heinrich, PHYSTAT-LHC, Review of Banff Challenge

3
SIMPLE PROBLEM?

• Gaussian
• exp-0.5(x-µ)2/s2
• No restriction on µ, s known exactly
• µ ? x0 k s
• BUT Poisson µ se b
• s 0
• e and b with uncertainties
• Not like 2 3 ?
• N.B. Actual limit from experiment Expected
  (median) limit

4
Methods (no systematics)

• Bayes (needs priors e.g. const, 1/µ, 1/vµ, µ,
  ..)
• Frequentist (needs ordering rule,
• possible empty intervals, F-C)
• Likelihood (DONT integrate your L)
• ?2 (s2 µ)
• ?2(s2 n)
• Recommendation 7 from CERN CLW Show your L
• 1) Not always practical
• 2) Not sufficient for frequentist methods

5
Bayesian posterior ? intervals
Upper limit
Lower limit
Central interval
Shortest
6
90 C.L. Upper Limits
m
x
x0
7
Ilya Narsky, FNAL CLW 2000
8
DESIRABLE PROPERTIES

• Coverage
• Interval length
• Behaviour when n lt b
• Limit increases as sb increases

9
?lnL -1/2 rule

• If L(µ) is Gaussian, following definitions of s
  are equivalent
• 1) RMS of L(µ)
• 2) 1/v(-d2L/dµ2)
• 3) ln(L(µs) ln(L(µ0)) -1/2
• If L(µ) is non-Gaussian, these are no longer the
  same
• Procedure 3) above still gives interval that
  contains the true value of parameter µ with 68
  probability
• Heinrich CDF note 6438 (see CDF Statistics
  Committee Web-page)
• Barlow Phystat05

10
COVERAGE How
often does quoted range for parameter include
params true value? N.B. Coverage is a property
of METHOD, not of a particular exptl
result Coverage can vary with Study coverage
of different methods of Poisson parameter
, from observation of number of events n Hope
for
100
Nominal value
11
COVERAGE If true for all
correct coverage
Plt for some undercoverage

(this is serious !)
Pgt for some overcoverage
Conservative Loss of rejection power
12
Coverage L approach (Not frequentist)
P(n,µ) e-µµn/n! (Joel Heinrich CDF note
6438) -2 ln?lt 1 ? P(n,µ)/P(n,µbest)
UNDERCOVERS
13
Frequentist central intervals, NEVER
undercovers(Conservative at both ends)
14
Feldman-Cousins Unified intervalsFrequentist,
so NEVER undercovers
15
Probability orderingFrequentist, so NEVER
undercovers
16

• (n-µ)2/µ ? 0.1 24.8
  coverage?
• NOT frequentist Coverage 0 ? 100

17
COVERAGE

• N.B. Coverage alone is not sufficient
• e.g. Clifford (CERN CLW, 2000)
• Friend thinks of number
• Procedure for providing interval that
  includes number 90 of time.

18
COVERAGE

• N.B. Coverage alone is not sufficient
• e.g. Clifford (CERN CLW, 2000)
• Friend thinks of number
• Procedure for providing interval that
  includes number 90 of time.
• 90 Interval -? to ?
• 10 number 102.84590135..

19
INTERVAL LENGTH

• Empty ? Unhappy physicists
• Very short? False impression of sensitivity
• Too long? loss of power
• (2-sided intervals are more complicated because
  shorter is not metric-independent e.g. 0?4
  or 4 ?9)

20
90 Classical interval for Gaussian s 1 µ
0 e.g. m2(?e)
21
Behaviour when n lt b

• Frequentist Empty for n lt lt b
• Frequentist Decreases as n decreases below b
• Bayes For n 0, limit independent of b
• Sen and Woodroofe Limit increases as data
  decreases below expectation

22
FELDMAN - COUSINS

• Wants to avoid empty classical intervals ?
• Uses L-ratio ordering principle to resolve
  ambiguity about which 90 region? ?
• Neyman Pearson say L-ratio is best for
  hypothesis testing
• Unified ? No Flip-Flop problem

23
Xobs -2 now gives upper limit
24
(No Transcript)
25
(No Transcript)
26
Flip-flop
Black lines Classical 90 central
interval Red dashed Classical 90 upper limit
27
(No Transcript)
28
Poisson confidence intervals. Background 3
Standard Frequentist Feldman -
Cousins
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
Recommendations?

• CDF note 7739 (May 2005)
• Decide method in advance
• No valid method is ruled out
• Bayes is simplest for incorporating nuisance
  params
• Check robustness
• Quote coverage
• Quote sensitivity
• Use same method as other similar expts
• Explain method used

43
(No Transcript)
44
(No Transcript)
45
Caltech Workshop, Feb 11th
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
Tomorrow is last day of this visit

• Contact me at
• l.lyons_at_physics.ox.ac.uk

56
Peasant and Dog

• Dog d has 50 probability of being 100 m. of
  Peasant p
• Peasant p has 50 probability of being within
  100m of Dog d

d
p
x
River x 0
River x 1 km
57
Given that a) Dog d has 50 probability of
being 100 m. of Peasant, is it true that b)
Peasant p has 50 probability of being within
100m of Dog d ?