FDR, Evidence Theory, Robustness

1
FDR, Evidence Theory, Robustness
2
Multiple testing

• The probability of rejecting a true null
  hypothesis at 99 is 1.
• Thus, if you repeat test 100 times, each time
  with new data, you will reject sometime with
  probability 0.63
• Bonferroni correction, FWE controlin order to
  reach significance level 1 in an experiment
  involving 1000 tests, each test should be checked
  with significance 1/1000

3
FDR Example - independence
Fdrex(pv,0.05,0) 10 signals suggested. Smallest
p-value notsignificant with Bonferroni
correction (0.019 vs 0.013)
4
FDR Example - dependency
Fdrex(pv,0.05,1) 10 signals suggestedassuming
independenceall disappear with correction term
5
Ed Jaynes devoted a large part of his career to
promoteBayesian inference. He also championed
theuse of Maximum Entropy in physics Outside
physics, he received resistance from people who
hadalready invented other methods.Why should
statistical mechanics say anything about our
daily human world??
6
Zadehs Paradoxical Example

• Patient has headache, possible explanations are
• M-- Meningitis C-- Concussion T-- Tumor.
• Expert 1 P( M )0 P( C )0.9 P( T )0.1
• Expert 2 P( M )0.9 P( C )0 P( T )0.1
• Parallel comb 0 0
  0.01
• What is the combined conclusion?
  Parallelnormalized (0,0,1)?
• Is there a paradox??

7
Zadehs Paradox (ctd)

• One expert (at least) made an error
• Experts do not know what probability zero means
• Experts made correct inferences based on
  different observation sets, and T is indeed the
  correct answer f(?o1, o2) c
  f(o1?)f(o2 ?)f(?)
• but this assumes f(o1,o2 ?)f(o1 ?) f(o2
  ?) which need not be true if granularity of ?
  istoo coarse (not taking variability of f(oi ?)
  into account).One reason (among several) to look
  at Robust Bayes.

8
Robust Bayes

• Priors and likelihoods are convex sets of
  probability distributions (Berger, de Finetti,
  Walley,...) imprecise probability
• Every member of posterior is a parallell
  combination of one member of likelihood and one
  member of prior.
• For decision making Jaynes recommends to use
  that member of posterior with maximum entropy
  (Maxent estimate).

9
Generalisation of Bayes/KalmanWhat if

• You have no prior?
• Likelihood infeasible to compute (imprecision)?
• Parameter space vague, i.e., not the same for all
  likelihoods? (Fuzziness, vagueness)?
• Parameter space has complex structure (a simple
  structure is e.g., a Cartesian product of reals,
  R, and some finite sets)?

10
Some approaches...

• Robust Bayes replace distributions by convex
  sets of distributions (Berger m fl)
• Dempster/Shafer/TBM Describe imprecision with
  random sets
• DSm Transform parameter space to capture
  vagueness. (Dezert/Smarandache, controversial)
• FISST FInite Set STatistics Generalisesobservat
  ion- and parameter space to product of spaces
  described as random sets.(Goodman, Mahler,
  Ngyuen)

11
Ellsbergs ParadoxAmbiguity Avoidance
Urna A innehåller 4 vita och 4 svarta kulor, och
4 av okänd färg (svart eller vit)
Urna B innehåller 6 vita och 6 svarta kulor
?
?
?
?
Du får en krona om du drar en svart kula. Ur
vilken urnavill du dra den?
En precis Bayesian bör först anta hur ?-kulorna
är färgade och sedansvara. Men en majoritet
föredrar urna B även om svart byts mot vit
12
Hur används imprecisa sannolikheter?

• Förväntad nytta för beslutsalternativ blir
  intervall i stället för punkter maximax,
  maximin, maximedel?

u
Bayesian
optimist
pessimist
a
13
(No Transcript)
14
Hur används imprecisa sannolikheter?

• Förväntad nytta för beslutsalternativ blir
  intervall i stället för punkter maximax,
  maximin, maximedel?

u
Bayesian
optimist
pessimist
a
15
Dempster/Shafer/Smets

• Evidence is random set over over ?.
• I.e., probability distribution over .
• Probability of singleton Belief allocated to
  alternative, i.e., probability.
• Probability of non-singelton Belief allocated
  to set of alternatives, but not to any part of
  it.
• Evidences combined by random intersection
  conditioned to be non-empty (Dempsters rule).

16
Correspondence DS-structure -- set of
probability distributions
For a pdf (bba) m over 2?, consider allways of
reallocating the probability mass of
non-singletons to their member atoms This gives
a convex set of probability distributions over
?. Example ?A,B,C
set of pdfs
bba
A 0.1B 0.3 C 0.1AB 0.5
A 0.10.5xB 0.30.5(1-x)C 0.1
for all x?0,1
Can we regard any set of pdfs as a bba? Answer
is NO!! There are more convex sets of pdfs than
DS-structures
17
Representing probability set as bba 3-element
universe
Rounding up use lower envelope. Rounding down
Linear programming Rounding is not unique!!
Black convex set Blue rounded up Red rounded
down
18
Another appealing conjecture

• Precise pdf can be regarded as (singleton)
  random set.
• Bayesian combination of precise pdfs corresponds
  to random set intersection (conditioned on
  non-emptiness)
• DS-structure corresponds to Choquet capacity
  (set of pdfs)
• Is it reasonable to combine Choquet capacities by
  (nonempty) random set intersection (Dempsters
  rule)??
• Answer is NO!!
• Counterexample Dempsters combination cannot be
  obtained by combining members of prior and
  likelihood
• Arnborg JAIF vol 1, No 1, 2006

19
Consistency of fusion operators
Axes are probabilities of A and B in a 3-element
universe
P(B)
Operands (evidence)
Robust Fusion
Dempsters rule
Modified Dempsters rule
Rounded robust
DS rule
MDS rule
P(A)
P(C )1-P(A)-P(B)