Exact calculations of expected power for the Benjamini-Hochberg procedure

1
Exact calculations of expected power for the
Benjamini-Hochberg procedure

• DH Glueck1, A Karimpour-Fard1, L Hunter1,
• J Mandel1 and KE Muller2

1University of Colorado Denver and 2University
of Florida
December 1, 2007
2
Power for multiple comparisons

• Efron, Storey and Tibshirani (2001) JASA
• Lee and Whitmore (2002)
• Finner Roters (2002)
• Sarkar (2002, 2004, 2006)
• Zien, Fluck, Zimmer and Lengauer (2002)
• Lehmann Romano (2005), Romano Shaikh (2006)

3
Benjamini and Hochberg procedure
4
Decisions for an experiment
5
Expected power
Expected power is the average number of null
hypotheses we should reject that we do reject.
6
Exact power
Glueck et al., CIS, 2008
7
Why this method for power?

• Exact
• Useful for clinical trials and other applications
  where accuracy is important.
• Works for any test statistics or for mixtures of
  test statistics.
• Avoids Type 3 errors- answering wrong question.
• If you plan to analyze your data using
  Benjamini and Hochberg, you need to conduct your
  power analysis by a method that does power for
  Benjamini and Hochberg