Power and Sample Size

1
Power and Sample Size
I HAVE THE POWER!!!

• Boulder 2006
• Benjamin Neale

2
To Be Accomplished

• Introduce concept of power via correlation
  coefficient (?) example
• Identify relevant factors contributing to power
• Practical
• Empirical power analysis for univariate twin
  model (simulation)
• How to use mx for power

3
Simple example

• Investigate the linear relationship (r)
• between two random variables X and Y r0 vs. r?0
  (correlation coefficient).
• draw a sample, measure X,Y
• calculate the measure of association r (Pearson
  product moment corr. coeff.)
• test whether r ? 0.

4
How to Test r ? 0

• assumed the data are normally distributed
• defined a null-hypothesis (r 0)
• chosen a level (usually .05)
• utilized the (null) distribution of the test
  statistic associated with r0
• tr ? (N-2)/(1-r2)

5
How to Test r ? 0

• Sample N40
• r.303, t1.867, df38, p.06 a.05
• As p gt a, we fail to reject r 0
• have we drawn the correct conclusion?

6
type I error rate probability of deciding r ?
0(while in truth r0) a is often chosen to
equal .05...why?
DOGMA
7
N40, r0, nrep1000 central t(38), a0.05
(critical value 2.04)
8
Observed non-null distribution (r.2) and null
distribution
9
In 23 of tests of r0, tgt2.024 (a0.05), and
thus draw the correct conclusion that of
rejecting r 0. The probability of rejecting
the null-hypothesis (r0) correctly is 1-b, or
the power, when a true effect exists
10
Hypothesis Testing

• Correlation Coefficient hypotheses
• ho (null hypothesis) is ?0
• ha (alternative hypothesis) is ? ? 0
• Two-sided test, where ? gt 0 or ? lt 0 are
  one-sided
• Null hypothesis usually assumes no effect
• Alternative hypothesis is the idea being tested

11
Summary of Possible Results

• H-0 true H-0 false
• accept H-0 1-a b
• reject H-0 a 1-b
• atype 1 error rate
• btype 2 error rate
• 1-bstatistical power

12
STATISTICS
Rejection of H0
Non-rejection of H0
H0 true
R E A L I T Y
HA true
13
Power

• The probability of rejection of a false
  null-hypothesis depends on
• the significance criterion (?)
• the sample size (N)
• the effect size (?)

The probability of detecting a given effect size
in a population from a sample of size N, using
significance criterion ?
14
Standard Case
Sampling distribution if HA were true
Sampling distribution if H0 were true
alpha 0.05
POWER 1 - ?
?
?
T
Non-centrality parameter
15
Increased effect size
Sampling distribution if HA were true
Sampling distribution if H0 were true
alpha 0.05
POWER 1 - ? ?
?
?
T
Non-centrality parameter
16
Impact of more conservative
Sampling distribution if H0 were true
Sampling distribution if HA were true
alpha 0.01
POWER 1 - ? ?
?
?
T
Non-centrality parameter
17
Impact of less conservative
Sampling distribution if H0 were true
Sampling distribution if HA were true
alpha 0.10
POWER 1 - ? ?
?
?
T
Non-centrality parameter
18
Increased sample size
Sampling distribution if HA were true
Sampling distribution if H0 were true
alpha 0.05
POWER 1 - ? ?
?
?
T
Non-centrality parameter
19
Effects on Power Recap

• Larger Effect Size
• Larger Sample Size
• Alpha Level shifts ltBeware the False Positive!!!gt
• Type of Data
• Binary, Ordinal, Continuous
• Multivariate analysis
• Empirical significance/permutation

20
When To Do Power Calculations?

• Generally study planning stages of study
• Occasionally with negative result
• No need if significance is achieved
• Computed to determine chances of success