Statistics for the Social Sciences

1
Statistics for the Social Sciences

• Psychology 340
• Spring 2005

Effect sizes Statistical Power
2
Outline

• Effect size Cohens d
• Error types
• Statistical Power Analysis

3
Performing your statistical test
Real world (truth)
H0 is correct
H0 is wrong


Reject H0
Experimenters conclusions
Fail to Reject H0
4
Performing your statistical test
Real world (truth)
H0 is correct
H0 is wrong


5
Performing your statistical test
Real world (truth)
H0 is correct
H0 is wrong


The original (null) distribution
The new (treatment) distribution
The original (null) distribution
6
Performing your statistical test
Real world (truth)
H0 is correct
H0 is wrong


So there is only one distribution
So there are two distributions
The original (null) distribution
The new (treatment) distribution
The original (null) distribution
7
Effect Size

• Hypothesis test tells us whether the observed
  difference is probably due to chance or not
• It does not tell us how big the difference is

H0 is wrong
So there are two distributions
The original (null) distribution
The new (treatment) distribution

• Effect size tells us how much the two populations
  dont overlap

8
Effect Size

• Figuring effect size

The original (null) distribution
The new (treatment) distribution

• Effect size tells us how much the two populations
  dont overlap

9
Effect Size

• Standardized effect size

• Puts into neutral units for comparison (same
  logic as z-scores)

Cohens d
The original (null) distribution
The new (treatment) distribution

• Effect size tells us how much the two populations
  dont overlap

10
Effect Size

• Effect size conventions
• small d .2
• medium d .5
• large d .8

The original (null) distribution
The new (treatment) distribution

• Effect size tells us how much the two populations
  dont overlap

11
Error types
Real world (truth)
H0 is correct
H0 is wrong


Reject H0
Experimenters conclusions
Fail to Reject H0
12
Error types
Real world (truth)
H0 is correct
H0 is wrong


Type I error
Reject H0
Experimenters conclusions
Fail to Reject H0
Type II error
13
Statistical Power

• The probability of making a Type II error is
  related to Statistical Power
• Statistical Power The probability that the study
  will produce a statistically significant results
  if the research hypothesis is true (there is an
  effect)

• So how do we compute this?

14
Statistical Power
Real world (truth)
H0 is true (is no treatment effect)
15
Statistical Power
Real world (truth)
H0 is false (is a treatment effect)
The original (null) distribution
Reject H0
16
Statistical Power
Real world (truth)
H0 is false (is a treatment effect)
The new (treatment) distribution
The original (null) distribution
Failing to Reject H0, even though there is a
treatment effect
Reject H0
Fail to reject H0
17
Statistical Power
Real world (truth)
H0 is false (is a treatment effect)
The new (treatment) distribution
The original (null) distribution
Failing to Reject H0, even though there is a
treatment effect
Probability of (correctly) Rejecting H0
Reject H0
Fail to reject H0
18
Statistical Power

• Steps for figuring power

• 1) Gather the needed information mean and
  standard deviation of the Null Population and the
  predicted mean of Treatment Population

19
Statistical Power

• Steps for figuring power

• 2) Figure the raw-score cutoff point on the
  comparison distribution to reject the null
  hypothesis

From the unit normal table Z -1.645
Transform this z-score to a raw score
20
Statistical Power

• Steps for figuring power

• 3) Figure the Z score for this same point, but on
  the distribution of means for treatment Population

Remember to use the properties of the treatment
population!
Transform this raw score to a z-score
21
Statistical Power

• Steps for figuring power

• 4) Use the normal curve table to figure the
  probability of getting a score more extreme than
  that Z score

From the unit normal table Z(0.355) 0.3594
The probability of detecting this an effect of
this size from these populations is 64
22
Statistical Power
Factors that affect Power

• a-level
• Sample size
• Population standard deviation ?
• Effect size
• 1-tail vs. 2-tailed

23
Statistical Power
Factors that affect Power
a-level
Change from a 0.05 to 0.01
Reject H0
Fail to reject H0
24
Statistical Power
Factors that affect Power
a-level
Change from a 0.05 to 0.01
Reject H0
Fail to reject H0
25
Statistical Power
Factors that affect Power
a-level
Change from a 0.05 to 0.01
Reject H0
Fail to reject H0
26
Statistical Power
Factors that affect Power
a-level
Change from a 0.05 to 0.01
Reject H0
Fail to reject H0
27
Statistical Power
Factors that affect Power
a-level
Change from a 0.05 to 0.01
Reject H0
Fail to reject H0
28
Statistical Power
Factors that affect Power
a-level
So as the a level gets smaller, so does the
Power of the test
Change from a 0.05 to 0.01
Reject H0
Fail to reject H0
29
Statistical Power
Factors that affect Power
Sample size
Change from n 25 to 100
Reject H0
Fail to reject H0
30
Statistical Power
Factors that affect Power
Sample size
Change from n 25 to 100
Reject H0
Fail to reject H0
31
Statistical Power
Factors that affect Power
Sample size
Change from n 25 to 100
Reject H0
Fail to reject H0
32
Statistical Power
Factors that affect Power
Sample size
Change from n 25 to 100
Reject H0
Fail to reject H0
33
Statistical Power
Factors that affect Power
Sample size
Change from n 25 to 100
Reject H0
Fail to reject H0
34
Statistical Power
Factors that affect Power
Population standard deviation
Change from ? 25 to 20
Reject H0
Fail to reject H0
35
Statistical Power
Factors that affect Power
Population standard deviation
Change from ? 25 to 20
Reject H0
Fail to reject H0
36
Statistical Power
Factors that affect Power
Population standard deviation
Change from ? 25 to 20
Reject H0
Fail to reject H0
37
Statistical Power
Factors that affect Power
Population standard deviation
Change from ? 25 to 20
Reject H0
Fail to reject H0
38
Statistical Power
Factors that affect Power
Population standard deviation
Change from ? 25 to 20
Reject H0
Fail to reject H0
39
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
40
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
mtreatment
mno treatment
41
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
42
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
43
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
44
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
45
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
46
Statistical Power
Factors that affect Power
Effect size
Compare a small effect (difference) to a big
effect
Reject H0
Fail to reject H0
47
Statistical Power
Factors that affect Power
1-tail vs. 2-tailed
Change from a 0.05 two-tailed to a 0.05
two-tailed
Reject H0
Fail to reject H0
48
Statistical Power
Factors that affect Power
1-tail vs. 2-tailed
Change from a 0.05 two-tailed to a 0.05
two-tailed
Reject H0
Fail to reject H0
49
Statistical Power
Factors that affect Power
1-tail vs. 2-tailed
Change from a 0.05 two-tailed to a 0.05
two-tailed
p 0.025
p 0.025
Reject H0
Fail to reject H0
50
Statistical Power
Factors that affect Power
1-tail vs. 2-tailed
Change from a 0.05 two-tailed to a 0.05
two-tailed
p 0.025
p 0.025
Reject H0
Fail to reject H0
51
Statistical Power
Factors that affect Power
1-tail vs. 2-tailed
Change from a 0.05 two-tailed to a 0.05
two-tailed
p 0.025
p 0.025
Reject H0
Fail to reject H0
52
Statistical Power
Factors that affect Power
1-tail vs. 2-tailed
Change from a 0.05 two-tailed to a 0.05
two-tailed
p 0.025
p 0.025
Reject H0
Fail to reject H0
53
Statistical Power

• Factors that affect Power
• a-level So as the a level gets smaller, so does
  the Power of the test
• Sample size As the sample gets bigger, the
  standard error gets smaller and the Power gets
  larger
• Population standard deviation As the population
  standard deviation gets smaller, the standard
  error gets smaller and the Power gets larger
• Effect size As the effect gets bigger, the Power
  gets larger
• 1-tail vs. 2-tailed Two tailed functionally cuts
  the ?-level in half, which decreases the power

54
Why care about Power?

• Determining your sample size
• Using an estimate of effect size, and population
  standard deviation, you can determine how many
  participants need to achieve a particular level
  of power
• When a result if not statistically significant
• Is is because there is no effect, or not enough
  power
• When a result is significant
• Statistical significance versus practical
  significance

55
Ways of Increasing Power