Power, Power Curves and Sample Size

1
Power, Power Curves and Sample Size
2
Planning Reliable and Efficient Tests

• For any job, you need a tool that offers the
  right amount of power for the task at hand.
• You wouldn't use a telescope to examine a stamp
  collection, or a handheld magnifying glass to
  search for new galaxies, because neither would
  provide you with meaningful observations. To
  complicate matters, if detecting a galaxy really
  was your goal, the cost of gaining the necessary
  power might be more than you can afford.

3
Planning Reliable and Efficient Tests

• Anyone using statistical tests faces the same
  issues. You must consider the precision you need
  to meet your goals (should your test detect
  subtle effects or massive shifts?), and balance
  it against the cost of sampling your population
  (are you testing toothpicks or jet engines?).

4
Planning Reliable and Efficient Tests

• You also want the confidence in your results
  that's appropriate for your situation (testing
  seat belts demands a greater degree of certainty
  than testing shampoo). We measure this certainty
  with statistical power the probability your
  test will detect an effect that truly exists.

5
Planning Reliable and Efficient Tests

• Minitab's Power and Sample Size tools, with
  Power Curves, help you balance these issues that
  may compete for your limited resources. Here are
  three examples of how a quick Power and Sample
  Size test can help you save time and money
  getting results you can trust.

6
Don't Leave Success to Chance

• A paper clip manufacturer wants to detect
  significant changes in clip length. They sample
  thousands of clips because it is cheap and quick
  to do. But this huge sample makes the test too
  sensitive the broken line (next slide) shows it
  will sound the alarm if the average length
  differs by a trivial amount (0.05).

7
Don't Leave Success to Chance
8
Don't Leave Success to Chance

• This Power Curve shows they are wasting
  resources on excessive precision. A sample size
  of just 100 will detect meaningful differences
  (0.25) without crying wolf at every negligible
  blip.

9
Don't Leave Success to Chance
10
Don't Leave Success to Chance

• An aerospace company is designing an experiment
  to test a new rocket. Each rocket is very
  expensive, so it is critical to test no more than
  necessary.

11
Don't Leave Success to Chance
12
Don't Leave Success to Chance

• This Power Curve confirms an experiment with 6
  replicates will give researchers the power they
  need without spending more than they must.

13
Don't Leave Success to Chance
14
Don't Leave Success to Chance

• We've always done it this way. That's why a
  lumber company would sample 10 beams to test
  whether their strength meets the target.

15
Don't Leave Success to Chance
16
Don't Leave Success to Chance

• According to the Power Curve, this small sample
  size made their test incapable of detecting
  important effects. They must sample 34 beams to
  detect meaningful differences (0.50).

17
Don't Leave Success to Chance
18
See the Big Picture

• A power analysis helps you weigh your resources
  against your demands, and quantifies a test's
  ability to answer your question. It can expose
  design problems, like the lumber company's
  insufficient sample size. It can also reveal
  design solutions you hadn't considered.

19
See the Big Picture

• Take for instance the packaging plant of a snack
  company. Customers complain that the company's
  pretzel bags are sealed with glue that's too
  strong, so researchers use One-Way ANOVA to
  compare their current glue with three potential
  replacements.

20
See the Big Picture

• Differences in seal strength less than 10 are
  undetectable to most people, so their test only
  needs to detect a difference of 10. A power value
  of 80 is acceptable, but 90 is ideal.
• What sample size meets their needs?

21
See the Big Picture
30 samples of each glue ensure the test detects a
difference of 10 with 90 power.
22
See the Big Picture
Or, they could detect the same difference with 23
samples and 80 power. If this represents
considerable savings, the researchers may
consider using the smaller sample.
23
See the Big Picture

• The Power Curve illustrates this information,
  but it also charts every other combination of
  power and difference for a given sample size.

24
See the Big Picture
25
See the Big Picture

• The solid line indicates researchers can attain
  90 power with just 23 samples if they are
  willing to seek a difference of 12 instead of 10.
  This might just be the ideal choice.

26
See the Big Picture
27
How to Create Power Curves in Minitab

• Performing power analyses with Power Curves
  couldn't be simpler. You supply the factors you
  know, and Minitab calculates the one you omit.

28
How to Create Power Curves in Minitab

• Suppose a trainer wants to compare two training
  courses for forklift operators.
• She will use a two-sample t-test to compare the
  average scores that operators from each course
  earned on the final exam.

29
How to Create Power Curves in Minitab

• She knows she must detect a difference of 5 in
  either direction between the two courses with 80
  power, and historical data suggest a standard
  deviation of 5.
• But how many participants must she sample from
  each course?

30
How to Create Power Curves in Minitab
Choose Stat gt Power and Sample Size gt 2-Sample
t In Differences, type -5, 5 In Power values,
type 0.80 In Standard deviation, type 5 Click
OK
31
How to Create Power Curves in Minitab
32
How to Create Power Curves in Minitab
33
How to Create Power Curves in Minitab

• According to this Power Curve, sampling 17
  participants from each class enables her test to
  detect the difference she seeks with 80 power.

34
How to Create Power Curves in Minitab
35
Putting Power Curves to Use

• Without knowing the power of your test, it's
  hard to know if you can trust your results your
  test could be too weak to answer your question,
  or too strong for your needs. Minitab's Power
  Curves (available for many common statistical
  procedures) help you balance your resources
  against your goals and design a test you can
  trust that costs no more than necessary.

36
Putting Power Curves to Use

• Power Curves graph the dynamic relationships
  that define power, revealing the big picture and
  ensuring no option escapes your consideration.
  And perhaps most importantly, they make power
  analysis an easier and more accessible part of
  every project. Empower your test. Trust your
  results.