Introduction to Statistics

1
Introduction to Statistics

• Lecture Notes
• October 30, 2008
• Dr. Sinn

2
Two More Ideas

• Level of Significance
• How sure are we of test?
• Example a .05 (typical)
• Indicates we are willing to take a 5 chance of
  making an error (Type I)
• 2 types of errors are possible
• Type I Errors
• Type II Errors

3
Error Analysis Martha Stewart

• There were 4 possibilities.
• Actually Guilty Guilty Verdict
• Actually Innocent Guilty Verdict
• Actually Guilty Acquittal
• Actually Innocent Acquittal
• Did the verdict match what was actually true?

4
Marthas Matrix
H0 Innocent Ha Guilty Prosecutor
researcher
5
Type I Error

• False Rejection of the Null Hypothesis
• a is the probability of Type I error we find
  acceptable.
• a .05 means we will take a 5 of thinking the
  groups are different when they arent.
• Two other ways to think of Type I
• Medical Tests False Positive
• Incorrectly conclude H1 is true

6
Type II Error

• False Acceptance of the Null Hypothesis
• Error conclude the groups are same when they are
  actually different.
• Two other ways to think of Type II
• Medical Tests False Negative
• Incorrectly conclude H0 is true

7
Error Key Points

• Error rates are inversely related
• Initially, suppose a .05
• 5 chance of Type I Error
• To reduce error, we change to a .01
• 1 chance of Type I Error
• This is better, right?
• Problem Type II error rate increases.
• As a (Type I error rate) decreases
• ß (Type II error rate) increases
• Analysis of Type II error (ß) is for an advanced
  statistics classbut everyone must understand
  there is trade-off.

8
Who Cares?

• Consider 2 examples.
• Pharmaceutical Study
• Studying a drug with massive side effects.
• Double-Blind Study Control Group vs. Treatment
  Group.
• Is Treatment Group different (better) than
  Control?
• Educational Study
• Study a teaching method that is inexpensive.
• Control group vs. Treatment group.
• Is Treatment Group different (better) than
  Control?

9
Pharmaceutical Study

• Type I Error
• Conclude Drug helps when it doesnt.
• Result Subject patients to terrible side effects
  for no reason.
• Need low a, say a .01
• Type II Error
• Conclude Drug doesnt help when it does.
• Result Dont subject patients to terrible side
  effects when it might help.
• This is a better outcome than above.

10
Educational Study

• Type I Error
• Conclude Method helps students when it doesnt.
• Result Implement inexpensive program for no
  reason (not all that unusual in education,
  anyway, except that most programs are expensive).
• Can use a high a, say a .10
• Type II Error
• Conclude Method doesnt help when it does.
• Result Students dont get program when it might
  help them.
• This is a worse outcome than above.

11
Example How to set a?

• Medical tests have an accuracy rate.
• Suppose you are administering an HIV test on
  samples of blood that were donated to a
  hospitals blood bank.
• Clicker Question Where should you set a? High
  or Low?
• High set a 0.1
• Medium set a 0.05
• Low set a 0.01
• It cannot be determined from the information given

12
Think About It This Way

• Go back to Marthas Trial
• Innocent until proven guilty.
• Wrongful Conviction is Type I Error.
• This is what our judicial system tries to avoid.
• HIV Test
• Clear until proven otherwise.
• Telling a person they are HIV when they are not
  is Type I Error.
• What should you do?

13
Think About It This Way

• Type II Error in Marthas Trial
• The guilty go free.
• This is what our judicial system (hopefully)
  allows.
• Type II Error in HIV Test
• Someone with HIV donates blood.
• How should you set a?

14
Setting a

• Determine the real-world effects of making both
  Type I and Type II errors. Minimize the worst
  one. If both are equivalently bad, consider
• Indications for setting a High a (a gt .05)
• Small sample (n lt 25)
• Exploratory Study
• Medical Tests Side Effects Benefits of main
  effect far outweigh problems of side effects
• Medical Tests (Controls Type II Error Problems)
• Indications Low a (a lt .05)
• Large sample (n gt 250)
• Confirmatory (high precision) Study
• Medical Tests Side Effects Benefits of main
  effect far outweighed by problems of side effects

15
Activity

• For the worksheet
• Describe Type I Error
• Describe Type II Error
• State the error rates in terms of real-world
  research implications.
• Which of the two do you feel to be a more severe
  problem?

16
Setting a

• a is the Type I error rate
• As Type I error is decreased, Type II increases.
• a levels
• Low (.001, .005, .01) Controls Type I
• Moderate (.05)
• High (.075, .1, .125) Controls Type II