Threats to the Validity of Hypothesis Testing: Confounding

1
Threats to the Validity of Hypothesis Testing
Confounding   Are the Groups Alike at the Start
of the Study?     Scott Wetstone,
M.D.   Community Medicine and Health
Care University of Connecticut School of Medicine
2

• Objectives
• The student should be able to
• Define and recognize confounding variables.
• Define unbalanced confounding variables and
  explain how they threaten the validity of a
  study.
• List methods that deal with unbalanced
  confounding variables.
• Detect obvious unbalanced confounding variables
  in research articles.

3
Group Comparison Method   Dependent Variable
Death Rate Independent Variable
Smoking   Control Group Experimental
Group  Non-smokers Smokers
Pexperimental
Pcontrol
rexperimental
rcontrol
Observed Group Difference
4
Critical Assumption
The members of each group are alike in all
possible ways except for the dosage of the
independent variable.
5
(No Transcript)
6
Example Statoma, a newly discovered form of brain
neoplasm, has a 75 response rate when treated
with standard chemotherapy. An investigator
wanted to compare this standard therapy with two
other treatment regimens 1) the standard drug
therapy plus radiation, and 2) a new drug. Both
the new therapies were reported to have a small
rate of impotency as a side effect as well as
being potentially lethal in a very small number
of cases. Patients from a Statoma clinic were
allowed to choose which of the three treatment
groups they entered.
7
Response rates for the three treatments  
standard treatment 85
standard treatment plus radiation 65
new drug 50  
8
Confounding Variable Another independent
variable (not the primary target of the research
study) that affects the value of the dependent
(outcome) variable.
9
Confounding Variables Almost always
EXIST! However, they do not
necessarily threaten the validity of a research
study.
10
Unbalanced Confounding Variable When a group has
a higher amount of the confounding variable than
another group. This unbalanced confounding
variable could be the cause of the observed
group difference!
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20

• Dealing with Confounding Variables
• Before group assignment
• Randomization (group assignment)
• Homogeneous subjects (inclusion
  exclusion criteria)
• Related sample designs (stratification
  matching)
• After group assignment
• Subgroup analysis
• Adjustment

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Detecting Unbalanced Confounders (Table 1)

• Inclusion of Potential Confounders
• (common sense professional judgment)
• Screening diagnostic tests for statistical
  significance
• Interval data t test
• Count data binomial chi-square tests

25
Demographic Characteristics
Differences between study groups were not
significant.
26
POWER (?) The ability to detect a true
effect,  IF it really exists Type II Error
(?)  Failure to detect a true effect, IF it
really exists  ? 1 - ?
27
Factors Affecting Power FACTOR p b a  
n NE s NE m1-m2
NE a Statistical
test YES YES NE data type dist YES YES NE
28
Demographic Characteristics
Differences between study groups were not
significant.
29
Two Standard Error Rule
30
Demographic Characteristics
Differences between study groups were not
significant.
31
         
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
Infant Behavior
See discussion of infant behavior in Methods for
definition of terms.
39
Infant Behavior
See discussion of infant behavior in Methods for
definition of terms.
40
Infant Behavior
See discussion of infant behavior in Methods for
definition of terms.
41
Infant Behavior
See discussion of infant behavior in Methods for
definition of terms.
42
(No Transcript)
43
(No Transcript)
44
Successes probability culm prob 0
0.00024 0.00024 1 0.00293 0.00317 2
0.01611 0.01929 3 0.05371 0.07300 4
0.12085 0.19385 5 0.19336 0.38721 6
0.22559 0.61279 7 0.19336 0.80615 8
0.12085 0.92700 9 0.05371 0.98071 10
0.01611 0.99683 11 0.00293 0.99976 12
0.00024 1.00000
45
Demographic Characteristics
Differences between study groups were not
significant.
46
Demographic Characteristics
Differences between study groups were not
significant.
47
(No Transcript)
48
(No Transcript)
49
Shift in Placebo Group 2 whites ? non-whites
50
(No Transcript)
51

• Summary
• The student should be able to
• Define and recognize confounding variables.
• Define unbalanced confounding variables and
  explain how they threaten the validity of a
  study.
• List methods that deal with unbalanced
  confounding variables.
• Detect obvious unbalanced confounding variables
  in research articles.

52
(No Transcript)
53
(No Transcript)