MultipleGroup Threats to Internal Validity

1
Multiple-Group Threats to Internal Validity
2
The Central Issue

• when you move from single to multiple group
  research the big concern is whether ther groups
  are comparable
• usually this has to do with how you assign units
  (e.g., persons) to the groups (or select them
  into groups)
• we call this issue selection or selection bias

3
The Multiple Group Case
alternative explanations
Administer Program
Measure Outcomes
Measure Baseline
Do not Administer Program
Measure Outcomes
Measure Baseline
alternative explanations
4
Example

• Compensatory Education in Math for 1st Graders
• Pre-Post Program-Comparison Group Design
• Measures (O) are standardized achievement tests
  (at start of grade 1 and start of grade 2 Forms
  A B)

5
Selection Threats

• any factor other than the program that leads to
  posttest differences between groups
• for example, because of group differences, kids
  in one group watch Sesame Street more frequently
  and pick up more math concepts

6
Selection-History Threat

• any other event that occurs between pretest and
  posttest that the groups experience differently
• for example, kids in one group pick up more math
  concepts because they watch more Sesame Street

7
Selection-Maturation Threat

• differential rates of normal growth between
  pretest and posttest for the groups
• they are learning at different rates, even
  without program

8
Selection-Testing Threat

• differential effect on the posttest of taking the
  pretest
• the test may have primed the kids differently
  in each group or they may have learned
  differentially from the test, not the program

9
Selection-Instrumentation Threat

• any differential change in the test used for each
  group from pretest and posttest
• for example, change due to different forms of
  test being given differentially to each group,
  not due to program

10
Selection-Mortality Threat

• differential non-random dropout between pretest
  and posttest
• for example, kids drop out of the study at
  different rates for each group

11
Selection-Regression Threat

• different rates of regression to the mean because
  groups differ in extremity
• for example, program kids are disproportionately
  lower math scorers and consequently have greater
  regression to the mean

12
(No Transcript)