Lecture 8

1
Lecture 8 Internal Validity Threats continued

• EPSY 640
• Texas AM University

2
TESTING

• PRETESTING OR REPEATED TESTING
• proactive or retroactive interference
• memory
• change in cognitive schema or structure

3
TESTING

• SOLOMON 4 GROUP DESIGN

1 R O X O 2 R O O 3 R X O 4 R O
4
TESTING

• Willson and Putnam (1982) conducted a
  meta-analysis of studies using pretests and
  concluded that pretesting can effect both
  cognitive and psychological testing. The largest
  effects can be expected to occur within about two
  weeks of pretesting, and effects on achievement
  and intelligence tests is expected to be greater
  than effects for attitude or psychological tests.
  

5
MORTALITY

• LOSS OF CASES FROM GROUPS NONRANDOMLY
• POTENTIAL FOR PARTICULAR CONDITION TO CAUSE
  DROPOUTS
• NEED FOR INITIAL INFORMATION ON ALL PARTICIPANTS
  TO EXAMINE CHARACTERISTICS OF DROPOUTS

6
REGRESSION TOWARD MEAN

• EQUIVALENT TO IMPERFECT CORRELATION BETWEEN
  VARIABLES
• EXISTS WHENEVER A SAMPLE IS SELECTED AND ITS MEAN
  IS NOT EQUAL TO THE POPULATION MEAN, THEN
  MEASURED ON ANOTHER VARIABLE

7
REGRESSION TOWARD MEAN

• REGRESSION OCCURS WITH
• GIFTED AND TALENTED RESEARCH
• RESEARCH IN SPECIAL EDUCATION
• RESEARCH ON ANY NONREPRESENTATIVE GROUPS

8
REGRESSION TOWARD MEAN AND MATCHING

• MATCHING ALWAYS PRODUCES REGRESSION EFFECTS
  UNLESS THE TWO SAMPLES MATCHED HAVE THE SAME
  POPULATION MEANS

9
REGRESSION TOWARD MEAN AND MATCHING

• In a study of a new intelligence test researchers
  gave the test to students who had been previously
  evaluated with another intelligence test and
  found to have scored over 120, qualifying them
  for gifted and talented status. The researchers
  found that the students averaged below 120 on the
  new test and concluded that the new test did not
  adequately assess gifted and talented status.
  What they actually found was the regression
  effect due to the lack of perfect correlation
  between the two intelligence tests. A simple
  calculation from their reported results conformed
  quite well to the expected lowering of score from
  one assessment to the other.

10
Statistical regression due to matching
group 1 group 2
matching mean for selecting variable
Outcome correlates .7 with selecting
variable Group 1 predicted score on outcome
.7SD Group 2 predicted score on outcome .35
SD Thus, .35SD difference has been created with
no treatment
1SD
½ SD
11
HISTORY

• CHANGES AND EVENTS OUTSIDE TREATMENT CONDITIONS
  THAT AFFECT OUTCOMES
• Local or national events of stress
• Weather
• Political change

12
HISTORY

• Campbell and Ross (1968) and Glass (1968)
  evaluated the effect of a speed limit law in
  Connecticut enacted to begin in January, 1955.
  Politicians claimed within several months that
  the law had reduced significantly auto deaths,
  using data from the previous several years to
  support their claim. Indeed, statistical analysis
  supported a significant drop in deaths in the
  months following the implementation of the law,
  even accounting for typical drops occurring after
  the Christmas season. Glass (1968) noted that an
  analysis of the surrounding states showed a
  similar drop even though no laws had been enacted
  in them. An investigation of weather conditions
  at the time showed that a series of blizzards at
  the time severely restricted driving,
  particularly high speed driving that would have
  contributed otherwise to auto deaths. He and
  Campbell and Ross (1968) discounted the legal
  experimental account in favor of the historical
  explanation. Willson (1973), reported in Glass,
  Willson, and Gottman (1975) demonstrated that the
  legal effect was not significant when adjusted
  for the comparable effects in other adjacent
  states

13
(No Transcript)
14
(No Transcript)
15
(No Transcript)