12 Experimental Control and Internal Validity

1
12 Experimental Control and Internal Validity

• What are the potential threats to the validity
  of research?
• What is experimental control?
• What effect do extraneous variables have on the
  validity of research?
• What is meant by confounding? Why does
  confounding reduce an
• experiments internal validity?
• What are some methods of controlling for
  extraneous variables in
• experimental designs?
• What are some methods for increasing the
  validity of experimental
• manipulation?
• What are manipulation checks and confound checks,
  and what can they
• tell us?
• What are some common artifacts in experimental
  research, and how can
• they produce confounding?

2
Threats to the Validity of Research
Threats to construct validity
Although it is claimed that the measured
variables and the experimental manipulations
relate to the conceptual variables of interests,
they actually may not.
Threats to statistical conclusion validity
Conclusions regarding the research may be
incorrect because a Type 1 or Type 2 error was
made.
Threats to internal validity
Although it is claimed that the independent
variable caused change in the dependent
variable, the changes in the dependent variable
may have actually been caused by a confounding
variable.
Threats to external validity
Although it is claimed that the results are more
general, the observed effects may actually only
be found under limited conditions or for specific
groups of people.
3
Experimental Control
Extraneous Variables
Variables other than the independent variable
that cause changes in the dependent variable.
Example participants ability, mood,
motivation, and differences in how the
experimenter treats the participants or how they
react to the experimental setting.
Confounding Variables
Variables other than the independent variable on
which the participants in one experimental
condition differ systematically or on average
from those in other conditions. Example Room
sizes, Speed of computers, and number of
participants.
4
Control of Extraneous Variables
Limited-Population Designs
Experimental designs in which participants are
sampled from a limited population so as
to standardize participants background. Example
College Students.
Before-After Designs
Experimental designs in which participants are at
first asked to complete baseline measure and
then are assigned to a experimental condition.
Example Word Memory
Matched-Group Designs
Experimental designs in which participants are
measured on the variable of interest before the
experiment begins and then are assigned to
conditions on the basis of their scores on that
variable. Example IQ test.
Standardization of Conditions
Experimental manipulation in which all
participants in all levels of the independent
variable are treated in exactly the same way.
Examples Script, Protocol, and Automated
Experiments
5
Creation of Valid Manipulations
Impact The manipulation which creates the
hoped-for changes in the conceptual variables.
Experimental Realism The extent to which the
experimental Manipulation involves the
participants in the research.
Manipulation Checks
Measures that determine whether the experimental
manipulation has had the intended impact on the
conceptual variables of interest.
Confound Checks
Measures that determine whether the manipulation
has unwittingly Caused differences on confounding
variables.
Pilot Testing
6
Threats to Internal Validity
Placebo Effects
Participants expectations about experiments,
which influence the dependent measure
independently of the actual effect of
manipulation. ltgtGiving the same instruction to
participants in control and in experiment
conditions.
Demand Characteristics
Aspect of the research that allows participants
to guess the research hypothesis. ltgt Cover
stories, Unrelated-experiments technique, Non
reactive measures.
Experimenter Bias
Experimenters influence on the dependent measures
when they know the research hypotheses and which
condition the participants they are running are
in. ltgt Naïve Experimenters and Blind Experimenters