Experiments and Observational Studies

1
Chapter 5

• Experiments and Observational Studies

2
Thought Questions
1, page 70
In conducting a study to relate two conditions
(activities, traits, etc.), researchers often
define one of them as the explanatory variable
and the other as the outcome or response
variable. In a study to determine whether
surgery or chemotherapy results in higher
survival rates for a certain type of cancer,
whether or not the patient survived is one
variable, and whether they received surgery or
chemotherapy is the other. Which is the
explanatory variable and which is the response
variable?
3
Thought Questions
2, page 70
In an experiment, researchers assign treatments
to participants, whereas in an observational
study they simply observe what the participants
do naturally. Give an example of a situation
where an experiment would not be feasible for
ethical reasons.
4
Thought Questions
3, page 70
Suppose you are interested in determining whether
or not a daily dose of vitamin C helps prevent
colds. You recruit 20 volunteers to participate
in an experiment. You want half of them to take
vitamin C and the other half to agree not to take
it. You ask them each which they would prefer,
and ten say they would like to take the vitamin
and the other ten say they would not. You ask
each of them to record how many colds they get
during the next ten weeks. At the end of that
time period, you compare the results reported
from the two groups. Give three reasons why this
is not a good experiment.
5
Thought Questions
4, page 70
When experimenters want to compare two
treatments, such as an old and a new drug, they
use randomization to assign the participants to
the two conditions. If you had 50 people
participate in such a study, how would you go
about randomizing them? Why do you think that
would be necessary, rather than having the
experimenter decide which people should get which
treatment?
6
Common Language

• Explanatory variable
• Response variable
• Treatments

7
Case Study 5.1

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Control patch
• Random assignment of treatments

8
Case Study 6.2

• Meditation and Aging
  (Noetic Sciences Review, Summer 1993, p. 28)
• Variables
• Explanatory Observed meditation practice
  (yes/no)
• Response Level of age-related enzyme
• Treatment not randomly assigned.

9
Randomized Experiment versus Observational Study

• Both typically have the goal of detecting a
  relationship between the explanatory and response
  variables.
• Experiment
• create differences in the explanatory variable
  and examine any resulting changes in the response
  variable
• Observational Study
• observe differences in the explanatory variable
  and notice any related differences in the
  response variable

10
Why Not Always Use a Randomized Experiment?

• Sometimes it is unethical or impossible to assign
  people to receive a specific treatment.
• Certain explanatory variables, such as handedness
  or gender, are inherent traits and cannot be
  randomly assigned.

11
Experiments Basic Principles

• Randomization
• to balance out extraneous variables across
  treatments
• Placebo
• to control for the power of suggestion
• Control group
• to understand changes not related to treatment
• Double-blind
• to control experimenter/respondent bias
• Pairing or blocking
• to reduce a source of variability in responses

12
RandomizationCase Study 5.1

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Control patch
• Random assignment of treatments

13
PlaceboCase Study 5.1

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Placebo Control patch
• Random assignment of treatments

14
Control GroupCase Study 6.1

• Mozart, Relaxation and Performance on Spatial
  Tasks
  (Nature, Oct. 14, 1993, p. 611)
• Variables
• Explanatory Relaxation condition assignment
• Response Stanford-Binet IQ measure
• Active treatment Listening to Mozart
• Control groups
• Listening to relaxation tape to lower blood
  pressure
• Silence

15
(not) Double-BlindedCase Study 6.1

• Mozart, Relaxation and Performance on Spatial
  Tasks
  (Nature, Oct. 14, 1993, p. 611)
• Variables
• Explanatory Relaxation condition assignment
• Response Stanford-Binet IQ measure
• Not double-blinded
• Participants know their treatment group
• Single-blinded
• Those measuring the IQ

16
Double-BlindedCase Study 5.1

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Double-blinded
• Participants dont know which patch they received
• Nor do those measuring smoking behavior

17
Pairing or BlockingCase Study 6.1

• Mozart, Relaxation and Performance on Spatial
  Tasks
  (Nature, Oct. 14, 1993, p. 611)
• Variables
• Explanatory Relaxation condition assignment
• Response Stanford-Binet IQ measure
• Blocking
• Participants practiced all three relaxation
  conditions. Each participant is a block.
• IQs re-measured after each relaxation period

18
Pairing or BlockingCase Study 5.1

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Pairing?
• Cannot block participants can only take one
  treatment
• Could use a matched-pair design

19
ExperimentsDifficulties and Disasters

• Extraneous variables
• Confounding variables
• Interacting variables
• Hawthorne, placebo and experimenter effects
• Ecological validity

20
Confounding Variables

• The problem
• may be inherent differences in the groups
  receiving the treatments that are inseparable
  from the treatments in their impact on the
  outcome.
• The solution
• randomize experimental units to receive different
  treatments.

21
Confounding VariablesCase Study 1.1

• Heart or Hypothalamus?
  (Scientific American, May 1973, pp. 26-29)
• Infants were not randomized to either hear the
  heartbeat sound or not
• Same nursery was used on subsequent days with
  different groups of babies
• Environment variables
• construction noise
• temperature

22
Interacting Variables

• The problem
• effect of explanatory variable on response
  variable may vary over levels of other variables.
• The solution
• measure and study potential interacting
  variables.
• does the relationship between explanatory and
  response variables change for different levels of
  these interacting variables?
• if so, report results for different groups
  defined by the levels of the interacting
  variables.

23
Interacting VariablesCase Study 5.1

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Researchers considered
• smoker at home
• found this to be an interacting variable

• other variables age, weight, depression
• no interactions found

24
Hawthorne, Placebo and Experimenter Effects

• The problem
• people may respond differently when they know
  they are part of an experiment.
• The solution
• Use placebos, control groups, and double-blind
  studies when possible.

25
Hawthorne, Placebo and Experimenter Effects
Case Study I

• 1920s Experiment by Hawthorne Works of the
  Western Electric Company
• What changes in working conditions improve
  productivity of workers?
• More lighting?
• Less lighting?
• Other changes?
• All changes improved productivity!

26
Hawthorne, Placebo and Experimenter Effects
Case Study II

• Experimenter Effects in Behavioral Research
  (Rosenthal, 1976, Irvington Pub., p. 410)
• Teachers given a list of student names
• told these were students who would show unusual
  academic development.
• IQ was measured at end of year
• First graders on list 15 points higher
• Second graders on list 9.5 points higher
• Older no striking difference
• Great expectations self-fulfilling prophecy

27
Ecological Validity

• The problem
• lack of generalizability due to
• unnatural settings
• willing group of volunteers as participants
• The solution
• Researchers should use natural settings with
  properly chosen sample.

28
Ecological ValidityCase Study 1.2

• Does Aspirin Prevent Heart Attacks? (NEJM, Jan.
  28, 1988, pp. 262-264)
• Participants were measured in their natural
  setting (at home)
• Only healthy male physicians were participants
• Results may not apply to
• male physical laborers
• women

29
Some Types of Observational Studies

• Retrospective
• Prospective
• Case-control

30
Observational Studies Difficulties and Disasters

• Confounding variables
• Can the results be extended to other groups?
• Recall remember the past

31
Confounding Variables

• The problem
• the impact of these variables cannot be separated
  from the impact of the explanatory variable on
  the response.
• The partial solution
• measure potential confounding variables
• determine if they have an impact on the response

32
Confounding VariablesCase Study

• Cellular Phones and Auto Accidents
  (New Scientist, Apr. 8, 1995, p. 11)
• Case control study
• cases 100 drivers who had accidents
• controls similar group who had no accidents
• Results
• 14 of cases had cellular phones
• 10.6 of controls had cellular phones
• the difference implies that having a phone in
  the carincreases the risk of an accident.
• observational study
• confounder how hectic were their lives?

33
Can Results be Extended?

• The problem
• many observational studies use convenience or
  volunteer samples.
• The partial solution
• recruit participants from a broad cross-section
  of the population of interest.

34
Can Results be Extended? Case Study 6.2

• Meditation and Aging
  (Noetic Sciences Review, Summer 1993, p. 28)
• Meditators were recruited from a university in
  Iowa which specializes in teaching meditation
• Non-meditators were recruited from a different
  setting in New York City

35
Remember the Past

• The problem
• subjects may be asked to recall events or
  details, thus it may be difficult to get accurate
  information.
• The partial solution
• for a retrospective study, use official records,
  if they exist

36
Remember the Past Case Study

• Personal Recall and the Limits of Retrospective
  Questions in Surveys
  (Pearson, Ross, Dawes in Questions about
  Questions, Tanur, 1992, Russell Sage Foundation,
  pp. 65-94)
• Chronic headache sufferers were asked to rate
  their level of pain (diaries were kept for a
  week)
• current pain
• maximum and minimum pain during past week
• Those with high current pain overstated prior
  levels, those with low current pain understated
  prior levels

37
Key Concepts

• Critical evaluation of an experiment or
  observational study
• Common terms
• explanatory vs. response variables
• treatments, randomization
• Randomized experiment
• basic principles and terminology
• possible complications
• Observational studies
• different types
• possible complications