Experiments, Good and Bad

1
Chapter 5

• Experiments, Good and Bad

2
Thought Question 1
In conducting a study to relate two conditions
(activities, traits, etc.), researchers often
define one of them as the explanatory variable
and the other as the outcome or response
variable. In a study to determine whether
surgery or chemotherapy results in higher
survival rates for a certain type of cancer,
whether or not the patient survived is one
variable, and whether they received surgery or
chemotherapy is the other. Which is the
explanatory variable and which is the response
variable?
3
Thought Question 2
In an experiment, researchers assign treatments
to participants, whereas in an observational
study they simply observe what the participants
do naturally. Give an example of a situation
where an experiment would not be feasible for
ethical reasons.
4
Thought Question 3
When experimenters want to compare two
treatments, such as an old and a new drug, they
use randomization to assign the participants to
the two conditions. If you had 50 people
participate in such a study, how would you go
about randomizing them? Why do you think that
would be necessary, rather than having the
experimenter decide which people should get which
treatment?
5
Common Language

• Explanatory variable
• Response variable
• Subjects
• Treatments

6
Case Study

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Control patch
• Random assignment of treatments

7
Case Study

• Meditation and Aging
  (Noetic Sciences Review, Summer 1993, p. 28)
• Variables
• Explanatory Observed meditation practice
  (yes/no)
• Response Level of age-related enzyme
• Treatment not randomly assigned.

8
Randomized Experiment versus Observational Study

• Both typically have the goal of detecting a
  relationship between the explanatory and response
  variables.
• Experiment
• create differences in the explanatory variable
  and examine any resulting changes in the response
  variable
• Observational Study
• observe differences in the explanatory variable
  and notice any related differences in the
  response variable

9
Why Not Always Use a Randomized Experiment?

• Sometimes it is unethical or impossible to assign
  people to receive a specific treatment.
• Certain explanatory variables, such as handedness
  or gender, are inherent traits and cannot be
  randomly assigned.

10
Experiments Basic Principles

• Randomization
• to balance out extraneous variables across
  treatments
• Placebo
• to control for the power of suggestion
• Control group
• to understand changes not related to treatment

11
RandomizationCase Study

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Control patch
• Random assignment of treatments

12
PlaceboCase Study

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Placebo Control patch
• Random assignment of treatments

13
Control GroupCase Study

• Mozart, Relaxation and Performance on Spatial
  Tasks
  (Nature, Oct. 14, 1993, p. 611)
• Variables
• Explanatory Relaxation condition assignment
• Response Stanford-Binet IQ measure
• Active treatment Listening to Mozart
• Control groups
• Listening to relaxation tape to lower blood
  pressure
• Silence

14
Confounding (Lurking) Variables

• The problem
• in addition to the explanatory variable of
  interest, there may be other variables that make
  the groups being studied different from each
  other
• the impact of these variables cannot be separated
  from the impact of the explanatory variable on
  the response

15
Confounding (Lurking) Variables

• The solution
• Experiment randomize experimental units to
  receive different treatments (possible
  confounding variables should even out across
  groups)
• Observational Study measure potential
  confounding variables and determine if they have
  an impact on the response(may then adjust for
  these variables in the statistical analysis)

16
Confounding VariablesCase Study

• Heart or Hypothalamus?
  (Scientific American, May 1973, pp. 26-29)
• Infants were not randomized to either hear the
  heartbeat sound or not
• Same nursery was used on subsequent days with
  different groups of babies
• Environment variables
• construction noise
• temperature

17
Statistical Significance

• If an experiment or observational study finds a
  difference in two (or more) groups, is this
  difference really important?
• If the observed difference is larger than what
  would be expected just by chance, then it is
  labeled statistically significant.
• Rather than relying solely on the label of
  statistical significance, also look at the actual
  results to determine if they are practically
  important.

18
Key Concepts

• Critical evaluation of an experiment or
  observational study
• Common terms
• explanatory vs. response variables
• treatments, randomization
• Randomized experiments
• basic principles and terminology
• problem with confounding variables