Experiments, Good and Bad

1
Chapter 5

• Experiments, Good and Bad

2
Thought Question 1
In studies to determine the relationship between
two conditions (activities, traits, etc.), one of
them is often defined as the explanatory
(independent) variable and the other as the
outcome or response (dependent) variable. In an
experiment to determine whether the drug
memantine improves cognition of patients with
moderate to severe Alzheimers disease, whether
or not the patient received memantine is one
variable, and cognitive score is the other.
Which is the explanatory variable and which is
the response variable?
3
Thought Question 2
In an observational study, researchers observe
what individuals do (or have done) naturally,
while in an experiment, they randomly assign the
individuals to groups to receive one of several
treatments. Give an example of a situation
where an experiment would not be feasible and
thus an observational study would be needed.
4
Thought Question 3
In testing the effect of memantine on the
cognition of Alzheimers disease patients (from
TQ 1), how would you go about randomizing 100
patients to the two treatment groups (memantine
group placebo group)? Why is it necessary to
randomly assign the subjects, rather than having
the experimenter decide which patients should get
which treatment?
5
Common Language

• Response variable
• what is measured as the outcome or result of a
  study
• Explanatory variable
• what we think explains or causes changes in the
  response variable
• often determines how subjects are split into
  groups
• Subjects
• the individuals that are participating in a study
• Treatments
• specific experimental conditions (related to the
  explanatory variable) applied to the subjects

6
Case Study

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Control patch
• Random assignment of treatments

7
Case Study

• Meditation and Aging
  (Noetic Sciences Review, Summer 1993, p. 28)
• Variables
• Explanatory Observed meditation practice
  (yes/no)
• Response Level of age-related enzyme
• Treatment not randomly assigned.

8
Randomized Experiment versus Observational Study

• Both typically have the goal of detecting a
  relationship between the explanatory and response
  variables.
• Experiment
• create differences in the explanatory variable
  and examine any resulting changes in the response
  variable
• Observational Study
• observe differences in the explanatory variable
  and notice any related differences in the
  response variable

9
Why Not Always Use a Randomized Experiment?

• Sometimes it is unethical or impossible to assign
  people to receive a specific treatment.
• Certain explanatory variables, such as handedness
  or gender, are inherent traits and cannot be
  randomly assigned.

10
Experiments Basic Principles

• Randomization
• to balance out extraneous variables across
  treatments
• Placebo
• to control for the power of suggestion
• Control group
• to understand changes not related to the
  treatments

11
RandomizationCase Study

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Control patch
• Random assignment of treatments

12
PlaceboCase Study

• Quitting Smoking with Nicotine Patches (JAMA,
  Feb. 23, 1994, pp. 595-600)
• Variables
• Explanatory Treatment assignment
• Response Cessation of smoking (yes/no)
• Treatments
• Nicotine patch
• Placebo Control patch
• Random assignment of treatments

13
Control GroupCase Study

• Mozart, Relaxation and Performance on Spatial
  Tasks
  (Nature, Oct. 14, 1993, p. 611)
• Variables
• Explanatory Relaxation condition assignment
• Response Stanford-Binet IQ measure
• Active treatment Listening to Mozart
• Control groups
• Listening to relaxation tape to lower blood
  pressure
• Silence

14
Confounding (Lurking) Variables

• The problem
• in addition to the explanatory variable of
  interest, there may be other variables that make
  the groups being studied different from each
  other
• the impact of these variables cannot be separated
  from the impact of the explanatory variable on
  the response

15
Confounding (Lurking) Variables

• The solution
• Experiment randomize experimental units to
  receive different treatments (possible
  confounding variables should even out across
  groups)
• Observational Study measure potential
  confounding variables and determine if they have
  an impact on the response(may then adjust for
  these variables in the statistical analysis)

16
Confounding VariablesCase Study

• Heart or Hypothalamus?
  (Scientific American, May 1973, pp. 26-29)
• Infants were not randomized to either hear the
  heartbeat sound or not
• Same nursery was used on subsequent days with
  different groups of babies
• Environment variables
• construction noise
• temperature

17
Statistical Significance

• If an experiment or observational study finds a
  difference in two (or more) groups, is this
  difference really important?
• If the observed difference is larger than what
  would be expected just by chance, then it is
  labeled statistically significant.
• Rather than relying solely on the label of
  statistical significance, also look at the actual
  results to determine if they are practically
  important.

18
Key Concepts

• Critical evaluation of an experiment or
  observational study
• Common terms
• explanatory vs. response variables
• treatments, randomization
• Randomized experiments
• basic principles and terminology
• problem with confounding variables