Title: CHAPTER 4 Designing Studies
1CHAPTER 4Designing Studies
2Experiments
- DISTINGUISH between an observational study and an
experiment. - EXPLAIN the concept of confounding.
- IDENTIFY the experimental units, explanatory and
response variables, and treatments in an
experiment. - EXPLAIN the purpose of comparison, random
assignment, control, and replication in an
experiment. - DESCRIBE a completely randomized design for an
experiment. - DESCRIBE the placebo effect and the purpose of
blinding in an experiment. - INTERPRET the meaning of statistically
significant in the context of an experiment. - EXPLAIN the purpose of blocking in an experiment.
DESCRIBE a randomized block design or a matched
pairs design for an experiment.
3Observational Study vs. Experiment
An observational study observes individuals and
measures variables of interest but does not
attempt to influence the responses. An
experiment deliberately imposes some treatment on
individuals to measure their responses.
When our goal is to understand cause and effect,
experiments are the only source of fully
convincing data. The distinction between
observational study and experiment is one of the
most important in statistics.
4 5Observational Study vs. Experiment
- Observational studies of the effect of an
explanatory variable on a response variable often
fail because of confounding between the
explanatory variable and one or more other
variables. - Well-designed experiments take steps to prevent
confounding.
Confounding occurs when two variables are
associated in such a way that their effects on a
response variable cannot be distinguished from
each other.
To explain confounding you should explain how the
variable you chose is associated with the
explanatory variable and also how it affects the
response variable.
6- AP Exam Tip on p.236
- Diet is a confounding variable because people
with bad diets are more likely to have a heart
attack - Need to explain why diet is also associated with
the choice to take hormones. - Would need to say something about why a better
diet might be more common in the group of women
who choose to take hormones. - AP common error on p.236
- For the hormone example you would need to explain
not only that wealthier people tend to go to the
doctor more often and to be healthier overall,
but also that wealthier people are more likely to
get hormone replacement. Thus, it could actually
be the overall healthiness of wealthy women that
caused the reduction in heart attacks, not any
hormones they may have taken. - CYU on p.237
7The Language of Experiments
- An experiment is a statistical study in which we
actually do something (a treatment) to people,
animals, or objects (the experimental units) to
observe the response. Here is the basic
vocabulary of experiments.
A specific condition applied to the individuals
in an experiment is called a treatment. If an
experiment has several explanatory variables, a
treatment is a combination of specific values of
these variables. The experimental units are the
smallest collection of individuals to which
treatments are applied. When the units are human
beings, they often are called subjects.
Often times the explanatory variables in an
experiment are called factors. If an
experiment has several factors, the combinations
of each level of each factor form the treatments.
8- See example on p.239
- http//www.ted.com/talks/michael_norton_how_to_buy
_happinesst-107021 - Describes a multifactor experiment to determine
if money can buy hapiness. - There are 2 factors, amount of money how it is
spent.
9How to Experiment Badly
- Many laboratory experiments use a design like the
one in the online SAT course example on p.240
Experimental Units
Treatment
Measure Response
In the lab environment, simple designs often work
well. Field experiments and experiments with
animals or people deal with more variable
conditions. Outside the lab, badly designed
experiments often yield worthless results because
of confounding.
10How to Experiment Well
- The remedy for confounding is to perform a
comparative experiment in which some units
receive one treatment and similar units receive
another. Most well designed experiments compare
two or more treatments. - Comparison alone isnt enough, if the treatments
are given to groups that differ greatly, bias
will result. The solution to the problem of bias
is random assignment.
In an experiment, random assignment means that
experimental units are assigned to treatments
using a chance process.
See example on p.241
11Principles of Experimental Design
Principles of Experimental Design
The basic principles for designing experiments
are as follows 1. Comparison. Use a design that
compares two or more treatments. 2. Random
assignment. Use chance to assign experimental
units to treatments. Doing so helps create
roughly equivalent groups of experimental units
by balancing the effects of other variables among
the treatment groups. 3. Control. Keep other
variables that might affect the response the same
for all groups. 4. Replication. Use enough
experimental units in each group so that any
differences in the effects of the treatments can
be distinguished from chance differences between
the groups.
12 13Completely Randomized Design
In a completely randomized design, the treatments
are assigned to all the experimental units
completely by chance. Some experiments may
include a control group that receives an inactive
treatment or an existing baseline treatment.
Experimental Units
14- See example on p.245
- Read exam tip on p.246
- CYU on p.247
15Experiments What Can Go Wrong?
The logic of a randomized comparative experiment
depends on our ability to treat all the subjects
the same in every way except for the actual
treatments being compared. Good experiments,
therefore, require careful attention to details
to ensure that all subjects really are treated
identically.
The response to a dummy treatment is called the
placebo effect. In a double-blind experiment,
neither the subjects nor those who interact with
them and measure the response variable know which
treatment a subject received.
16- See example on p.247
- http//www.cbsnews.com/videos/treating-depression-
is-there-a-placebo-effect/
17Inference for Experiments
In an experiment, researchers usually hope to see
a difference in the responses so large that it is
unlikely to happen just because of chance
variation. We can use the laws of probability,
which describe chance behavior, to learn whether
the treatment effects are larger than we would
expect to see if only chance were operating. If
they are, we call them statistically significant.
An observed effect so large that it would rarely
occur by chance is called statistically
significant. A statistically significant
association in data from a well-designed
experiment does imply causation.
18Blocking
When a population consists of groups of
individuals that are similar within but
different between, a stratified random sample
gives a better estimate than a simple random
sample. This same logic applies in experiments.
A block is a group of experimental units that are
known before the experiment to be similar in some
way that is expected to affect the response to
the treatments. In a randomized block design, the
random assignment of experimental units to
treatments is carried out separately within each
block.
19Matched Pairs Design
A common type of randomized block design for
comparing two treatments is a matched pairs
design. The idea is to create blocks by matching
pairs of similar experimental units.
A matched pairs design is a randomized blocked
experiment in which each block consists of a
matching pair of similar experimental units.
Chance is used to determine which unit in each
pair gets each treatment. Sometimes, a pair in
a matched-pairs design consists of a single unit
that receives both treatments. Since the order of
the treatments can influence the response, chance
is used to determine with treatment is applied
first for each unit.
20Experiments
- DISTINGUISH between an observational study and an
experiment. - EXPLAIN the concept of confounding.
- IDENTIFY the experimental units, explanatory and
response variables, and treatments in an
experiment. - EXPLAIN the purpose of comparison, random
assignment, control, and replication in an
experiment. - DESCRIBE a completely randomized design for an
experiment. - DESCRIBE the placebo effect and the purpose of
blinding in an experiment. - INTERPRET the meaning of statistically
significant in the context of an experiment. - EXPLAIN the purpose of blocking in an experiment.
DESCRIBE a randomized block design or a matched
pairs design for an experiment.