Experimental Design

1
Chapter 2

• Experimental Design

2
Definitions

• 1) Observational study - observe outcomes without
  imposing any treatment

2) Experiment - actively impose some treatment in
order to observe the response
3
Ive developed a new rabbit food, Hippity Hop.
Makes fur soft shiny!
Increases energy!
100 of daily vitamins essential oils!
4
Can I just make these claims?
NO
What must I do to make these claims?
Do an experiment
Who (what) should I test this on?
Rabbits
What do I test?
The type of food
5
3)Experimental unit the single individual
(person, animal, plant, etc.) to which the
different treatments are assigned
4) Factor is the explanatory variable
5) Level a specific value for the factor
6

• 6) Response variable what you measure

7) Treatment a specific experimental condition
applied to the units
7
I plan to test my new rabbit food. What are my
experimental units? What is my factor? What is
the response variable?
Rabbits
Type of food
How well they grow
8
Ill use my pet rabbit, Lucky!
Since Luckys coat is shinier he has more
energy, then Hippity Hop is a better rabbit food!
9
8) Control group a group that is used to
compare the factor against can be a placebo or
the old or current item
9) Placebo a dummy treatment that can have no
physical effect
10
Now Ill use Lucky my friends rabbit, Flash.
Lucky gets Hippity Hop food Flash gets the old
rabbit food.
WOW! Lucky is bigger shinier so Hippity Hop is
better!
11
The first five rabbits that I catch will get
Hippity Hop food and the remaining five will get
the old food.
The Hippity Hop rabbits have scored higher so
its the better food!
12
Number the rabbits from 1 10.
Place the numbers in a hat.
The first five numbers pulled from the hat will
be the rabbits that get Hippity Hop food.
The remaining rabbits get the old food.
8
5
3
7
9
I evaluated the rabbits found that the rabbits
eating Hippity Hop are better than the old food!
13

• 10) blinding - method used so that units do not
  know which treatment they are getting

11) double blind - neither the units nor the
evaluator know which treatment a subject received
14
Hippity Hop Rabbit Food makes fur soft and shiny,
increases energy for ALL types of rabbits!
Can I make this claim?
15
Principles of Experimental Design

• Control of effects of extraneous variables on the
  response by comparing treatment groups to a
  control group (placebo or old)
• Replication of the experiment on many subjects to
  quantify the natural variation in the experiment
• Randomization the use of chance to assign
  subjects to treatments

16
The ONLY way to show cause effect is with a
well-designed, well-controlled experiment!
The ONLY way to show cause effect is with a
well-designed, well-controlled experiment!!
The ONLY way to show cause effect is with a
well-designed, well-controlled experiment!!!
17
Example 1 A farm-product manufacturer wants to
determine if the yield of a crop is different
when the soil is treated with three different
types of fertilizers. Fifteen similar plots of
land are planted with the same type of seed but
are fertilized differently. At the end of the
growing season, the mean yield from the sample
plots is compared.
Experimental units? Factors? Levels? Response
variable? How many treatments?
Plots of land
Type of fertilizer
Fertilizer types A, B, C
Yield of crop
3
18
Example 2 A consumer group wants to test cake
pans to see which works the best (bakes evenly).
It will test aluminum, glass, and plastic pans in
both gas and electric ovens. Experiment
units? Factors? Levels? Response variable? Number
of treatments?
Cake batter
Two factors - type of pan type of oven
Type of pan has 3 levels (aluminum, glass,
plastic type of oven has 2 levels (electric
gas)
How evenly the cake bakes
6
19
Example 3 A farm-product manufacturer wants to
determine if the yield of a crop is different
when the soil is treated with three different
types of fertilizers. Fifteen similar plots of
land are planted with the same type of seed but
are fertilized differently. At the end of the
growing season, the mean yield from the sample
plots is compared. Why is the same type of seed
used on all 15 plots? What are other potential
extraneous variables? Does this experiment have a
placebo? Explain
It is part of the controls in the experiment.
Type of soil, amount of water, etc.
NO a placebo is not needed in this experiment
20
Experiment Designs

• Completely randomized all experimental units
  are allocated at random among all treatments

Random assignment
21
Treatment B
Treatment A
Treatment C
Treatment D
Randomly assign experimental units to treatments
Completely randomized design
22

• Randomized block units are blocked into groups
  and then randomly assigned to treatments

Random assignment
23
Treatment B
Treatment A
Treatment A
Treatment B
Put into homogeneous groups
Randomly assign experimental units to treatments
Randomized block design
24

• Matched pairs - a special type of block design

• match up experimental units according to similar
  characteristics randomly assign on to one
  treatment the other automatically gets the 2nd
  treatment
• have each unit do both treatments in random order
• the assignment of treatments is dependent

25
Treatment B
Treatment A
Pair experimental units according to specific
characteristics.
Next, randomly assign one unit from a pair to
Treatment A. The other unit gets Treatment B.
This is one way to do a matched pairs design
another way is to have the individual unit do
both treatments (as in a taste test).
26

• 12) Confounding variable the effect of the
  confounding variable on the response cannot be
  separated from the effects of the explanatory
  variable (factor)

27
Suppose we wish to test a new deodorant against
one currently on the market.
28
Treatment B
Treatment A
One group is assigned to treatment A the other
group to treatment B.
Treatment A
Treatment B
Confounding does NOT occur in a completely
randomized design!
Treatment group are confounded
29
Example 4 An article from USA Today reports the
number of victims of violent crimes per 1000
people. 51 victims have never been married, 42
are divorced or separated, 13 are married, and 8
are widowed. Is this an experiment? Why or why
not? What is a potential confounding variable?
No, no treatment was imposed on people.
Age younger people are more at risk to be
victims of violent crimes
30
Example 5 Four new word-processing programs are
to be compared by measuring the speed with which
standard tasks can be completed. One hundred
volunteers are randomly assigned to one of the
four programs and their speeds are measured. Is
this an experiment? Why or why not?
Yes, a treatment is imposed.
What type of design is this? Factors?
Levels? Response variable?
Completely randomized
one factor word-processing program with 4 levels
speed
31
Example 5 Four new word-processing programs are
to be compared by measuring the speed with which
standard tasks can be completed. One hundred
volunteers are randomly designed to one of the
four programs and their speeds are measured. Is
there a potential confounding variable?
Can this design be improved? Explain.
NO, completely randomized designs have no
confounding
32
Example 6 Suppose that the manufacturer wants to
test a new fertilizer against the current one on
the market. Ten 2-acre plots of land scattered
throughout the county are used. Each plot is
subdivided into two subplots, one of which is
treated with the current fertilizer, and the
other with the new fertilizer. Wheat is planted
and the crop yields are measured.
What type of design is this? Why use this
method? When does randomization occur?
Matched - pairs design
Randomly assigned treatment to first acre of each
two-acre plot
33
Randomization reduces bias by spreading any
uncontrolled confounding variables evenly
throughout the treatment groups.
Is there another way to reduce variability?
Blocking also helps reduce variability.
Variability is controlled by sample size. Larger
samples produce statistics with less variability.
34
High bias low variability
High bias high variability
Low bias low variability
Low bias high variability