Experiments and Observational Studies

1
Chapter 13

• Experiments and Observational Studies

2
Review

• Population Parameter
• This is a numerical summary of a population
• Proportion, mean, standard deviation
• We dont know this number and usually never will.
• Sample Statistic
• This is a numerical summary computed from a
  subset of the population(a sample).
• sample proportion, sample mean, sample standard
  deviation
• We know this number because we can compute it. We
  use this number to try and get an idea of what
  the population parameter(which we dont know)
  might be.

3
Observational Study

• Observing data in the wild.
• Researchers don't assign treatments to subjects
  they simply observe them.
• Results show association, but do not prove
  causation.
• Also called an Investigative Study.
• Sample surveys are observational studies.

4
Observational Studies

• Two types
• Retrospective Study
• Subjects are selected
• Previous behaviors or conditions are determined
• Prospective Study
• Subjects are selected
• Subjects are followed to observe future outcomes

5
Experiments

• A way to prove a cause and effect relationship
  between two or more variables
• Manipulate explanatory variable
• Observe response
• In an experiment you are applying a treatment to
  the subjects in an observational study you just
  observe them

6
Experiment Terminology

• Experimental Units individuals on whom the
  experiment is performed
• Factors explanatory variable(s)
• Need at least one factor for every experiment

7
Experiment Terminology

• Levels specific values that the experimenter
  chooses for a factor
• Need at least two levels for each factor
• Treatments different levels of a single factor
  or combinations of levels of two or more factors

8
Example OptiGro Fertilizer

• An ad for OptiGro plant fertilizer claims that
  with this product we will grow juicier, tastier
  tomatoes. We'd like to test this claim and see
  if we can get by with a half-dose.
• What are the experimental units in this example?
• Tomato plants (the thing we apply the treatment
  to)
• What is the response we are interested in?
• Juiciness and Tastiness of the tomatoes

9
Example OptiGro Fertilizer

• What is the factor?
• Fertilizer
• What are the levels of this factor?
• No Fertilizer (to see if the fertilizer even
  works)
• Full-Dose
• Half-Dose (want to see if the half dose is as
  good as the full)
• What are the treatments?
• No Fertilizer, Full-Dose, Half-Dose

10
Example OptiGro Fertilizer

• The makers of OptiGro want to ensure that their
  product will work under a wide variety of
  watering conditions. Let's include this as a
  second factor specifically lets look at daily
  watering and watering every other day.
• How many factors are there now?
• 2
• What are the factor(s)?
• Fertilizer and water
• What are factor levels
• Fertilizer
• no fertilizer, half dose, full dose
• Water
• every day, every other day

11
Example OptiGro Fertilizer

• What are the treatments in the new experiment?
• Half-Dose of fertilizer, daily watering
• Full-Dose, daily watering
• No fertilizer, daily watering
• Half-Dose, water every other day
• Full-Dose, water every other day
• No fertilizer, water every other day

12
Experiment Terminology

• Control Group experimental units assigned to a
  baseline treatment level
• Provides basis for comparison
• Baseline either a gold standard or a placebo
• Placebo a null treatment known to have no
  effect
• Placebo Effect tendency of many human subjects
  to show response even when administered a placebo

13
Experiment Terminology

• Blinding - the researcher disguises treatments
• Single Blind - either the subject or the
  evaluator is unaware of the treatment
• Double Blind - both the subject AND the evaluator
  are in the dark
• Reduces personal bias

14
Experiment Terminology

• Confounding - A condition where the effects of
  two variables on the response cant be
  distinguished from each other
• Lurking Variable - A variable that effects the
  relationship between the response variable and
  the explanatory variable and is not included
  among the variables in a study.

15
Principles of Experimental Design

• Control
• Make all conditions as similar as possible for
  all treatment groups
• The only difference between groups should be the
  treatments
• Controlling outside influences reduces
  variation in the responses, making it easier to
  detect differences among the groups due to the
  treatments

16
Principles of Experimental Design

• Randomization
• Experimental units should be randomly assigned to
  treatment groups
• Order of the trials should also be randomized
• Some uncontrolled sources of variation will be
  controlled through randomization
• Experiments without randomization may have biased
  results

17
Principles of Experimental Design

• Replication
• In an experiment, each treatment is given to
  several different experimental units
• Entire experiment should be repeated on a
  different group of experimental units

18
Blocking

• This can be considered another principle of
  experimental design, but not required in an
  experimental design
• Group similar individuals together and then
  randomize within each of these blocks
• Reduces the effects of identifiable attributes of
  the subjects that cannot be controlled

19
Example

• The leg muscles of men aged 60 to 75 were 50 to
  80 stronger after they participated in a
  16-week, high-intensity resistance-training
  program twice a week.
• Experiment
• Men aged 60 to 75 (experimental units)
• Exercise (1 level) (factors)
• 1 treatment 16-week resistance-training
• Strength levels pre- and post-exercise program
  (response)
• Not blinded
• Applies only to men 60 to 75 who participate in
  similar exercise programs. (Nature and scope of
  conclusion)
• Randomization?

20
Example

• In 2001 a report in the Journal of the American
  Cancer Institute indicated that women who work
  nights have a 60 chance of developing breast
  cancer. Researchers based these findings on the
  work histories of 763 women with breast cancer
  and 741 women without the disease.
• Observational Study
• Retrospective
• Women unknown selection process with in
  formation taken from work histories
• (subjects studied and how they were selected)
• Risk of breast cancer
• parameter of interest
• Observational study, no way to know that working
  nights causes breast cancer.
• Nature and scope of the conclusion.

21
Example

• Some gardeners prefer to use non-chemical methods
  to control insect pests in their gardens. Two
  kinds of traps have been designed and Researchers
  want to know which one is more effective. They
  randomly choose 10 locations in a large garden
  and place one of each kind of trap at each
  location. After a week they count the number of
  bugs in each trap.
• Experiment.
• Locations in a garden. (experimental units)
• 1 factor traps(2 levels). (factors)
• 2 treatments. ( of treatments)
• Number of bugs in trap. (response variable
  measured)
• Blocked by locations. (type of design)
• Not blind.
• One type of trap is more effective than the other
  (Nature and scope of conclusion)

22
Diagrams

• One way to portray an experiment is through a
  diagram.
• UNITS ? TREATMENT ? RESPONSE

Group 1
Treatment 1
Random Assignment
Compare response
Group 2
Treatment 2
23
Example Shingles

• A research doctor believes a new ointment will be
  more effective than the current medication in
  treating shingles (a painful skin rash). Eight
  patients have volunteered for our study. We will
  design an experiment to help the doctor verify
  her claim.

24
Example Shingles

• Experimental Unit Person (we have 8)
• Response Severity of shingles
• Factor ointment
• Levels current ointment and new ointment
• Why do we not want a placebo here?
• Treatments current ointment and new ointment
• Treatments are the same as levels here because we
  only have one factor

25
Example - Shingles

• Control
• Ointment taken every day for both groups
• All other care is similar
• Replication
• 4 patients in each treatment group
• Randomization
• Randomly assign 4 patients to each treatment
  group
• How do we do this?

26
Example - Shingles

• Patients Alex, Bethany, Carl, Denise, Evelyn,
  Fred, George, Hannah
• Number the patients 1 through 8
• Use random numbers to assign treatments 41098
  18329 78458 31685 55259
• First 4 patients we select get new treatment, the
  rest get the current treatment

27
Example - Shingles

• By ones4 1 0 9 8 1 8 3 2 9 7 8 4 5 8 3 1 6 8 5
  5 5 2 5 9
• Throw out 0 and 9 (they dont correspond to
  patients)
• Throw out repeats
• New ointment 4, 1, 8, 3
• Denise, Hannah, Alex, Carl
• Current ointment 2, 5, 6, 7 (everyone else)
• Bethany, Evelyn, Fred, George

28
Example - Shingles
Group 1 4 patients
Treatment 1 New Ointment
Randomly Assign the 8 patients to groups
Compare severity of shingles
Group 2 4 patients
Treatment 2 Current Ointment
29
Example - Shingles

• Could we make this experiment double-blind?
• Double-blind means that neither the patient nor
  the person evaluating the results knows who is
  receiving each treatment. To make this
  double-blind we would have to assume that the two
  ointments look alike same color, unmarked tubes,
  similar texture, similar odors, etc.
• If one ointment had a distinctive odor, it would
  probably not be possible to make the experiment
  double-blind.

30
Example Blocking

• Cancer treatment (3 treatments)
• We suspect that men and women respond differently
  so divide subjects into men and women
• Randomly assign each block to three groups
• The three groups in each block receive only one
  treatment
• Compare survival rates

31
Examples Blocking
Group 1
Treatment 1
Compare survival rates
Random Assign.
Men
Group 2
Treatment 2
Group 3
Treatment 3
Subjects
Group 1
Treatment 1
Compare survival rates
Random Assign.
Women
Group 2
Treatment 2
Group 3
Treatment 3
32
Statistical Significance

• If the differences between the treatment groups
  are big enough, we will attribute the differences
  to the treatments
• How big is big enough?
• Answer more precisely later
• Looking for differences large enough to be due to
  something other than random variation