Types of Studies and Study Design

1
Types of Studies and Study Design
2
Research classifications

• Observational vs. Experimental
• Observational researcher collects info on
  attributes or measurements of interest, but does
  not influence results.
• Experimental researcher deliberately
  influences events and investigates the effects of
  the intervention, e.g. clinical trials and
  laboratory experiments.

We often use these when we are interested in
studying the effect of a treatment on individuals
or experimental units.
3
Experiments Observational Studies

• We conduct an experiment when it is (ethically,
  physically etc) possible for the experimenter to
  determine which experimental units receive which
  treatment.

4
Experiments Observational Studies

• Experiment Terminology
• Experimental Unit Treatment
  Response

patient drug cholesterol patient
pre-surgery antibiotic
infection mouse radiation mortality
5
Experiments Observational Studies

• In an observational study, we compare the units
  that happen to have received each of the
  treatments.

6
Experiments Observational Studies
Observational Study

• e.g. You cannot set up a control (non-smoking)
  group and treatment (smoking) group.

7
Experiments Observational Studies

• Note
• Only a well-designed and well-executed
  experiment can reliably establish causation.
• An observational study is useful for identifying
  possible causes of effects, but it cannot
  reliably establish causation.

8
1. Completely Randomized Design

• The treatments are allocated entirely by chance
  to the experimental units.

9
1. Completely Randomized Design

• Example
• Which of two varieties of tomatoes (A B) yield
  a greater quantity of market quality fruit?
• Factors that may affect yield
• different soil fertility levels
• exposure to wind/sun
• soil pH levels
• soil water content etc.

10
1. Completely Randomized Design

• Divide the field into plots and randomly
  allocate the tomato varieties (treatments) to
  each plot (unit).
• 8 plots 4 get variety A

UPHILL
(A)
(B)
(A)
(A)
(A)
(A)
(B)
(B)
(B)
(A)
Randomly assign A B varieties in each strip of
similar elevation.
11
1. Completely Randomized Design

• Note
• Randomization is an attempt to make the
  treatment groups as similar as possible we can
  only expect to achieve this when there is a large
  number of experimental units to choose from.

12
2. Blocking

• Group (block) experimental units by some known
  factor and then randomize within each block in an
  attempt to balance out the unknown factors.
• Use
• blocking for known factors (e.g. slope of field
  in previous example)
• and
• randomization for unknown factors to try to
  balance things out.

13
2. Blocking

• Example 2 Multi-Center Clinical Trial
• Suppose a Mayo clinical trial comparing two
  chemotherapy regimens in treatment of patients
  with colon cancer will be conducted using cancer
  patients in Scottsdale, AZ and Rochester, MN.

14
2. Blocking
Scottsdale Rochester
1 (B)
2 (A)
4 (B)
3 (A)
6 (A)
5 (A)
8 (B)
7 (B)

• How should we allocate treatments to the 12
  patients?

Randomly assign treatments to 4 the patients from
Scottsdale and then to the 8 Rochester patients.
15
2. Blocking

• Example 3 Comparing Three Pain Relievers for
  Headache Sufferers
• How could blocking be used to increase precision
  of a designed experiment to control to compare
  the pain relievers?
• What are some other design issues?

16
Example 4 Comparing 17 Different Leg Wraps on
Used on Race Horses

• 17 boots tested, each boot is tested n 5
  times. Why?
• Because of the time constraints all boots were
  not tested on the same day.
• 8 tested 1st day, 5 tested 2nd day, 4 tested 3rd
  day.
• Leg was placed in freezer and thawed before the
  2nd and 3rd days of testing.

17
Horse Leg Wraps (contd)

• What problems do you foresee with this
  experimental design? Discussion Question 1
• What actually happened?

What are the implications of these results?
Discussion Question 2
18
Horse Leg Wraps (contd)
FINAL BOOT COMPARISONS
19
Horse Legs Wraps (contd)

• What should have been done?
• Discussion Question 3

20
3. People as Experimental Units

• Example Cholesterol Drug Study Suppose we
  wish to determine whether a drug will help lower
  the cholesterol level of patients who take it.
• How should we design our study?
• Discussion Question 4

21
Polio Vaccine Example
22
Polio Vaccine Example
Dr. Jonas Salk, vaccine pioneer 1914-95
Iron Lung
23
The Salk Vaccine Field Trial

• 1954 Public Health Service organized an
  experiment to test the effectiveness of Salks
  vaccine.
• Need for experiment
• Polio, an epidemic disease with cases varying
  considerably from year to year. A drop in polio
  after vaccination could mean either
• Vaccine effective
• No epidemic that year

24
The Salk Vaccine Field Trial

• Subjects 2 million, Grades 1, 2, and 3
• 500,000 were vaccinated
• (Treatment Group)
• 1 million deliberately not vaccinated
• (Control Group)
• 500,000 not vaccinated - parental permission
  denied

25
The Salk Vaccine Field Trial

• NFIP Design
• Treatment Group Grade 2
• Control Group Grades 1 and 3 No Permission
• Flaws ?
• Polio contagious, spreading through contact.
  i.e. incidence could be greater in Grade 2 (bias
  against vaccine), or vice-versa.
• Control group included children without parental
  permission (usually children from lower income
  families) whereas Treatment group could not (bias
  against the vaccine).

26
The Salk Vaccine Field Trial

• Double-Blinded Randomized Controlled
  Experimental Design
• Control group only chosen from those with
  parental permission for vaccination
• Random assignment to treatment or control group
• Use of placebo (control group given injection of
  salted water)
• Diagnosticians not told which group the subject
  came from (polio can be difficult to diagnose)
• i.e., a double-blind randomized controlled
  experiment

27
The Salk Vaccine Field Trial

• The double-blind randomized controlled
  experiment (and NFIP) results

28
3. People as Experimental Units

• control group
• Receive no treatment or an existing treatment
• blinding
• Subjects dont know which treatment they receive
• double blind
• Subjects and administers / diagnosticians are
  blinded
• placebo
• Inert dummy treatment

29
3. People as Experimental Units

• placebo effect
• A common response in humans when they believe
  they have been treated.
• Approximately 35 of people respond positively to
  dummy treatments - the placebo effect

30
Observational Studies

• There are two major types of observational
  studies

prospective
and retrospective studies
31
Observational Studies

• 1. Prospective Studies
• (looking forward)

• Choose samples now, measure variables and follow
  up in the future.
• E.g., choose a group of smokers and non-smokers
  now and observe their health in the future.

32
Observational Studies

• 2. Retrospective Studies
• (looking back)

• Looks back at the past.
• E.g., a case-control study
• Separate samples for cases and controls
  (non-cases).
• Look back into the past and compare histories.
• E.g. choose two groups lung cancer patients and
  non-lung cancer patients. Compare their smoking
  histories.

33
Observational Studies

• Important Note
• 1. Observational studies should use some form of
  random sampling to obtain representative samples.
• Observational studies cannot reliably establish
  causation.

34
Controlling for various factors

• A prospective study was carried out over 11 years
  on a group of smokers and non-smokers showed that
  there were 7 lung cancer deaths per 100,000 in
  the non-smoker sample, but 166 lung cancer deaths
  per 100,000 in the smoker sample.
• This still does not show smoking causes lung
  cancer because it could be that smokers smoke
  because of stress and that this stress causes
  lung cancer.

35
Controlling for various factors

• To control for this factor we might divide our
  samples into different stress categories. We
  then compare smokers and non-smokers who are in
  the same stress category.
• This is called controlling for a confounding
  factor.

36
Example 1

• Home births give babies a good chance NZ
  Herald, 1990
• An Australian report was stated to have said that
  babies are twice as likely to die during or soon
  after a hospital delivery than those from a home
  birth.
• The report was based upon simple random samples
  of home births and hospital births.
• Q Does this mean hospitals are dangerous places
  to have babies in Australia? Why or why not?
  Discussion Question 5

37
Example 2

• Lead Exposure Linked to Bad Teeth in Children
  USA Today
• The study involved 24,901 children ages 2 and
  older. It showed that the greater the childs
  exposure to lead, the more decayed or missing
  teeth.
• Q Does this show lead exposure causes tooth
  decay in children? Why or why not?
• Discussion Question 6

38
Example 2 contd

• Lead Exposure Linked to Bad Teeth in Children
  USA Today
• Researcher
• We controlled for income level, the proportion
  of diet due to carbohydrates, calcium in the diet
  and the number of days since the last dental
  visit.

39
Limitations on Scope of Inference
40
Discussion Question 7 Determine Whether Age at
1st Pregnancy is a Risk Factor for Cervical Cancer
How might we proceed?
41
Discussion Question 8 Determine what job
related factors Mayo nurses are most dissatisfied
with.

• How might we proceed?

42
Discussion Question 9 Determine if a new
pre-operative antibiotic reduces the risk of
infection for patients undergoing knee
replacement.

• How might we proceed?

43
Surveys and Polls(and the errors inherent in
them)
44
Sampling
45
Sources of Nonsampling Errors

• Selection bias
• Population sampled is not exactly the population
  of interest.
• e.g. KARE 11 poll, telephone interviews

46
Sources of Nonsampling Errors

• Non-response bias
• People who have been targeted to be surveyed do
  not respond.
• Non-respondents tend to behave differently to
  respondents with respect to the question being
  asked.

47
1936 U.S. Election

• Country struggling to recover from the Great
  Depression
• 9 million unemployed
• 1929-1933 real income dropped by 1/3

48
1936 U.S. Election

• Candidates

• Franklin D Roosevelt (Democrat)
• Deficit financing - Balance the budget of the
  people before balancing the budget of the Nation

• Albert Landon (Republican)
• The spenders must go!

49
1936 U.S. Election

• Roosevelts percentage
• Digest prediction of the election result
• Gallups prediction of the Digest prediction
• Gallups prediction of the election result
• Actual election result

43
44
56
62

• Digest sent out 10 million questionnaires to
  people on club membership lists, telephone
  directories etc.
• received 2.4 million responses
• Gallup Poll used another sample of 50,000
• Gallup used a random sample of 3,000 from the
  Digest lists to predict Digest outcome

50
Sources of Nonsampling Errors

• Self-selection bias
• People decide themselves whether to be surveyed
  or not.
• Much behavioural research can only use
  volunteers.

51
Sources of Nonsampling Errors
52
Sources of Nonsampling Errors
53
Sources of Nonsampling Errors

• Question effects
• Subtle variations in wording can have an effect
  on responses.
• Eg Should euthanasia be legal?
• vs Should voluntary euthanasia be legal?

54
New York Times/CBS News Poll (8/18/80)

• Do you think there should be an amendment to
  the constitution prohibiting abortions?
• Yes 29 No 62
• Later the same people were asked
• Do you think there should be an amendment to
  the constitution protecting the life of the
  unborn child?
• Yes 50 No 39

55
Sources of Nonsampling Errors

• Interviewer effects
• Different interviewers asking the same question
  can obtain different results.
• Eg sex, race, religion of the interviewer

56
Interviewer Effects in Racial Questions

• In 1968, one year after a major racial
  disturbance in Detroit, a sample of black
  residents were asked
• Do you personally feel that you trust most
  white people, some white people or none at all?
• White interviewer
• 35 answered most
• Black interviewer
• 7 answered most

57
Sources of Nonsampling Errors

• Behavioural considerations
• People tend to answer questions in a way they
  consider to be socially desirable.
• e.g. pregnant women being asked about their
  drinking habits

58
Behavioural Considerations in Election

• Official vote counts show that 86.5 million
  people voted in the 1980 U.S. presidential
  elections.
• A census bureau survey of 64,000 households some
  weeks later estimated 93.1 million people voted.

59
Sources of Nonsampling Errors

• Transferring findings
• Taking the data from one population and
  transferring the results to another.
• e.g. Twin Cities opinions may not be a good
  indication of opinions in Winona.

60
Sources of Nonsampling Errors

• Survey-format effects
• Eg question order, survey layout, interviewed
  by phone or in- person or mail.

61
Sampling
62
Survey Errors
Nonsampling Errors
Sampling/Chance/ Random Errors
63
Sampling / Chance / Random Errors

• errors caused by the act of taking a sample
• have the potential to be bigger in smaller
  samples than in larger ones
• possible to determine how large they can be
• unavoidable (price of sampling)

64
Nonsampling Errors

• can be much larger than sampling errors
• are always present
• can be virtually impossible to correct for after
  the completion of survey
• virtually impossible to determine how badly they
  will affect the result
• must try to minimize in design of survey (use a
  pilot survey etc.)

65
Surveys / Polls

• A pilot survey is a small survey that is carried
  out before the main survey and is often used to
  identify any problems with the survey design
  (such as potential sources of non-sampling
  errors).

66
Surveys / Polls

• A report on a sample survey/poll should include
• target population (population of interest)
• sample selection method
• the sample size and the margin of error
• the date of the survey
• the exact question(s)