Research design

1
Research design

• Gordon Prescott

2
Why do we see conflicting results in studies?

• Results and inferences are dependent on the study
  design
• Method of selection of subjects for the study
• Size of the sample used for the study
• Conduct of study

3
Study design

• The aim of study design is
• to maximise attribution
• to minimise all sources of error
• to be practical

4
Factors to be aware of ...

• Bias
• systematic error
• Confounding
• A variable which is associated with both exposure
  (intervention) and outcome
• e.g. contraceptive use - smoking Myocardial
  Infarction (MI)
• Chance
• random error

5
Some common types of studies conducted

• Experimental designs
• Randomised controlled trials (RCT)
• Parallel group
• Crossover
• Block designs
• Observational studies
• Cohort
• Case-control
• Cross-sectional

6
Outline of lecture

• Focus first on the data structures obtained from
  RCTs (parallel and crossover) and observational
  designs
• Limit this to dichotomous outcomes
• Consider experiments where repeated samples or
  replications for all sources of variation
• Sample size issues

7
Research questionsRCT and observational studies

• Examples
• Hormone replacement treatment (HRT) and Breast
  cancer
• Trial of Vitamin D and calcium supplementation
  and hip fracture
• Use of statins for the prevention of Myocardial
  Infarction
• Use of the oral contraceptive pill and deep vein
  thrombosis (DVT)

8
Measures of effect size (dichotomous outcome)

• Absolute risk reduction (ARR)
• The difference in risk of a given event, between
  two groups
• Number Needed to Treat (NNT)
• It is defined as the number needed to treat in
  order to prevent one additional adverse event
  (e.g. death)
• Relative risk (RR)
• Is the ratio of the risk of a given event in one
  group of subjects compared to another group
• Relative risk reduction
• (1-RR) x 100
• The proportion of the initial or baseline risk
  which was eliminated by a given
  treatment/intervention or by avoidance of
  exposure to a risk factor
• Odds ratio (OR)
• Is the ratio of the odds of a given event in one
  group of subjects compared to another group

9
Interpretation of effect sizes

• Consider the null hypothesis
• ARR
• Difference in risk
• Ho RiskA RiskB 0
• OR/RR
• Ratio
• Ho riskA/riskB 1 or oddsA/oddsB 1

10
Risk vs. odds

• The risk (or rate) of an event occurring is
• the number with the event _
  
• total number of people exposed
• The odds of an event is
• number with event _
• number without the event
• Out of 10 people,
• 2 have headache rate 0.2 odds0.25
• 4 have headache rate 0.4 odds0.67

11
Randomised controlled trial parallel group
12
RCT crossover trial
13
Data structure

• Parallel group trials give independent samples
• Crossover trials give paired samples

14
(No Transcript)
15
Example

• A multi-centre, randomised placebo-controlled
  trial of the beta blocking drug Timolol, reported
  the number of deaths in 18 months of follow-up
  among patients who had recently suffered a
  myocardial infarction
• (New England Journal of Medicine. 1981304
  801-7).

16
Example

• Outcome
• Treatment Died Survived
  Total
• Timolol 98 847
  945
• Placebo 152 787 939
• What is the risk of death in the Timolol group?
• What is the risk of death in the placebo group?
• What is the difference in risk of mortality
  (ARR)?
• What is the relative risk of mortality?
• What statistical test would you apply to test
  whether there is a difference in mortality
  between the two groups?
• What is the number needed to treat with Timolol
  to prevent one additional person dying?

17
Calculations

• Treatment Died Survived Total
• Timolol 98 847 945
• Placebo 152 787 939
• Risk of death with Timolol 98/945 0.104
  (10.4)
• Risk of death with Placebo 152/939 0.162
  (16.2)
• Absolute risk reduction through use of Timolol
• (152/939) - (98/945) 0.058 (5.8)
• 95 CI 2.8, 8.9
• Relative risk 0.104/0.162 0.64 (95 CI 0.51,
  0.81)
• Number of patients needed to be treated (NNT)
  with
• Timolol to prevent one death 100/5.8
  1/0.058 17
• Chi square test can be used for significance test

18
Crossover trials
The crossover design (each patient receives both
treatments in random order, often with a washout
period between treatments).
19
Crossover (Paired samples of size n)
Outcome on treatment A
Outcome on Treatment B

• Rate of failure with treatment A (wx)/(wxyz)
• Rate of failure with treatment B (wy)/(wxyz)
• Particular interest focuses on the numbers of
  patients with discordant findings (x and y)

20
Observational studies

• A question which is often posed in epidemiology
  is
• Does exposure A cause disease B?
• It is unethical to randomise these subjects to
  these exposures (e.g. smoking) so instead we have
  to make do with the information which is
  available
• We do not have the experimental design i.e.
  randomisation
• Therefore, causal relationships are harder to
  demonstrate
• Differences may exist between the exposure groups
  that could have an impact on the outcome

21
Confounding

• An important part of an observational study
  investigating a relationship between an exposure
  and a disease is to check for possible
  confounding factors.
• Such factors are associated with both the
  exposure and the disease (e.g. a study of
  whether or not smoking is a cause of liver
  cirrhosis would need to take account of the
  confounding influence of alcohol consumption)

22
Cohort study
diseased
exposed
non-diseased
Group of subjects disease free at the start of
the study
comparison
not exposed
diseased
non-diseased
23
Relative Risk

• For cohort studies the relative risk is used.
• The relative risk is
• the risk of disease in exposed group
• relative to
• the risk of disease in the unexposed group.
• Example
• The Caerphilly cohort study followed up
  approximately 2,500 middle-aged Welsh men to
  examine the association between several risk
  factors (measured at entry to the study) and the
  subsequent risk of ischaemic heart disease in a
  five-year period.

24
Cohort study

• Risk of disease in exposed 101/1387
• Risk of disease in unexposed 50/1114
• The relative risk associated with smoking is
  obtained by the risk ratio
• (101/1387) / (50/ 1114) 1.62 95 CI 1.17
  to 2.26
• Smokers have an increased risk of IHD compared
  with non-smokers. Smokers are 1.6 times more
  likely than non smokers to have IHD

25
Case-control study
Risk factors
Cases with disease under study
comparison
Previous exposures
Controls without disease under study
26
Case-control study

• For case-control studies the odds ratio is used.
• The odds ratio is the ratio of
• the odds of exposure in the diseased group
  compared to
• the odds of exposure in the non-diseased group.

27
Unmatched case-control study
28
Case-control example

• The ECTIM study was a case-control study of 610
  men who had suffered a myocardial infarction and
  733 controls.
• One of the factors assessed in these men was the
  gene encoding for angiotensin-converting enzyme
  (ACE), and each man was classified as Yes or No
  for a particular ACE genotype.

29
Case-control example
30
Case-control study

• Odds of ACE genotype in diseased group
  197/413
• Odds of ACE genotype in undiseased group
  200/533 
• The estimate of the relative risk of myocardial
  infarction associated with this ACE genotype is
  given by the odds ratio 
• (197/413) / (200/533) 1.27 95CI 1.00 to
  1.62
• Cases are more likely to be exposed to the ACE
  genotype than cases. The odds of being exposed
  to the ACE genotype is greater in the cases

31
Individually matched case-control study
32
Observational studies

• Statistical adjustment may be required for
  confounding factors
• Multiple regression models
• Logistic regression (next week)
• Stratification
• Mantel Haenszel methods
• Further reading Kirkwood and Sterne Chapt 18
• Altman DG
• Petrie and Sabine

33
Randomised block experiments

• Many sources of variation
• Time, temperature, resting / following exercise,
  observers, etc
• Replication is required per combination of
  experimental conditions
• Must be independent to one another
• This will give greater precision
• There will often be non-experimental conditions
• Age of patient
• Consider what varies across observations
  (experimental conditions) and what varies between
  subjects (covariates)

34
Statistical analysis

• Same number of replications per combination of
  experimental conditions makes analysis easier
  design is said to be balanced
• Multiple regression or ANOVA commonly adopted
• Number of experimental conditions relates to one
  way, two way etc ANOVA

35
Examples

• Study conducted to examine effect of three diets
  and the timing of measurement (first thing
  am/after midday meal).
• Subjects were allocated to a diet and two
  measurements were taken on each patient
• For each diet/timing combination 4 subjects
  measured
• Important to distinguish between subject and
  within subject comparisons
• Between subject diets
• Within subjects time of assessment

36
Diagram

• Diet 1 Diet 2 Diet 3
• Fast Food Fast Food Fast Food
• x x x x x x
• x x x x x x
• x x x x x x
• x x x x x x
• Fast Fasting (before food)
• Food After food

37
Statistical analysis

• Ho No effect of diet on outcome
• mean (diet1) mean (diet2) mean (diet3)
• Ho No effect of timing on outcome
• mean (fasting) mean (after food)
• Two way analysis of variance could be conducted
  to partition variation into diet, timing,
  residual

38
Why consider sample size?

• Recall random error - chance
• It is possible determine what sample size should
  be taken, if we wish to achieve a given level of
  precision.
• This is because precision can be increased by
  reducing the size of the standard error.
• The size of the standard error is based on the
  size of the sample.
• The larger the sample size the smaller the
  standard error.

39
Sample size to estimate a population parameter

• Initial estimate of population parameter (e.g.
  from a pilot study)
• What degree of accuracy required? (e.g. to
  within 5)

40
Sample size for population percentage

• True in Precision 95 CI Sample size
• population (in terms of )
• 5 0.5 4 to 6 1900
• 5 1.5 2 to 8 212
• 20 0.5 19 to 21 6400
• 20 1.5 17 to 23 712
• 20 2.5 15 to 25 256
• 50 0.5 49 to 51 10000
• 50 1.5 47 to 53 1112
• 50 2.5 45 to 55 400
• 50 5 40 to 60 100
• adapted from Crombie IK (1996)
• e.g. 5 - 1.96x0.5, 5 1.96x0.5 4 to 6

41
Factors important in calculating sample size

• Study design
• Outcome measures
• Statistical test
• Minimum clinical effect
• Significance level (Type I error)
• Statistical power (Type II error)
• (Initial estimates of effectiveness and
  variability)

42
Significance level and power

• Significance level
• The probability that the statistical test returns
  a significant result when there is no difference
  between the treatments
• Power
• The probability that a study of a given size will
  detect a real difference of a given magnitude as
  statistically significant

43
Sample size for a comparative study

• The proportion with the feature in the control
  group (binary outcome)
• Measure of variability (continuous outcome)
• Minimum clinical difference
• The smallest difference in outcome between the
  two treatments that would be deemed to be
  clinically relevant
• Significance level
• Power

44
Example 1

• A randomised controlled trial to assess the
  effectiveness of laparoscopic versus open hernia
  repair
• Primary outcome measure is proportion of patients
  who have returned to normal activities at 2 weeks
  after operation

45
Sample size calculation

• Study design RCT
• Outcome Proportion of patients returned to
  usual activities at 2 weeks following open hernia
  repair
• Statistical test Chi squared test
• Estimate of level of outcome in control group
  (standard care) 30
• Minimum clinical difference 10
• Type I error 0.05 (5 significance)
• Type II error 0.1 (90 power)
• 500 patients required in each group

46
Example 2

• A study is to be conducted to evaluate a new drug
  for hypertension compared with the standard drug.
• The outcome will be systolic blood pressure at
  one month after treatment starts.

47
Sample size calculation

• Study design - RCT
• Outcome - SBP at one month
• Statistical test independent groups t-test
• Minimum clinical difference 10 mmHg
• Estimate of variability of SBP 30 mmHg
• Standardised difference 10/30 1/3
• Power - 80
• Significance level - 5

48
Example 2 continued

• From statistical formulae, the required sample
  size is 300 patients in total.
• 150 patients are required in both groups to yield
  80 power of detecting a difference of 10 mm Hg
  (0.3 standard deviation) in systolic blood
  pressure at the 5 significance level.
• 90 power - 200 patients in each group

49
Summary sample size

• Acceptable power is 0.8, 0.9 (80 or 90)
• Sample size is calculated from the clinically
  relevant difference, standard deviations of each
  group, power and significance level.
• Power is increased by
• larger sample size,
• larger clinically relevant difference,
• larger significance level (5, plt0.05 as
  significant),
• smaller standard deviations (e.g. more accurate
  measurements)

50
Relative Risk

• For cohort studies the relative risk is used.
• The relative risk is the risk or rate of disease
  in exposed group relative to the risk or rate of
  disease in the unexposed group.
• The prevalence of disease can be found using a
  cohort study
• A RR of 1 means that the risk of the event is
  equal in the groups being compared.
• A RR gt1 suggests the event (disease) is more
  common in the exposed group than the non-exposed
• An odds ratio lt1 suggests the event is less
  common in the exposed group than the non-exposed

51
Odds ratio

• For case-control studies the odds ratio is most
  often used to show relationships.
• The odds ratio is the ratio of the odds of
  exposure in the diseased group compared to the
  odds of exposure in the non-diseased group.
• An odds ratio of 1 means that the odds of the
  event is equal in the groups being compared.
• An odds ratio gt1 suggests the event (exposure) is
  more common in the diseased group than the
  non-diseased
• An odds ratio lt1 suggests the event is less
  common in the diseased group than the
  non-diseased

52
General comments

• The odds, risks and rates are the probabilities
  or chances of an individual having a particular
  event such as death or an illness.
• The odds ratio and the relative risk are very
  similar when a disease or exposure is rare.