Design and conduct of randomised controlled trials

1
Design and conduct of randomised controlled trials

• David Torgerson
• Director, York Trials Unit
• djt6_at_york.ac.uk
• www.rcts.org

2
Justification

• Randomised controlled trials are the key to
  evidenced based practice.
• The basic principles of the RCT are relatively
  straight forward.
• HOWEVER, many elementary mistakes are made that
  undermine their design.
• Whilst placebo trials are widely thought of as
  being the gold-standard other trial designs are
  as important if not more important to provide
  evidence for decision making.

3
The Need for Clinical Trials
4
Background

• Traditionally most interventions have been
  evaluated using a pre-test post-test or before
  and after design.
• Participants are tested treated and then tested
  again any improvements are attributable to the
  intervention.
• Currently this is probably the most POPULAR
  evaluative method.

5
What is wrong with before and after studies

• "It is incident to physicians, I am afraid,
  beyond all other men, to mistake subsequence for
  consequence. (Samuel Johnson, 1734)

6
Problems

• Problems include
• Temporal changes
• Regression to the mean.

7
Temporal Change

• Most people get better or worse irrespective of
  any medical attention.
• Similarly in other fields (e.g. education)
  self-learning irrespective of teaching occurs.
• Any intervention or treatment is mixed up with
  these temporal changes difficult to disentangle.

8
Regression to the Mean

• As well as temporal changes before and after
  studies are confounded by a statistical
  phenomenon known as Regression to or towards the
  mean

9
Regression to the mean

• This is a GROUP phenomenon and occurs when the
  group are measured with an inexact measurement
  tool and then remeasured. Those individuals with
  extreme values will have a high probability of
  regressing towards the mean on the second
  measurement.

10
Components of RTM

• Statistically we have two components to a test
  the true value plus a random error component.
• The larger the value of the random error the
  greater will be RTM.
• Many clinicians and others are mislead because of
  RTM.

11
Economists and RTM

• I suspect that the regression fallacy is the
  most common fallacy in the statistical analysis
  of economic data
• Milton Friedman 1992

12
Regression to Mean using bone density
Cummings et al. JAMA 20002831318-21
13
Same data dichotomised
Cummings et al. JAMA 20002831318-21
14
Example of clinician confusion

• In routine care women not responding to a
  bisphosphonate are given high dose vitamin D to
  try and reverse bone loss.

Heckman et al. BMC Musculoskeletal Disorders
200236.
15
(No Transcript)
16
Authors conclusion

• In patients not responding to bisphosphonates
  1000 vitamin D daily may improve BMD.
• So what if they are wrong? Vit D is harmless?
• BUT an RCT of high dose vitamin D showed a
  significant INCREASE in hip fractures among the
  treated group!

17
Another clinical fallacy

• A recent conference presentation, using
  observational before and after data, made the
  suprising observation that bisphosphonate therapy
  may reduce the number and incidence of falls.

18
Effect of Bisphosphonate on Falls
Population Risk of Falls
19
Effect of Bisphosphonate on Falls
Population Risk of Falls
Change p lt 0.001
20
Do bisphosphonates reduce falls?

• This suprising positive effect of
  bisphosphonates warrants further study as one
  possible additional explanation as to why
  fracture rates are reduced with bisphosphonate
  treatment
• Explanation is probably Regression to Mean.

21
Two further abstracts at same conference

• One was an RCT of vitamin D injections. Placebo
  injections were also given. Patients had low BMD
  placebo group increased by 3 in one year (not
  diff from active group).
• Next abstract before and after with calcitriol
  bone mass increased by 1.21, among women
  selected with low BMD. Conclusion Calcitriol
  prevented bone loss at hip and spine
• Did it? Probably RTM.

22
Policy Changes

• Regression to the mean is an excellent method of
  proving something works
• Failing hospitals can have an expensive
  management change and there is a good chance that
  regression to the mean will do the job.
• A NHS authority sent for retraining the 10 of
  cytologists who had the highest error rate
  training was highly effective!

23
Solution to weakness of before and after studies

• To control both for regression to the mean and
  temporal effects it is ESSENTIAL that evaluations
  have a contemporaneous control group.

24
Choice of Control Groups

• Not ALL control groups are equal. Only control
  groups formed by randomisation or a similar
  procedure (e.g., minimisation) are robust.

25
Selection Bias

• All non randomised studies are susceptible to
  selection bias.
• Selection bias means that comparator groups are
  not equal in all variables except for the
  treatment. Selection for treatment may be based
  on a variable that predicts outcome.

26
Selection Bias What are its Effects?
27
Hormone Replacement Therapy

• Non-randomised studies have shown that HRT is
  associated with reductions in cardiovascular
  disease, dementia and all cause mortality.
• This view prevailed until a large (n 16,000)
  RCT was undertaken.

28
Trial Evidence

• Early trial in men of oestrogens in 1970s stopped
  increased mortality due to cardiovascular events.
• HERS trial in mid 1990s among women with Angina
  showed INCREASED risk of cardiovascular risk.
• Trial of 16,000 women (WHI study) stopped early
  in 2001 because of INCREASE in cardiovascular
  deaths.

29
HRT what happened?

• Women who took HRT tended to be different from
  those who did not. They took more exercise had a
  better diet both of which reduces cardiovascular
  disease.
• The association was due to these factors NOT HRT.
• Because of these SELECTION effects we need to
  undertake randomised trials.

30
Why Randomisation?

• Randomisation WILL on average create two or more
  groups that are equivalent in all important
  characteristics that affect outcome.
• Because of this the introduction of a treatment
  in one group will therefore produce and UNBIASED
  estimate of effect.

31
Randomisation

• Production of equivalent groups will control for
  regression to the mean, temporal changes and
  selection bias.

32
Failure of medicine

• Because of LACK of RCTs in many areas unproven,
  harmful, treatments have been given to many
  patients.
• History of medicine is littered with examples of
  introducing unproven treatments to patients
  before controlled rigorous evaluation.

33
Opposition to RCTs

• The use of the RCT in health care has met
  sustained opposition from many clinicians.
• This has led to many disasters.

34
Opposition

• I found but one person who rigidly adherred to
  the idea of a placebo control and he is a
  bio-statistician who, if he did not adhere to
  this view, would have had to admit his own
  purposelessness in life (Jonas Salk).

35
Health Care Disasters

• Opposition to RCTs has declined over the years,
  partly due to a number of catastrophes, from
  unevaluated treatments.
• Harmful treatments are still in widespread use
  today we just dont know which ones.

36
Disasters among babies

• Routine practice in 40s and 50s to give premature
  infants pure oxygen. At the same time it was
  noted that there was an epidemic of blindness
  among babies. Linked to oxygen use.
• Routine practice in 50s to give prophylactic
  antibiotics to premature infants, caused brain
  damage and death.
• BOTH of these problems only discovered AFTER an
  RCT was undertaken.

37
Heart Attacks

• After heart attack survivors can be placed into
  two groups
• Those with arrhythmia those without.
• Arrhythmia patients have a much HIGHER risk of
  subsequent death.
• There are drugs that can correct cardiac
  arrhythmias.
• These were widely used on millions of patients.

38
CAST Trials

• For decades researchers tried to evaluate
  arrhythmia drugs.
• Not ETHICAL argued many cardiologists.
• Finally, CAST trial was started, terminated early
  in 1989 because of INCREASE in deaths on active
  drug.
• 100s of thousands died because of this.

39
Can it still HAPPEN?

• Supplementation with folate increases risk of MI
  (2005)
• Use of intravenous steroids after head injury
  increases mortality (2004)
• HRT INCREASES strokes, myocardial infarction and
  dementia (2002).
• Counselling women after difficult labour
  INCREASES depression (2001)

40
Development of the Controlled Trial.

• Many researchers over the centuries have seen the
  need for a control group to avoid the inherent
  biases in the before and after study.
• Controlled trials have been conducted for several
  hundred years probably occasionally using
  randomisation.

41
Scurvy

• Scurvy was a very prevalent condition among
  sailors before the 19th Century.
• A controlled trial in the middle of the 18th
  Century of 12 sailors showed that the two sailors
  allocated to receive lime or orange juice
  recovered compared to their ship mates allocated
  to vinegar or salt water.

42
Medical Care 1810

• Standard medical care in the early 1800s
  included routine bleeding.
• Practised by many if not most doctors in the 18th
  and early 19th Centuries.
• Chalmers reports a controlled trial undertaken
  during the Peninsular War comparing bleeding with
  no bleeding of sick soldiers.

43
Mortality after Bleeding
44
Cambridge-Somerville

• In 1937 a classic experiment the
  Cambridge-Somerville trial was launched.
• The aim was to show that social worker
  intervention among delinquent boys would reduce
  criminality.

45
Design

• 650 boys were identified by their teachers has
  having delinquent behaviour that put them at
  later risk of criminal activity.
• 325 pairs were formed and one from each pair was
  allocated a social worker supported by
  psychiatrists.

46
Results early follow-up of boys indulging in
crime.
Pale bar indicates intervention grop
47
Results later follow-up

• In 1975 boys were followed up again when middle
  aged men.
• 58 of intervention group had NOT had a criminal
  conviction
• BUT 68 of control group had NOT had a
  conviction.
• If a control group had not been used success of
  the intervention would be assured.

48
Patulin Trial

• Perhaps the first placebo controlled trial was
  the 1944 MRC patulin trial for the treatment of a
  common cold.
• 1449 patients included in the trial. Percent
  cured or improved at 48 hours was 77 for placebo
  and 73 for patulin.

49
Streptomycin

• The first randomised trial in health care of a
  BENEFICIAL treatment was the MRCs steptomycin
  trial.
• Streptomycin expensive antibiotic, unknown
  efficacy for pulmonary TB.
• UK could not afford to buy much due to virtual
  postwar bankruptcy.
• Randomisation was seen as fair method of
  allocating scarce drug.

50
Streptomycin Results
Streptomycin Control
Improvement 69 33
No change 4 6
Worse 20 34
Death 7 27
51
Streptomycin

• Note 1/3rd of the control patients improved
  despite no antibiotic
• Patulin Trials were methodologically important
• Used random allocation
• Blinding of doctors, patients, assessors
• Allocation of treatment was concealed
• Placebo (for patulin).

52
Summary

• Failure to do timely controlled trials has cost
  many 000s of lives.
• All new treatments need to be evaluated using the
  RCT.