Bias control in Randomised trials

1
Bias control in Randomised trials components
associated with overestimated intervention
benefits

• Lise Lotte Gluud
• Cochrane Hepato-Biliary Group
• Department of Medicine
• Copenhagen University Hospital Gentofte
• Denmark

2
Low risk of bias
High risk of bias
Low risk of random error
High risk of random error
3
Remember

• Randomised trials are the gold standard for
  intervention comparisons but not all randomised
  trials are the same
• On average randomised trials without adequate
  bias control overestimate intervention benefits

4
Components of bias control

• Several components reflect the control of bias
  but why do we need this information and how much
  do we need to know
• Evidence-based practitioner must know a quick way
  to analyse internal validity
• Systematic reviewer must have an in-depth
  knowledge of bias

5
Bias in clinical trials

• For the evidence-based practitioner
• Selection bias
• Detection bias
• For the systematic reviewer
• Performance bias
• Ascertainment bias
• Attrition bias
• Reporting bias and many many more...

6
The most important one

• Selection bias!!!
• May occur when participants are selected for an
  intervention based on variables associated with
  the outcome
• Is avoided through randomisation with
• Adequate allocation sequence generation
• Adequate allocation concealment

7
Why randomise?
Clinical trial comparing treatment A with
treatment B. The results suggest that treatment A
is better than B, but can we believe the results?
8
Why randomise

• Why did these 17 patients recover?
• They were treated with A or they were younger,
  stronger, and more compliant?

9
Why randomise

• Why did these 12 patients not recover?
• They were treated with B or they had
  co-morbidities or were less compliant?

10
Confounding by indication

• In any given population, patients who receive
  treatment will be different from patients who are
  not treated
• Which patients do you normally treat?
• The very old with co-morbidities ?
• The noncompliant patient ?

11
Confounding by indication

• Confounding by indication means that we cant
  know whether the difference in outcomes between
  the patient groups reflects
• Different baseline characteristics
• The treatments assessed

12
Selection bias

• The same factors that lead to confounding by
  indication in clinical practice can influence the
  results in clinical trials
• In clinical trials, selection bias may be defined
  as a systematic error (bias) in the selection of
  patients to the interventions compared

13
Selection bias

• To know if any difference in the outcomes of
  patients in the comparison groups reflect the
  intervention or the baseline prognosis the groups
  must be similar regarding
• Known prognostic factors
• Unknown prognostic factors

14
Randomisation

• The way to avoid selection bias
• Randomisation is a term associated with the
  allocation of patients to intervention groups
• NOTE!!
• Randomization ? random selection

15
Randomisation

• Randomisation means that each patient has a known
  usually equal chance of receiving the
  interventions being compared and that the
  allocation of the next patient is unpredictable

16
Randomisation methods

• Randomisation requires
• Allocation sequence generation
• Computer generated random numbers, or table of
  random numbers
• Allocation concealment
• Central independent unit, identically appearing
  coded drug containers, or serially numbered
  opaque sealed envelopes

17
Allocation sequence generation

• Random number table
• Can be found in statistical books or online from
  trusted source
• Each digit occurs with same frequency
• Adjacent digits are completely independent of one
  another
• Moderately long sections show substantial
  regularity
• Randomness is a property of the table

18
Random number table
19
Allocation sequence generation

• How to perform simple randomisation using a
  random number table
• Use the table and give treatment A to equal
  numbers and B to unequal numbers
• Begin at an arbitrary point

20
Allocation sequence generation

• Randomised trial comparing treatment A with
  treatment B
• How to use a random number table for simple
  randomisation
• Each patient has an equal chance of receiving A
  or B
• Similar to tossing a coin

21
Random number table
Select numbers for randomisation of 8 patients
22
Random number table
Treatment A to 5 patients and B to 3 patients
23
Randomisation

• Simple randomisation may lead to differences in
  the number of patients allocated to A or B
• This may be avoided if patients are randomised
  through block or restricted randomisation
• Use the same table as before but randomise
  patients in groups

24
Block randomisation

• Two treatments A versus B
• Block size of four
• Omit all numbers except 1 to 6
• 1 A A B B 2 A B A B 3 A B B A
• 4 B B A A 5 B A B A 6 B A A B

25
Block randomisation

• Block size may vary
• The number of patients in the two groups cant
  differ by more than half the block size (which is
  normally a multiple of the number of treatments)
• Does not guarantee similar patient
  characteristics that may occur by chance

26
Stratified randomisation

• Separate lists for subgroups of patients strata
• Should be based on block randomisation within
  each stratum
• If simple randomisation is used the object of
  stratification is missed because there is no
  control of the balance between treatments

27
Stratified randomisation

• If several strata is used separate lists for
  combination of categories are made but remember
  that
• smaller trials can only have one or two
  categories
• If you have too many some strata may be empty

28
Allocation concealment

• The allocation of the next patient has to be
  unpredictable
• This may be achieved through
• Central randomisation
• Coded identical numbered drug containers or
  bottles
• Serially numbered opaque sealed envelopes

29
Allocation concealment

• Subversion has been reported in several trials
• Sealed envelopes may be opened or the allocation
  number seen through trans-illumination
• Researchers have broken into filing cabinets or
  called randomisation units with special
  requests......

30
What do you think about this description?

• Patients were randomised in a double blind
  manner using a table of random numbers

31
What do you think about this description?

• Patients were allocated to propranolol or
  placebo in a random manner.....

32
What do you think about this description?

• Patients were randomised in a double blind
  manner using a table of random numbers

33
What do you think about this description?

• Randomisation was performed with stratification
  for viral genotype and load through an
  independent researcher

34
What do you think about this description?

• Restricted randomisation was performed with a
  block size of two stratified by age gt 50 using a
  table of random numbers

35
What do you think about this description?

• The duty anaesthetist randomly assigned
  .(patients to)using a customised randomisation
  programme situated on the delivery suite

36
But the program had a fault

• Lessons from the COMET 1 trial
• Effect of low-dose mobile versus traditional
  epidural techniques on mode of delivery a
  randomised controlled trial Lancet 200135819.

Another 1000 women were included in COMET 2
37
Randomisation

• Another way to determine whether there was
  adequate randomisation or differences occurring
  by chance is to determine whether allocation
  groups are comparable after randomisation

38
Bias control

• Adequate randomisation reduces the risk of
  selection bias but the validity of the trial can
  still be threatened by other forms of bias
• Detection bias
• Attrition bias
• Performance bias
• Dissemination bias

39
Blinding

• Expectations may bias researchers and patients in
  their performance during the trial and their
  assessment of the outcome

40
Blinding

• Several persons in trials may be blinded
• Double blinding has been described as keeping
  patients and investigators unaware of the
  allocated treatment but definitions vary
  considerably

41
Blinding

• A study of 200 trials described as double blind
  found that in 19 there was no blinding of
  patients, health care providers, or outcome
  assessors
• Haahr Clin Trials 2006

42
Blinding

• Achieved through administration of identically
  appearing interventions or placebo
• 56 students received a blue or a pink placebo.
  Those receiving the blue felt less alert than
  those taking the pink placebo
• Blackwell Lancet 1972

43
Blinding

• The effect of blinding should be assessed
• Blinding was broken in trial on vitamin c for
  common cold due to taste of vitamin and placebo
  capsules
• Karlowski JAMA 1975

44
What do you think about this description?

• Patients were randomised to nicotine or a
  placebo chewing gum..

Campbell Practitioner 1987
45
What do you think about this description?

• Patients were randomised to nicotine or a
  placebo chewing gum..

Regular chewing gum
46
Blinding

• A study of 1599 blinded trials found that 2
  reported that blinding was tested usually by
  asking patients or investigators to guess the
  allocated intervention
• 23 found that blinding was not successful
• Hróbjartsson Int J Epidemiol 2007

47
Blinding

• Double blinding may apparently be achieved in
  trials on drugs
• But then what about drugs with known adverse
  events
• Can blinding be achieved in trials on surgery?
• Is blinding of outcome assessors enough if we
  cant double blind?

48
Attrition Bias

• Most trials lose participants after randomisation
• Attrition bias can distort the baseline
  comparability of comparison groups
• Adverse events lead to dropouts especially among
  non-compliant patients

49
Attrition bias

• Causes of attrition bias
• Investigators may choose to exclude patients if
  they disagree with the allocation
• Patients may be determined ineligible in
  retrospect
• There may be competing events

50
Attrition Bias

• Intention to treat analysis reduces the problems
  with attrition bias
• Has to include all patients randomised
  irrespective of follow up
• For patients with missing data a decision has to
  be made regarding how they are included in the
  analysis

51
Attrition bias
plt.01
plt.01

• Five year follow up data from randomised trial on
  clofibrate for heart disease
• Piantadosi Clin Trials 1997

52
Attrition bias

• The table shows mortality
• Wilcox BMJ 1980

53
Attrition bias
Kemney J Clin Onc 2002
54
Attrition bias

• From the text comparisons were made between the
  intent-to-treat and patients considered
  assessable
• Intention to treat 53 patients in the treatment
  group and 56 in the control group
• Assessable patients 30 in the treatment group
  versus 45 in the control group

55
Attrition bias

• Of the 45 assessable patients who survived
  surgery in the control arm 35 (77.8) have
  experienced recurrence
• In the chemotherapy arm 16 (53.3) of the 30
  assessable patients who survived surgery had
  recurrences

56
Dissemination bias

• When the dissemination of research is influenced
  by the results
• Statistically significant results are more likely
  to be published
• Rapidly, in English, more than once, in high
  impact journals, and subsequently cited by others

57
Outcome reporting bias

• Cohort study comparing protocols for 102
  randomised trials described in 122 published
  reports
• 50 of efficacy and 65 of harm outcomes per
  trial were incompletely reported
• Chan JAMA 2004

58
Outcome reporting bias

• The odds of statistically significant outcomes
  being fully reported
• Efficacy outcomes OR 2.4 (CI 1.4 to 4.0)
• Harm outcomes OR 4.7 (CI 1.8 to 12.0)
• In 62 of trials at least 1 primary outcome was
  changed, introduced, or omitted

59
Outcome reporting bias

• In a survey of the trialists for the 102 trials
• 86 of survey responders denied the existence of
  unreported outcomes despite clear evidence to the
  contrary
• Chan JAMA 2004

60
Some examples

• Changed outcomes
• with severe cerebral bleeding changed from
  primary to secondary
• with graft occlusion introduced not described
  in the protocol
• Prespecified primary outcome omitted
• Event-free survival rate

61
Reporting bias

• Study of protocols for 43 randomised trials
  funded by pharmaceutical firms found that the
  sponsor had
• Access to accumulating data during 16 trials
  (disclosed in 1 publication)
• The right to stop the trial at any time for any
  reason in 16 trials (not disclosed in any trial
  publications)
• Gøtzsche JAMA 2006

62
Competing interests

• Study of 159 trials published in the BMJ compared
  authors conclusions using a 6 point scale
• Conclusions were more positive towards the
  experimental intervention in trials funded by for
  profit organisations alone
• Kjaergard BMJ 2002

63
Competing interests

• Study of 370 randomised drug trials found that
  the experimental drug was recommended as the
  treatment of choice in
• 51 of trials reporting for-profit funding alone
  compared with 16 to 35 of other trials Plt.001
• Als-Nielsen JAMA 2003

64
Competing interests

• The association between funding and conclusions
  did not reflect
• The reported results
• The quality of the trials
• The disease area
• The reported adverse events

65
How to change conclusions

• One way to make conclusions more favourable is to
  change outcome measures and the analytical
  strategy
• The ISIS 2 trial found that asprin has no effect
  for myocardial infarction in persons with the
  star signs Gemini or Libra
• ISIS 2 Lancet 1988

66
Fraud

• VIOXX Rofecoxib was introduced by Merck in 1999
• Scientists at Merck were concerned that the drug
  alters the ratio of prostacyclin to thromboxane

67
Vioxx

• Following internal review, Merck requested that
  conclusions from a study on rofecoxib were
  changed
• systemic biosynthesis of prostacyclin ... was
  decreased by rofecoxib" to "Cox-2 may play a
  role in the systematic biosynthesis of
  prostacyclin

68
Vioxx

• In 1998 Merck submitted 9 small studies including
  patients with a low risk of heart disease and
  short duration of follow up for FDA approval
• None assessed cardiovascular risk although
  initial studies suggested that the drug may
  increase thrombus formation

69
Vioxx

• In 1999 the VIGOR trial was launched
• Compared VIOXX with NSAID
• The trial included more than 8000 persons
• No SOP for collecting information on
  cardiovascular events
• No cardiologist on the data safety monitoring
  board

70
Vioxx

• The first safety analysis presented to the safety
  board showed a 79 increased risk of death or
  serious cardiovascular events P.007
• The board recommended additional analyses of
  serious cardiovascular events

71
VIOXX

• The VIGOR trial results showed that
• VIOXX
• Was better at relieving symptoms of rheumatoid
  arthritis
• Reduced GI events with 50
• Increased the risk of myocardial infarction RR
  5.00 (CI 1.68 to 20.13)

72
Vioxx

• The risk of MI was obscured by
• Reporting an interim analysis with different
  termination dates for cardiovascular and
  gastrointestinal events (1 month longer)
• Removed three MI occurring after the reported
  follow up (in VIOXX group)
• Changed the HR to indicate NSAID was the
  experimental drug (RR 0.2)

73
Ketek

• Ketek (telithromycin) was developed for upper
  respiratory tract infections by Sanofi-Aventis
• Submitted for FDA approval but FDA asked for
  additional data on potential adverse events
  especially liver damage

74
Ketek

• A randomised trial, study 3014 on Ketec versus
  amoxicillinclavulanate was performed
• 1800 doctors were paid 400 per patient
• The published results suggested that Ketek was
  safe
• The new data were submitted for approval

75
Ketek

• However!!
• Routine inspection revealed that one doctor
  fabricated data suggesting that more than 400
  patients had been included leading to a 57 month
  sentence

76
Ketek

• Inspections of nine other sites revealed serious
  violations of trial conduct, raising substantial
  concerns about the overall integrity of the study
• 4 of the 10 inspected sites were referred for
  criminal investigation

77
Ketek

• Due to the criminal investigations managers were
  barred from revealing the information and Ketek
  were approved by the FDA in 2004
• But since 2005 several cases of severe liver
  damage were recorded...

78
Ketek

• In April 2007
• 23 cases of acute severe liver injury
• 12 cases of acute liver failure
• 4 patients died due to liver failure
• Could this somehow have been avoided???

79
Bias in randomised trials

• Randomised trials are the gold standard for
  intervention comparisons BUT not all randomised
  trials are the same
• Trials described as randomised without adequate
  bias control may generate misleading conclusions

80
Bias in randomised trials

• Important components of bias control
• Randomisation
• Allocation sequence generation
• Allocation concealment
• Blinding
• Follow up
• Funding

81
Bias in randomised trials

• For the more thorough reviewer
• Find the unpublished trials and the unpublished
  results plus information about what actually
  occurred
• Did authors change outcomes or make other
  changes?
• Always be sceptical!

82
(No Transcript)