Appraising a randomised controlled trial

1
Appraising a randomised controlled trial

• Alison Avenell

2
Why critically appraise?

• Methodological quality of trials may vary
• Trials may be biased
• Critical appraisal skills are important to
  analyse, interpret and use clinical information
  and research findings

3
CRITICAL APPRAISAL
FIND
USE
APPRAISE
4

• What do we mean by quality of a study?

5
Trial quality

• The quality of a trial is the extent to which
  its design and conduct are likely to prevent bias
• Variation in quality might explain variation in
  results

6
Quality assessment of studies

• Quality assessment refers to the internal
  validity of a study
• High quality studies are more likely to yield
  valid information

7
Internal validity

• the extent to which systematic error (bias) is
  minimised in a clinical study

8
What is quality?

• ...the confidence that the trial design,
  conduct and analysis has minimized or avoided
  biases in its treatment comparisons
• (Moher er al, Controlled Clin Trials 1995 16
  62-73)

9
Potential sources of bias

• Selection bias
• Performance bias
• Detection bias
• Attrition bias

10
Selection bias

• Systematic differences between intervention
  groups
• Bias in the way intervention groups are assembled
  (biases are likely to occur when participants
  receiving treatment are chosen by investigators)

11
Randomisation
INTERVENTION
POPULATION
NO INTERVENTION/PLACEBO
RANDOMISED CONTROLLED TRIAL
12
Randomisation

• Generation of allocation sequences
• Ideally resulting sequence is completely
  unpredictable
• Inadequate if sequence is predictable

13
Concealment

• Concealment of allocation sequence
• Adequate if patients and investigators cannot
  foresee assignment
• Inadequate if both parties can foresee assignment

14
Performance bias

• Systematic differences in the care provided apart
  from the intervention being evaluated
• Contamination and co-intervention can affect
  study results
• Solution blinding

15
Detection bias

• Systematic differences in outcome assessment
• The investigator assessing the outcomes of a
  trial should be blinded to whether the
  participants have been in the intervention or
  control group
• Solution blinding

16
Detection bias blinding

• Single blinding of patients
• Double care provider also blinding
• Triple outcome assessors also blind
• Quadruple analyst also blind

17

• Concealment
• is not the same
• as blinding!!

18
Attrition bias

• Differences between groups in the numbers of
  withdrawals/dropouts
• Jeopardise comparability of groups

19
Intention-to-treat analysis

• All participants are included in the arm to which
  they were allocated, whether or not they received
  (or completed) the intervention given to that
  arm. Intention-to-treat analysis prevents bias
  caused by the loss of participants, which may
  disrupt the baseline equivalence established by
  randomisation
• (The Cochrane Library Handbook glossary)

20
Consolidated standards of reporting trials
(CONSORT)

• Ideally the report of a RCT needs to convey
  relevant information concerning the design,
  conduct, analysis, and generalisability of the
  trial. This information should provide the reader
  with the ability to make informed judgements
  regarding the internal and external validity of
  the trial.
• (Begg, JAMA, 1996)
• http//www.consort-statement.org/

21
STUDY QUESTION

• Was the study addressing a clearly focused
  question?
• Participants/intervention/outcomes?
• Does it make sense?
• Prior hypothesis?

22
TREATMENT ALLOCATION

• Was assignment/allocation to treatment groups
  randomised?
• Method of randomisation?
• Allocation concealed?
• Balance between treatment groups?

23
INTERVENTION

• Were the intervention groups similar at the start
  of the study?
• Were the participants treated identically in the
  different groups?
• Demographic characteristics?
• Severity, co-morbidity, prognosis?
• Treatment regimens described?
• Co-interventions?

24
OUTCOMES

• Clinically relevant outcomes reported?
• - Which outcomes?
• Clinical outcomes/patient-relevant
  outcomes/surrogate outcomes?
• - How measured?
• Blind, reproducible assessment?

25
ASSESSMENT

• Were all participants who entered the trial
  accounted for at the end?
• Withdrawals/dropouts after trial entry?
• Were participants analysed in the groups in which
  they were originally randomised?

26
FOLLOW-UP

• Duration of follow-up?
• Sufficient to detect all important outcomes?

27
RESULTS

• Appropriate statistical methods?
• - Prior hypothesis-post hoc findings?
• - Adjustment for prognostic variables?
• Main results of the study?
• - How large is the treatment effect?
• - Statistical and clinical significance
  reported?

28
STATISTICAL AND CLINICAL SIGNIFICANCE

• Sample size - to detect clinically important
  difference in outcome
• Type I error (or false positive) results show a
  difference between groups when in fact no such
  difference exists
• Type II error (of false negative) results fail
  to detect a difference between groups which does
  actually exist

29
APPLICABILITY (external validity)

• Is study applicable and generalisable to other
  settings (your setting)?
• Participants similar for demographics, severity,
  co-morbidity?
• Who was included/excluded?
• Intervention relevant and practical (setting,
  cultural/geographical differences)

30
USEFUL REFERENCES

• Greenhalgh T. How to read a paper. Assessing the
  methodological quality of published papers. BMJ
  1997 315 305-308.
• Jadad AR. Randomised Controlled Trials. London
  BMJ Books, 1998.
• Cochrane Handbook for Systematic Reviews of
  Interventions http//www.cochrane.org/resources/ha
  ndbook/index.htm