Critical appraisal

1
Critical appraisal

• Terry Irwin

2
We are influenced as much if not more by the
experience of individual patients as we are by
the evidence

• Freeman BMJ 2001

3
Evidence based practice

• The process of systematically finding,
  appraising and using contemporaneous research
  findings as a basis for clinical decisions.

EBM Working Group JAMA 1992
4
Evidence based practice

• Conscientious, explicit and judicious use of
  current best evidence in making decisions about
  the care of individual patients

Sacket 1996
5
Evidence based medicine

• 1 Medline citation in 1996
• 3000 citations in 2001

6
The evidence gap

• Many gaps in knowledge
• Perceived
• Real
• These gaps are filled with decisions based on
  experience and reasoning

7
Understanding EBM

• Convert information needs into answerable
  questions
• Track down the best evidence with which to answer
  these questions
• Critically appraise the evidence for its validity
  and importance
• Integrate this appraisal with clinical expertise
  and patient values to apply the results to
  clinical practice
• Evaluate performance

8
Evidence based medicine

• Does not equate with randomised controlled
  trials!
• These are not always useful for questions about
  diagnosis, prognosis or harm, only for
  interventions

9
Guidelines

• Systematically developed
• Specific
• Evidence based
• User friendly
• Major decision points only
• Relevant to individual patients
• Relative risks often quoted
• Better to quote event rate
• Concise and accessible

Jackson and Feeder BMJ 2002
10
Medline/Pubmed

• Covers the majority of published studies
• Up to date
• Enormous
• Laborious to search
• Inexperienced searches miss most articles

11

• Cochrane
• Archie Cochrane 1972
• Established 1992
• Cochrane database of systematic reviews
• DARE
• Data of Abstracts of Reviews of Effectiveness
• National Health Centre for Reviews and
  Dissemination, York
• Primarily meta-analyses
• OMNI
• Internet gateway to systematic reviews
• Other guidelines
• National Institutes of Health
• Specialty Associations

12
Systematic reviews

• Now superseded simple narrative reviews?
• ALL available literature assessed, published and
  unpublished
• Assess each study and correlate results
• Present an unbiased assessment of the evidence
• Must be carried out meticulously
• Very time-consuming

13
Value of systematic reviews

• If all of the studies on cardiac thrombolysis
  had been systematically reviewed, the therapy
  would have been recommended in the mid 1970s.

14
Meta-analysis

• A statistical technique for combining the
  findings from independent studies
• Used to assess clinical effectiveness of
  interventions it does this by combining data
  from RCTs.
• Give more precise estimates of treatment effects
• Depends on the quality of the systematic review

15
Evidence quality

• Case report
• Audit
• Cohort study
• Comparative trial
• Randomised (controlled) trial
• Systematic review
• Meta-analysis

16
Grading of evidence

• Ia From meta-analysis or RCTs
• Ib From at least one RCT
• IIa From at least one well designed controlled
  trial without randomisation
• IIb From at least one other well designed
  quasi-experimental study
• III Comparative studies, Correlation studies and
  Case studies
• IV From expert committee reports, opinions, or
  clinical experience of respected authorities

Health Services Research Unit, Aberdeen
17
Grading of recommendations

• Levels Ia and Ib
• Levels IIa, IIb or III
• Level IV

18
Academic viva

• You can easily prepare for this part of the
  examination
• There may be no correct answer, only opinion
• There is nothing wrong with strong opinions!
• Think about the format of your journal club

19
Academic viva

• Two publications from major (surgical) journals
• Often BJS
• One hour to read and digest
• Be prepared to discuss
• Make notes
• The examiners are briefed to take one hour over
  the same task

20
Standard appraisal

• Are the aims clearly stated?
• Was the sample size justified?
• Are the measurements valid and reliable?
• Are the statistical methods described?
• Did untoward events occur?
• Were the basic data adequately described?
• Do the numbers add up?
• Was statistical significance assessed?
• What do the main finding mean?
• How are the null findings interpreted?
• Are important effects overlooked?
• How do the results compare with other studies?
• What implications does the study have for your
  practice?

21
(No Transcript)