Evidence in Practice Understanding systematic reviews

1
Evidence in PracticeUnderstanding systematic
reviews

• Poole House, Nunthorpe Thursday 15th June 2006
• Amanda McGough
• Clinical Effectiveness Lead
• Hambleton and Richmondshire PCT
• Evidence in Practice Group
• Amanda McGough, John Blenkinsopp, Tony Roberts,
  Anne Lister
• www.evidencebasedpractice.org.uk

2
What is a systematic review?

• A research study in itself
• Protocol
• Researchers/clinicians
• Gathers data
• Classed as secondary research as it draws data
  from other research papers
• Primary research gathers data direct from
  population

3
Why bother with reviews?

• Sheer volume of literature published each year
• Different studies may show differing results
• New information
• Good systematic review should help to explain

4
(No Transcript)
5
Publication bias

• A tendency among investigators, peer reviewers
  and journal editors to allow the direction and
  statistical significance of research findings to
  influence decisions regarding submission and
  acceptance for publication.

6
Measures of effectiveness

• These measures can be expressed as Relative
  Risks, Odds Ratios and Hazard Ratios
• If the ratio is 1 then the effects of the
  intervention are the same as the effects of the
  control group
• If the effect is greater than 1 then the effect
  in the intervention group is greater than in the
  control group
• NOTE The effect may be good eg more
  breastfeeding or bad eg stop breast feeding
• So if the outcome is mortality, youre looking
  for ratios smaller than 1

7
Death after heart attack
Cumulative Year RCTs Pts 1960 1965
Odds Ratio
2
1
0.5
?
1 23 2 65 3 149 4 316 7 1793
?
?
?
?
Treatment Control
8
Death after heart attack
Cumulative Year RCTs Pts 1960 1965
1970 1975 1980 1985 1990
Textbook Recommendations Rout Specif Exp NOT
Odds Ratio
2
1
0.5
?
1 23 2 65 3 149 4 316 7 1793 10 2544 11 2651 15 33
11 17 3929 22 5452 23 5767 27 6125 33 6571 65 4718
5 70 48154
21 5 10 2 8 7 8 12 4 3 1 1
?
?
?
1 1 2 8 7 2
?
?
p lt 0.01
?
?
?
1 1 1 2 8 1
?
?
?
p lt 0.001
?
5 15 6
?
?
p lt 0.00001
?
Treatment Control
9
Reviews of heart attack treatments
The clinical value of thrombolysis... remains
uncertain. Oxford Textbook of Medicine 1987
10
The Cochrane Collaboration
Preparing, maintaining and disseminating systemati
c reviews of the effects of health care
11
Whats important in appraising reviews?

• Establish the PICO structure of the systematic
  review
• Does it match your PICO?
• Appraise for validity
• have the right sort of studies been included?
• have all the trials been included?
• have the included studies been appraised for
  quality?
• If the results have been combined statistically
  (meta-analysis), were the studies similar enough
  to justify this?

12
Whats important in appraising reviews?

• Look at the results
• Absolute Risk Reduction
• Number Needed to Treat
• Odds Ratio
• Role of 95 Confidence Intervals
• Estimate the risk to your patient(s)
• Consider all the other issues when you make your
  decision

13
How does the evidence help?
14
Imagine this scenario

• Your CHD network is trying to persuade local
  Trusts and PCTs to develop and commission a
  package of care to manage the care of patients
  with heart failure.

15
Group work

• Evaluate the rigour of the review process
• Have a go at calculating the numbers
• Consider the 95 Confidence Intervals - you need
  to be able to interpret them
• Use the nomogram for different scenarios
• Have fun!