Systematic Reviews of the Literature and Meta-analyses:

1
Systematic Reviews of the Literature and
Meta-analyses.problems or panacea?

• Daren K. Heyland, MD, FRCPC, MSc

Queens University, Kingston, Ontario
2

• Updated Jan 2009
• Summarizes gt200 trials studying 21283 patients
• 34 topics 17 recommendations

www.criticalcarenutrition.com
3
Clinical Practice Guidelines
evidence
integration of values

Validity Homogeneity Safety Feasibility Cost
practice guidelines
4
In Search of Truth...Does it work?

• Begins with a hypothesis or question
• Does Drug X reduce the incidence of problem Y in
  patients with condition Z
• Application of experimental or observational
  methods to determine the answer
• Results of our observations leads to conclusions
  that are correct (truth) or incorrect (due to
  bias or chance)

5
Levels of Evidence

• Systematic reviews
• RCTs
• Cohort Studies
• Case Control
• Case Series

less bias/strong inferences
more bias/weaker inferences
6
198 RCTs Reviewed in Critical Care Nutrition
Guidelines
7
PLOS 20085 e4
8
Learning Objectives

• Will be able to appraise and incorporate results
  of systematic reviews into clinical decision
  making.
• understand the role of systematic reviews in
  research and policy settings.
• List the strengths and weakness of meta-analyses

9
Overview

• Definition and Classification
• Usefulness
• Methodological Quality
• Making Inferences
• Conclusions

10
Systematic Review

• Form of scientific investigation to assess the
  effectiveness of healthcare interventions
• Integrative research
• Subjects original or primary studies
• Employs methods that limit bias and reduce random
  error

11
(No Transcript)
12
Systematic Reviews and Meta-analysis
Systematic Reviews
Narrative Reviews
Meta-analysis
13
Number of Systematic Reviews Published
14
The Frailties of Narrative Reviews

• If the original studies of thrombolytics
  therapies had been subject to a systematic
  review, the treatment effect would have been
  apparent in the 1970s instead of 1980s.
• Narrative reviews omitted effective therapies and
  endorsed ineffective therapies.

Antman JAMA1992268240 and Lau NEJM 1992327248
15
Clinical Decision Making and Systematic Reviews

• Case Scenario
• 77 y.o. male with presumptive Dx of Urosepsis
• PMHX MI, Prostate
• BMI 21
• After initial resuscitation
• FiO2 100, PO2 55
• MAP 65, CVP 13, levophed 20 mcg/kgk/min
• rising Cr, 20 ml of urine, acidemic
• High NG drainage
• Going to start on EN but not likely to tolerate
  Role for early supplemental PN?

16
Clinical Decision Making and Systematic Reviews

• Problem
• 100s of citations across scores of journals
  published over the last 20 years In diverse
  patient populations or diverse settings with
  variable or inconsistent results!

How do you make sense of this all?
17
Impact of Caloric Debt
Adequacy of EN

• ? Caloric debt associated with
• ? Longer ICU stay
• ? Days on mechanical ventilation
• ? Complications
• ? Mortality

Rubinson CCM 2004 Villet Clin Nutr 2005 Dvir
Clin Nutr 2006 Petros Clin Nutr 2006
18
2007 International Nutrition Practice Survey

• Point prevalence survey of nutrition practices in
  ICUs around the world conducted Jan. 27, 2007
• Enrolled 2772 patients from 158 ICUs over 5
  continents
• Included ventilated adult patients who remained
  in ICU gt72 hours

19
Hypothesis

• There is a relationship between amount of energy
  and protein received and clinical outcomes
  (mortality and of days on ventilator)
• The relationship is influenced by nutritional
  risk
• BMI is used to define chronic nutritional risk

20
What Study Patients Actually Recd

• Average Calories in all groups
• 1034 kcals and 47 gm of protein
• Result
• Average caloric deficit in Lean Pts
• 7500kcal/10days
• Average caloric deficit in Severely Obese
• 12000kcal/10days

21
Relationship Between Increased Calories and 60
day Mortality
BMI Group Odds Ratio 95 Confidence Limits 95 Confidence Limits P-value
Overall 0.76 0.61 0.95 0.014
lt20 0.52 0.29 0.95 0.033
20-lt25 0.62 0.44 0.88 0.007
25-lt30 1.05 0.75 1.49 0.768
30-lt35 1.04 0.64 1.68 0.889
35-lt40 0.36 0.16 0.80 0.012
gt40 0.63 0.32 1.24 0.180
Legend Odds of 60-day Mortality per 1000 kcals
received per day adjusting for nutrition days,
BMI, age, admission category, admission diagnosis
and APACHE II score.
22

• RESULTS WHO IS AT RISK?

23
RCT Level of Evidence that More EN Improved
Outcomes

• RCTs of aggressive feeding protocols
• Results in better protein-energy intake
• Associated with reduced complications and
  improved survival

• Taylor et al Crit Care Med 1999 Martin CMAJ 2004

• Meta-analysis of Early vs Delayed EN
• Reduced infections RR 0.76 (.59,0.98),p0.04
• Reduced Mortality RR 0.68 (0.46, 1.01) p0.06

• www.criticalcarenutrition.com

24
More is Better!
Our Field of Dream
If you feed them (better!) They will leave
(sooner!)
25
ICU patients are not all created equalshould we
expect the impact of nutrition therapy to be the
same across all patients?b
26

• What if you cant provide adequate nutrition
  enterally?
• to TPN or not to TPN,
• that is the question!

27
Current practice in nutritional support in septic
patients Results of national, prospective
multicenter German Study

• Point prevalence study
• 454 ICUs from 310 hospitals in Germany
• 399 patients septic patients included
• Median APACHE II 26
• 68 had no GI pathology
• 46 in shock
• Overall mortality 55.2

• Elke CCM 2008361762

28
Current practice in nutritional support in septic
patients Results of national, prospective
multicenter German Study
P0.005

• Point prevalence study
• 454 ICUs from 310 hospitals in Germany
• 399 patients septic patients included
• Median APACHE II 26
• 68 had no GI pathology
• 46 in shock
• Overall mortality 55.2

Multivariate analysis PN independent predictor
for mortality (OR 2.09, 95 CI 1.29-3.37)
29
Early Supplemental PN is Associated with
Increased Infection in Critically Ill Trauma
Patients

• Retrospective, multicenter, cohort study of 597
  severely injured patients
• Compared those that recd PN within 7 to those
  who did not.
• Also compared early PN group to subgroup of EN
  tolerant (tolerated 1000 kcal any day during
  first week)
• Adjusted for differences in key baseline
  demographics

Sena J Am Coll Surg 2008207459
30
Early Supplemental PN is Associated with
Increased Infection in Critically Ill Trauma
Patients
No Early PN Early PN Odds Ratio P value
Overall Adjusted
Nosocomial Infections 27 56 2.1 (1.3-3.5) P0.003
Late ARDS 1 8 3.4 (1.0-11.0) P0.04
Death 8 23 1.5 (0.8-3.0) P0.24
EN tolerant analysis
Nosocomial Infections 42 69 2.5 (1.1-5.9) P0.03
Late ARDS 2 9 5.4 (1.1-27.4) P0.04
Death 8 19 2.7 (0.8-9.3) P0.10
Differences not due to differences in glycemic
control
31
Prospective Studies of Supplemental PNEffect on
Mortality
www.criticalcarenutrition.com
32

• What if you cant provide adequate nutrition
  enterally?
• to TPN or not to TPN,
• that is the question!

33
Use of Supplemental PN in Sepsis?

• Results of meta-analysis
• Results of single RCTs of Septic Patients
• Results of observational studies
• Consideration of Individual Patient
  Characteristics

34
Using Systematic Reviews in Clinical Practice

• Summarizes large body of knowledge
• Answers specific clinical question
• Less likely to be biased than narrative reviews
• More accurate and precise estimate of treatment
  effect

35
Using Systematic Reviews in Research Setting

• Research Question
• What is the effect of Glutamine and Antioxidant
  supplementation on survival in critically ill
  patients?
• Methods
• A meta-analysis

36
Effect of Glutamine in Critically Ill A
Systematic Review of the Literature

• Comprehensive search
• Selection criteria
• Randomized
• Surgical or critically ill adults
• Glutamine (EN or PN) vs. placebo
• Clinically important outcomes

20 RCTs
37
Effect of Glutamine A Systematic Review of the
Literature
Mortality
Updated Jan 2009, see www.criticalcarenutrition.co
m
38
Effect of Glutamine A Systematic Review of the
Literature
Infectious Complications
Updated Jan 2009, see www.criticalcarenutrition.co
m
39
Effect of Glutamine A Systematic Review of the
Literature
Hospital Length of Stay
Updated Jan 2009, see www.criticalcarenutrition.co
m
40
Results of Subgroup Analysis
Mortality Infection
EN (n9) 0.81 (0.48-1.34) P0.41 0.83 (0.64-1.08) P0.16
PN (n17) 0.71 (0.55-0.92) P0.008 0.76(0.62-0.93) P0.008
PNgtgtgtEN?
41
REducing Deaths from OXidative StressThe
REDOXS study
antioxidants
Factorial 2x2 design
glutamine
R
Concealed Stratified by
1200 ICU patients
R
placebo
Evidence of
site
organ failure
antioxidants
Fed enterally
R
placebo
placebo
42
Using Systematic Reviews in Research Setting

• Summarizes what is known identifies gaps
• Background of grant proposals
• Generates hypotheses
• Estimate of treatment effect N
• Subgroup analysis

43
Using Systematic Reviews in Policy Making
As an ICU, should you make an
arginine-supplemented diet available for general
use in your institution?
44
Meta-analyses of Arginine-supplemented Diets

• 22 RCTs of IEDs
• All arginine-containing IED, not just
  IMPACT/IMMUNAID
• Non english, more recently published studies
• Excluded duplicates
• Excluded single agents

Heyland JAMA 2001286944
45
Overall Effect on Mortality

• RR 1.10 (0.93-1.31)

46
Overall Effect on Complications

• RR 0.66 (0.54-0.80)

47
1.18 (0.88,1.58)
48
Effect of Arginine-supplemented Diets in the
Critically Ill Patient
Mortality
Updated Jan 2009, see www.criticalcarenutrition.co
m
49
Effect of Arginine-supplemented Diets in the
Critically Ill Patient
Infectious Complications
Updated Jan 2009, see www.criticalcarenutrition.co
m
50
Effect of Arginine-supplemented Diets in the
Critically Ill Patient
Hospital Length of Stay
Updated Jan 2009, see www.criticalcarenutrition.co
m
51
Using Systematic Reviews in Policy Making

• Greatest generalizability
• Consistent with perspective of policy makers
• Related to other forms of integrative research

52
Assessing the Validity of Systematic Reviews
Validity fxn inputs, process, results
53
Assessing the Validity of Systematic Reviews

• Inputs
• selection of studies
• clinical homogeneity
• explicit, reproducible criteria
• methodological quality of studies
• outdated/unmeasured co-interventions

54
Assessing the Validity of Systematic Reviews

• Process
• comprehensive search strategy
• publication/timing bias
• data excess
• language bias
• judgements about inclusion explicit/reproducible
• data abstraction reproducible

55
Assessing the Validity of Systematic Reviews

• Results
• few studies
• few clinical endpoints
• statistical heterogeneity

56
Methdological Quality of Meta-analyses
lots of bias
little bias
weak inferences
strong inferences
Strong clinical recommendations
57
Making Inferences from a Meta-Analysis of RCTs
Weaker Inferences
Stronger Inferences

• Small number of trials
• Weak trial methodology
• Outdated/unmeasured co-interventions
• Surrogate endpoints
• Statistical heterogeneity
• Fixed effects model

• Large number of trials
• Strong trial methodology
• Current/documented co-interventions
• Clinically important endpoints
• Statistical homogeneity
• Random effects model

58
Meta-analysis vs. Large RCTs

• if no subsequent randomized, clinical trial,
  the meta-analysis would have led to the adoption
  of an ineffective treatment in 32 cases and
  rejection of useful treatment in 33 cases.
• LeLorier NEJM 1997337536
• I still prefer conventional narrative reviews
• Editorial, NEJM

59
Meta-analysis vs. Large RCTs
RCT 2
RCT 1
RCT 5
RCT 4
RCT 3

• RR

60
Meta-analysis vs. Large RCTs

• Argument is with Meta-analysis, not the concept
  of systematic reviews
• Assumes the latest single large trial is the GOLD
  standard
• Assumes RCT and Meta-analysis are measuring the
  same thing
• Differences in Generalizability
• Bias exists in both TOOLS.

61
Resolving Discrepancies Between a Meta-analysis
and a Subsequent Large RCT

• Recent meta-analysis found calcuim
  supplementation to be effective in preventing
  preeclampsia
• Large RCT found no risk reduction in health
  nulliparous women
• Exploration of heterogeneity across studies
• Stratify for high and low baseline risk

JAMA 1999282664
62
Resolving Discrepancies Between a Meta-analysis
and a Subsequent Large RCT
JAMA 1999282664
63
JAMA 2008300933
64
(No Transcript)
65
Role of Systematic Reviews in Medical Education

• Good source of medical knowledge
• Promotes EBM practices
• Helps locate original articles
• Facilitates critical appraisal of original
  research
• Considered a scholarly research activity

66
Conclusions

• Important tool to determine the effectiveness of
  therapeutic interventions
• Need to understand the strengths, weaknesses and
  limitations
• Useful in clinical and policy decision making and
  research setting
• Encourage use of and generation of systematic
  reviews amongst learners.