The International Liaison Committee on Resuscitation (ILCOR) Process for Evidence Based Review and the Role of Public Comment

1
C0009 NRP Current Issues Seminar Monumental
Changes on the Horizon

• The International Liaison Committee on
  Resuscitation (ILCOR) Process for Evidence Based
  Review and the Role of Public Comment
• Myra H. Wyckoff, MD
• UT Southwestern Medical Center at Dallas
• Co-Chair of ILCOR Neonatal Task Force 2016-2020

2
Faculty Disclosure Information In the past 12
months, I have no relevant financial
relationships with the manufacturer(s) of any
commercial product(s) and/or provider(s) of
commercial services discussed in this CME
activity.I do not intend to discuss an
unapproved/investigative use of a commercial
product/device in my presentation.
3
Objectives

• Understand the purpose of ILCOR
• Understand how PICO questions are identified and
  refined for systematic review
• Identify components of the GRADE process that is
  used by ILCOR for systematic reviews
• Understand where to find the systematic reviews
  on the AHA ILCOR website
• Understand the role of public comment when the
  systematic reviews are posted

4
Achieving Consensus on Resuscitation Science

• Since 2000, the AAP with the American Heart
  Association, participates with the International
  Liaison Committee on Resuscitation (ILCOR) for a
  complete review of resuscitation science every 5
  years.
• 23 new questions reviewed for 2015

5
ILCOR 2015 Neonatal Working Group
6
Guidelines for Neonatal Resuscitation

• New Consensus on Science and Neonatal
  Resuscitation Guidelines available online since
  October 14, 2015
• Printed Guidelines supplement published in
  Circulation, Resuscitation and in Pediatrics
• Download at www.heart.org/cpr

7
NRP Guidelines are set by the AAP NRP Steering
Committee
NRP 7th Edition will rollout in Spring 2016
8
Ask An Answerable Question

• Development of clear, important questions for
  which to search for evidence is paramount
• PICO format
• Patient (the exact population)
• Intervention (usually experimental group)
• Comparator (usually the usual practice or a
  placebo)
• Outcome (impact on what)

9
ILCOR Evaluation Process

• Identify and develop the PICO questions that need
  scientific review
• May have to be refined after initial evidence
  search
• Rank Outcomes of interest for importance
• Examples
• Time to return of Heart Rate gt 100 bpm
• Survival
• Neurodevelopmental Outcome at 2-3 years of age
• Temperature on Admission
• Need for Intubation

10
ILCOR Evaluation Process

• Prioritize the questions and assign reviewers
• Resources are finite (Particularly TIME!!!)
• Minimum requirements for every search strategy
  are specified and done by professional librarians
• Medline, Embase, and Cochrane Systematic Reviews
• Hand searches
• EndNote reference software and access to a
  resuscitation reference library
• Every reviewer rates the level and quality of
  evidence using a standardized evidence evaluation
  (GRADE system).

11
Understanding GRADE

• GRADE Most widely used method for appraising
  studies to be included in systematic reviews and
  guidelines
• Recommended by Institute of Medicine-to give a
  common process and language for published
  guidelines
• GRADE is a method used by systematic reviewers
  and guideline developers to assess
• the quality of evidence
• Decide whether to recommend an intervention
• Grade is different from other appraisal tools
  because it
• Separates quality of evidence and strength of
  recommendation
• The quality of evidence is assessed for each
  outcome of interest
• Observational studies can be up-graded if they
  meet certain criteria

12
Using GRADE

• Step 1
• Assign an a priori ranking of high to
  randomized controlled trials and low to
  observational studies.
• Randomized trials are initially assigned a higher
  grade because they are usually less prone to bias
  than observational studies

13
Using GRADE

• Step 2
• Downgrade or upgrade initial ranking.
• It is common for randomized controlled trials and
  observational studies to be downgraded because
  they suffer from identifiable bias.
• Also, observational studies can be upgraded when
  multiple high-quality studies show consistent
  results.

14
Reasons to Down Grade in Step 2

• Risk of Bias
• Lack of clearly randomized allocation sequence
• Lack of blinding
• Lack of allocation concealment
• Failure to adhere to intention to treat analysis
• Keeps folks from switching to the treatment they
  believe is best
• Large losses due to follow-up
• Exaggerates effect size. There is a positive bias
  conferred when participants benefitting from the
  treatment are more likely to stay in the trial

15
Other Reasons to Down Grade in Step 2

• Inconsistency
• Variability in results from different trials
• Hard to tell if benefits outweigh the harms
• Indirectness
• Population studied is not exactly the population
  we want to apply the data to
• Outcome that is available in the literature is
  not exactly the outcome we care about

16
Other Reasons to Down Grade in Step 2

• Imprecision
• When wide confidence intervals mar the quality of
  the data
• Publication Bias
• When negative findings remain unpublished.

17
Reasons to Up Grade in Step 2

• Large Effect
• When the effect is so large that bias common to
  observational studies cannot possibly account for
  the result
• Dose-response relationship
• When the response is proportional to the degree
  of exposure

18
GRADE Step 3

• Step 3 Assign final grade for the quality of
  evidence as
• high We are very confident that the effect in
  the studies reflects the actual effect
• moderate We are quite confident that the
  effect in the studies is close to the true
  effect, but it is also possible it is
  substantially different
• low The true effect may differ significantly
  from the estimate
• very low The true effect is likely to be
  substantially different from the estimated effect

19
GRADE Step 4

• Step 4 Consider other factors that impact on the
  strength of the recommendation for a course of
  action
• Balance between desirable and undesirable effects
• In cases where the benefit to harm ratio are less
  clear, then patient values and preferences, as
  well as costs need to be carefully considered

20
GRADE Step 5

• Step 5 Make a strong or weak recommendation
• For or against

21
GRADE

• Unless the recommendation is based on moderate or
  strong evidence.
• We need to be advocating for more research
• The recommendations should not squelch research
  

22
An Example ILCOR Evidence Review Using the GRADE
System
23
Umbilical Cord Milking(NRP 849)

• EVREV 1 Marya Strand, MD, MS COI222
• EVREV 1 Takahiro Sugiura, MD COI224
• Taskforce Neonatal

24
COI Disclosure

• Marya Strand, MD, MS COI222
• Commercial/industry
• None
• Potential intellectual conflicts
• None
• Takahiro Sugiura, MD COI224
• Commercial/industry
• None
• Potential intellectual conflicts
• None

25
2010 Treatment Recommendation

• Evidence from a single, small, randomized
  controlled trial (Hosono 2008) suggest that
  among premature newborns (24-28 weeks gestation),
  milking the umbilical cord immediately after
  birth may decrease the need for packed red blood
  cells transfusions during the initial hospital
  stay. This single study was not adequately
  powered to fully assess the safety or efficacy of
  this intervention.
• Secondary analysis of data from this small
  randomized trial (Hosono 2009) suggests that
  among premature newborns (24-28 weeks gestation),
  milking the umbilical cord immediately after
  birth may increase hemoglobin at birth, improve
  hemodynamic stability during the first 5 days of
  life, and decrease the incidence of chronic lung
  disease at 36 weeks postmenstrual age.

26
2015 PICO

• Outcomes
• 9Infant death
• 8Neurodevelopmental outcome at 2-3 years
• 8Severe Intracranial Hemorrhage
• 7All grade Intracranial Hemorrhage
• 7Cardiovascular stability (initial MBP)
• 7Temperature on admission
• 6Hematologic indices
• Initial hemoglobin, PRBC transfusion
• 6Hyperbilirubinemia
• Phototherapy, Exchange transfusion

• Population Very preterm infants (28 wks EGA)
• Intervention Umbilical cord milking
• Comparison Immediate umbilical cord clamping

27
Inclusion/Exclusion Articles Found

• Inclusions/Exclusions
• Randomized or observational studies
• Infants born at 28 weeks gestation
• Search terms initially identified 690 potential
  articles
• Number Included in Evidence Profile tables
• RCTs 4
• non-RCTs 1

28
Draft Treatment Recommendations

• There is insufficient published human evidence to
  suggest the routine use of cord milking for
  infants born at less than 29 weeks of gestation
  but cord milking may be considered a reasonable
  alternative to immediate cord clamping to improve
  initial mean blood pressure, hematological
  indices and ICH. However, there is no evidence
  for improvement or safety in long term outcomes.
  (Weak recommendation, low level of evidence)
• All studies included in this evidence review
  milked 20 cm of umbilical cord toward the
  umbilicus 3 times while the infant was held at
  the level of the introitus or below the level of
  the placenta prior to cord clamping.

29
Risk of Bias in Studies
RCTs
Non-RCTs
30
Outcome- Infant Death
RCTs
Non-RCTs
31
Evidence Profile Table-Infant Death
RCTs
Non-RCT
32
Outcome-Intracranial Hemorrhage (ICH)
ltSevere ICH-RCTgt
Any ICH-RCT
Any ICH non-RCT
33
Evidence Profile Table-ICH
RCT
Non-RCT
34
Outcome - Hematologic Indices
Initial hemoglobin RCT
Initial hemoglobin non-RCT
Received transfusion RCT
35
Evidence profile table -hematologic indices
RCT
Non-RCT
36
Proposed Consensus on Science Statements

• For the critical outcome of death we found low
  quality evidence (downgraded for imprecision)
  from 3 RCTs Hosono 2008 Katheria 2014, 1045
  March 2013, 1 that there is no difference in
  death (OR 0.76, 95 CI 0.25-2.29).
• We did not identify any evidence to address the
  critical outcome of neurologic outcome at 2-3
  years.
• For the critical outcome of cardiovascular
  stability we found low quality evidence
  (downgraded for imprecision) from 2 studies
  Hosono 2008 Katheria 2014 that the initial
  mean blood pressure was 5.43 mm Hg higher
  (1.98-8.87 mm Hg) in the intervention group.

37
Proposed Consensus on Science Statements

• For the important outcome of IVH, we found low
  quality evidence (downgraded for imprecision)
  from 2 RCT studies Hosono 2008 March 2013 of a
  reduction of any IVH (OR 0.37, 95 CI 0.18-0.77)
  but no difference (from 1 study Hosono 2008) in
  severe IVH (OR 0.44, 95 CI 0.07-2.76).
• For the important outcome of hematologic indices,
  we found low quality evidence (downgraded for
  imprecision) from 2 studies Hosono 2008
  Katheria 2014 that cord milking increased the
  initial hemoglobin measurement (2.27 g/dl, 95 CI
  1.57-2.98 g/dl) and low quality evidence
  (downgraded for imprecision) from 3 studies
  Hosono 2008 Katheria 2014 March 2013 that
  cord milking decreased need for transfusion (OR
  0.2, 95 CI 0.09-0.44).

38
Draft Treatment Recommendations

• We suggest against the routine use of cord
  milking for infants born at less than 29 weeks of
  gestation but cord milking may considered a
  reasonable alternative to immediate cord clamping
  to improve initial mean blood pressure,
  hematological indices and ICH. However, there is
  no evidence for improvement or safety in long
  term outcomes. (Weak recommendation, low level of
  evidence)
• All studies included in this evidence review
  milked 20 cm of umbilical cord toward the
  umbilicus 3 times while the infant was held at
  the level of the introitus or below the level of
  the placenta prior to cord clamping.

39
Values and Preferences Statement

• In making this recommendation we place a high
  value on the simplicity/economy of this
  intervention with no demonstrated negative
  outcome, acknowledging the lack of evidence
  regarding critical long-term outcomes.

40
Knowledge Gaps

• Specific research required
• There are insufficient subjects in the reviewed
  studies to make strong recommendations for or
  against this intervention. We need the results of
  several on-going international trials that are
  not yet completed.
• We need information on important long-term
  neuro-developmental outcomes
• Cord milking vs Delayed cord clamping
• One RCT Rabe 2011 demonstrated similar
  hematologic indices between cord milking and
  delayed cord clamping.

41
How many of you heard about the opportunity for
the public to review the ILCOR Preliminary
Reviews and offer comment?

• 1-Yes, I heard about it!
• 2-No, I heard nothing

42
The Role of Public Comment

• All reviews posted to the AHA SEERS website 2
  weeks prior to the Consensus Conference 
• The public, especially those who teach and use
  resuscitation skills but were not part of the
  ILCOR process were encouraged to visit the SEERS
  home page at http//www.ilcor.org/seers to view
  these recommendations (and to provide feedback).
•  Comments submitted by the time of the CoSTR
  meeting were discussed in the CoSTR debates.
  Public commenting was open until February 28,
  2015 and was reviewed prior to finalizing wording
  of ILCOR CoSTR document.

43
How many of you went to the AHA website and
posted a public comment regarding a Neonatal
ILCOR Review?

• 1-Yes, I did it!
• 2-No, did not happen

44
ILCOR CoSTR Meeting

• The in-depth review for each of 23 PICO questions
  was completed and presented in Dallas in Feb.
  2015
• Neonatal Task Force debated the evidence
  reviewers findings until consensus reached on
  wording for
• the consensus on what the available science meant
• treatment recommendation
• values and preferences that went into making that
  treatment recommendation
• and knowledge gaps
• This wording used to build the ILCOR CoSTR
  Document

45
Conclusions

• The ILCOR evidence review process is long but
  results in comprehensive, meaningful review of
  the best science to apply to our resuscitation
  algorithms.
• The work is never done. The ILCOR Neonatal Task
  Force meets this December to begin debate on the
  most pressing PICO questions to review for 2020.

46
Potential Change in Your Practice

• YOU can play an important part in the process by
  offering public comment both regarding the PICO
  questions selected as well as the preliminary
  evidence reviews and treatment recommendations.
• Public Invitation will be sent through AAP list
  serves including the NRP Instructor Database