Transforming Guidelines Into Electronic Tools

1
Transforming GuidelinesInto Electronic Tools

• Richard N. Shiffman, MD, MCIS
• Yale Center for Medical Informatics
• New Haven, Connecticut, USA

2
Overview

• Cultural contrasts
• Computer-based Decision Support
• Ways developers can help implementers
• Implementers have some tools that may help
  developers
• URL and email are on my last slide!

3
(No Transcript)
4
Suave and debonair
5
Cultural difference food
6
(No Transcript)
7
We Read Different Journals
8
We Meet in Different Places(at the same time!)
9
Language differences

• Guideline prose
• Woolly, fudgy
• Both inadvertent and deliberate
• C, JAVA, ASBRU, GLIF,
• Precise
• Ambiguity causes crashes

More About This Later
10
SAME GOALDIMINISHING INAPPROPRIATE PRACTICE
VARIATIONPROMOTING KNOWLEDGE-BASED CARE
Hip Fx
(McGlynn. NEJM 2003)
Pneumonia
Diabetes
Asthma
Colorectal Cancer
CHF
Hypertension
Senile Cataract
Receiving Recommended Care
11
Strategies

• Authors Sculpting recommendations to match
  evidence
• Implementers Operationalizing recommendations in
  a system that influences behavior

12
High Probability of Effectiveness

• Patient-specific reminder at time of consultation
• Grimshaw JM and Russell IT. Effect of clinical
  guidelines on medical practice a systematic
  review of rigorous evaluations. Lancet 1993

13
Computer-Based Decision SupportSystematic Reviews
Mary Johnston McMaster JAMA 1994
Derek Hunt McMaster JAMA 1998
Amit Garg Univ. Western Ontario JAMA 2005
Ken Kawamoto Duke BMJ 2005
Basit Chaudhry UCLA Ann Intern Med 2006
14
Findings

• Computer-based decision support regularlybut not
  alwaysimproves the process of care
• Outcomesthough infrequently measuredsometimes
  improve

15
CDSS Definition

• A system that compares patient characteristics
  with a knowledge base and then guides a health
  provider by offering patient-specific and
  situation-specific advice

CP Friedman JC Wyatt
16
Functions of CDSS
Alerting Out-of-range lab values
Reminding Schedule preventive service
Critiquing Rejecting an electronic order
Interpreting EKG
Predicting Mortality from severity score
Diagnosing Differential based on symptoms
Assisting Tailoring dosages based on renal function
Suggesting Ventilator adjustments
Randolph AG. JAMA 1999
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Thoughtful Assistant
24
(Electronic) Implementers Need More Help from
Developers

• Language
• Obstacles to Implementation
• Define Recommendation Strength

25
Translation of Guideline Knowledge for Decision
Support

• Different recommendations would be given for the
  same patient using encoded representations of
  vaccine- and breast mass workup guidelines
  formulated by different members of the (same)
  laboratory.

Patel VL.JAMIA 1998
26
Why is attention to implementability important?

• We found that it was possibleif not
  inevitablethat two encoders would encode the
  same guideline in two different ways. Sources of
  variation included differences in the order in
  which data elements are collected, differences in
  the level of detail represented, differences in
  the use of atomic sentences or composite
  sentences in criteria, differences in the
  specification of data elements, and omissions due
  to human error

27
is not contraindicated Single dose of PCV-7 for high risk children of any age
may benefit from RSV prophylaxis for infants 32 weeks gestation or less
may be beneficial Meningococcal polysaccharide vaccine for travelers
will benefit from RSV monoclonal antibody for children 24 months of age or less with hemodynamically significant congenital heart disease
may be helpful Testing for asymptomatic seroconversion after varicella exposure and receipt of VZIG
most experts recommend RSV prophylaxis for infants 32 to 35 weeks gestation with risk factors
some experts recommend Pertussis vaccine for children who have had natural pertussis
some experts suggest Duration of face-to-face contact that qualifies for significant varicella exposure
some experts prefer Serologic testing for anti-HBsAg antibody after primary vaccine series in perinatally exposed infants
some experts consider Safety of influenza vaccine during early pregnancy
experts differ in opinion HBIG for the incompletely immunized child exposed to a discarded needle in the community
the manufacturer recommends Avoidance of salicylates after varicella vaccine
28
Statement of fact is NOT a recommendation

• Adjuvant hormone therapy for locally advanced
  breast cancer results in improved survival in the
  long term.
• Clinicians should prescribe adjuvant hormone
  therapy for locally advanced breast cancer
  (when/unless?)

29
Ambiguity Resolution by classification

• Ambiguous statements are interpretable in more
  than one discrete way
• Ill meet you at the bank.
• MS morphine sulfate, magnesium sulfate
  multiple sclerosis
• LAD left axis deviation, lymphadenopathy
• Vague- lack a crisp threshold in a single
  dimension
• Fever, tall
• Underspecified - lack specificity in multiple
  dimensions
• sufficiently ill to warrant immediate
  antimicrobial therapy

30
Deliberate vaguenessand underspecification

• Insufficient evidence
• Inability to reach consensus
• Legal concerns (standard of care)
• Economic reasons
• Ethical/religious issues (e.g., concept of
  burden, futility of care)

• What if authors were transparent about
• the reason for AVUL?

31
Authors Should Be Explicit About

• WHEN under what circumstances
• WHO should
• Do WHAT
• To WHOM
• HOW
• WHY

32
AUTHORS SHOULD USE STRONG VERBS

• Active voice
• Passive masks the actor
• Appropriate choice of deontic operators
• The clinician must, must not the Committee
  strongly recommends
• The clinician should, should not the Committee
  recommends
• The clinician may the Committee suggests
• The dreaded consider

33
GLIA (GuideLine Implementability Appraisal)

• Helps to identify obstacles to implementation
• Provides feedback to guideline authors to
  anticipate and address these obstacles before a
  draft guideline is finalized
• Assists implementers to select guidelines and to
  focus attention on anticipated obstacles
• GLIA is available from http//gem.med.yale.edu/gli
  a

34
How does GLIA complement AGREE?

• Limited to issues related to implementation
• Emphasis on individual recommendation (rather
  than the guideline as a whole)
• Concordance
• Of 23 AGREE items (and 31 GLIA items)
• 5 have equivalent questions in GLIA
• 3 have similar questions in GLIA

35
GuideLine Implementability Appraisal(GLIA)

• Decidability - precisely under what circumstances
  to do something
• Executability - exactly what to do under the
  circumstances defined)
• Effect on process of care - the degree to which a
  recommendation impacts upon the usual workflow of
  a care setting)
• Presentation and formatting - the degree to which
  the recommendation is easily recognizable and
  succinct
• Measurable outcomes - the degree to which the
  guideline identifies markers or endpoints to
  track the effects of implementation of this
  recommendation

36
GLIA Constructs (2)

• Apparent validity - the degree to which a
  recommendation reflects the intent of the
  developer and the strength of evidence
• Novelty/innovation - the degree to which a
  recommendation proposes behaviors considered
  unconventional by clinicians or patients
• Flexibility - the degree to which a
  recommendation permits interpretation and allows
  for alternatives in its execution
• Computability - the ease with which a
  recommendation can be operationalized in an
  electronic information system

37
Guideline Authors are committed to appraising the
quality of scientific evidence

• As they should!
• But they regularly fall short in helping users
  understand how to use the information
• Implementers are rarely interested in evidence
  quality per se
• Implementers need authors assessment of strength
  of recommendation

38
Tripod of Concepts
Confidence (weve got benefits harms
right Evidence Quality)
Benefits vs. Harms Assessment
Importance of Adherence Recommendation Strength
39
American Academy of PediatricsGrading
Recommendation Strength
Evidence Quality Preponderance of Benefit or Harm Balance of Benefit and Harm
A. Well designed RCTs or diagnostic studies on relevant population
B. RCTs or diagnostic studies with minor limitationsoverwhelmingly consistent evidence from observational studies
C. Observational studies (case-control and cohort design)
D. Expert opinion, case reports, reasoning from first principles
X. Exceptional situations where validating studies cannot be performed and there is a clear preponderance of benefit or harm
Strong
Option
Rec
Option
No Rec
Strong
Rec
40
Clinicians and Strong Recommendations

• Benefits of the recommended approach clearly
  exceed the harms
• Quality of the evidence is excellent
• Clinicians should follow such guidance unless a
  clear and compelling rationale for acting in a
  contrary manner is present
• Optimal source for P4P

41
Clinicians and Recommendations

• Benefits exceed the harms
• Quality of the evidence on which this
  recommendation is based is not as strong
• Clinicians generally should follow such guidance
  but also should be alert to new information and
  sensitive to patient preferences.

42
Clinicians and Options

• Evidence quality is suspect or well-designed,
  well-conducted studies have demonstrated little
  clear advantage to one approach versus another
• Options offer flexibility in decision making
  about appropriate practice, although they may set
  boundaries on alternatives
• Patient preference should have a substantial role
  in influencing clinical decision making
• Hard to hold clinicians accountable (P4P)

43
Electronic Implementers and Recommendation
Strength

• Strong recommendation
• Cannot close form until issue is addressed
• Recommendation
• Recommended action is default some effort
  required to override
• Option
• Radio buttons, checkboxes for choices within
  limited range

44
(Electronic) Implementers have some tools that
can help developers

• GEM
• EXTRACTOR
• eGLIA
• Action-types
• Implementation Section

45
Logical Analysis with Highlighters

• UTI Recommendation 3
• If an infant or young child 2 months to 2 years
  of age with unexplained fever is assessed as
  being sufficiently ill to warrant immediate
  antimicrobial therapy, a urine specimen should be
  obtained by SPA or bladder catheterization the
  diagnosis of UTI cannot be established by a
  culture of urine collected in a bag. (Strength of
  evidence good) Urine obtained by SPA or urethral
  catheterization is unlikely to be contaminated...

46
XML From a small number of discrete colors to an
unlimited palette
47
XML

• Multi-platform, Web-based, open standard
• Tags enclose and describe text
• ltinclusion.criteriongthematurialt/inclusion.criterio
  ngt
• Human-readable, yet can be processed by machine
• Markup can be performed by non-programmers
• Hotconsiderable energies invested in X-tech

48
GEM

• Knowledge model for guideline documents
• GEM adopted as a standard by ASTM in 2002 GEM II
  updated and re-standardized in 2006
• Models heterogeneous information contained in
  guidelines
• Multi-level hierarchy (gt100 elements) indicates
  relationships

49
GEM II-Top Level
50
Conditional
51
GEM Cutter
52
MORO
53
GQAQ Report Card
54
Markup OnceReuse Often

• Markup with GEM Cutter Editor
• Guideline quality appraisal with GEM-Q
• Feedback to developers with Extractor

55
Feedback to authors
Decision Variable Values Decidable?
Age lt6m, 6m-2yr,gt2yr
Attend day care Y, N
Recently treated with abx Y, N
Severe illness Y, N
Followup can be assured Y, N
Action Action Executable?
Amoxicillin 45 mg/kg/d in 2 divided doses Amoxicillin 45 mg/kg/d in 2 divided doses
Observation is appropriate Observation is appropriate
56
Action-Types in405 Recommendations
Essaihi A. Proc AMIA 2004
Action
57
Action Distribution
No Rec
Prepare
Advocate
Document
Monitor 3
Conclude 3
Dispose 4
Refer 5
58
Example Application of Action-Types

• Action-type Prescribe
• Drug information
• Safety alerts (allergy, drug-drug, drug-disease,
  drug-lab)
• Formulary check
• Dosage calculation
• Pharmacy transmission
• Patient education
• Corollary orders

59
Key Messages

• Developers and implementers (esp electronic)
  represent different cultures
• Electronic DSS work
• Electronic implementers need help from developers
• Watch your language!
• Anticipate problems in implementation GLIA
• Define recommendation strength
• Implementers have some tools that may help
  developers
• GEM a standardized document representation
• EXTRACTOR
• Action-typing to define recurring patterns

60
Thank You!

• http//gem.med.yale.edu
• richard.shiffman_at_yale.edu