Automating Clinical Practice Guidelines with ATHENA Decision Support System

1
Automating Clinical Practice Guidelines with
ATHENA Decision Support System

• Knowledge-based clinical decision support
• Mary K. Goldstein, MD, MS in HSR
• VA Palo Alto and Stanford University
• ATP III Conference, Pittsburgh

2
Disclosures/Disclaimers

• Views expressed are those of the speaker and not
  necessarily those of the Department of Veterans
  Affairs or other funding agencies or affiliated
  institutions
• Speaker does not have commercial interest in
  systems discussed
• See later slide for grant funding
• Jointly owned by VA and Stanford hope to develop
  for open-source
• Reference list in handout
• Many papers cited are openly available in
  full-text through pubmedcentral
• Patient data in slides is artificial data, not
  actual patient data
• Screenshots are intended to show general layout
  rather than detail

3
Acknowledgements

• ATHENA Project Leadership
• Mary K. Goldstein, MD, MSc VA Palo Alto Health
  Care System and Stanford University Dept of
  Medicine (Principal Investigator)
• Brian B. Hoffman, MD, VA Boston-West Roxbury and
  Harvard Medical School (Co-Principal
  Investigator)
• ATHENA Decision Support System Development
• Stanford Medical Informatics EON Group
• Mark Musen, MD PhD Samson Tu, MS, and others
• ATHENA Group at VA Palo Alto and Stanford
• Bob Coleman, MS Pharm Susana Martins, MD, MS
  Lond
• Many other collaborators

4
Funding Sources

• Development of ATHENA DSS
• NLM LM05708 (PI Musen) for development of EON
  architecture and collaboration on building ATHENA
  DSS, built with EON Technology for
  guideline-based decision support systems
• VA HSRD Career Development Award for Dr.
  Goldsteins time VA Palo Alto Health Care System
  for other staff time
• Implementation and Clinical Trials
• VA HSRD CPI 99-275 "Guidelines for Drug Therapy
  of Hypertension Multi-Site Implementation(PI
  Goldstein/Hoffman) and IMV 04-062 VISN
  Collaborative for Improving Hypertension
  Management with ATHENA-HTN (PI
  Goldstein/Hoffman)
• Also used in separate clinical trial at Durham
  VAMC (PI Oddone/Bosworth)
• VA Palo Alto Health Care System IRMS for support
  in integration with VistA/CPRS
• NLM funding for Stanford Medical Informatics EON
  Group collaboration in implementation and further
  testing

5
Objectives for this Talk

• Describe a knowledge-based clinical decision
  support system
• Discuss issues in implementing real-time clinical
  decision support in a health-care system
• Explain cluster-randomized design for clinical
  trial of new IT implementation

6
Presentation Outline

• Describe ATHENA DSS
• Clinician perspective
• Underlying technology
• Knowledge-based systems and Knowledge Management
  in an organization
• Issues in Implementation
• Working with the organization to implement
• Patient safety in new health IT
• Evaluation

7
What is ATHENA-DSS?

• Automated clinical decision support system (DSS)
• Knowledge-based system automating guidelines
• Built with EON technology for guideline-based
  decision support, developed at Stanford Medical
  Informatics
• For patients meeting eligibility criteria for
  particular system
• Hypertension, chronic pain management
• Designed to be integrated with existing EHR
• Provides patient specific information and
  recommendations at the point of care
• Uses more of patients clinical data than typical
  reminders
• Potentially can provide information to health
  care professionals and/or to patients
• Purpose is quality improvement

Athena in Greek mythology is a symbol of good
counsel, prudent restraint, and practical insight
8
HTN advisory
9
See also Goldstein and Hoffman, in Hypertension
Primer, J.L. Izzo, Jr and H.R. Black, Editors.
2003, Williams Wilkins Baltimore.
10
Patient SummaryWindow
11
Evidence Summaries
ACP
BMJ Clinical Evidence
From earlier version of ATHENA-HTN
Reprinted with permission BMJ Publishing Group
12
Experience with ATHENA-DSS

• ATHENA-Hypertension
• Developed, implemented, evaluated
• Clinical trial over 15-months at three medical
  centers
• 96 clinician-clusters, 11,000 patients, 40,000
  clinic visits
• Now installing for study at 5 new medical centers
• ATHENA-Opiate Therapy
• About to start first clinical trial

13
Sites for First Clinical Trial

• Palo Alto (in 7 cities), San Francisco,
  and Durham VAMCs (total 9 separate sites)

San Francisco VA, CA
Palo Alto VA, CA
Durham VAMC, North Carolina
14
Steps to Implementing

• Develop automated knowledge base in relevant
  clinical domain (hypertension)
• Map patient data variables to knowledge base
  (guideline) terms
• Test offline with guideline interpreter for
  accuracy of advisories generated
• Set up system to extract patient data
• Set up system to display advisories within
  electronic health record (EHR)
• Test in clinics with key clinicians
• Goldstein et al JAMIA 2004

15
ATHENA Architecture
Electronic Medical Record System Patient Data
VISTA hierarchical Database in M
Treatment Recommendation
CPRS
Guideline Interpreter
ATHENA HTN Guideline Knowledge Base
SQL Server Relational database Data Mediator
16
Steps to Implementing Knowledge Base

• Develop automated knowledge base in relevant
  clinical domain (hypertension)
• Protégé knowledge acquisition tool
• Model the knowledge of medical conditions, drugs,
  etc
• Model the clinical algorithm as clinical
  scenarios with action choices
• Protégé is available as an open-source download

• http//protege.stanford.edu/download/download.html
  

• Goldstein et al Proc AMIA 2000

17
Example ontology in Protégé
Classes define concepts in the domain
Slots define attributes and relationships
Facets define constraints on slots
18
ATHENA Protégé top level
ATHENA Hypertension
Blood Pressure Targets
Patient risk categories
Drug classes
19
ATHENA Protégé GL managementdiagram
20
Indexing Medical Knowledge

• Easy for clinicians to inspect and modify
• Allows specification of generic decision
  criteria
• Prefer drugs that have compelling indication

21
Use of Concept Model Defining Guideline-Specific
Concepts
22
Data Standards in ATHENA-DSS

• The meta-concepts in the EON model used for
  ATHENA knowledge bases are not limited to
  specific data standards
• Can use the data standards used in the underlying
  clinical information source
• For ATHENA-HTN in VA, we use
• ICD 9 codes and CPT codes
• LOINC codes for lab values
• VA National Drug file codes for pharmacy

23
Knowledge Management (KM) in ATHENA Project

• Initial Knowledge Acquisition (KA)
• ATHENA team member encodes published guideline
  into a knowledge base (KB) using Protégé
• Generates Rules document detailing the encoded
  information in very specific narrative in English
• Process reveals areas of insufficient
  specificity, conflicting information, etc
• Conference calls with domain experts to resolve
  questions/discrepancies
• Knowledge base revised offline testing performed
• Knowledge Maintenance
• Periodic conference calls to identify new
  knowledge (eg from new clinical trials) that has
  been vetted for use
• Cycles of discussions with domain experts,
  updates to KB, offline testing prior to fielding
  updated system

24
Steps to Implementing

• Develop automated knowledge base in relevant
  clinical domain (hypertension)
• Map patient data variables to knowledge base
  (guideline) terms
• Test offline with guideline interpreter for
  accuracy of advisories generated
• Set up system to extract patient data
• Set up system to display advisories within EHR
• Test in clinics with key clinicians

25
Overview of Offline Testing
Martins et al Proc AMIA 2006
26
Steps to Implementing

• Develop automated knowledge base in relevant
  clinical domain (hypertension)
• Map patient data variables to knowledge base
  (guideline) terms
• Test offline with guideline interpreter for
  accuracy of advisories generated
• Set up system to extract patient data
• Set up system to display advisories within EHR
• Test in clinics with key clinicians

27
ATHENA Architecture
Electronic Medical Record System Patient Data
VISTA hierarchical Database in M
Treatment Recommendation
CPRS
Guideline Interpreter
ATHENA HTN Guideline Knowledge Base
SQL Server Relational database Data Mediator
28
Building ATHENA System From EON Components
M (Mumps) extract
VA VISTA (DHCP)
EON Servers
SQL Patient Database
VA CPRS
ATHENA Clients
Temporal Mediator
ATHENA Clients
Event Monitor
Event Monitor
Pre- computed Advisories
Guideline Interpreter
Data Converter
Advisory Client
Advisory Client
ATHENA GUI
nightly data extraction
ATHENA HTN Guideline Knowledge Base
Protégé
29
Steps to Implementing

• Develop automated knowledge base in relevant
  clinical domain (hypertension)
• Test offline with guideline interpreter for
  accuracy of advisories generated
• Set up system to extract patient data
• curly braces problem
• Set up system to display advisories within EHR
• Test in clinics with key clinicians
• Deploy in clinics and run several months

30
Patient Safety in New Health IT

• Health IT has potential to improve patient
  safety
• But also is an opportunity to introduce new forms
  of error
• Cheng et al Proc AMIA 2003
• Important to design for patient safety
• Offline testing before deployment
• Systems for early detection of problems
• Monitoring system after wider deployment
• Goldstein et al JAMIA 2002

31
Monitoring the System

• Automated checks
• Unrecognized drugs in patient data
• Errors in automated data transfers
• Feedback from clinician users
• Comment window on top screen
• Allows problems to be detected early
• Also allows analysis of reasons for non-adherence
  to guidelines
• Chan et al, AHRQ Pt Safety, 2005
• Lin et al, Proc AMIA, 2006

32
Attention to Organizational Context

• Project goals in alignment with organizations
  goals
• E.g., Performance standards
• Iterative technical design in response to
  organizational input
• Translating Research into Practice, JAMIA 2004

33
Evaluation of Clinician Reaction

• Clinicians used the system extensively
• Data logged by system
• Speaks to usability and usefulness
• Clinicians reported ATHENA-HTN affected their
  prescribing decisions
• Questionnaire data

34
Evaluation of Impact on Prescribing

• Clinical trial for formal evaluation
• Random allocation
• To control for secular trends
• Cluster-randomization
• Patients clustered within clinics
• Potential contamination within provider if
  randomized by patient
• Unit of delivery of the intervention is provider,
  or even clinic/site
• Unit of analysis should match unit of
  randomization
• Unit of ascertainment may be patient

35
Deploying Client
n 40
n 193
n 118
36
Important Findings

• This project has demonstrated the feasibility of
  integrating automated decision support in a
  complex clinical domain for chronic disease
• The tools used for this decision support system
  can be used to develop decision support for other
  clinical domains
• ATHENA-Opiate Therapy for managing chronic pain

37
Summary

• Knowledge-based decision support systems can
  generate real-time clinical advisories with
  patient-specific information
• Deploying system requires working with the health
  care organization
• Evaluation must take account of clustering of
  patients within provider

38
Additional Slides and Literature Citations in
Handout
39
Hypertension Model Program

• We selected hypertension as a model for guideline
  implementation because
• Hypertension is highly prevalent in VA population
  and other adult medical practice
• There are excellent evidence-based guidelines for
  management
• There is also evidence that the guidelines are
  not well-followed

40
Clinical Decision Support

• Point-of-care decision support
• One component of guideline implementation
• Provides information to the clinician at the time
  of medical-decision-making
• Potential clinical decision support systems can
  be evaluated along multiple dimensions (see next
  slide)

41
Dimensions of Decision Support
Uses existing electronic data
Complex logic and reasoning
No explanation or evidence presented
Knowledge In code
Transferable to diverse EMRs
Requires access to separate system
Integrated into clinical workflow
Limited to single EMR
Presents explanation / evidence-base for
recommendations
Requires separate data entry
Knowledge easy to browse/update
Limited to simple rules
42
(No Transcript)
43
Triggers for ATHENA Pop-Up
Goal The right patient, the right clinician, in
the right clinical setting

• Patient Selection for nightly data extract from
  Vista (current configuration)
• Hypertension diagnosis
• Scheduled Primary Care Clinic Visit 5-day
  window
• Hypertension Advisory displayed as a pop-up
  window in CPRS-GUI on client computers installed
  for it or in Thin Client if user login configured
  for it
• Primary Care clinician (in primary care clinic)
  logs onto CPRS-GUI and opens record of patient
  with advisory available

44
Who Can Modify What

• Clinical-Administrators
• Knowledge in KB, for example
• Strength of indications and contraindications
• Warnings/messages
• Formulary preferred drugs
• Evidence Summary
• Triggers for the pop-up window
• For example
• Markedly elevated BP only
• All hypertensive patients
• No trigger clinician activation only

• Clinician End-Users
• Patients clinical data
• Update BP
• Add or delete
• Diagnoses
• Meds
• ADRs
• Labs
• As What-If not saved to CPRS at present except
  for option to save BP

45
Guideline Encoding Standards

• The HL7 Clinical Guideline Special Interest Group
  has not yet agreed on a standard for encoding
  guidelines
• EON architecture allows alternative
  decision-criteria languages
• ATHENA-HTN (built with EON) uses a set of
  templates for encoding common expressions
• These could be replaced by similar templates that
  can be seen as stereotypical GELLO expressions
• For some medical domains, need additional
  expression capability beyond GELLO for temporal
  abstractions and comparisons
• such as duration of uninterrupted
  antihypertensive monotherapy
• Tu SW and Musen MA MEDINFO 2001