For the Practice Change Fellows Program

1
The Importance of Measurement in Health Care

• For the Practice Change Fellows Program
• September 25, 2008
• Washington, DC
• Dennis A. Ehrich, MD, FACC
• Vice President for Medical Affairs
• St. Josephs Hospital Health Center
• Syracuse, New York

2
Agenda for the Afternoon

• 1-Why we measure in health care?
• 2-The Model for Improvement
• 3-Selecting ones measures
• 4-Time ordered statistics and understanding
  variation
• 5-Displaying and tracking results
• 6-Deciding whether To design a new process or
  improve an existing process

3
Why We Measure in Health Care
4
The Model for Improvement
Set aims that are measurable, time-specific, and
apply to a defined population
Establish measures to determine if a specific
change leads to improvement
Select changes most likely to result in
improvement
Test the changes
T. Nolan et al. www.ihi.org
5
The Use of Iterative PDSA Cycles
Implementing the Changes
Rapid-cycle CQI
Multiple Simultaneous Tests of Change
T. Nolan et al. www.ihi.org
6
Spreading the Change
1-Executive sponsorship 2-Planning and set-up
3-Spread within the target population-social
network theory 4-Continuous monitoring and
feedback during the spread process 5-Capturing
and sharing organizational learning  
T. Nolan et al. www.ihi.org
7
Donabedians Quality Triangle-Its Relevance to
Process Improvement
-Avedis Donabedian, MD, MPH (1919-2000)
8
Donabedians Triad

• Structure
• Organization
• People
• Equipment/Technology
• Process
• The steps taken in accomplishing the change and
  achieving the outcome
• Results must be client-focused
• Must deliver results reliably
• Outcomes
• Clinical (mortality, complications)
• Client perception or satisfaction
• Financial

9
Selecting Your Measures
10
The Three Domains of Measurement

• Structural Measures
• Process measures
• Outcomes Measures
• Balancing measures

Donabedian
11
The Three Domains of Measurement

• Structural Measures
• Describe the environment. How many?
• Square footage of a clinical unit
• Number of staff
• Staff qualifications and competencies
• Presence or absence of technology and its
  characteristics
• Process Measures
• Process cycle time
• The percentage of patients for whom the process
  achieves its desired result

Donabedian
12
The Three Domains of Measurement

• Outcome Measures
• The impact of the change initiative on mortality,
  readmissions to the hospital, ED visits
• The satisfaction scores of clients and staff
• The cost per case, average LOS, revenue per case
• Balancing Measures
• Unintended outcomes that are consequences of the
  new program
• Unanticipated mortality, morbidity or cost
• Has the shifting of resources in an organization
  compromised other client or patient populations?

Donabedian
13
Aim Selecting A Measure Operational
Definitions Data Collection Plan Data
Collection Data Analysis
The Quality Measurement Roadmap
ACTION
Modified from Lloyd, Robert Quality Health Care
A Guide to Using Indicators
14
Selecting a Measure -When selecting a measure,
have clarity as to whether the measure is one of
structure, process or outcome -And select a
balanced panel of indicators that reflect the
dimensions of performance being evaluated and
the change concept(s) being employed
15
What Dimension of Performance Do You Want to
Measure?

• Appropriateness
• Availability
• Continuity
• Effectiveness
• Efficiency
• Respect and caring
• Financial/Viability
• Safety
• Time lines

Joint Commission (1996)
16
What Dimension of Performance do You Want to
Measure?

• Safety
• Effectiveness
• Patient-centeredness
• Timeliness
• Efficiency
• Equity

IOM Crossing the Quality Chasm (2001)
17
What is the Change Concept?

• Eliminate waste
• Improve work flow
• Shorten a waiting list
• Change the work environment
• Improve the Provider/Client interface
• Manage time
• Focus on variation
• Error proofing a process
• Focusing on product or service

The Improvement Guide by Langley, Nolan, Nolan,
Norman and Provost. Jossey-Bass
18
Relating a Change Concept to a Specific Measure
19

• Establishing Operational Definitions That Are
  Agreed Upon By All Stakeholders

20
Operational Definitions

• Is clear and unambiguous
• Specifies the measurement method, procedures and
  equipment when appropriate
• Clinical data (chart reviews) vs. administrative
  data
• Client logs vs. a computer database
• Define specific criteria for the data to be
  collected
• Define all inclusions and exclusions
• For percentages or rates, or ratios, define the
  criteria for inclusion in the numerator and
  denominator
• Always ask How might somebody be confused by
  this definition?

Lloyd, R. Quality Health Care (2004) Jones and
Bartlett
21
Examples of Unclear Definitions

• Timely completion of the screening process
• A complete medication list
• The readmission rate
• Medication error
• Cost impact
• From the acute care hospital
• A patient fall
• Surgical start time

Lloyd, R. Quality Health Care (2004) Jones and
Bartlett
22
Data Analysis

• How will the measurements be expressed?
• Quantities, rates, ratios, proportions,
  percentages
• What type of statistics will be used?
• Descriptive statistics
• Measures of central tendency
• Mean, median, mode
• Measures of variation or spread
• Minimum, maximum, range, standard deviation
• Inferential statistics
• t-tests
• ANOVA
• Chi Square

23
Data Display

• Table
• Bar chart
• Histogram
• Line chart
• Pie chart
• Pareto diagram
• Time-ordered data
• Run chart
• Control chart

24
Comparative Data

• Internal targets-trended data
• External comparisons-benchmarking
• Best practices
• National or regional population averages

25
External Benchmarking
Joint Commission
Calculation of the Confidence Interval
Estimates t s/ vn Where t 3 (the sigma
number for 99 confidence interval) s The
hospitals standard error of the mean and n
The number of patients in the hospitals
denominator
CMS
26
Data Reporting

• Data reporting plan
• Who will receive the results?
• How often will they receive the results?
• How will it be formatted?
• Dashboard
• Paper reports
• Spider diagram
• How will the data be disseminated?
• E mail
• Internet
• Intranet

27
Displaying Time-Ordered Statistics and
Understanding Variation
28
Tools for Displaying Time-ordered Data

• Run charts
• Plot of data over time with the median of the
  data set plotted as a center line
• Control charts
• Plot of data over time with the mean as the
  center line and with upper and lower control
  limits

29
Run Charts

• Easily constructed by hand or in available
  spreadsheet programs
• Provides a good idea of improvement in a change
  initiative
• Less sensitive to significant changes (special
  cause variation) than the control chart

30
Control Charts

• More sensitive to special cause variation than a
  run chart
• Requires specialized computer software to create
• There are 9 types of control charts used in
  health care, depending upon whether the data
  collected is distributed normally, is continuous
  (numerical) or discreet (attributes) and whether
  the events measured are frequent or infrequent
• Have their own set of rules to identify special
  cause variation

31
Understanding Variation

• All data, collected over time, varies
• Random variation (common cause)
• The changes occurring are intrinsic to the
  process being measured
• Non-random variation (special cause)
• The changes are being imposed on the system by
  some external factor
• May be unintended and un anticipated or may be by
  design
• Before process improvement can be implemented,
  the process must be in control (free of special
  cause variation)

32
Common Cause (Random) Variation in a Run Chart
33
Special Cause Variation in a Run Chart
34
Special Cause Variation in a Control Chart
Daily record of Blood Pressure
Upper Control Limit 205 mmHg
Mean 173 mmHg
Lower Control Limit 142 mmHg
Special Cause Variation 138 mm Hg
35
Special Cause Variation in a Control Chart
36
Deciding Whether To Design A New Process or
Improve An Existing Process
37
Initial Considerations

• Is the process under consideration local?
• Within a department
• On a clinical unit
• Is the project organization wide?
• A process change in a work system that impacts
  the entire organization
• Requires commitment of people, funds, or new
  technologies

38
Organization-Wide Initiatives

• Must be consistent with the organizations
  Mission, Vision, and Values
• Must be aligned with the organizations strategic
  plan

39
Strategic Goals
Q U A L I T Y
P A T I E N T s
P E O P L E
G R O W T H
M A R G I N




40
Measurement and the Strategic Plan
Analyze the inputs
Obtain Inputs
Determine the organizational strategies for each
strategic goal
Determine the departmental tactics, measures, and
targets
Determine the organizational measures,
performance Targets and benchmarks
Determine HR Requirements
Formulate the IT Capital Budget
Map the data source Locate or design the system
Write the interfaces Populate the dashboards
Staffing requirements Grow or Purchase Training
requirements