Title: For the Practice Change Fellows Program
1The Importance of Measurement in Health Care
- For the Practice Change Fellows Program
- September 25, 2008
- Washington, DC
- Dennis A. Ehrich, MD, FACC
- Vice President for Medical Affairs
- St. Josephs Hospital Health Center
- Syracuse, New York
2Agenda for the Afternoon
- 1-Why we measure in health care?
- 2-The Model for Improvement
- 3-Selecting ones measures
- 4-Time ordered statistics and understanding
variation - 5-Displaying and tracking results
- 6-Deciding whether To design a new process or
improve an existing process
3Why We Measure in Health Care
4The Model for Improvement
Set aims that are measurable, time-specific, and
apply to a defined population
Establish measures to determine if a specific
change leads to improvement
Select changes most likely to result in
improvement
Test the changes
T. Nolan et al. www.ihi.org
5The Use of Iterative PDSA Cycles
Implementing the Changes
Rapid-cycle CQI
Multiple Simultaneous Tests of Change
T. Nolan et al. www.ihi.org
6Spreading the Change
1-Executive sponsorship 2-Planning and set-up
3-Spread within the target population-social
network theory 4-Continuous monitoring and
feedback during the spread process 5-Capturing
and sharing organizational learning Â
T. Nolan et al. www.ihi.org
7Donabedians Quality Triangle-Its Relevance to
Process Improvement
-Avedis Donabedian, MD, MPH (1919-2000)
8Donabedians Triad
- Structure
- Organization
- People
- Equipment/Technology
- Process
- The steps taken in accomplishing the change and
achieving the outcome - Results must be client-focused
- Must deliver results reliably
- Outcomes
- Clinical (mortality, complications)
- Client perception or satisfaction
- Financial
9Selecting Your Measures
10The Three Domains of Measurement
- Structural Measures
- Process measures
- Outcomes Measures
- Balancing measures
Donabedian
11The Three Domains of Measurement
- Structural Measures
- Describe the environment. How many?
- Square footage of a clinical unit
- Number of staff
- Staff qualifications and competencies
- Presence or absence of technology and its
characteristics - Process Measures
- Process cycle time
- The percentage of patients for whom the process
achieves its desired result
Donabedian
12The Three Domains of Measurement
- Outcome Measures
- The impact of the change initiative on mortality,
readmissions to the hospital, ED visits - The satisfaction scores of clients and staff
- The cost per case, average LOS, revenue per case
- Balancing Measures
- Unintended outcomes that are consequences of the
new program - Unanticipated mortality, morbidity or cost
- Has the shifting of resources in an organization
compromised other client or patient populations?
Donabedian
13 Aim Selecting A Measure Operational
Definitions Data Collection Plan Data
Collection Data Analysis
The Quality Measurement Roadmap
ACTION
Modified from Lloyd, Robert Quality Health Care
A Guide to Using Indicators
14Selecting a Measure -When selecting a measure,
have clarity as to whether the measure is one of
structure, process or outcome -And select a
balanced panel of indicators that reflect the
dimensions of performance being evaluated and
the change concept(s) being employed
15What Dimension of Performance Do You Want to
Measure?
- Appropriateness
- Availability
- Continuity
- Effectiveness
- Efficiency
- Respect and caring
- Financial/Viability
- Safety
- Time lines
Joint Commission (1996)
16What Dimension of Performance do You Want to
Measure?
- Safety
- Effectiveness
- Patient-centeredness
- Timeliness
- Efficiency
- Equity
IOM Crossing the Quality Chasm (2001)
17What is the Change Concept?
- Eliminate waste
- Improve work flow
- Shorten a waiting list
- Change the work environment
- Improve the Provider/Client interface
- Manage time
- Focus on variation
- Error proofing a process
- Focusing on product or service
The Improvement Guide by Langley, Nolan, Nolan,
Norman and Provost. Jossey-Bass
18Relating a Change Concept to a Specific Measure
19- Establishing Operational Definitions That Are
Agreed Upon By All Stakeholders
20Operational Definitions
- Is clear and unambiguous
- Specifies the measurement method, procedures and
equipment when appropriate - Clinical data (chart reviews) vs. administrative
data - Client logs vs. a computer database
- Define specific criteria for the data to be
collected - Define all inclusions and exclusions
- For percentages or rates, or ratios, define the
criteria for inclusion in the numerator and
denominator - Always ask How might somebody be confused by
this definition?
Lloyd, R. Quality Health Care (2004) Jones and
Bartlett
21Examples of Unclear Definitions
- Timely completion of the screening process
- A complete medication list
- The readmission rate
- Medication error
- Cost impact
- From the acute care hospital
- A patient fall
- Surgical start time
Lloyd, R. Quality Health Care (2004) Jones and
Bartlett
22Data Analysis
- How will the measurements be expressed?
- Quantities, rates, ratios, proportions,
percentages - What type of statistics will be used?
- Descriptive statistics
- Measures of central tendency
- Mean, median, mode
- Measures of variation or spread
- Minimum, maximum, range, standard deviation
- Inferential statistics
- t-tests
- ANOVA
- Chi Square
23Data Display
- Table
- Bar chart
- Histogram
- Line chart
- Pie chart
- Pareto diagram
- Time-ordered data
- Run chart
- Control chart
24Comparative Data
- Internal targets-trended data
- External comparisons-benchmarking
- Best practices
- National or regional population averages
25External Benchmarking
Joint Commission
Calculation of the Confidence Interval
Estimates t s/ vn Where t 3 (the sigma
number for 99 confidence interval) s The
hospitals standard error of the mean and n
The number of patients in the hospitals
denominator
CMS
26Data Reporting
- Data reporting plan
- Who will receive the results?
- How often will they receive the results?
- How will it be formatted?
- Dashboard
- Paper reports
- Spider diagram
- How will the data be disseminated?
- E mail
- Internet
- Intranet
27Displaying Time-Ordered Statistics and
Understanding Variation
28Tools for Displaying Time-ordered Data
- Run charts
- Plot of data over time with the median of the
data set plotted as a center line - Control charts
- Plot of data over time with the mean as the
center line and with upper and lower control
limits
29Run Charts
- Easily constructed by hand or in available
spreadsheet programs - Provides a good idea of improvement in a change
initiative - Less sensitive to significant changes (special
cause variation) than the control chart
30Control Charts
- More sensitive to special cause variation than a
run chart - Requires specialized computer software to create
- There are 9 types of control charts used in
health care, depending upon whether the data
collected is distributed normally, is continuous
(numerical) or discreet (attributes) and whether
the events measured are frequent or infrequent - Have their own set of rules to identify special
cause variation
31Understanding Variation
- All data, collected over time, varies
- Random variation (common cause)
- The changes occurring are intrinsic to the
process being measured - Non-random variation (special cause)
- The changes are being imposed on the system by
some external factor - May be unintended and un anticipated or may be by
design - Before process improvement can be implemented,
the process must be in control (free of special
cause variation) -
32Common Cause (Random) Variation in a Run Chart
33Special Cause Variation in a Run Chart
34Special Cause Variation in a Control Chart
Daily record of Blood Pressure
Upper Control Limit 205 mmHg
Mean 173 mmHg
Lower Control Limit 142 mmHg
Special Cause Variation 138 mm Hg
35Special Cause Variation in a Control Chart
36Deciding Whether To Design A New Process or
Improve An Existing Process
37Initial Considerations
- Is the process under consideration local?
- Within a department
- On a clinical unit
- Is the project organization wide?
- A process change in a work system that impacts
the entire organization - Requires commitment of people, funds, or new
technologies
38Organization-Wide Initiatives
- Must be consistent with the organizations
Mission, Vision, and Values - Must be aligned with the organizations strategic
plan
39Strategic Goals
Q U A L I T Y
P A T I E N T s
P E O P L E
G R O W T H
M A R G I N
40Measurement and the Strategic Plan
Analyze the inputs
Obtain Inputs
Determine the organizational strategies for each
strategic goal
Determine the departmental tactics, measures, and
targets
Determine the organizational measures,
performance Targets and benchmarks
Determine HR Requirements
Formulate the IT Capital Budget
Map the data source Locate or design the system
Write the interfaces Populate the dashboards
Staffing requirements Grow or Purchase Training
requirements