Measurement for Improvement

1
Measurement for Improvement
18 March 2008
Mike Davidge
2
The prologue



What actually is the question?



3
aims measurements interventions
4
How does measurement help us?

• By showing the variation that lives within our
  data (and therefore our processes)
• By indicating whether the way we react to that
  variation is taking us in the right direction

Source Robert Lloyd IHI 2006
5
The measurement journey
6
Why are we measuring?
Research
Improvement
Judgement
The answer to this question will guide our entire
measurement journey
7
Measurement for judgement
waiting over 4 hours in AE England by week
2003/04
target
The week Trusts were measured for performance
ratings
8
Outpatient waiting times
9
The 3 reasons for measurement
Why is measurement for improvement different?
Source Robert Lloyd IHI 2006
10
Take two numbers
11
Take two numbers
12
29
12
We have 2 quarterly data points - is this an
improvement?
Higher is better
13
Are we assuming something like this?
14
But it could be like this ...
15
Or this ...
16
Or this!
17
Case study Housing repairs

• Example courtesy of John Seddon in Freedom from
  Command Control

18
The scenario

• A Local Authority manages a large stock of
  housing
• They manage repairs using 2 measures
• Budget
• Time to repair
• Time to repair is a government mandated BVPI
  (best value performance indicator) measured in
• emergencies completed in 24 hrs
• urgents completed in 7 days
• routines completed in 28 days
• The LA achieving all BVPIs

19
Hunky dory?

• What does the customer want?
• Repair done quickly and properly
• How do we know we are delivering this?
• Lets measure the end to end time from request
  to job complete
• But theres a problem the IT system couldnt
  provide the data they only had the BVPI
  percentages so had to track each job manually

20
SPC chart showing true performance
21
Not so hunky dory

• Special causes not different system unstable
• Achieve targets by distorting the data system
• Ignore if tenant is out
• Reclassifying emergencies as urgents
• Splitting a job into several, each one opened
  closed
• Got a problem? Have a management restructure!
• Actually destabilised the system even more

22
So what next?

• They understood they had a problem
• They looked at the process and discovered it was
  broke
• 40 failure demand at the call centre
• Tradesmen scheduling work to maximise earnings
• Queuing to get the right materials
• Led to redesigning the process
• Within weeks all repairs completed within 8 days

23
They didnt know they had a problem until they
measured the right thing in the right way!
24
And that reminds me of

• What gets measured gets managed

even when its pointless to measure and manage
it and even if it harms the purpose of the
organisation to do so.
25
The 3 reasons for measurement
Why is measurement for improvement different?
Source Robert Lloyd IHI 2006
26
national
aims measurements interventions
Save 1000 lives and 50,000 episodes of harm
HSMR and Global Trigger Tool
6 Content Areas
Source IHI
27
local
aims measurements interventions
Use the measures and interventions for each of
the content areas
28
How to measure?
Start collecting the data and using it for local
decision making

• C
• A
• R

• ollect the right data in a sustainable way
• nalyse the right measures presented in a
  helpful way
• eview whether your changes are having the
  desired effect

29
The How questions

• How many of your project leads can
• Collect the relevant data
• Analyse data effectively (create run charts)
• Review it regularly (interpret charts and make
  the right decisions)
• How much time do your Information analysts have
  to support them?
• How many of the content area measures can be
  obtained via your corporate IT systems?

30
In summary

• Measures follow aims clarity here means you
  know what 42 means
• Measurement for improvement is different from the
  other types of measurement we are used to
• Plot your data over time to understand current
  situation and track changes
• Ensure your project teams have the right
  measurement skills
• Re-align your Information department to provide
  longer term sustainability