The Measurable and or the Meaningful

1
The Measurable and / or the Meaningful

• Dee Mangin and Les Toop
• IPAC 2008

2
Current regrettable situation

• Stand off between clinicians and their
  representative organisations and the DHBs /
  Ministry on how and by whom clinical practice in
  primary care in New Zealand should be determined
• Make no mistake. This is huge and will not go
  away anytime soon

3
(No Transcript)
4
Presentation Outline

• The background and the Players
• Two models
• Measurable vs Meaningful indicators
• Why the Angst?
• Contrasting the models
• Consequences, intended and unintended
• Who chooses the model
• IF P4P who chooses which indicators

5
Everyone wants

• The highest quality care for all
• To prevent illness and premature death where
  possible
• To alleviate suffering when it occurs
• A public who are satisfied with their care
• A healthy and vibrant health workforce
• Most, not all, would add
• To do the above in an equitable fashion within
  the constraints of finite resources

6
The Battleground of Control

• Disagreement between partial funder and provider
  about how best to achieve essentially the same
  goals
• Two distinct models have emerged in recent times
• Pay for Performance UK QOF, PMP
• If you want something particular done in General
  Practice you must specify and pay for it, how it
  is achieved is of lesser importance
• Professional educative model
• If you provide professionals with evidence,
  audit comparative feedback, an opportunity to
  discuss with their peers and an appropriate
  incentive, they will do the right thing

7
The Players

• GROUND UP
• Patients
• GPs and PNs
• IPAs
• Colleges

• TOP DOWN
• The population
• Politicians
• Ministry of Health
• DHB(s)
• Public Health lobby

PHOs
8
Blurring Boundaries of General Practice and
Public Health
9
Driven by a Series of Tensions

• Population statistics / Patient centredness
• Professionalism / Managerialism
• History / Ideology (moving feast)
• Perverse partial funding streams
• Culture of mutual mistrust
• Complexity vs impatience
• Misuse of a paucity of poor quality data

10
The Measurable and / or the Meaningful
11
Difficulty with Clinical Performance Indicators
Measurable
Process outcomes (outputs)
Biological outcomes
Health outcomes
Meaningful
12
Prime example of data misuse and the mistrust it
engenders
13
(No Transcript)
14
Van der Lei stated the first law of informatics

• Data shall be used only for the purpose for
  which they were collected
• And the collateral
• If no purpose was defined prior to the
  collection of data, then those data should not be
  used
• Simon de Lusignan, Chris Mimnagh
• Breaking the first law of informatics
• the Quality and Outcomes Framework
• (QOF) in the dock
• Informatics in Primary Care 2006141536

15
(No Transcript)
16
(No Transcript)
17
STATIN UPTAKE BY DHB PHARMAC DATA
18
(No Transcript)
19
(No Transcript)
20
It appears that

• Funders here and overseas, most notably in the
  UK, have made a unilateral ideologically driven
  decision that General Practice will go down a
  managerialist path of central external
  micromanagement using (pay for) clinical
  performance indicators.
• The obvious questions and the ones causing all
  the current angst that follow are
• which indicators and who chooses?

21

• Perhaps it is time to pause, draw breath and
  reflect on the differences of this experiment
  with what was the emerging alternative evidence
  based approach and to examine the possible
  consequences

22
Attributes Pay 4 Performance

• Based on providing EBM
• Lends itself to contracts and accountability
• Can tick the quality box and sigh with relief
• Quick results to easy targets
• Aligned to population health needs
• Content driven by state priorities and limited to
  what is measurable
• Tendency to value what is recorded rather than
  outcome
• Single disease model
• Shared decision making not incorporated
• Based on mistrust and Doctor coercion
• Model has weak evidence base for effectiveness
• Open to gaming

23
Attributes Pay 4 Professionalism

• Evidence based - encourages critical thinking
• Real time responsiveness to changes in evidence
• Strong(RCT) evidence base local and internl
  evidence - results in change in clinical practice
• Uses the measurable in audit and feedback but not
  as targets
• Content driven by meaningful clinical and local
  priorities
• Patient preference not penalised
• Is low risk and low cost/opportunity cost
• Quality tick has to be given to a process not
  measures
• Balance of control clinician groups - requires
  trust
• Less incentive to generate data
• Not very exciting

24
Content driven by state priorities and limited to
what is measurable

• Performance something you do when someone is
  watching you
• the risk is that other areas will be seen as
  unimportant and neglected

25
UK Study

• We looked at travel medicine, and realised that
  it did not require to be delivered in terms of
  the contract Why are we doing this? Who is
  paying us to do this? So we dont do it anymore.

26
Enforcing a Single Disease Model
27

• About half of those over 65 have at least 3
  coexisting chronic conditions. About one in five
  have 5 or more
• 80 of people who die from heart disease are now
  over 65
• Hypothetical 79 year old woman
• COPD
• Type 2 diabetes
• Hypertension
• Osteoarthritis
• Osteoporosis

Boyd et al Jama 2005
28

• 19 doses of 12 different medications
• Taken at five times during the day
• 14 non pharmacological activities
• 10 different possibilities for significant
  medicine interactions either with other medicines
  or other diseases

29

• Conclusions
• Standards that define quality of patient careby
  placing emphasis on high rates of adherence to
  guidelines and targets rather than weighing the
  burden, risks and benefits of complex therapies
  in shared decision making could ultimately
  undermine quality of care.

30

• OR
• What seems quality care from the perspective of
  the population, the Ministry, and the DHB is not
  necessarily in the best interests of the
  individual
• If it turns to custard, who bears the
  responsibility?

31
Coercive Approach

• Persuasion
• where a person becomes convinced by a series of
  logical arguments
• Coercion
• where decisions are influenced by a credible
  threat

32
We developed this zero tolerance of blood
pressure. No-one is allowed to say Its a little
bit up, leave it . its not acceptable. Senior
GP Roland M, et al. Primary Health Care
Research and Development 2006 7 70-78
33

• I find myself considering whether to start an
  elderly female patent on a fourth
  antihypertensive in order that she will fall as I
  predict she will and I can then exception report
  her in order to maintain the QOF target

UK Professor of General Practice
34

• Theres a real drive to get GPs to lower blood
  pressure as much as possible. Over Christmas I
  found my aunt who is on antihypertensives
  collapsed on the floor. My father had had the
  same experience. They are clearly being
  overtreated but my father had to lower the drugs
  himself as his GP wouldnt. This sort of
  treatment policy cannot be beneficial
• Dr J Moncreiff UK Psychiatrist

35

• are you doing this for me doctor or am I doing
  this for you?

36
Shared decision making

• Everyone has the right to personalised advice
  that is evidence informed and has at its heart
  shared decision making. Choosing to follow the
  non advised path is a fundamental right.

37
Gaming

• The QOF cuff
• The QOF arm
• Denominator fixing
• Real (text field) and QOF values
• Repeated measures

38
Letter from America

• theyre really annoyed as theyre held
  accountable for patient behaviours they cant
  influence
• ..also since some patients refuse or get
  annoyed when chlamydia screening is raised, some
  doctors are starting to ask women to bring in a
  urine sample and not telling them that theyre
  screening for chlamydia in order to meet the
  target

39
Evidence

• QOF seems to be large scale and expensive
  experiment with no protocol, no informed consent,
  no control group, no measurement of the adverse
  effects, and hence no measure of the net effects.
  Unfortunately, uncontrolled experiments don't
  seem to require ethics committee approval.
• Paul P Glasziou, Professor, Evidence Based
  Medicine University of Oxford BMJ 29 Nov 2007

40
Shepherd et al NEJM 2007
41
Guthrie et al 2007
42
Family Practice Feb 2008
43
Plt0.001
QOF
15
44
(No Transcript)
45
Other Studies

• Stroke high/low QOF points did not predict
  adherence to guideline recommendations in manual
  note search (audit) Williams et al Informatics
  in Primary Care 2006
• Range of conditions inconsistent effect on
  mortality / morbidity (observational study)
  deprivation more predictive Dowing et al BMC
  HSR 2007
• Secondary care heart disease (RCT in US) no
  change in clinical behaviour
• Glickman et al JAMA 2007
• Electronic record implemented if P4P criteria
  (RCT in US)
• Lee et al Health Policy Reform and Performance
  Improvement 2007

46
RCT Evidence

• Interactive and mixed educational sessions are
  associated with a significant effect on practice
  (SES 0.67, 95 CI 0.01, 1.45)
• Audit and feedback academic detailing moderate
  effect size
• Cochrane Review
• In addition
• Peer led small group education with audit and
  feedback statistically significant effect within
  3 months off intervention. Effect size 7-40.
• Richards et al Family Practice 2003

47
Augmentin vs Amoxil (tablets)
Richards, Toop et al Family Practice.
200320(2)199-206
48
For Liver Function Tests
Pegasus Health 2006
49
Other questionsnet benefit vs net harm

• Workforce
• Workforce capacity- opportunity costs of time,
  money, motivation and loss of innovation
• Workforce skills critical thinker vs box ticker
• Trust vs mistrust
• Control and responsibility

50
Unresolved questions

• Which outcomes?
• How is the measurable best used?
• Who decides?
• Contractual obligation vs professional
  responsibility?
• What is the best use of taxpayer money?
• When will general practice be viewed as part of a
  solution rather than part of the problem?

51
Essentials common to both models

• Good IT systems
• Direct access to good quality data
• Access to independent evidence
• Audit and comparative feedback
• Better integration with secondary care

52

• P4P is a model for promoting evidence based
  quality improvement in general practice
• an often poorer quality imitation of the real
  thing"

53
P4Performance

• Record of smoking status
• Record of smoking cessation offered

54
P4Professionalism (1999)

• NRT Quit rates in GP only 10 (c/f literature
  14-22)
• Programme improvement, flexible team delivery
• 3670 enrolled in 2000 in 94 member practices (227
  GPs)
• Implemented in research design (cohort study)

55
Quit rate

• 6 month quit rate 36 (95 CI 31-41)

Richards DA et al NZMJ NZMJ 2 May 2003
56
The world gone mad From latest QOF update UK

• Smoking 4 The percentage of patients with any
  one or any combination of the following
  conditions coronary heart disease, stroke or
  TIA, hypertension, diabetes, COPD, CKD, asthma,
  schizophrenia, bipolar affective disorder or
  other psychoses who smoke whose notes contain a
  record that smoking cessation advice or referral
  to a specialist service, where available, has
  been offered within the previous 15 months.

57
A. GP

• Support me in providing the tools I need for
  patient centred decision making that is evidence
  informed

58
(No Transcript)
59
The greatest challenge facing contemporary
medicine is for it to retain or regain its
humanity- its centre- without losing its
essential foundation in scienceto find a middle
way. James Willis. The Paradox of Progress.
1994.