QOF Assessment a refreshed look

1
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

2
QOF Assessment a refreshed look

• Rebecca Thornley
• Primary Care Contracting Advisor
• East of England

3
House keeping

• Fire Exits
• Mobile phones
• Refreshments available at 11.10am and 3.15pm
• Lunch will be served from 1.00pm 1.45pm
• Aim to finish at 4.00pm

4
Aims of the day

• These events are aimed at QOF Assessor Teams,
  Clinical Governance Leads, Data Analysts, PCT QOF
  Leads and SHA Primary Care Leads
• Revisiting the programme of education and support
  for PCT QOF Leads
• Workshops will highlight the lessons learnt from
  QOF since its implementation in April 2004 and
  will specifically cover -
• QMAS and QOF Assessor Reports
• Sharing approaches to QOF
• Working with Internal Audit
• Examples of good practice
• Data triangulation and analysis

5
Refreshed resources

• Events
• Website
• Research
• Communications

6
Events

• During September and October PCC will be running
  6 regional events
• Pick up via QOF networks additional sharing of
  approaches/good practice/problem solving
• Additional, locally tailored events can be
  organised via your local Primary Care Contracting
  Advisor

7
Website

• The current QOF resources available via the PCC
  website require some review and refreshment
• It is proposed that the section itself be
  extensively revamped to place it as a major
  information and learning site to PCTs and act as
  a living resource which is regularly
  maintained, has an interactive content and would
  include -
• Detail on what PCTs can and should do
• Description of model techniques from the
  assessment to prepayment verification and counter
  fraud processes and the relationships between
  them PCT good practice
• Access to NatPaCT QOF training resources
• New models of data management
• It is hoped that the refreshed web resource will
  be available during November

8
Research

• PCT documentation
• Questionnaire to PCT QOF Leads
• Responses to this questionnaire will be used in
  future policy development and negotiation

9
Communications

• Establish QOF Newsletter
• maintain an efficient and effective means of
  communicating with PCT QOF Leads
• First edition planned for October/November 2006
• This resource would be emailed to QOF Leads on a
  bi-monthly basis and would include -
• Examples of good practice
• Internal Audit/Counter Fraud updates
• QMAS briefings and updates
• Information Commission downloads
• FAQs
• Please go to Registration if you would like to
  sign up to receive this publication

10
www.primarycarecontracting.nhs.ukrebecca.thornle
y_at_pcc.nhs.ukTelephone 07770444025
11
Lunch and Post-its

• You now have the opportunity to post questions
  and/or what you would like to cover in the
  breakout sessions
• Post on board for specific topic
• Data Analysis
• QMAS and QOF Assessor Reports
• Internal Audit and PCT good practice

12
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

13
QOF MIXTURESAssessment and quality

• Philip Leech
• Consultant Adviser for Primary Care

14
Assessment review Culture and
quality Assessment, PPV and 5 checks
15
What we did

• Gather material
• Questionnaire
• Call for templates and protocols
• Gather opinions from key stakeholders
• Established an expert group

16
What we found

• Everyone trying hard
• Some really very good work but
• no consistency
• not much of a result
• a lot of worry
• baffled by the data

17
Why?

• Everyone in turmoil
• Budgets
• Job
• Loss of experience
• Loss of priority

18
So, what are we going to do?

• New explanation of purpose
• New tool(s)
• Document extracts
• New materials on the website
• Road shows
• Follow up

19
Progress so far.
20
Changes in management of coronary heart disease
1998-2003
Campbell et al. British Medical Journal 2005
331 1121-1123.
21
Percentage of practices reaching 80 cervical
cytology target Baker et al. J. Epidemiology and
Community Health 2003 57 417-423
22

• Major UK initiatives
• National standards
• Clinical governance
• Annual appraisal
• Contracts
• Public release
• Patient safety
• Collaboratives
• Inspection

100 quality
All of these things - no magic bullet
quality achieved
Baseline quality
23
Practice performance in first year of new
contract
Quality points per practice, out of a maximum of
1050
www.ic.nhs.uk/services/qof
24
QOF..

• Supports quality of care in an ordered and
  systematised way that will produce hugely better
  patient outcomes
• Will help to develop a high quality public health
  database
• produces a too heavily weighted bonus payment
  that has inhibited other developments in practice
• risks neglect of clinical areas not incentivised
  (e.g. osteoporosis).

25
Issues in assessor review Practice culture

• If you deliver good clinical care the points
  should look after themselves
• We certainly will not stop just when we have
  achieved an indicator
• We need to ensure patients not represented in QOF
  get just as good care
• We need to know about all our patients, so we
  will try to use exception reporting properly
• We want to use QOF information to improve our
  services and our commissioning

26
Issues in assessor reviewPCT culture

• This is about patient care, not financial balance
• This is an opportunity to build relationships
  with practices the annual review is only a
  punctuation mark in a continuous process of
  quality improvement
• This is the chance to develop the practice for
  the next year and integrate QOF with other
  quality initiatives
• This is the way to show fairness, with firmness

27
Issues in assessor review Process

• Annual review .will involve significant
  preparation and organisation for the PCT
• Delivering Investment in General Practice,
  December 2003, Section E (pg 74)

28
Issues in assessor review Process

• What to do..
• Prepare!
• Who to involve
• Read the documentation
• Know the rules
• Understand the data
• Grade A evidence
• Prescribing
• Referral
• Public health
• Assessor reports

29
Issues in assessor review Process

• Knowledgeable (context and quality)
• Trained (generic, specific), and expert
• Clinician
• Internal audit
• Quality manager
• Lay person
• Accountable
• Manager
• Clinician
• (Lay person)

30
Issues in assessor review Process

• What to do..
• Prepare!
• Who to involve
• Read the documentation
• Know the rules
• Understand the data
• Grade A evidence
• Prescribing
• Referral
• Public health
• Assessor reports

31
Issues in assessor review Process 7 documents

• GMS Contract 2003
• Delivering Investment in General Practice
• DH December 2003
• General Guidance on QOF Annual Review Process
• DH 12-05-04
• Technical Annex to Annual QOF Review Process
• DH 12-05-04

• SFE Consolidated April 2006 onwards
• Revisions to GMS Contract 2006/07
• PCCA Practice QOF Assessment Visits

32
Issues in assessor review Process

• Indicators
• Previous year(s) QOF Report
• PACT data Pharmaceutical Advisors
• PH data PH team
• HES data PCT
• Prevalence
• QMAS Exception Reporting, Patient Experience

33
Issues in assessor review Process

• QOF Assessor Validation Report
• Grade A evidence B and C as well
• OOHs data link to HES
• Clinical Audits (PCT and Practice)
• Business Rules
• Local knowledge
• PALs patient complaints

34
Issues in assessor review Process

• What to do..
• Think about the service the patient is getting
• Be consistent across the PCT
• Have a quality dialogue
• Stand up for yourself
• Feed back at board level
• Keep smiling you can go back if you need to

35
Issues in assessor review Process

• What not to do..
• Use expensive, paid for clinician time to do
  mundane analyses
• 20 sets of notes for every indicator
• No discrimination between practices
• Tick boxes
• Shift position
• Continue if there is something bad
• Be put off by confidentiality issues

36
Issues in assessor review Training and tools

• Generic and specific
• QMAS
• QOF assessor validation reports
• Auditors tools
• Guidance

37
Consider

• QOF Visits, PPV and 5
• Confidentiality
• Establishing Accuracy
• Exception reporting
• Expected prevalence

38
Exception reporting for clinical indicators

• Patient refused
• Not clinically appropriate
• Newly diagnosed or recently registered
• Already on maximum doses of medication

39
Other issues

• How does QOF fit with other incentives?
• QOF developments - what is it best at doing?
• How could we improve QOF?

40
If you think I made myself clear, you must have
misunderstood what I said (Greenspan to Congress)
Philip_at_primarycareleads.com
41
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

42
QOF Assessment a refreshed look

• REFRESHMENTS

43
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

44
QOF Assessment a refreshed look 2006
QOF Assessor Validation Reports Their place in
the practice annual quality review
Cheryl Cowley Primary Care Programme NHS
Connecting for Health
45
Purposes of QOF Annual Review Visit

• to review the contractors current achievement
  and to provide the PCT with an assessment of
  achievement by March 31
• to confirm that data collection and quality (and
  hence any payments made on the basis of this
  data) are accurate
• to discuss the contractors aspiration for the
  following year
• Ref General Guidance on QOF Annual Review
  Process DH 12-05-04

46
What is data?

• Data
• A group of known, given, or ascertained
  facts.. (Ref Penguin dictionary)
• Data item
• A single unit of data (Ref Peter Collins
  dictionary of IT)

47
Information versus data

• DATA
• stored facts
• inert (it exists)
• technology based
• gathered from various sources

• INFORMATION
• presented facts
• active (it enables doing
• business based
• transformed from data

48
Data, information, knowledge

• 1234567.99 is data
• Your bank balance has jumped 8087 to 1234567.99
  is information
• Nobody owes me that much money is knowledge
• .and wisdom?

49
Purposes of QOF Annual Review Visit

• The first purpose entails verification,
  which will therefore be an element of the visit
• The second purpose, to be meaningful, entails
  consideration of learning, development and
  support needs, and other problem solving.
• Ref ScHARR GMS QOF Annual Review Visits
  Emerging Conclusions 2004

50
(No Transcript)
51
Information sources for the practice QOF Review
Process

• Pharmacist PACT data
• Director of Public Health Local Public Health
  data
• National v local disease prevalence
• PCT team Hospital Episode Statistics (HES) data
• Grade A submitted evidence previous years
  evidence and achievement
• PCT local practice clinical governance agenda
• PCT protected time clinical meetings

52
Information sources for the practice QOF Review
Process (cont.)

• Practice clinical meetings
• PCT and practice clinical audits
• PCT data quality facilitator knowledge/activity
  local IT training needs assessments
• Local deanery practice engagement
• Out of hours data
• PALS - Patient complaints patient praise
• QMAS monthly reporting functionality
• QOF Assessor Validation Reports

53
PCT Collating Evidence

• Once the PCT has received and analysed this
  information it will identify areas for
  discussion. It is sensible to resolve as many
  issues as possible with the contractor before the
  visit to ensure the agenda is kept to a minimum
  of key issues forming the agenda for the visit.
• Ref General Guidance on QOF Annual Review
  Process DH 12-05-04

54
Purpose of QOF Assessor Reports

• Support decision of PCT for
• intensity of the visit
• need for visit
• Email suite of reports to QOF Lead prior to the
  visit
• Evaluation of reports prior to visit
• Enable weekly reports
• Minimise resource gaining consent

55
Code of Confidentiality

• Where practices are unable to anonymise
  records, those who act on behalf of PCTs are
  entitled to access non-anonymised records either
  for NHS management purposes (if they are
  themselves under a duty of confidentiality) or to
  check a persons legal entitlement to payment. It
  is not necessary for practices or PCTs to inform
  individual patients that individual records will
  be accessed for such purposes, nor to seek the
  consent of individual patients to obtain access
  to individual records in such circumstances.
  Nor, in the view of the Department, would such
  access constitute a breach of the data Protection
  Act."

56
Three hurdles.

• Software has been procured for the purpose
• Informed patient consent
• If 1 or 2 not possible then anonymise records
• ONLY if 1, 2, and 3 cannot be achieved

57
Communications

• All PCT QOF Leads
• Initial briefing bulletin July 2005
• Second briefing with
• - Produce data validation reports for QOF
  Assessors
• Establishing Accuracy in QOF data
• NHS CFH website

58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
Process

• Practices download and install software
• EMIS and Microtest patched
• Slow connections sent a CD
• Reports scheduled remotely
• Reporting wizard
• View reports
• Email reports
• Run another set of reports
• Before visit practices email reports to QOF Lead

65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
QOF Records Report

• Random selection 20 patient records e.g
• CHD 1 Stroke/TIA 2 Hypertension 1 Diabetes 0
  COPD 2 HF/LVD 1 Thyroid 1 Depression 2 MH 0
  CKD 0 Dementia 1 4 or more repeats 1 Cancer 1
  Palliative care 0 Pts from pract pop 1 Learning
  disabilities 1 AF 3
• Females 8 Males 12
• Age bands lt20 1 20yrs 0 30yrs 1 40yrs 1
  50yrs 4 60yrs 6 70yrs 4 80yrs 3 90yrs 0

73
QOF Records Report..cont

• Random selection 20 patient records e.g.
• CHD 3 Stroke/TIA 1 Hypertension 2 Diabetes 3
  COPD 1 HF/LVD 2 Thyroid 1 Depression 2 MH 1
  CKD 1 Dementia 1 4 or more repeats 1 Cancer 2
  Palliative care 1 Pts from pract pop 2 Learning
  disabilities 1 AF 1
• Females 11 Males 9
• Age bands lt20 1 20yrs 1 30yrs 0 40yrs 4
  50yrs 6 60yrs 4 70yrs 2 80yrs 1 90yrs 1
• Last 2 years data entry

74

• www.connectingforhealth.nhs.uk/qof/
• Cheryl.Cowley_at_cfh.nhs.uk

75
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

76
QOF Assessment a refreshed look

• QA Session

77
Lunch and Post-its

• You now have the opportunity to post questions
  and/or what you would like to cover in the
  breakout sessions
• Post on board for specific topic
• Data Analysis
• QMAS and QOF Assessor Reports
• Internal Audit and PCT good practice

78
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

79
Data Analysis as part of the Assessment Process

• Jill Burke
• Insight Solutions

80
Who are we?

• Established since 2001
• Independent IT training consultancy
• Not affiliated to any of the clinical suppliers
• Designated Clinical System trainers for all major
  systems
• Working with 2500 practices
• Working in over 75 PCOs/LHBs
• Accredited training Company
• main contract software suppliers
• Welsh Assembly Government
• nGMS Contract Training specialists
• IT Training Specialists for Primary Care

81
Ethical Disclaimer

• All information provided during this presentation
  is an opinion and is, therefore, optional. It is
  designed to enhance your abilities to work with
  the nGMS Contract should you wish
• It is based on how QoF 2 stands today! There are
  likely to be many changes, it is the
  responsibility of the practice to ensure they
  keep up-to-date with any future changes made
• The decision to implement any changes rest
  entirely with each practice/clinician

82
Why?

• Assessments in the past have often been routine
• Assessors going through learning curve
• Practices expecting you to look at patient data
  for evidence
• Largest investment in Primary Care
• Practice income higher than ever before

83
Where are we now?

• QoF2
• Established a process of assessing
• Identified a need to move this forward
• 2 years of back data and information from QoF
• years of other information
• Prescribing data
• Prevalence
• 2 years experience of assessing
• But not necessarily using all the information
  together

84
Where do we want to be?

• Assessing practices
• Accurately, consistently and relevantly
• Justifying payments
• Confident practice have achieved as a result of
  hard work
• Not just ticking boxes

85
Benefits outside of QoF?

• Chronic Disease management
• Powerful information for commissioning of
  services
• Community Matrons
• Admissions Prevention
• Practice Based Commissioning
• Accurate Disease Prevalence
• Budget Setting
• Better Patient Services
• Patient Pathways

86
Data AnalysisA new dimension
87
Three Approaches

• Comparing expected and observed prevalence rates
• Identifying high levels of exception reporting
• Comparing prescribing and prevalence rates
• Approaches will identify lines of enquiry not
  definitive answer to questions of gaming
• Making assessments worthwhile

88
1. Prevalence Rates

• QMAS reports crude prevalence rates not
  adjusted for risk factors (age, sex ethnicity
  etc)
• Very high / low rates might be explained by these
  factors
• Doncaster PCT have generated a tool that refines
  expected prevalence rates for 7 conditions based
  on practice level socio-demographic factors

89
Data required

• Practice Level
• Reported prevalence rates (QMAS)
• Age sex breakdown
• Deprivation
• Ethnicity
• PCT will need to data entry into spreadsheet
  time consuming but worthwhile results

90
Model Output
91
(No Transcript)
92
(No Transcript)
93
2.Exception Reporting

• Exception reporting codes concerns
• over-use
• under use
• misuse
• Comparison can be made with reference to PCT
  average level of exception reporting and size of
  disease register
• QMAS provides data on levels of exception
  reporting

94
Automatically identifies indicators where the
practice varies from PCT average
95
Locate data in QMAS
96
Download into Spreadsheet
97
A few simple calculations
98
Produce Control Chart
99
(No Transcript)
100
3. Comparing Prescribing and Prevalence Rates

• One would expect strong association between
  prevalence rates and prescribing rates for
  chronic diseases
• Variance from expected might indicate inaccurate
  registers
• School of Medicines Management (Keele Uni) have
  investigated this approach

101
CHD Register Size vs Chapter 2 Prescribing
EXAMPLE
102
CHD Register Size vs Chapter 2 Prescribing
Poor Data Quality? Patients not identified on
Registers
Poor Data Quality Inflated registers?
103
Practice to benefit

• If prevalence is high and incorrect, practice
  targeted to deliver services to patients who not
  in need
• More work
• Less success
• Fewer points
• If prevalence is low and incorrect , practice may
  be doing the work but not getting recognition
• Less per point

104
Who benefits?

• Patient
• Practice
• PCT

105
Patient to Benefit

• Correct services offered to relevant patients
• Patient care to improve
• Practices dont waste patients time

106
PCT to Benefit

• Correct investment of public money
• Patient services improved
• Practice achievement improves

107
Other possible checks

• Hypertension prevalence vs anti-hypertensive
  prescribing
• BP5 vs anti-hypertensive large amount of points
• Asthma and COPD prevalence vs BNF Chapter 3
  prescribing
• Diabetes prevalence vs diabetic prescribing
• Epilepsy prevalence vs anti-epileptic prescribing
• CHD8 compared with lipid lowering prescribing

108
What to target

• With so many areas / indicators / disease
  registers be selective
• Identify which registers / indicators are best
  for target
• High points potentially expensive for PCTs if
  there is concern
• Speciality within practice/PCT
• Diabetes
• Asthma etc.

Use data analysis to establish this
109
High Point Indicators

• CHD5 BP less that 150/90 - 19
• CHD8 Cholesterol less than 5 - 17
• BP4 BP recorded in past 9 months - 20
• BP5 BP less than 150/90 - 57
• DM20 HbA1c 7.5 or less - 17
• MH9 review in past yr - 23
• Asthma3 variability/reversibility - 15
• Asthma6 review - 20
• DEM2 review - 15
• DEP2 assessment - 25
• AF3 anticoagulation/platelet - 15
• Smoking - 68

110
Best use of data

• Pointless to have all this data and use it in
  isolation
• Cross referencing all sources
• E.g. Prescribing versus prevalence
• Prescribing versus achievement
• Exception reporting versus Prevalence
• Practice to practice comparisons

111
Summary

• Data will need to be collected, prepared,
  analysed and understood before the assessment
• Assessors should have the questions ready to ask
• May need practice input for some data
• Standard approach across PCT for all practices
• Target areas pre defined
• Training if required
• Assessors need to see the benefit
• May require specific training e.g. Excel

112
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006

113
Good practice and internal audit
QOF
John Dixon
114
Good practice and internal audit
Our

115
Life in a cardboard box
116
Ideals

• Rigour
• Fair and consistent
• Summative and formative element
• Clinical domains
• Non-clinical domains
• Other data
• Triangulation
• Use beyond the QOF

117
Feasibility constraints

• Rigour
• Fair and consistent
• Summative and formative element
• Clinical domains
• Non-clinical domains
• Other data
• Triangulation
• Use beyond the QOF

• Rigour
• Fair and consistent
• Summative and formative element
• Clinical domains
• Non-clinical domains
• Other data
• Triangulation
• Use beyond the QOF

• Resources
• Value for

• Rigour
• Fair and consistent
• Summative and formative element
• Clinical domains
• Non-clinical domains
• Other data
• Triangulation
• Use beyond the QOF

• Resources

• Resources

118
Value for ?

• More efficient data capturing
• Augmented relationships with practices
• Sharing of good practice
• Learning PCT and practices
• .and the long term VFM of the QOF?

119
Value for ?

• give and take!

120
(No Transcript)
121
Cost 2004/5
61,000
122
Cost 2005/6
61,000
37,000
123
Cost 2006/7
61,000
37,000
21,000
124
Front office
Back office
v
125
Variance

• "Almost all measures lose variance and hence the
  capacity to discriminate good from bad
  performance

MEYER, M. (2002) Finding performance The new
discipline in management. In A. NEELY (ed.)
Business Performance Measurement Theory and
Practice. Cambridge University Press.
126
Variance
Number lt 90 max
3 lt 950
2 lt 495
14 lt 166
127
Back office systems

• Communication data recording
• Risk analysis of non-clinical indicators
• Comparative analysis of prevalence exception
  reporting data
• Pre-payment verification process
• Patient experience indicators
• Organisational knowledge

128
Communications and data recording

• Paperless system
• Standardised electronic (Excel) recording system
  / audit trail (QVR)
• Single entry of data
• Generic visit agenda
• No pre-visit meetings
• No post-visit meetings
• Standard visit report generated from initial data
  entry
• All communication by email
• All reports, verification records available
  electronically and e-indexed electronic QOF
  guidelines
• Pre-payment verification reminder of outstanding
  indicators generated electronically form original
  record

129
(No Transcript)
130
Space for final request
131
Appropriate comment re outstanding indicator
entered on to PPV Summary reflects findings after
visit
132
(No Transcript)
133
Risk analysis
134
Prevalence and exception reporting

• What is normal?
• What is acceptable?
• What demands clarification?

135
Prevalence and exception reporting
QMAS
Practice prevalence and exceptions data
Establish national and PCT average ( normal)
EXCEL
DEFINE
Acceptable range average /- x
Acceptable number of domains outside range
Practices with data which requires clarification
136
Prevalence and exception reporting

• 8 practices had prevalence rates falling at least
  15 below from the national average in 7 or more
  clinical domains
• 6 practices had prevalence rates falling at least
  10 below from the PCT average in 7 or more
  clinical domains
• 6 practices had exception rates falling at least
  20 above the national average in 7 or more
  clinical domains

137

• All variation limits set to 100

138

• Prevalence variation limit National raw prev
  rate /- 15
• Prevalence variation limit PCT average raw prev
  /- 10
• Exception variation limit National average /-
  20

139
(No Transcript)
140
Raw exceptions data extracted from QMAS
141
Exception rates
142
Prevalence data extracted from QMAS
143
Prevalence rates
144
Pre-Payment Verification Aspiration v Achievement
145
Audit ( Consistency)

• QOF assessment team
• Communication
• Quality assurance panel
• External auditors
• Internal auditors
• Cross PCT 5 random counter fraud checks
• The future.is bright?

146
Triangulation

• Information group
• 2005/6 use of additional data
• Prescribing report
• Hospital activity data 1st OPD and admissions
• Out-of-hours activity

147
Triangulation?
QOF
Knowledge
RESOURCES OPERATIONAL SKILLS
OTHER DATA
148
(No Transcript)
149
QOF Assessment a refreshed look

• De Vere Hotel Cambridge
• 20 September 2006