Clinical Performance Development in NHS Direct 2003-4

1
Clinical Performance Development in NHS Direct
2003-4

• Dr Andrew Lee
• NAGPC Annual Conference
• March 2004

2
Clinical performance indicators

• Standard national reports replaced individual
  site versions
• Reported at national, network, site, nursing
  team, and individual nurse level
• Separate 0845 and OOH Clinical performance
  targets
• Sorting of symptomatic calls
• Use of NHS CAS

3
Gold Standard Sorting Study
4
Aims

• Establish a benchmark of existing NHS practice in
  clinical risk tolerance for primary presentations
  (first point of contact) by telephone
• Identify and quantify any appropriate difference
  in the sorting profile for calls made to access
  GP OOH services and that for other NHS Direct
  calls
• Use the results to inform target setting for NHS
  Direct clinical performance indicators

5
Gold Standard Sorting Study

• First Stage - Benchmarking
• 113 GPs from England and Wales
• Random sample of 600 NHS Direct calls
• 300 0845 and 300 GP OOH
• Clinical details of each call reviewed
  independently by 10 reviewers

6
Gold Standard Sorting Study

• Second stage Reviewing the Differences
• Cases with significant disagreement between
  reviewers and NHSD identified
• Consensus review of these by senior NHSD medical
  and nursing clinicians taking account of
• NHS guidelines, clinical consistency and context
  of service model

7
Gold Standard Dispositions

• Dispositions used to calculate Gold Standard
• NHSD actual dispositions where consensus amongst
  reviewers same
• NHSD actual dispositions where no consensus at
  all amongst reviewers
• Dispositions chosen by senior NHSD clinicians in
  Consensus review stage

8
GP consensus?

• Unanimity amongst reviewers on level of care in
  only 7 of cases
• At least 21 consensus amongst reviewers on a
  level of care in 46 of cases
• At least 21 consensus amongst reviewers on
  whether referral to urgent care (999/AE/GPOOH)
  needed or not in 74 of cases

9
When GP consensus re Urgent referral/Not

• High concordance with NHS Direct
• NHSD actual disposition was same as reviewer
  consensus in 78 of cases
• Gold standard is same as reviewer consensus in
  96 of cases

10
Use of NHS CAS

• Compared the accuracy and risk of the three
  patterns of assessment
• Algo used and followed
• Algo used but a different disposition chosen
• No algorithm used
• Risk measured as potentially critical misses
  i.e. a Gold Standard disposition of Emergency or
  Urgent Care (999, AE, GP 4 or 12) with an actual
  NHSD disposition below this level (in hours care
  or self care)

11
NHS CAS results

• Algorithm content safe
• Algorithm delivers gold standard sorting or
  nearer to it than altering in over 80 of use
• Not using an algorithm produced a rate of
  potentially critical misses nearly three times
  higher than when an algorithm was used whether
  the recommended disposition was followed or not

12
Comparison with other studies

• SWOOP study of nurse-led OOH triage (1997)
  49.2 referred to GP OOH service (calls before
  midnight)
• NHSD Gold standard for GP OOH (combination of GP
  within 2 hours and GP within 6 hours) 43
  ( some GP
  next day will go to OOH at weekends)
• Sheffield MCRU Appropriateness Study (2003) 13
  overtriage and 1 undertriage (equivalent to
  a net reduction in urgency of 12)
• NHSD Gold standard reduction in
  emergencyurgent care 9.7 (0845) and 11.1 (OOH)

13
NHS CAS developments

• Disposition wording changed to make timeframes
  consistent with OOH access standards (1, 2 and 6
  hours)
• Twice yearly major clinical content version
  releases ongoing
• Version 12 (April/May 2004) contains first Gold
  Standard related revisions

14
Summary

• GPs have difficulty agreeing on appropriate
  sorting
• NHS Directs sorting is not very different from
  what GPs can broadly agree on
• NHS Direct has a good safety record
• NHS Directs sorting can improve
• NHS Direct is taking actions to achieve that
  improvement
• NHS CAS will help nurses deliver that improvement