TMH Clinical Audit Workshop (11th Dec 99) Introduction to Clinical Audit and Audit Cycle Dr L C Leung, CONS(Surgery)

1
TMH Clinical Audit Workshop(11th Dec
99)Introduction to Clinical Audit and Audit
CycleDr L C Leung, CONS(Surgery)
2
Clinical Audit

• A clinically led initiative which seeks to
  improve the quality and outcome of patient care
  through structured peer review whereby clinicians
  examine their practices and results against
  agreed explicit standards and modify their
  practice where indicated.

3
(No Transcript)
4
Data Analysis Activities

• 1. Data handling
• 2. Research
• 3. Surgical epidemiology
• 4. Outcomes investigation
• 5. Clinical audit

5
Data handling

• Example
• Develop a computerised database and grading
  system for Plastic Surgery Outcomes

6
Data Analysis Activities

• 1. Data handling
• 2. Research
• 3. Surgical epidemiology
• 4. Outcomes investigation
• 5. Clinical audit

7
Research vs Clinical Audit

• Am I singing the right song
• Is X as effective as Y
• X is always more effective than Y
• To investigate

• Am I singing this song right
• Are we doing X, not Y
• We did X in 75 of cases
• To improve

8
Research

• Example
• To investigate whether plain radiographs plus
  ultrasound is as effective as an IVU in the
  detection of hydronephrosis.

9
Data Analysis Activities

• 1. Data handling
• 2. Research
• 3. Surgical epidemiology
• 4. Outcomes investigation
• 5. Clinical audit

10
Surgical Epidemiology

• Activity analysis is not audit
• Example
• Audit of paediatric surgery in a DGH.
• A retrospective analysis was carried out of
  activity over 5 years, including types of
  surgery, deaths, complications and status of
  operator.

11
Data Analysis Activities

• 1. Data handling
• 2. Research
• 3. Surgical epidemiology
• 4. Outcomes investigation
• 5. Clinical audit

12
Outcomes Investigation

• Local validation study
• Mortality and morbidity analysis
• Patient satisfaction surveys

13
Local Validation Studies

• Example
• Early results of X groin hernia repair done by
  trainee surgeons in a DGH.
• No early recurrences at median follow-up of 7.6
  months.
• 76 of patients had no complications.
• Median time to cessation of analgesia was 4
  days.

14
Mortality and Morbidity

• Example
• Retrospective analysis of 75 gastrectomies
  carried out over 17 years by a single-handed
  surgeon in a cottage hospital.
• Mean age was 63.2 years Malefemale ratio 1.21
  Overall mortality rate 4 etc.

15
Patient satisfaction surveys

• Example
• 70 of patients replied, and 91 of these were
  satisfied or very satisfied with the operation.
  95 indicated they would be happy to have the
  same operation again.

16
(No Transcript)
17
Step 1 Selection of topic
Clinical Audit Process

• Dr. M. L Szeto, COS (MG)

18
Audit cycle --Select topic

• Select topic includes
• 1. Identify topics
• 2. Deciding the topic

19
Suggestions for identifying topics --- Maxwell
Dimensions of Quality(1984) 1. Access - to
services Location and coverage of service 2.
Relevance - to need Of services to the
healthcare needs for the population 3. Equity
- fairness Fairness in provision for different
groups of people
20
Suggestions for identifying topics --- Maxwell
Dimensions of Quality(1984) 4.
Efficiency Economy in the use of resources 5.
Acceptability To patients and relatives 6.
Effectiveness Of care provided
21
Suggestions for identifying topics -- Donabedian
(1966) Structure Amount and type of resources,
such as furniture, equipment, staffing,
premises, etc. Process Quantity and types of
activities of medical care, investigations or
procedures Outcome Relevant indicator of
clinical care to demonstrate improvement in
current or future health
22
DECIDING THE TOPIC Consider the following factors

• 1. Clinical concern
• Wide variation in clinical practice
• Major changes recently
• High risk
• Conditions demand rapid diagnosis or treatment
• Achievable results not achieved
• Complex or difficult management
• Involve other specialties
• 2. Financially important
• High volume
• High cost

23
DECIDING THE TOPICConsider the following factors

• 3. Practical considerations
• Measurable activity
• Achievable standards
• Adequate sample available
• Change can be effected
• Results worth the effort
• 4. Group support
• Enthusiastic and interesting
• Expertise available
• Acceptable effort required

24
Rings of evaluation of audit topics
Clinical concern
Cost
Level
5 very high 4 high 3 moderate 2 low 1 very low
Group support
Feasibility
25
Set audit parameters
Focus of audit - process of patient management -
clinical outcome and effectiveness of care Who
should be involved - Doctors, nurses, allied
health professionals. Multi-disciplinary
approach should be adopted. What and how to
audit - Regular random review of casenotes and
records - Regular review of areas mortality,
infection, clinical incidents, complications -
Systematic criterion-based audit
26
Setting standards

• Evidence of setting standards can be obtained
  through
• Literature review
• Comparison with other hospitals / countries
• Clinical judgement
• Assessment of current practice

27
Types of standards

• External
• Medical literatures
• World Health Organisation
• Academy / Colleges / National guidelines
• Internal
• Local benchmarks
• Modify external standards

28
Compare external with internal standards
29
Clinical Audit Process
Step 3 Design Measure
By Dr. H H YAU, COS(AE)
30
1. Objective(s) Setting

• Express what the group intends to
• achieve by the audit
• Design the audit methodology
• to satisfy the audit objective(s)

31
2. What to measure?

A. Outcome Measures B. Process Measures C.
Process-Outcome Measures
32
2A. Outcome Measures

• May be the most meaningful measures to quality of
  care.
• However, ..
• Important outcomes may happen long after care,
• outcomes may be affected by factors that
  outside the control of the practitioners,
• difficult to isolate which services have
  contributed to the outcomes,
• outcome-only data can be misunderstood by
  public.

33
2B. Process Measures
- Process Measures give clearer pictures of care
delivery but more time consuming to audit. -
More valid esp. when research has demonstrated
a direct link with outcomes. - Critical factors
or steps can be measured - Causes of poor
outcome can be identified during the care
process.
34
2C. Process-outcome Measures
Measure the Process together with the Outcome (It
is preferable) Process-outcome and critical
process measures are more useful to identify
areas for improvement. Use up-to-date reference
(e.g., protocols, care pathways, and outcome
statistics), relevant evidence, collective
judgement to develop audit measures.
35
3. How to Measure?
A. Explicit Measures B. Implicit Measures C.
Two-Phase Strategy
36
3A. Explicit Measures
Agreed formally by the group as the basis for
data collection. Criteria are defined
objectively unambiguously (e.g., no. of
re-operations for the same condition in a
defined period) Representing parameters of good
or poor outcome Can be used by an assistant
trained to collect data (save clinicians
time)
37
3B. Implicit Measures

• Use clinicians knowledge and judgement
• Subjective in nature
• Individual cases or situations can be analysed.
• (e.g., screen out planned re-operate cases)
• Audit findings more believable by clinicians

38
3C. Two-Phase Strategy
Best approach - use explicit and followed by
implicit measures.
1. Screen the cases by using explicit
measures (can be carried out by clinicians
assistant). 2. Then, conduct structured
peer review by using implicit measures to
judge whether acceptable care has been
given. 3. Identify shortcomings in those
problem cases.
39
4. Data Collection
A. Data Definition - precise, well agreed B.
Data Source - medical records, IDS C. Sampling
- balance workload
sample size D. Validity - lead to right
conclusion E. Reliability - pilot trial
(lt10 error)
40
5. Documentation

• Develop an audit tool for the collection of
• data (e.g., use of checklists or audit forms)
• Record the rating results, marking scheme
• and data definition on the forms for
• second or future review.
• Write an audit report to show the key
• components of the audit cycle.

41
6. Ethics and Confidentiality
With Patient participants - - Voluntary
basis - Fully informed in advance Audit data
and report must not identify individual patient
and staff names. - assign case no. or alphabet
to represent patient or staff identities
42
Clinical Audit Process
Step 4 Evaluate the results (Compare with
Standards) Ms Bonnie WONG, M(IA)1
43
A. Analysing Data (I)

• 1. Group Data into useful information
• For example
• of non-compliance for
• individual group of patients
• or staff members,
• range of deviations,
• types of complications, or
• site of incisions.

44
A. Analysing Data (II)
2. Analyse the overall pattern or common
trend of the actual practice. - e.g., which
shift of duty happen most, which age group of
patients, or which team of staff involved most.
45
A. Analysing Data (III)
3. Adjust the preliminary audit findings. -
Take account of special situations, determine
Allowable Exceptions for clinically-sound
reasons through structured peer review.
46
B. Presenting the Audit Results
Structured Peer Review
Prelim results
Number of cases meeting the audit measures
No. of cases were determined clinically
acceptable (Allowable exceptions)

meeting Audit Measures (Standards)

x
100
Total no. of cases reviewed in the audit
47
C. Document the Exceptions
1. State the criteria for giving the allowance
or exceptions, or 2. Explain the reasons for
the cases that were judged as exceptions in
the report.
48
D. Compare with Standards (I)
Target Standards - How far away from the
pre-set target? (Outcome Measures) - Are we
higher or lower than the reference/ agreed
standards? - How much is the gap?
49
D. Compare with Standards (II)
Criterion-based Standards - What are the of
meeting these criteria? (Process Measures) -
Which criteria scored with unsatisfactorily
low compliance rate? - What are the
potential contributing factors?
50
E. Identify Causes of shortcomings
What are the potential contributing factors for
the shortcomings? - Formal Group Discussion -
Breakdown problems into organisational or
individual staff problems (Demings rule 94
Vs 6) - Identify root-causes lead to the
shortcomings.
51
F. Examples of systemic problems

• Unclear direction, objectives or working
  guidelines
• Poor communication between staff supervisors
• Lack of monitoring feedback mechanism
• Insufficient staff supervision
• Insufficient training development
• Inaccessible to updated information
• Ineffective work arrangement or time scheduling
• Inappropriate staff or skill mix
• Inappropriate tools, equipment or facilities
• Lack of incentives or encouragement

52
G. Identify Needed Improvements

• Based on the analysis of the root-causes of
  problems,
• develop strategies for improvements.
• Next is action!

53
Step 5 Implement Improvement Measure

• Mr Billy Yu, DOM (Surgery)

54
Implement Improvement Measures

• 1. Devise a strategy for action
• 2. Implement action

55
1. Devise a strategy for action

• A strategy can include
• 1. Consistent and rapid feedback on performance
  of a process,
• 2. Development and implementation of new
  guidelines
• 3. Changes in procedures, forms, or staff
  education

56
2. Implement Action

• An audit action plan should include
• 1. What has to change?
• Procedure, schedule, practice,
  training, etc.
• 2. Ready to change?
• People, time, resources, etc.

57
Implement Action

• 3. What is the most likely way to achieve change
  and what are the alternative ways
• 4. How to implement the change
• - education-teaching/counseling
• - monitoring/encouraging
• - correcting/praising

58
Implement action

• Factors to consider when implement action
• 1. Who needs to act and for whom.
• 2. When will the action take place.
• 3. How and when do the group know if things are
  going according to plan.
• 4. How will the group know if the action taken
  actually worked.

59
Implement Action

• 1. Who needs to act and for whom
• frontline staff -HO/MO/RN/WS/HCA
• top management-HCE/COS
• other discipline/unit/department-DR/MRO/HRU

60
Implement Action

• 2. When will the action take place
• readiness of people, other resources
• urgency for meeting standard
• wish of top management

61
Implement Action

• 3. How and when will the group know if things are
  going according to plan
• continuous auditing
• check when action objectives are met in every
  subsequent audit

62
Implement Action

• 4. How will the group know if the action taken
  actually worked
• e.g. repeat auditing analysis
• survey nursing staff
• survey H.O.
• check outcome

63
Step 6 Re-audit

• Repeat the audit process for improvement
• re-audit as frequently as required
• as quickly as possible
• until needed improvements are achieved and
  sustained
• Repeat audit cycle several times till changes
  made are working as intended

.
64
Step 7 Compare improved service with standard
Ongoing audit cycle

standard
line
change
change
change
of success
time
65
Evaluation of a Clinical Audit Programme
Dr. S. K. O COS, Clinical Oncology
66
Monitor Effects
67
(No Transcript)
68
(No Transcript)
69
2. Valued and respected by stakeholders
Successful audit programme has credibility and
value from important stakeholder. Clinicians,
managers, purchasers and patients all value
the programme for achievement.
70
3. Covering a full range of provider services,
departments and professions not just include
some or few 4. Providing documented,
demonstrable improvement in quality of
care Impact on care on patients through
documented changes in clinical and organizational
practice. (KIRERAN WALSHE, Health Services
Management Centre, Birmmghan, U.K.)
71
(III) The Process 1. Standard Setting Define
the degree of compliance with chosen criteria
? International standard ? Local standard /
consensus ? Agreed standard Compare
with the standard
72
(III) The Process 2. Presentation of data to a
group - cases meeting the criteria for
audit ( sample size, methodology) -
Stimulate group discussion ? Best be
multidisciplinary - Peer review at
presentation / meeting ? People in same
medical specialty with comparable
training and experience
73
(III) The Process 3. Identify short comings
System Vs. People
? Support ? Knowledge
? Review ? Attitude
? Time ?
Appropriateness of
application of knowledge

74
(III) The Process 4. Identify the change
needed - Change required to improve patient
care. - Feedback to staff with care, best
handled by professionals and peers rather
than managers. - Information disseminated to
peers in same or different hospitals.
- Documentation
75
SUMMARY EVALUATION OF CLINICAL
AUDIT PROJECT 1. Was the project
evaluated after completion? 2. Were objectives
met? 3. Was the impact of project
worthwhile? 4. Review the process. Any
significant problems encountered ?
Successfully resolved? 5. Implement change.
Monitor progress.
76
Clinical Audit project illustration

• Audit on Breast Mass Excision

Dr. K.H. Kwan SMO, Surgery
77
Clinical Audit Cycle
Identification
of Topic
Set
Standard
Measure
Current Service
Compared
With Standard
Implement
Improvement
Measures
Measure
Improve Service
78
Selection of Topic

• Clinical Audit Cycle ( Step 1)

79
Selection of Topic

• Reason for choosing this topic
• Excision of breast mass is one of the important
  work of Plastic Surgery Team.
• Over 25 breast masses were excised every month.
• MM meeting indicated frequent complication after
  breast mass excision

80
Set standardClinical Audit Cycle (Step 2)

• Sources of standard
• International
• Other regional hospital
• Peer

81
Measure Current ServiceClinical Audit Cycle
(Step 3)

82
Outcome Measure

• Complication
• Wound infection, haematoma, gaped wound.

83
Measure Current Service

• Retrospective study on the result of breast mass
  excision from 1.7.98 - 30.11.98.

84
Measure Current Service

• Result
• Number of breast mass excised 102
• Number of complication 14
• Complication rate 13.72
• Complication by individual surgeon
• A0/7 (0) D 14/62 (22.6)
• B 0/7 (0) E 0/18 (0)
• C 0/8 (0)

85
Compare with Standard

• Clinical Audit Cycle (Step 4)

86
Compare with Standard

• Standard lt1
• Present service 13.72

87
Implement Improvement Measure

• Clinical Audit Cycle (Step 5)
• How?

88

• 1. Identify the possible explanation for service
  deficit.
• 2. Implement improvement measures.

89
(No Transcript)
90
(No Transcript)
91
Implementation
Time Frame Immediate Immediate Immediat
e

• Action
• Meeting held among team members
• Limit number of booking each session.
• Provide supervision to surgeon D until technique
  mature

• Responsible Person
• All medical staff
• All medical staff
• Cons and SMO

92
Measure improved Service

• Clinical Audit Cycle (Step 6)
• Re-audit

93
Measure Improved Service

• Result
• Retrospective study on result of breast mass
  excision from 1.1.99-31.5.99
• Number of operation 125
• Complication 4
• Complication rate 3.2

94
Measure Improved Service

• Result
• Complication by surgeons
• B 0/6 (0)
• D 2/42 (4.76)
• E 0/39 (0)
• F 2/38 (5.2)

95
Measure Improved Service

• 1.7.98 - 30.11.98
• Overall complication rate
• 13.72
• By Surgeon D
• 22.6

• 1.1.99 - 31.5.99
• Overall complication rate
• 3.2
• By Surgeon D
• 4.76

96
Compare Improved Service with Standard

• Clinical Audit Cycle (Step 7)

97
Compare Improved Service with Standard

• Standard lt1
• Present Service 3.2

98
What to do next?
99
Complication by surgeon

• B 0/6 (0)
• D 2/42 (4.76)
• E 0/39 (0)
• F 2/38 (5.2)

100
Implement Improvement Measures

• Continuous supervision on Surgeon D.
• Better supervision for new join surgeon.
• Whole process will be reassessed 6 months later.

101
Repeat the Cycle till Standard is Reached
102
Case Study Dr. Y.C. Wun, SMO, OT