External Quality Assessment EQA

1
External Quality Assessment (EQA)
2
The Quality System
Information Management
Occurrence Management
3
The Problem

• A newly appointed supervisor of the laboratory
  has begun reviewing past EQA performance reports
  The reports indicated a trend in poor performance
  on interpreting gram stains.
• How should the laboratory investigate this
  problem?
• What are the implications for patient test
  results performed and reported during the same
  time period?
• What corrective actions might be taken?

4
EQA

• Definitions
• Responsibilities
• EQA benefits
• EQA methods
• Documentation

5
External Quality Assessment

• A system of objectively checking the
  laboratorys results by means of an external
  agency

6
EQA Methods

• Proficiency Testing
• Rechecking/Retesting
• On-site Evaluation

7
Proficiency Testing

• A program in which multiple samples are
  periodically sent to members of a group of
  laboratories for analysis and/or identification
  whereby each laboratorys results are compared
  with those of other laboratories in the group
  and/or with an assigned value, and reported to
  the participating laboratories and others.
  (NCCLS)

8
Rechecking/Retesting

• Sampling of specimens or slides for rechecking
  by a reference laboratory
• Rechecking of slides is blinded
• Retesting (e.g. HIV) may not require blinding

9
Onsite Evaluation and Review

• Periodic visits to the laboratory to assess
  laboratory practices
• Why?
• Obtain a realistic picture of laboratory
  practices
• Provide assistance with problem areas

10
Laboratory Responsibilities

• Should participate in organized interlaboratory
  comparisons, e.g., EQA schemes
• Participation in EQA should encompass the extent
  and complexity of examination procedures used by
  the laboratory
• ISO Standard prEN ISO/DIS 15189.2

11
Management Responsibilities

• Develop policy
• Assign responsibility
• Monitor the results of external quality
  assessments
• Participate in the implementation of corrective
  actions
• ISO Standard prEN ISO/DIS 15189.2

12
EQA Benefits

• Allows comparison of performance and results
• Serves as an early warning-system for problems
• Identifies systematic kit problems
• Provides objective evidence of laboratory quality
• Serves as an indicator of where to direct
  improvement efforts
• Identifies training needs

13
Proficiency Testing Process

• Central organization sends out challenge
  specimens for testing
• Laboratories analyze specimens, return results to
  central organization
• Results evaluated, laboratories sent scores,
  performance report
• Laboratories take appropriate corrective actions
  and document

14
PT Analysis / Reporting Protocols

• Handle in the same manner as patient/client
  samples
• Follow your testing strategy
• Use same method
• Analysis by the tech who routinely performs
  testing
• Conduct testing within required timeframe
• Communicate problems to supervisors, PT
  organizers
• Inter-laboratory discussion of PT specimens or
  results are not permitted

15
Rechecking/Retesting

• Most commonly used for smears, e.g., TB
• Must consider
• Sample size
• Proper specimen handling
• Procedures to ensure blinding (for smears)
• Procedures to resolve discrepancies
• Details outlined in EQA for AFB Microscopy

16
Comparison of random blinded re-checking to
non-blinded rechecking of AFB Smears in Mexico
Martinez et al. Int J Tuberc Lung Dis (In press)
17
Results of laboratory quality assurance, 2002
HIV Sentinel Surveillance, Botswana
18
On-Site Evaluation

• Conducted by
• PT organization
• Laboratory leaders, e.g., provincial, district
  supervisors
• Consider
• Frequency, e.g., monthly, quarterly
• Use of checklists
• Follow-up to visits
• Monitoring corrective actions
• Training

19
On-site Evaluation Results AFB Microscopy
BEFORE
AFTER
Aziz, M. and G. Bretzel, Int J Tuberc Lung Dis
20026(4)340-349
20
Evaluation of EQA ReportsWhere are the problems?

• Pre-analytic
• Specimen compromised during preparation,
  transport, or after receipt by improper storage
  or handling
• Analytic
• Reagents, instruments, methods, calibration, QC -
  Analytical error
• Calculation
• Competency of Staff
• Post-analytic
• Report format
• Interpretation
• Data
• Clerical/transcription error

21
Investigating Unacceptable EQA Performance

• Clerical error
• Methodological problem
• Technical problem
• Problem with PT materials
• Problem with evaluation of results
• No explanation

22
EQA Documentation

• Establish written procedures for
• Handling of samples
• Analysis of sample
• Recording of results on report forms
• Verification of clerical accuracy
• Use of statistical tools
• Handling corrective actions

23
International Guidelines for External Quality
Assessment (EQA) of AFB Smear microscopy
World Health Organization (WHO) International
Union Against TB and Lung Disease (IUATLD) Royal
Netherlands Tuberculosis Association
(KNCV) Association of Public Health Laboratories
(APHL) Centers for Disease Control and Prevention
(CDC) Japan Anti-Tuberculosis Association (JATA)
24
Proposed Quality System for Rapid HIV Testing EQA

• Proficiency testing
• onsite administration)
• Retesting (serum or dried blood spot)
• statistical sample
• priority of new sites/personnel
• Onsite evaluation
• emphasize visitation for support and evaluation

25
effective EQA never comes cheap Van Deun, A.
External quality assessment of sputum smear
microscopy a matter of careful technique and
organizations. IJTLD 2003 7(6), 507-8.