TB Susceptibility

1
Review of External Quality Assessment (EQA)
Systems
2
International Guidelines for External Quality
Assessment (EQA) of AFB Smear microscopy
World Health Organization (WHO) International
Union Against TB and Lung Disease (IUATLD) Royal
Netherlands Tuberculosis Association
(KNCV) Association of Public Health Laboratories
(APHL) Centers for Disease Control and Prevention
(CDC) Japan Anti-Tuberculosis Association (JATA)
3
External Quality Assessment

• Early warning-system for problems
• Measure of laboratory quality
• Valuable benchmarking tool (standardization and
  traceability)
• Indicator of where to direct improvement efforts
• Monitor of changes in technology and testing
  practices (evaluation component)

4
DL
DL
DL
DL
DL
DL
PL
PL
DL
DL
DL
DL
DL
DL
DL
NRL
DL
DL
PL
DL
DL
DL
DL
DL
DL
DL
DL
PL
DL
PL
DL
DL
DL
DL
DL
DL
5
Take Home Message

• The primary responsibility for National TB
  Reference Laboratories is supporting AFB
  microscopy networks through implementation of EQA
  and training

6
International GuidelinesEQA AFB Microscopy

• External Quality Assessment (EQA) Components
• onsite evaluation
• rechecking
• proficiency testing

7
Key Features of EQA guidelines

• Resource analyses and phased approach for EQA
• Identify laboratories with performance
  problems---not intended to identify individual
  errors or validate patient diagnosis
• Emphasizes the need for onsite evaluation of
  laboratories using standard indicators
• Recognizes panel/proficiency testing as a method
  of EQA

8
Key Features of EQA Guidelines Rechecking

• Emphasis on blinded random sample
• Emphasis on the smallest possible sample size
  that provides some information on test
  performance (sensitivity)
• Provides information to measure
  performance---based on the number and types of
  errors

9
EQA -- AFB MicroscopyOn-site Evaluation --
Background

• Many countries lack the resources for annual
  visits of peripheral laboratory by central
  laboratory staff
• Laboratories in most countries are visited by a
  non-laboratory District supervisor
• Optimum evaluation is performed by trained
  laboratory staff in supervisory role

10
EQA AFB Microscopy Guidelines Key
FeaturesOn-site Evaluation

• Develop a standard checklist of questions and
  indicators
• Include minimal evaluation that can be performed
  by non-laboratory trained personnel (e.g.,
  inventory supplies, reagents, equipment)
• Include detailed evaluation that can be performed
  by supervisory laboratory staff
• Train laboratory and non laboratory staff to
  assure consistent application

11
On-site Evaluation Results - Uganda
AFTER
BEFORE
Aziz, M. and G. Bretzel, Int J Tuberc Lung Dis
20026(4)340-349
12
On-site Evaluation Performed by Non-laboratory
Staff Examples

• Are all staining reagents available and within
  expiration dates?
• How are wire loops cleaned?
• Is the laboratory register present and all
  columns completed properly?
• How is maintenance on the microscope performed?

13
(No Transcript)
14
On-site Evaluation Performed by Laboratory Staff
Examples

• Does the technician verify that the container is
  properly labeled?
• How are slides labeled?
• How often is the carbol fuchsin filtered?
• How many fields are examined to report a negative
  smear?

15
EQA - AFB MicroscopyProficiency Testing -
Background

• Uncommon in resource-limited countries
• Prepared smears (South Africa) or patient slides
  (Senegal) sent from central laboratory
• Consistent challenge of laboratory test
  performance
• PT test performance may be different from testing
  routine patient specimens

16
EQA AFB Microscopy Guidelines Key
FeaturesProficiency Testing

• Laboratory may re-use patient slides but a
  procedure is provided to produce test slides for
  consistent slide sets
• Recommended slide set is 10 slides 5 stained and
  5 unstained
• Simple forms for slide production and collection
  of test results

17
PT Implementation in Mexico

• Inspected 587 of 637 laboratories
• 604 microscopists given a 2 hour, 10 slide test
• 54 had score gt80
• 33 had score 60-79
• 13 had score lt60
• 536/604 (88.7) finished all 10 slides
• 216 persons with score lt80 received training
  followed by second PT average scores improved
  from 65 to 90 (P-value lt 0.0001).

Ref Martinez-Guarneros et al, Int J Tuber Lung
Dis 2003 7(6) 516-521
18
EQA - AFB MicroscopyRechecking - Background

• Recommended by IUATLD and WHO
• Usually 100 of positive and 10 of negative
  smears
• Usually un-blinded adds bias
• Reviews patient testing including smear
  preparation, staining, and interpretation
• Lan N.T.N. et al, 1999 Int J Tuberc Lung Dis
  3(1) 55-61

19
EQA AFB Microscopy Guidelines Key
FeaturesRechecking

• Emphasizes blinding and random sample using the
  laboratory register
• Sample size is based on Lot Quality Assurance
  Sampling (LQAS) with parameters selected for test
  volume and desired sensitivity
• Positives and negatives sampled
• Minor errors (FP or FN with 1-9 AFB/ 100 f) are
  included as a measure of performance

20
Comparison of random blinded re-checking to
non-blinded rechecking of AFB Smears in Mexico
Provided by A. Martinez (InDRE) and B. Madison
(CDC)
21
Comparison of random blinded re-checking to
non-blinded rechecking of AFB Smears in Mexico
Provided by A. Martinez (InDRE) and B. Madison
(CDC)
22
Sample Size Comparison of the old method and
new EQA guidelines
Sample 100 positive and 10 negative smears
23
EQA - AFB Microscopy Guidelines Key
FeaturesResource Analysis

• Inventory available resources (actual/projected)
• Manpower, supplies, communication,
  administrative, financial
• Examine effectiveness of current EQA activities
• Gather laboratory service information
• Planningoptions for the evolution of EQA
• Pilot test and document changes
• Expansion based on availability of resources

24
Resources to implement on-site evaluation The
NTP will need to consider

• Capacity of the reference laboratory staff to
  provide on-site evaluation of all intermediate
  laboratories at least annually.
• Capacity of intermediate laboratories to provide
  on-site inspection of the peripheral laboratories
  at least annually, and more frequently to correct
  problems identified.
• Availability of properly trained non-laboratory
  personnel to make supervisory visits at least
  quarterly (as required for DOTS).
• Capacity to implement necessary QI measures.

25
Resources to implement panel testingThe NTP will
need to consider

• Available financial support.
• Proficiency of reference laboratory staff to
  perform ZN AFB smear microscopy.
• Capacity of the reference laboratory staff to
  prepare panel testing slide sets for the
  laboratories to be evaluated
• Available mechanisms to deliver slides to the
  peripheral sites, including mail and couriers.
• Capacity of the reference laboratory staff to
  review and evaluate results from peripheral
  laboratories, and provide recommendations and
  follow-up for corrective action.

26
Resources to implement blinded rechecking The
NTP will need to consider

• Available financial support.
• Capacity of peripheral laboratories to store
  smears for rechecking.
• Availability of properly trained personnel to
  collect appropriate samples of slides from
  peripheral sites.
• Capacity of the reference laboratory staff at
  central and intermediate level laboratories to
  reread smears from peripheral sites, including
  second rereading to resolve discrepancies as
  needed.
• Capacity of reference laboratories to provide
  results of rechecking as well as feedback to
  implement effective corrective action.

27
EQA AFB Microscopy GuidelinesResource
AnalysisPhased Approach

• Assure the five elements of DOTS
• Develop a central reference and intermediate
  laboratories to carry out EQA
• Determine the existing capacity for EQA
• Train district health officials to evaluate the
  minimal functions of microscopy laboratories

28
EQA AFB Microscopy GuidelinesResource
AnalysisPhased Approach (cont)

• Proficiency testing to evaluate performance
• Pilot rechecking program
• Determine resources additional PT or phased
  implementation of rechecking

29
EQA References 1998 -2003
1. Van Deun A, Portaels F. , Limitations and
requirements for quality control of sputum smear
microscopy for acid-fast bacilli. Int J Tuberc
Lung Dis. 1998 Sep2(9)756-65. 2. Nguyen TN, et
al. Quality control of smear microscopy for
acid-fast bacilli the case for blinded
re-reading. Int J Tuberc Lung Dis. 1999
Jan3(1)55-61. 3. Nguyen TN, et al. The
importance of quality control of sputum smear
microscopy the effect of reading errors on
treatment decisions and outcomes. Int J Tuberc
Lung Dis. 1999 Jun3(6)483-7. 4. de Kantor IN,
et al. Periphery to centre quality control of
sputum smear microscopy and 'rapid fading' of
Ziehl-Neelsen staining. Int J Tuberc Lung Dis.
2000 Sep4(9)887-9. 5. de Kantor IN, et al.
More on periphery to centre quality control of
sputum smear microscopy and 'rapid fading' of
Ziehi-Neelsen staining. Int J Tuberc Lung Dis.
2001 Apr5(4)387-9. 6. Fujiki A, et al.
Quality control of sputum smear examination in
Cebu Province. Int J Tuberc Lung Dis. 2002
Jan6(1)39-46. 7. Aziz M, Bretzel G. Use of
a standardised checklist to assess peripheral
sputum smear microscopy laboratories for
tuberculosis diagnosis in Uganda. Int J Tuberc
Lung Dis. 2002 Apr6(4)340-9. 8. Buzingo T, et
al Systematic restaining of sputum smears for
quality control is useful in Burundi. Int J
Tuberc Lung Dis. 2003 May7(5)439-44. 9.
Paramasivan CN, et al. Quality assurance studies
in eight State tuberculosis laboratories in
India. Int J Tuberc Lung Dis. 2003
Jun7(6)522-7. 10. Martinez-Guarneros A, et
al. Implementation of proficiency testing in
conjunction with a rechecking system for external
quality assurance in tuberculosis laboratories in
Mexico.Int J Tuberc Lung Dis. 2003
Jun7(6)516-21.