Calibration,%20Calibration%20Verification,%20Analytical%20Measurement%20Range,%20and%20Clinically%20Reportable%20Range

1
Calibration, Calibration Verification,
Analytical Measurement Range, and Clinically
Reportable Range
For the CAP Instrumentation and Chemistry
Resource Committees
1
2
The Truth, Please

• Whats your response when
• You receive the results of a CAP Linearity
  Survey?
• You come to Checklist Question CHM.13400 (one
  of several relating to Calibration Verification)?

2
3
The Truth, Please

• Whats your response when
• You receive the results of a CAP Linearity
  Survey?
• You come to Checklist Question CHM.13400 (one
  of several relating to Calibration
  Verification)?
• If youre anything like me, its not
• Wow, let me at it. I was really looking forward
  to taking care of this!

• Who I Am Gary L. Horowitz, MD, FCAP Director,
  Clinical Chemistry, Beth Israel Deaconess Medical
  Center Associate Professor of Pathology, Harvard
  Medical School Chair, CAP Chemistry Resource
  Committee

3
4
Terms

• Calibration

4
5
Terms that Instill Fear

• Calibration
• Calibration Verification
• Analytical Measurement Range (AMR)
• Clinically Reportable Range (CRR)

5
6
Terms that Instill Fear

• Calibration
• Calibration Verification
• Analytical Measurement Range (AMR)
• Clinically Reportable Range (CRR)

• CAP Linearity Surveys

6
7
Why Were Here Today

• confusing, but important, concepts
• audioconference format allows anyone to visit
  the web as often as needed
• objectives
• explain the concepts
• reveal how they relate to the accreditation
  process
• show you in detail how the CAP Linearity Surveys
  enable you to address these issues for your
  laboratory
• ambitious goals, but we have two excellent
  speakers, experts
• in Laboratory Medicine
• in CAP Surveys

7
8
Introductions

• Greg Miller, PhD
• Professor of Pathology, Virginia Commonwealth
  University
• Consultant Member, CAP Chemistry Resource
  Committee

8
9
Introductions

• Greg Miller, PhD
• Professor of Pathology, Virginia Commonwealth
  University
• Consultant Member, CAP Chemistry Resource
  Committee
• Anthony Killeen, MD, PhD, FCAP
• Associate Professor of Pathology, University of
  Minnesota
• Member, CAP Instrumentation Resource Committee
• Former Member and Chair, CAP Chemistry Resource
  Committee
• Program developed as joint effort of these two
  CAP Resource Committees

9
10
Objectives

• Differentiate between the terms calibration,
  calibration verification, analytical measurement
  range (AMR) and clinically reportable range
  (CRR).
• Clarify the accreditation requirements regarding
  these items.
• Explain techniques to perform the appropriate
  verifications.
• Clarify CAP Linearity (LN) Survey reports.

10
11
CLIA 493.2

• Reportable range span of test result values
  over which the lab can establish or verify the
  accuracy of the measurement response.

11
12
CAP (two concepts) CLIA 493.2
Analytical measurement range range of analyte
values that a method can directly measure on
the specimen without any dilution, or other
pretreatment not part of the usual assay process.

• Reportable range span of test result values over
  which the lab can establish or verify the
  accuracy of the measurement response.

12
13
CAP (two concepts) CLIA 493.2
Clinically reportable range range of analyte
values that are reported as a quantitative
result, allowing for specimen dilution or other
pretreatment used to extend the actual AMR.

• Reportable range span of test result values over
  which the lab can establish or verify the
  accuracy of the measurement response.

13
14
CAP CLIA
Calibration set of operations that establish the
quantitative relationship between the reagent
system / instrument response and the
concentration or activity values of an analyte.

• Calibration process of testing and adjusting to
  establish a correlation between the measurement
  response and the concentration.

14
15
CAP (two concepts) CLIA 493.2

• Calibration verification assaying materials of
  known concentration to substantiate the systems
  calibration throughout the reportable range.

15
16
CAP (two concepts) CLIA 493.2

• Calibration verification assaying materials of
  known concentration to substantiate the systems
  calibration throughout the reportable range.

Calibration verification process of confirming
that the current calibration settings remain
valid for a method.
16
17
CAP (two concepts) CLIA 493.2

• Calibration verification assaying materials of
  known concentration to substantiate the systems
  calibration throughout the reportable range.

Analytical measurement range verification
process of confirming that the assay system will
correctly recover the concentration or activity
of the analyte over the AMR.
17
18
CLIA 493.1255 Calibration and calibration
verification procedures

• (a) Perform and document calibration procedures.
• Follow manufacturers instructions.
• Use criteria established by lab during validation
  of method when put in service.
• Whenever calibration verification fails to
  meet the labs specifications.

18
19
CLIA 493.1255 Calibration and calibration
verification procedures

• (b) Perform and document calibration verification
  procedures.
• Follow manufacturers instructions
• Use criteria established by lab during validation
  of the method when put in service
• Include minimum, mid-point and maximum value to
  verify the laboratorys reportable range (this is
  AMR)
• At least every 6 months or when the following
  occur
• Change of reagents unless it does not affect the
  range to report patient results (this is AMR), or
  QC values (this is CAL)
• Major preventive maintenance (this is CAL)
• QC trend, shift or out of limits (this is CAL)
• Schedule for verifying reportable range (this is
  AMR) requires more frequent calibration
  verification

19
20
Calibration (two point)
Analytical Measurement Range extends beyond the
high calibrator
Reportable Results
SIGNAL
CONCENTRATION
CALIBRATOR TARGET VALUES

• May be measurement system specific

20
21
Calibration (two point)

• Example typical serum glucose method
• Low calibrator reagent blank (0 mg/dL)
• High calibrator 356 mg/dL
• AMR 10-600 mg/dL
• The AMR is based on the expectation that the
  signal is linearly related to glucose
  concentration over the 10-600 mg/dL range.
• Verification of the AMR expectation is required
  because the calibrators do not define the
  complete measurement range.

21
22
Calibration (multi-point)
Analytical Measurement Range
Reportable Results Note that when calibrators
span the AMR, the calibration is fully defined
over the AMR
SIGNAL
CONCENTRATION
CALIBRATOR TARGET VALUES

• May be measurement system specific

22
23
Accuracy (Trueness) Traceability

• Manufacturer provides calibration traceability to
  the highest order reference method or reference
  material available
• Manufacturers product calibrator has method (and
  sometimes reagent lot) specific target values
• Cannot mix reagents and calibrators from
  different manufacturers

23
24
Terminology

• Trueness the mean bias between a routine method
  and a reference method determined from
  replicate measurements of multiple patients
  samples
• Imprecision is removed (or minimized) by
  replication
• Accuracy the difference between a routine
  method and a reference method for a single
  measurement on a single patients sample
• Difference includes influence of both bias and
  imprecision for a single measurement

Refer to the CLSI harmonized terminology database
for more detailed definitions at
www.clsi.org/resources
24
25
Calibration Verification

• Verify that measurement system calibration has
  not changed since the last calibration event
• And, where applicable (many routine methods),
• Verify that method calibration is in conformance
  to the manufacturers specifications
• (i.e. is consistent with the manufacturers
  calibration traceability)
• Special case (uncommon) when a lab can verify
  calibration traceability to an accuracy based
  reference system

25
26
Commutable Reference/ PT/ QC Material

• Has equivalent mathematical relationships among
  the results of different measurement procedures
  for a reference material and for native clinical
  samples (International Standards Organization,
  Clinical and Laboratory Standards Institute
  derived).
• Produces a numeric result from a routine method
  that is the same as would be obtained for a
  native clinical sample that has the same quantity
  of an analyte (measurand) present.

26
27
Non-commutable materials may have a matrix bias

• Bias in a result that is attributable to
  modification of the usual clinical sample matrix
  caused by preparation of a reference material, PT
  material or QC material
• CAP has investigated matrix bias in PT materials
  using fresh frozen off-the-clot samples (that are
  expected to be commutable with clinical samples)
• CAP Home (www.cap.org) gt CAP Reference Resources
  and Publications gt Chemistry
• CAP has developed accuracy based PT Surveys from
  this research

27
28
CLSI EP14 to validate commutability
10
95 prediction interval
8
6
Test Method
4
Clinical Specimen
2
0
0
2
4
6
8
10
Comparison Method (1? RMP if available)
28
29
CLSI EP14 to validate commutability
10
95 prediction interval
8
6
Test Method
4
Clinical Specimen RM Commutable
2
0
0
2
4
6
8
10
Comparison Method (1? RMP if available)
29
30
CLSI EP14 to validate commutability
10
95 prediction interval
8
6
Test Method
4
Clinical Specimen RM Commutable RM Not-Commutable
2
0
0
2
4
6
8
10
Comparison Method (1? RMP if available)
30
31
Effect of non-Commutable PT materials
Patient bias 0.2





PT bias -9.5
Reference Method

L
a
b

M
e
d
.

1
9
9
3

1
1
7

3
4
5
-
5
1
32
CAP Accuracy Based Surveys

• Glycohemoglobin (GH2) uses fresh whole blood
  with National Glycohemoglobin Standardization
  Program target values
• Creatinine Accuracy Calibration Verification and
  Linearity (LN24) uses off-the-clot serum with
  NIST target values
• Accuracy Based Lipids (ABL) uses off-the-clot
  serum with CDC reference method target values
• Additional accuracy based Surveys are in
  development for analytes with clinical practice
  guidelines

32
33
Calibration Verification
Run assay materials with measurement system and
compare results to measurement system appropriate
target values
33
34
Calibration Verification Materials

• Method product calibrator(s)
• Method vendor provided materials for calibration
  verification
• Previously tested clinical specimens
• Reference materials with matrix and target values
  appropriate for the method
• PT (or PT validated) materials with matrix and
  target values appropriate for the method,
  including materials designed for calibration
  verification

34
35
Verification of Calibration Trueness (or
accuracy for individual samples)

• Requires a reference material that is commutable
  with native clinical samples (not commonly
  available)
• Most PT materials are not commutable due to the
  cost to prepare commutable materials and the
  practical need to have many analytes in the
  same material

35
36
LN Surveys for Calibration Verification

• Lab verifies conformance to the manufacturers
  calibration, using
• Commutable reference materials that have been
  validated to give results equivalent to those for
  native clinical samples (e.g. CAP LN 24
  Creatinine accuracy based Survey with NIST target
  values)
• Non-commutable materials with target values
  appropriate for a specific method (e.g. most CAP
  LN Survey materials with peer group evaluation)

36
37
Calibration Verification Concentration Levels

• Approximately the same as the calibrators used
  for method calibration
• i.e. to confirm correct calibration
• Can also verify the AMR if concentrations, or
  activities, of the calibrators cover the full
  measurement range.
• LN Survey provides a series of defined
  concentrations to verify calibration over the AMR

37
38
Frequency of Calibration Verification

• Manufacturers instructions
• Troubleshoot a QC problem
• Calibration OR calibration verification at least
  every 6 months (CLIA), or when
• Change in critical reagents, component, or
  maintenance
• If cal lt6 months do not need calibration
  verification unless part of manufacturers
  calibration process

38
39
Analytical Measurement Range Verification
Non-linear region
Specimen concentration relationship is linear
over the AMR
CONCENTRATION RECOVERED
CONCENTRATION EXPECTED Based on (a) admixture
ratio (b) assigned value
39
40
Method Calibration and AMR Verification can be a
single process
ANALYTICAL MEASUREMENT RANGE
Calibrator must 1. Have at least 3 values that
span the AMR, and 2. Have target values specific
for the method (and reagent lot, if necessary).
Analytical signal reportable values
No separate verification of AMR is necessary
Method Calibrator
40
41
AMR Verification
1. Specimens (can use patients) with low and high
values 2. Mix 11 to get midpoint value 3. Mix
lowmid and midhigh to get additional
values 4. Expected values are Sample
Proportion Concentration expected Low 0 20
mg/dL 0.75 L 0.25 H 25 160 mg/dL 0.50 L
0.50 H 50 299 mg/dL 0.25 L 0.75 H 75 438
mg/dL High 100 578 mg/dL
Criteria for acceptability recovery of
concentration in proportion to the amount present
(linear relationship)
41
42
AMR Verification

• AMR verification can be based on demonstrating
  linearity either by measuring recoveries or by
  demonstrating expected proportions

42
43
AMR Verification
Linear relationship verifies recovery of correct
concentration relationship over AMR.
43
44
Materials for AMR Verification

• Linearity material of appropriate matrix
• PT (or PT validated) materials
• Previously tested clinical specimens
• Previously tested clinical specimens, altered by
  admixture, dilution, spiking
• Reference materials with matrix and target
  values appropriate for the method
• Method product calibrator(s)

44
45
AMR how close to low and high limits

• Goal is to verify acceptable clinical performance

• For low limit use clinical judgment, e.g.
• creatinine, 0.2 or 0.3 mg/dL

• For high limit within 10-15 of upper limit of
  AMR
• Challenging to find residual samples for some
  analytes need to use clinical judgment, e.g.
• blood gas (use commercial materials)
• cardiac markers, e.g. Troponin (use LN Survey)

45
46
CAP LN Surveys

• Most evaluate performance vs. peer group due to
  commutability limitations (matrix effect bias)
• LN24 offers matrix-free creatinine challenges
  with reference method values assigned
• Verify that calibration conforms to
  manufacturers specifications
• Verify that calibration is correct over the
  analytical measurement range
• When accuracy based (e.g., LN24), verify that
  calibration is traceable to a reference system

46
47
Example of LN2 Report (Calcium)
47
48
Example of LN2 Report (Calcium)
48
49
Example of LN24 Report (Creatinine)
49
50
Example of LN24 Report (Creatinine)
50
51
Example of a Non-Linear Result
Creatinine
51
52
Calibration Verification and Linearity Users Guide
52
53
Clinically Reportable Range

• Range of analyte values that will be reported as
  a quantitative result, allowing for specimen
  dilution or other pretreatment to extend the AMR
• Policy based on the laboratory directors medical
  judgment regarding the clinical use requirements
  for a laboratory test

53
54
Clinically Reportable Range

• Truncation or extension of the AMR based on
  clinical requirements.
• Established once when method is put in service
• Does not need to be verified on a recurring
  schedule
• Values outside the CRR are reported as lt or gt a
  numeric value.
• Checklist question is Are dilution protocols
  and diluents (or concentration protocols)
  specified for all methods for which the CRR
  exceeds the AMR?

54
55
Clinically Reportable Range

• Example hCG may have a CRR with both low and
  high limits.
• AMR is 3-1,000 mIU/mL (method specification).
• CRR is 5-1,000,000 mIU/mL (clinical usefulness).
• Results of lt3, 3, or 4 are reported as lt5
  mIU/mL.
• Results gt1,000 are diluted and rerun to obtain
  quantitative values up to 1,000,000 mIU/mL.
• Results gt1,000,000 are reported as gt1,000,000
  mIU/mL.
• NOTE CRR limits are the judgment of the lab
  director and may be different for different labs

55
56
Clinically Reportable Range

• Example AST may have a CRR with only a low
  limit.
• AMR is 4-900 IU/L (method specification).
• CRR is 4 IU/L to any value (clinical usefulness).
• Results of lt4 are reported as lt4 IU/L.
• Results gt900 are diluted and re-assayed.
• The upper CRR is not specified because specimens
  are diluted until a quantitative value is
  obtained.

56
57
Summary

• Calibration or Calibration Verification at least
  every 6 months.
• Calibration verification is not necessary if
  calibration is performed at least every 6
  months.

57
58
Summary

• Analytical Measurement Range verification at
  least every 6 months.
• May be required when change a critical reagent
  lot or other analytical parameter that can
  influence the AMR.
• Not required when a recalibration is performed.
• Not required when method calibrators include at
  least 3 concentrations that span the full AMR.
• REQUIRED for initial method validation.

58
59
Summary

• Clinically Reportable Range is established once
  when a method is introduced based on clinical
  requirements as defined by the lab director.
• SOP for dilution or other specimen pre-treatment
  must be defined.
• Not required to verify CRR on a recurring schedule

59
60
Summary
For additional information on the content of the
Audioconference or questions about the CAP
Calibration Verification/Linearity Surveys,
please contact the CAP By telephone Customer
Contact Center at 800-323-4040, option 1 By
email contactcenter_at_cap.org By mail Surveys
Program 325 Waukegan Road Northfield IL
60093-2750