UNCERTAINTY AND ' THE PROOF OF UNDERSTANDING LABORATORY MEASUREMENT'

1
UNCERTAINTY AND .
THE PROOF OF UNDERSTANDING
LABORATORY MEASUREMENT.

• WA BRANCH
• Tuesday, 8th February, 2005
• Dr Ken Sikaris
• Melbourne Pathology.

2
UNCERTAINTY AND .
THE PROOF OF UNDERSTANDING
LABORATORY MEASUREMENT.

• WA BRANCH
• Tuesday, 8th February, 2005
• Dr Ken Sikaris
• Melbourne Pathology.

3
AACB SRACMeasurement Uncertainty Working Group

• Graham White Flinders Medical Centre,
  Adelaide AACB
• Convenor
• Tom Hartley Royal Hobart Hospital, Hobart AACB
• John Whitfield Royal Prince Alfred Hospital,
  Sydney AACB
• Ian Farrance PathCare, Geelong NPAAC
• John Glasson IMVS, Adelaide AIMS
• Tony Barker LabPLUS, Auckland, NZ RCPA
• Ken Sikaris Melbourne Pathology QAP
• Georgina Kanizaj-Clark
• Jenny Kox NATA Melbourne. NATA
• from March, 2004

Introduction
4
OUTLINE

• 1. What is MU?
• 2. How is MU estimated?
• 3. How can MU be reported?
• 4. What is the clinical value of MU?

Introduction
5
What is MU?
1. What is MU?
6
(No Transcript)
7
THE UNCERTAINTY PRINCIPLE
1. What is MU?
8
1. What is MU?
9
ISO GUM 1995

• Guide to the expression of Uncertainty of
  Measurement
• CIPM Comm Int des Pods et Mesures 7781
• BIPM Int Bur Weights and Measures
• IEC Int Electrochemical Comm
• IFCC Int Fed Clin Chem
• ISO Int Org Standardisation
• IUPAC Int Union Pure Appl Chemistry
• IUPAP Int Union Pure Appl Physics
• OIML Int Org Legal Metrology

1. What is MU?
10
Sources

• References
• VIM (Vocabulary) 1989 / 04
• GUM (UM Guide) 1995 / 04
• Standards
• ISO 17025 (Lab Standards) 1999
• ISO 15189 (Medical Labs) 2003
• Guides
• NATA Technical Circular 2 2003
• AACB Guide 2004

1. What is MU?
11
The term uncertainty

• the word uncertainty means doubt about the
  validity of a result.
• MU will also be used for quantitative measures of
  the concept.
• GUM 2.2.1

1. What is MU?
12
International Vocabulary of Basic and General
Terms in Metrology VIMDraft April 2004 3rd
Edition

• 2.11 (3.9)
• measurement uncertainty
• uncertainty of measurement
• uncertainty
• parameter that characterizes the dispersion of
  the quantity values that are being attributed to
  a measurand, based on the information used

1. What is MU?
13
Uncertainty (of Measurement)

• Parameter,
• associated with the result of a measurement,
• that characterises the dispersion of the values
  that could reasonably be attributed to the
  measurand.
• GUM 2.2.3

1. What is MU?
14
Other terms

• The error in a sample measurement
• Result True value.
• This is not known because
• The true value for the sample
• This is not known
• eg Na 134 135 136 137 138 mmol/L
• The result is only an estimate of a true value
  and only complete when accompanied by a statement
  of uncertainty.

GUM 2.2.4
GUM 3.2.1
1. What is MU?
15
Types of Error
GUM 3.2.2

• Random error
• Cannot be eliminated, only reduced.
• Unpredictable temporal and spatial variations
• Systematic error
• Cannot be eliminated, only reduced.
• Can be quantified
• If significant in size relative to required
  accuracy, a correction factor can be applied to
  compensate
• Then it is assumed that systematic error is zero.
• It is assumed that the result of a measurement
  has been corrected for all recognised significant
  systematic effects

GUM 3.2.3
GUM 3.2.4
1. What is MU?
16
LFTsFemale DOB 30/1/1934
Date 29/01 28/04 14/05 02/07 Units Range S
BILI 38 29 27 34 umol/L (2-20) S ALP 234
192 206 193 U/L (30-120) S GGT 93 83 87
74 U/L (5-45) S ALT 124 137 113 103 U/L
(5-40) S AST 187 202 167 166 U/L
(5-40) Some clinicians (and patients) believe
that the results from laboratory assays have
little of no uncertainty.
1. What is MU?
17
Introduction to GUM

• When reporting the result of a measurement of a
  physical quantity, it is obligatory that some
  quantitative indication of the quality of the
  result be given so that those who use it can
  assess its reliability.

GUM 0.1
1. What is MU?
18
ISO/IEC DIS 17025

• 5.4.7.2
• apply procedures to estimate uncertainty or
  measurement

2. How is MU estimated?
19
NATA Technical Circular 2December 2003

• Uncertainty of Measurement in Biological,
  Forensic, Medical and Veterinary Testing.
• This technical policy is applicable to all
  facilities in the fields of Biological, Forensic,
  Medical and Veterinary testing and will apply to
  all assessments carried out from 1 January 2004.

1. What is MU?
20
How is MU estimated?
2. How is MU estimated?
21
AS ISO/IEC 17025 - 1999

• 5.4.6.2 Testing laboratories shall have and
  shall apply procedures for estimating uncertainty
  of measurement.
• The degree of rigor needed in an estimation of
  uncertainty of measurement depends on factors
  such as
• the requirements of the test method
• the requirements of the client
• the existence of narrow limits on which decisions
  on conformance to a specification are based.

2. How is MU estimated?
22
ISO 15189 2003(E)

• 5.6.2
• The laboratory shall determine the uncertainty of
  results, where relevant and possible.

2. How is MU estimated?
23
Eurachem / Citac Guide CG 4
1
2
3
4
2. How is MU estimated?
24
Estimating MU

• 1. Define the Measurand.
• 2. Identify all Sources of Uncertainty.
• 3. Quantify the Individual Uncertainties.
• 4. Calculate Combined Uncertainty

2. How is MU estimated?
25
Define the Measurand
2. How is MU estimated?
26
The measurand?

• This guide is primarily concerned with the
  expression of uncertainty in the measurement of a
  well defined physical quantity the measurand
  that can be characterised by an essentially
  unique value.

GUM 1.2
2. How is MU estimated?
27
The Measurand.

• The measurement should have one unique value
• Testosterone
• Reference method (GCMS) value
• ALT
• Reference method (IFCC) value
• PSA
• No Reference method.
• Multiple potential PSA method values.
• Unique method specific PSA value
• Measurand PSA as measured by Abbott Architect
  Assay

2. How is MU estimated?
28
Identify allSources of Uncertainty
2. How is MU estimated?
29
Sources of MU
GUM 3.3.2

• Incomplete definition of the measurand
• Imperfect realisation of the definition of the
  measurand
• Non-representative sampling
• Effects of environmental conditions on the
  measurement
• Personal bias in reading analogue instruments
• Finite instrument resolution or discrimination
  threshold
• Inexact values of measurement standards
• Inexact values of constants obtained from
  external sources
• Approximations incorporated into the measurement
• Variations in repeated observations under
  apparently identical conditions

2. How is MU estimated?
30
ISO/IEC DIS 17025

• 5.4.7.2
• attempt to identify all the components of
  uncertainty
• 5.4.7.3
• All uncertainty components which are of
  importance shall be taken into account
• Components include reference materials, methods,
  equipment, environment, sample condition.

2. How is MU estimated?
31
ISO 15189 2003(E)

• 5.6.2
• Sources that contribute to uncertainty may
  include
• sampling,
• sample preparation,
• sample portion selection,
• condition of the sample
• calibrators,
• reference materials,
• input quantities,
• equipment used,
• changes of operator,
• environmental conditions

2. How is MU estimated?
32
Biological Variation
2. How is MU estimated?
33
POSTANALYTICAL
PREANALYTICAL

REPORT Units Printing, Transcription, Transfer
SAMPLING Posture, Venous stasis, Drip
Arm, Labeling
SAMPLE HANDLING Anticoagulant, Anticoagulant
concentration, Mixing, Micro clots
SAMPLE PROCESSING Centrifugation, Transport, Tempe
rature, Time, Storage
RESULT INTERPRETATION Reference intervals, Age
Sex, Interpretative comments
BIOLOGICALVARIATION Pulsatility, Diurnal, Seasona
l, Fasting
ANALYSIS Precision Bias Interference Detection
limit Linearity Sporadic faults
SAMPLE PREPARATION Mixing, Aliquotting, Labeling,
Evaporation
RESULT HANDLING Transcription Data
download Calculations
ANALYTICAL
2. How is MU estimated?
34
General Approach ?

• Pre-analytical
• Change laboratory habits and not to expand the
  uncertainty estimate.
• Post-analytical
• Risk management procedures or failure rates and
  should be dealt with by general quality
  management policies.

2. How is MU estimated?
35
GUM 3.4.7 - Blunders

• Blunders in recording or analysing data can
  introduce significant unknown errors in the
  result of a measurement.
• Large blunders can usually be identified by a
  proper review of data,
• Small ones could be masked by, or even appear as,
  random variations.
• Measures of uncertainty are not intended to
  account for such mistakes.

2. How is MU estimated?
36
ISO 15189 2003(E)

• 5.8.3
• Comments (e.g. quality or adequacy of primary
  sample which may have compromised the result..)
• 5.8.5
• The report shall indicate if the quality of the
  primary sample received was unsuitable for
  examination or could have compromised the result

2. How is MU estimated?
37
Uncertainty of MEASUREMENT

• Imprecision Bias
• Uncertainty of Measurement

Calibration
Other effects
2. How is MU estimated?
38
Method Manuals

• Detection Limit
• Sensitivity
• Limit of repeatability / reproducibility.
• Selectivity
• Linearity
• Interferences / Cross Reactivities
• Precision
• Within Run, Between Run
• Uncertainty
• Imprecision, Instrument, Reagents, Calibration

2. How is MU estimated?
39
Quantifythe individual uncertainties
2. How is MU estimated?
40
2. How is MU estimated?
41
The mean
n S k1
_
1 n
_
q
qk

2. How is MU estimated?
42
The variance
n S k1
_
1 n-1
___
(qk-q)2
s2(qk)

2. How is MU estimated?
43
The standard deviation
n S k1
_
?
1 n-1
___
(qk-q)2
s (qk)

2. How is MU estimated?
44
Two Categories of Uncertainty

• A. those which are evaluated by statistical
  methods
• si2 Estimated variances
• B. those which are evaluated by other means
• ui2 Approximations of assumed variances
• GUM 0.7

2. How is MU estimated?
45
NATA Technical Circular 2December 2003

• Quantify the individual uncertainties
• according to the laboratorys policy and
  procedures.

2. How is MU estimated?
46
Practical considerations

• If all of the quantities on which the result of a
  measurement a varied, its uncertainty can be
  evaluated by statistical means.
• However because this is rarely possible in
  practice due to limited time and resources, the
  uncertainty of a measurement result is usually
  evaluated using a mathematical model of the
  measurement and the law of propagation of
  uncertainty.

GUM 3.4.1
2. How is MU estimated?
47
Type B evaluation

• Previously measured data.
• Experience with or general knowledge of the
  behavior and properties of relevant materials and
  instruments.
• Manufacturers specifications.
• Data provided in calibration and other
  certificates.
• Uncertainties assigned to reference data taken
  from handbooks.

GUM 4.3.1
2. How is MU estimated?
48
Type B components

• In many cases little or no information is
  provided about the individual components from
  which the quoted uncertainty has been obtained.
• This is generally unimportant .. since all
  standard uncertainties are treated in the same
  way when the combined standard uncertainty is
  calculated.

GUM 4.3.3
2. How is MU estimated?
49
Which is better A or B?

• It should be recognised that a Type B evaluation
  of a standard uncertainty can be as reliable as a
  Type A evaluation, especially in a measurement
  situation where a Type A evaluation is based on a
  comparatively small number of statistically
  independent observation.

GUM 4.3.2
2. How is MU estimated?
50
How many data points? GUM Table E1
2. How is MU estimated?
51
Sample Size vs CVestimatedTrue result CV 5
Sample Size
52
Uncertainty of Uncertainty
53
EQA vs IQC
2. How is MU estimated?
54
GUM 3.4.2

• Because the mathematical model may be incomplete,
  all relevant quantities should be varied to the
  fullest practical extent so that the evaluation
  on uncertainty can be based as much as possible
  on observed data.
• Good range of inputs.

2. How is MU estimated?
55
GUM 3.4.2

• Whenever feasible the use of empirical models of
  measurement founded on long term quantitative
  data, and the use of check standards and control
  charts that can indicate if a measurement is
  under statistical control, should be part of the
  effort to obtain reliable evaluations of
  uncertainty.
• Long period of evaluation.

2. How is MU estimated?
56
External QA vs Internal QC
Advantages
2. How is MU estimated?
57
Lab X (near QAP office) ALBUMIN
QA DATA QC DATA No. of Concentrations
8 2 Concentrations 24.9 51.6 25.8,
39.1 SD 0.65 0.55 CV
1.7 1.7 Number of Results
16 613, 615
2. How is MU estimated?
58
2. How is MU estimated?
59
2. How is MU estimated?
60
Lithium
QA QC CV 3.9 1.4 (39th
Percentile) Range 0.33 - 2.73 0.54, 1.92

2. How is MU estimated?
61
Creatine Kinase
QA QC CV 3.3 1.5 (19th
Percentile) Range 61 - 788 135, 451
2. How is MU estimated?
62
GGT
QA QC CV 4.7 2.3 (83th
Percentile) Range 16 - 175 33, 165
2. How is MU estimated?
63
Conjugated Bilirubin
QA QC CV 5.1 2.9 (27th
Percentile) Range 8 - 66 8, 91
2. How is MU estimated?
64
Lactate Dehydrogenase (L?P)
QA QC CV 2.1 1.3 (18th
Percentile) Range 155 593 168, 429
2. How is MU estimated?
65
CVQC vs CVQA
CVQA
CVQC
66
CalculateCombined Uncertainty
2. How is MU estimated?
67
Combined Uncertainty (uc)

• Standard uncertainty
• u (or s) standard deviation
• Combined (standard) uncertainty
• uc the sum of the known standard deviations

GUM 2.3.1
GUM 2.3.4
2. How is MU estimated?
68
Combining Individual Uncertainties SDs

• For sum (or difference)
• V X Y (V X Y)
• SDV2 SDX2 SDY2
• Use absolute SD (not CV)

2. How is MU estimated?
69
Coefficient of Variation (CV)

• CV SD x 100 Fractional SD
• MEAN
• For tests with a wide analytical range, may need
  more than one CV

2. How is MU estimated?
70
Combining Individual Uncertainties CVs

• For product (or quotient)
• V X x Y (V X / Y)
• CVV2 CVX2 CVY2
• Use CV (not absolute SD)

2. How is MU estimated?
71
Sum or Difference

• Anion Gap
• AG (Na K) (Cl HCO3)
• SDAG2 SDNa2 SDK2 SDCl2 SDHCO32

2. How is MU estimated?
72
Product or Quotient

• Creatinine Clearance
• Clearance (UCr x Vol) / ( PCr x Time)
• CVClearance2CVUCr2CVVol2CVPCr2CVTime2

2. How is MU estimated?
73
EDMAEuropean Diagnostic Manufacturer Association

• uresult ?(ucal2 umethod2 usample2
  uother2)
• ucal
• Manufacturer
• umethod
• Intralaboratory imprecision
• Variation between operators, instruments,
  reagents, labs
• (collaborative studies?)
• usample
• Pre-analytical, Biological
• uother
• Interferences

2. How is MU estimated?
74
Analytical Components

• Minimum approach short term
• uC(y) ?(uCalibration2 uImprecision2
  uInstrument2
  uReagent2)
• Minimum approach long term
• uC(y) ?(uCalibration2 uImprecision2)
• Where long term imprecision includes the
  instrument and reagent contributions

2. How is MU estimated?
75
Expanded Uncertainty (U)

• Expanded uncertainty
• The confidence limits around a result
• Coverage factor
• The number of SDs for the confidence limit
• U uc x k

GUM 2.3.5
GUM 2.3.6
2. How is MU estimated?
76
Coverage factor

• k1.00 68.27 confidence
• k1.64 90
• k1.96 95
• k2.00 95.45
• k2.58 99
• k3.00 99.73
• One can assume that taking k2 produces an
  interval having a confidence of 95 and taking
  n3 produces an interval having a confidence
  interval of 99.

GUM 6.3.3
2. How is MU estimated?
77
References

• Clin Chem 2004501396-1405
• Lead in Blood
• Clin Chem Lab Med 200240391-398
• Calcium and Glucose in Serum

2. How is MU estimated?
78
How canMU be reported?
3. How can MU be reported?
79
Introduction to GUM

• 0.1 - When reporting the result of a
  measurement of a physical quantity, it is
  obligatory that some quantitative indication of
  the quality of the result be given so that those
  who use it can assess its reliability.

3. How can MU be reported?
80
AS ISO/IEC 17025 - 1999

• 5.10.3.1
• where applicable, a statement on the estimated
  uncertainty of measurement information on
  uncertainty is needed in test reports
• when it is relevant to the validity or
  application of the test results,
• when a clients instruction so requires,
• or when the uncertainty affects compliance to a
  specification limit

3. How can MU be reported?
81
ISO 15189 2003(E)

• 5.8.3
• uncertainty of measurement should be provided
  upon request

3. How can MU be reported?
82
NATA Technical Circular 2December 2003

• Report Uncertainties
• Where it is relevant.
• Where the client wants it.
• Where it affects compliance.

3. How can MU be reported?
83
Reporting Uncertainty

• If instruments are known to be in conformance
  with their specifications or with the existing
  normative documents that apply, the uncertainties
  of their indications may be inferred from these
  specifications or from these normative documents.

GUM 7.1.3
3. How can MU be reported?
84
Reporting Conventions

• 1000 (30) mL
• Defines the result and the (combined) standard
  uncertainty
• 1000 /- 60 mL
• Defines the result and the expanded uncertainty
  (k2)
• 1000 /- 60 mL at 95 confidence level.
• Defines the expanded uncertainty at the specified
  confidence interval

3. How can MU be reported?
85
Clinical Application

• Other Reporting mechanisms
• Significant figures
• Commenting

3. How can MU be reported?
86
What is theclinical value of MU?
4. What is the clinical value of MU?
87
2. How is MU estimated?
88
Non-clinical uses of MU

• QC QA in production
• Law enforcement and regulations
• Basic and applied research
• Calibration to achieve traceability to national
  standards
• International reference standards and materials
• GUM 1.1

4. What is the clinical value of MU?
89
ISO/IEC DIS 17025

• 5.4.7.2
• The laboratory shall use methods which meet the
  needs of the client

4. What is the clinical value of MU?
90
ISO 15189 2003(E)

• 5.5.1
• The laboratory shall use examination procedures,
  which meet the needs of the users of
  laboratory services and are appropriate for the
  examinations.

4. What is the clinical value of MU?
91
NATA Technical Circular 2December 2003

• Laboratory tests are usually performed to assist
  the person requesting the test to make a
  decision.
• The result of the test is often compared to
• a limit,
• reference interval,
• or another result obtained previously.

4. What is the clinical value of MU?
92
Clinical Application Overview

• A Appropriateness for Use
• Analytical uncertainty biological variability
• B Diagnosis
• Clinical Decision Limit (eg Gluc gt6.9
  mmol/L)
• Reference Interval
• C Monitoring
• Changes in result / clinical condition
• D Clinical Reporting of Uncertainty
• Confidence Limits
• Significant figures
• Commenting
• E Confidence in laboratory trouble shooting

4. What is the clinical value of MU?
93
Day to day variability

• A series of four results taken from an
  individual
• Sodium mmol/L 137 139 136 138 135
  -147
• Potassium mmol/L 4.3 4.6 4.5 4.4 3.5
  - 5.0
• Urea mmol/l 4.0 4.4 4.1
  3.9 3.3 - 6.6
• Creatinine ?mol/L 88 97 89 92 64
  - 120
• Bilirubin ?mol/L 19 21 16
  20 up to 17
• What can we infer from these results?

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
94
Sources of random variation

• Biological within-subject Biological
  Variation
• Pre-analytical Preparation of subject
• Sample collection
• Analytical Imprecision
• Changes in bias

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
95
A single result represents a distribution

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
96
Data on biological variation

• Over the years, many compilations
• Ricos C, et al. Current databases on biologic
  variation pros, cons and progress.
• Scand J Clin Lab Invest 199959491-500
• 2001 update at http//www.westgard.com/guest21.htm

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
97
Data on biological variation

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
98
An example of a single result

• Serum cholesterol 6.60 mmol/L
• Total variation is due to
• Analytical imprecision
• Within-subject biological variability.
• CVT ?(CVA2 CVB2)

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
99
Influence of imprecision on a single result

• Dispersion is Z x ? (CVA2 CVB2)
• Z2, 95.5 confidence window
• CVB6.0
• 6.6
• If CVA was 0 - 95.5 window is 5.80 7.40
  mmol/L
• If CVA was 3 - 95.5 window is 5.72 7.48
  mmol/L
• If CVA was 5 - 95.5 window is 5.57 7.63
  mmol/L
• If CVA was 10 - 95.5 window is 5.05 8.14
  mmol/L

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
100
Influence of imprecision on dispersion of a
single result - cholesterol

• 0
• 3
• 5
• 10
• 5.0 6.0 7.0 8.0

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
101
Imprecision and Variability

• Increased imprecision ADDS to intrinsic test
  result biological variability.
• if CVA CVB,
• Total ? (CVA 2 CVB 2)
• ? (CVB 2 CVB 2)
• ? (2CVB 2)
• 1.414 x CVB
• so variation is made 41 bigger through analysis

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
102
The effect of imprecision on test result
variability

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
103
41 more dispersion
4. What is the clinical value of MU?
104
25 more dispersion
4. What is the clinical value of MU?
105
12 more dispersion
4. What is the clinical value of MU?
106
3 more dispersion
4. What is the clinical value of MU?
107
0 more dispersion
4. What is the clinical value of MU?
108
The effect of imprecision on test result
variability
Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
109
Appropriate Imprecision

• CVA/ CVB
• Minimum 0.25
• Desirable 0.50
• Optimum 0.75

4. What is the clinical value of MU?
110
The reference distribution, limits, and intervals

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
111
4. What is the clinical value of MU?
112
Effect of negative bias

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
113
Effect of positive bias

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
114
Effect of bias on reference values

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
115
Effect of bias on reference values
Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
116
Appropriate Imprecision and Bias

• CVA/CVB Bias/CVB
• Minimum 0.25 0.125
• Desirable 0.50 0.25
• Optimum 0.75 0.375

4. What is the clinical value of MU?
117
QUALITY LEVELS Terminology
OPTIMAL STANDARD
DESIRABLE
MINIMUM STANDARD
4. What is the clinical value of MU?
118
B Diagnosis

• Diagnosis based on result can be made by
• Reference Interval
• eg hyponatraemia
• Diagnostic cutoff
• eg diabetes

4. What is the clinical value of MU?
119
4. What is the clinical value of MU?
120
Reference Interval Confidence
Per Hyltoft Petersen et al, Upsala Med J
199398241-256
4. What is the clinical value of MU?
121
The six IFCC reference value recommendations

• Part 1 - the concept of reference values
• Part 2 - the selection of individuals
• Part 3 - the preparation of individuals and
  collection of specimens
• Part 4 - the control of analytical variation
• Part 5 - the statistical treatment of
  collected reference values
• Part 6 - presentation of observed values

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
122
Recommendations from IFCC Part 4

• Failure to employ methods that have known and
  suitably low levels of inaccuracy and imprecision
  reduce the clinical utility of reference values.
• It is important to establish and maintain a
  realistic awareness of the contribution that
  analytical inaccuracy and imprecision may have on
  the production of reference values.
• Every effort should be made to establish the
  level of analytical inaccuracy and imprecision
  tolerable for the intended uses.

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
123
Population characteristics

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
124
Effect of bias on nosological characteristics

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
125
Effect of Analytical Bias onPopulation Diagnosis
Per Hyltoft Petersen et al, Upsala Med J
199398221-240
4. What is the clinical value of MU?
126
Consequences of bias

• Proportion of false results at one reference
  limit is far larger than proportion at other
  reference limit.
• Conventionally, 2.5 are outside each reference
  limit.
• Any bias means that MORE than 5 will be outside
  the reference interval.

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
127
EXAMPLE Effect of imprecision on reference
interval

• If true biological 95 reference interval was
• serum sodium 135 147 mmol/L
• then, if Gaussian,
• SD 3 mmol/L (since RI is mean 1.96 SD)

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
128
Effect of imprecision on reference interval

• True Analytical Total Reference SD
  SD SD Interval
• 3.0 1.0 3.2 134.6 - 147.4
• 3.0 2.0 3.6 133.8 - 148.2
• 3.0 3.0 4.2 132.6 - 149.4
• 3.0 4.0 5.0 125.0 - 157.0

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
129
Analytical imprecision widens reference intervals

Biological
Biological
plus
analytical
False high
False
low
RI
2.5
2.5
4. What is the clinical value of MU?
Slide courtesy of Callum G Fraser
130
Effect of imprecision on proportion outside
reference limits

• Inferior imprecision leads to more false
  positives at both high and low values.
• Superior imprecision leads to more false
  negatives at both high and low values.

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
131
Effect of Analytical Imprecision onReference
Limit Diagnosis
Per Hyltoft Petersen et al, Upsala Med J
199398221-240
4. What is the clinical value of MU?
132
Effect of Imprecision on Cutoff Diagnosis

• Cutoff is absolute.
• Cholesterol gt 5.5 mmol/L
• Fasting Glucose gt 7.0 mmol/L
• Opiates gt 300 ug/L
• 9deltaTHC gt 15 ug/L
• Pregnant hCG gt 25 IU/L

4. What is the clinical value of MU?
133
Effect of Analytical Bias onCutoff Diagnosis
Per Hyltoft Petersen et al, Upsala Med J
199398221-240
4. What is the clinical value of MU?
134
Effect of Analytical Imprecision onCutoff
Diagnosis
Per Hyltoft Petersen et al, Upsala Med J
199398221-240
4. What is the clinical value of MU?
135
Effect of Analytical Imprecision on Cutoff
Diagnosis
Per Hyltoft Petersen et al, Upsala Med J
199398221-240
4. What is the clinical value of MU?
136
Analytical confidenceabove a cutoff
95 confidence
1.96SD
4. What is the clinical value of MU?
137
Analytical confidenceabove a cutoff
No confidence Borderline
lt1.96SD
RESULT
4. What is the clinical value of MU?
138
MONITORING

• Both Initial result and Final result have the
  same uncertainty
• Same bias cancels out
• Same imprecision (assumed)

4. What is the clinical value of MU?
139
Confidence in a change
FINAL
4. What is the clinical value of MU?
140
Analytical uncertainty of two results

• Total variation of test1 variation of test2
• z x ? (CVA12 CVA22)
• z x ? ( 2 x CVA2)
• z x ? 2 x CVA
• 1.96 x 1.414 x CVA 2.77 CVA

4. What is the clinical value of MU?
141
95 confidence in a change
2.8 SD
FINAL
4. What is the clinical value of MU?
142
Significant change

• Also referred to as
• Reference change value
• Critical difference
• Delta check ?
• CLINICAL CHANGE

4. What is the clinical value of MU?
143
Patient variability of two results

• Total variation of test1 variation of test2
• z x ?(CVA2 CVB2) z x ?(CVA2CVB2)
• z x ?(2 x (CVA2CVB2))
• ?2 x z x ?(CVA2CVB2)
• 2.8 x ?(CVA2CVB2)

4. What is the clinical value of MU?
144
Calculation of RCV 95 conf.

• Analyte CVA CVB RCV
• Albumin 0.8 3.1 8.6
• Alk Phos 1.4 6.4 18.1
• Bilirubin 1.0 25.6 70.9
• ALT 0.9 24.3 67.3
• AST 1.1 11.9 33.2

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
145
Is there a difference?

• A B
• Initial 3 months later
• FAST FAST
• Cholesterol 7.0 6.2 mmol/L
• Triglycerides 3.3 2.5 mmol/L
• HDLC 0.90 1.00 mmol/L
• LDLC 4.6 4.1 mmol/L

4. What is the clinical value of MU?
146
Analytical Uncertainty
4. What is the clinical value of MU?
147
Biological Variability
4. What is the clinical value of MU?
148
Confidence with cholesterol levels

• The variability of patient lipid levels
  continues to surprise doctors and patients
  alike.
• Guidelines for measurement of blood cholesterol
  levels recommend that at least two measurements
  are required to classify most patients .
• Sikaris K.A.,
• Confidence with cholesterol levels.
• Australian Family Physician 200029(6)582-3.

4. What is the clinical value of MU?
149
LFTsFemale DOB 30/1/1934
Date 29/01 28/04 14/05 02/07 Units Range S
BILI 38 29 27 34 umol/L (2-20) S
ALP 234 192 206 193 U/L (30-120) S
GGT 93 83 87 74 U/L (5-45) S ALT 124
137 113 103 U/L (5-40) S AST 187 202
167 166 U/L (5-40)
Are any of these results different to the
previous?
4. What is the clinical value of MU?
150
LFTsFemale DOB 30/1/1934
Date 29/01 28/04 14/05 02/07 Units Range S
BILI 38 29 27 34 umol/L (2-20) S ALP 234
192 206 193 U/L (30-120) S GGT 93 83 87
74 U/L (5-45) S ALT 124 137 113 103 U/L
(5-40) S AST 187 202 167 166 U/L (5-40)
CDA 4 25 8 12 15
Are any of these results different to the
previous?
Some results are analytically different,
4. What is the clinical value of MU?
151
LFTsFemale DOB 30/1/1934
Date 29/01 28/04 14/05 02/07 Units Range S
BILI 38 29 27 34 umol/L (2-20) S ALP 234
192 206 193 U/L (30-120) S GGT 93 83 87
74 U/L (5-45) S ALT 124 137 113 103 U/L
(5-40) S AST 187 202 167 166 U/L (5-40)
CDA 4 25 8 12 15
Are any of these results different to the
previous?
Some results are analytically different, But none
are clinically different.
4. What is the clinical value of MU?
152
Significant Figures?Hawkins RCW Johnson RN
Clin Chem 199036824.

• Can we really distinguish the critical difference
  between two results?
• An analytical difference in the patients results
  2.77 x SDA
• lt 1.9 then round to ones 126
• lt 9.9 then round to fives 125
• lt 19 then round to tens 130
• lt 99 then round to fifties 150
• lt 190 then round to hundreds 100
• A biological difference in the patients results
  2.77 x ?(SDA2 SDB2)

4. What is the clinical value of MU?
153
Badrick T, Wilson SR, Dimeski G Hickman PE,Ann
Clin Biochem 200441385-390.

• Objective Reporting of Appropriate Reporting
  Intervals.
• The majority of analytes are inappropriately
  reported when analytical precision alone is
  considered. The concept of uncertainty of
  measurement has not been adequately addressed.

154
LFTsFemale DOB 30/1/1934
Date 29/01 28/04 14/05 02/07 Units Range S
BILI 40 30 30 35 umol/L (2-20) S ALP 250
200 200 200 U/L (30-120) S GGT 95 85 90
75 U/L (5-45) S ALT 120 140 110 100 U/L
(5-40) S AST 190 200 170 170 U/L (5-40)
All are biologically the same. No significant
changes.
4. What is the clinical value of MU?
155
Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
156
The Dundee Approach

• no flag means inside RI AND no changes
• lt means lower than LRL
• gt means higher than URL
• means significant change at P lt 0.05
• ltlt means significantly lower than LRL
• gtgt means significantly higher than URL
• means significant change at P lt 0.01

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
157
Glucose Uncertainty Variability

• Analytical Uncertainty
• Glucose CVA2.4 (QAP)
• Biological variability
• Fasting blood glucose CVB 7
• (2h post-load glucose CVB15)
• Scand J Clin Lab Invest. 200262(8)623-30.

4. What is the clinical value of MU?
158
Commenting 1

• Fasting Glucose 8.5 mmol/L
• Analytical uncertainty 2.4
• Analytical confidence 8.5 /- 0.4 mmol/L
• Biological variability 7.0
• Biological confidence 8.5 /- 1.2 mmol/L
• Diabetic Fasting Glucose.

4. What is the clinical value of MU?
159
Commenting 2

• Fasting Glucose 7.5 mmol/L
• Analytical uncertainty 2.4
• Analytical confidence 7.5 /- 0.4 mmol/L
• Biological variability 7.0
• Biological confidence 7.5 /- 1.1 mmol/L
• Diabetic Fasting Glucose - Suggest repeat to
  confirm.

4. What is the clinical value of MU?
160
Commenting 3

• Fasting Glucose 7.0 mmol/L
• Analytical uncertainty 2.4
• Analytical confidence 7.0 /- 0.3 mmol/L
• Biological variability 7.0
• Biological confidence 7.0 /- 1.0 mmol/L
• Borderline Fasting Glucose - Suggest repeat to
  confirm.

4. What is the clinical value of MU?
161
The effect of replicates

• Replicates reduce the variation
• by the square root of the number
• CVT ?(CVA2/na CVB2/ns)

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
162
Effect of replicate assays
Cholesterol 5.0 mmol/L, CVA3.0, CVB6.0

• n CVA ?(CVA2/n CVB2) Dispersion
• 1 3.0 6.7 4.34 - 5.66
• 2 2.1 6.4 4.37 - 5.63
• 3 1.7 6.2 4.39 - 5.61
• 4 1.5 6.1 4.40 - 5.60

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
163
Effect of replicate samples

• n CVA ?(CVA2 CVB/n2) Dispersion
• 1 3.0 6.7 4.34 - 5.66
• 2 3.0 5.2 4.49 - 5.51
• 3 3.0 4.6 4.55 - 5.55
• 4 3.0 4.2 4.58 - 5.42

Slide courtesy of Callum G Fraser
4. What is the clinical value of MU?
164
Change in HbA1c - 1

• 21/1/2004
• HbA1c 7.9
• Fair diabetic control

4. What is the clinical value of MU?
165
Change in HbA1c - 2

• 21/1/2004 30/4/2004
• HbA1c 7.9 8.1
• Bad diabetic control

4. What is the clinical value of MU?
166
HbA1c Uncertainty
Slide courtesy of Ian Farrance
4. What is the clinical value of MU?
167
Significant HbA1c changes

• HbA1c
• CVA3.0
• CVB5.2
• Analytical Difference 2.77 CVA
• 8.0 /- 0.7
• Critical Difference 2.77 ?(CVA2 CVB2)
• 8.0 /- 1.3

4. What is the clinical value of MU?
168
Change in HbA1c - 3

• 21/1/2004 30/4/2004
• HbA1c 7.9 8.1
• No significant change in HbA1c, diabetic control
  is now bad.
• ??

4. What is the clinical value of MU?
169
Change in HbA1c - 4

• 21/1/2004 30/4/2004
• HbA1c 7.9 8.1
• Diabetic control remains borderline poor.

4. What is the clinical value of MU?
170
Laboratory Confidence

• How does understanding components of analytical
  uncertainty contribute to clinical confidence.
• Laboratory can solve QC failures faster.
• Faster TAT to clinician.
• Greater understanding of occasional analytical
  errors that are released
• Prevented
• Explained to clinician

4. What is the clinical value of MU?
171
RCPA/AACB QAP - Error Analysis
4. What is the clinical value of MU?
172
Summary (1)

• Clinical Biochemists have been aware of the
  degree of result dispersion and the contributory
  factors for decades.
• However, estimates of precision (CV) and bias
  have had little clinical relevance.
• Laboratories are responsible for
• Identifying their measurement uncertainty.
• Ensuring doctors are aware of it.
• Understanding its potential clinical impact.

173
Summary (2)

• Uncertainty is clinically important
• Any single test result has an uncertainty.
• Uncertainty must be kept within useful limits.
• Diagnosis is made allowing for uncertainty.
• Monitoring for significance changes is made by
  allowing for uncertainty.
• Ability to gain and maintain clinicians
  confidence depends on our understanding of
  uncertainty.

174
Acknowledgements

• AACB Working Party
• Graham White
• Ian Farrance
• Lloyd Penberthy
• NCCLS Project
• Members, Advisors Observers
• Callum Fraser
• For tonight !
• WA Branch AACB
• Mary Conroy, Tony Prior, Bill McConnell, Wayne
  Smit