Accreditation

1
Accreditation Validation

• Joris Van Loco
• Scientific Institute of Public Health
• Food Section

2
Method Validation

• Is method validation
• analyzing 6 samples ?
• Calculating the bias, repeatability,
  reproducibility, of a method ?
• Knowing the detection limits of the method ?
• knowing the uncertainty associated with a method?
• satisfying ISO 17025 assessors?

3
What is Method Validation?

• Method validation is the process of proving that
  an analytical method is acceptable for its
  intended purpose

4
Why is Method Validation Necessary?

• To prove what we claim is true
• To increase the value of test results
• To justify customers trust
• To trace criminals
• Examples
• To value goods for trade purposes
• To support health care
• To check the quality of drinking water

5
When and How should Methods be Validated

• New method development
• Revision of established methods
• When established methods are implemented in new
  laboratories

• Interlaboratory Comparison
• Single lab validation
• Full Validation
• Implementation Validation
• Method performance parameters are determined
  using
• equipment that is
• Within specification
• Working correctly
• Adequately calibrated
• Competent operators

6
Validation and Quality Control

• In house validation
• (Bias), recovery
• Repeatability
• Within lab reproducibility

• Internal QC
• Control charts

Long term within lab reproducibility

• Proficiency Testing
• Bias (trueness)
• Collaborative trial
• Reproducibility
• Bias (trueness)

7
Method Validation

• Accuracy
• Trueness (CRM)
• Recovery (spikes)
• Precision
• Repeatability
• (Within) reproducibility
• Selectivity ( Specificity)
• Detection capability
• LOD, LOQ, CC?, CC?
• Linearity calibration range
• Robustness
• Applicability stability

8
Method ValidationPerformance Characteristics
2002/657/CE
S Screening methods C Confirmatory methods
9
Linearity

• Purpose
• to evaluate the linear response of your
  instrument
• How
• Evaluating your calibration model
• Mandels fitting test
• Lack-of-Fit
• Residuals
• Conclusion
• Linear model
• ltgt other (i.e. quadratic) regression model

10
Linearity

• Residual plots (ei)
• with
• Statistical tests
• Lack-of-fit
• Mandels fitting test

11
Coefficient of correlation (r)

• Is NOT a suitable measure for linearity

12
Matrix Effect

• Purpose
• To evaluate whether you have a concentration
  dependent systematic error due the matrix
• i.e. ion suppression
• How
• comparison of standard curve with matrix matched
  standard curve
• Conclusion
• Standard solutions, spiked extracts or spiked
  samples for the calibration line.

13
Detection Limits
Detection limit DIN 32645 from blanks
from calibration data Funk dynamic
model IUPAC Coleman recursive formula
explicit formula
14
Detection Limits
A) DIN 32645 Detection limit by fast
estimation Capability limit Determination
limit by fast estimation Factor for fast
estimation
15
Detection Limits
B) Funk Detection limit dynamic
model Determination limit dynamic model
16
Detection Limits
C) IUPAC Detection limit
17
Detection limits How to

• Choose a definition and stick to it
• Describe the equation used in the validation file
• Problems
• statistics ltgt practical limitations
• statistics ltgt ID-criteria
• Practical LOD
• Analyzing samples with decreasing concentration
• Minimum concentration which fulfills the
  identification criteria practical limit of
  detection
• Repeat the experiment
• S/N
• i.e. LOD3xS/N

18
Quantitation Limit (LQ)

• The quantification limit is the minimum signal
  (concentration or amount) the can be quantified.
• the residual standard deviation (RSD) is included
  in the definition.
• The IUPAC default value for RSDQ 0.1 (or 10).
  ?LQ10sQ.

19
a- and b-error

• a-error
• risk of erroneously rejecting H0
• i.e. risk of the conviction of an innocent
• b-error
• risk of erroneously accepting H0
• i.e. risk of the non conviction of a criminal

20
Detection CapabilityCase of a permitted limit
(MRL)
CCa
MRL
CCb
1.64sMRL
1.64ssample
Signal orConcentration
a 5
a b 5
21
Determination of CCa and CCb with ISO 11843
yc
CCa
CCb
MRL
22
Detection CapabilityCase of a permitted limit
(MRL)
23
Detection CapabilityCase of no established
permitted limit or banned substance
Xblank
CCa
CCb
2.33sblank
1.64ssample
Signal orConcentration
a 1 ? b 5
24
Detection CapabilityCase of no established
permitted limit or banned substance

• Under the assumption of linearity, normality,
  independence, and homoscedasticity CCa and CCb
  are given by

• a 2.33
• b 1.64

with b the slope of the regression line, xMRL the
nominal concentration at the permitted limit t
the associated t-value, Sy the standard error of
the estimate, I the number of replicates per
concentration for the spiked samples i 1, 2, .
. . , I and J the number of concentrations for
the spiked samples.
25
Basic assumptions of the ISO 11843-2 (linear
regression model)

• Linearity
• Normality
• Independence
• Error free independent variable (concentration)
• Homoscedasticity (? heteroscedasticity)

26
Heteroscedasticity

• Variance f(x)
• Solution Weighted regression
• Evaluation
• Statistical tests (cochran, Breush-pagan,)
• Visual interpretation
• Residual plot
• Plot S or S² vs concentration
• Impact on CCa and CCb
• Variance at CCa and CCb is not correctly
  estimated
• CCa and CCb may be incorrect

27
Presence of Heteroscedasticity

• Nitroimidazoles in plasma (MNZ-OH)
• Residuals plot
• lt - shape
• Plot of S vs conc
• Linear relationship between S and concentration
• ?Heteroscedasticity
• Impact on CCa and CCb
• CCa and CCb are incorrectly calculated
• Sblank ? ? CCa ?
• CCb ? or ?

28
Other examples

• Nitroimidazoles in plasma

• Nitrofurans in honey

• Corticosteroids in liver

29
Weighted regression equations for CCa and CCb

• Solved by iteration

30
Conclusion detection capability

• Many definitions of detection limits
• detection limit ( CCa_banned substances)
• determination limit ( CCb_banned substances)
• Quantition limit
• Complicated statistics
• KISS
• demonstrate with real (spiked) samples at low
  concentration level ? practical limit of
  detection

31
Selectivity/Specificity

• Identity Signal to be attributed to the analyte
• GLC (change column/polarity), GC/MS, Infra-red
• Selectivity The ability of the method to
  determine accurately the analyte of interest in
  the presence of other components in a sample
  matrix under the stated conditions of the test.
• Specificity is a state of perfect selectivity

32
Selectivity

• The procedure to establish selectivity
• Analyze samples and reference materials
• Assess the ability of the methods to confirm
  identity and measure the analyte
• Choose the more appropriate method.
• Analyze samples
• Examine the effect of interferences

33
Selectivity Verification of the identification
criteria (2002/657/EC)

• MS criteria
• 3 or 4 identification points
• 1 precursor and 2 transition ions
• Relative ion intensities
• LC criteria
• Relative retention time (RRT) /- 2.5 (LC)
• UV criteria
• Spectrum match
• /- 3 nm

• CCb is concentration at or above the calculated
  CCb for which the ID criteria are fulfilled in
  95 of the cases.
• CCa is concentration at or above the calculated
  CCa for which the ID criteria are fulfilled in
  50 of the cases.

34
Ruggedness and Robustness

• Intra-laboratory study to check changes due to
  environmental and/or operating conditions
• Usually it is part of method development
• Deliberate changes in
• Temperature
• Reagents ( e.g. different batches)
• Extraction time
• Composition in the sample
• etc

35
Precision ISO 5725 1-6 (1994)

• Expresses the closeness of agreement (dispersion
  level, relative standard deviation) between a
  series of measurements from multiple sampling of
  the same homogeneous sample (independent assays)
  under prescribed conditions.
• Irrespective of whether mean is a correct
  representation of the true value.
• Gives information on random errors
• Evaluated at three levels
• repeatability
• intermediate precision (within laboratory)
• reproducibility (between laboratory)

36
Precision (cont.) ISO 5725 1-6 (1994)

• Repeatability precision under conditions where
  the results of independent assays are obtained by
  the same analytical procedure, on identical
  samples, in the same lab, by the same operator,
  using the same equipment and during short
  interval of time
• Intermediate precision ISO recognizes M-factor
  different intermediate precision conditions (M
  1, 2 or 3)
• M 1 only 1 of 3 factors (operator, equipment,
  time) is different
• M 2 or 3 2 or all 3 factors differ between
  determinations

37
Precision (cont.) ISO 5725 1-6 (1994)

• Reproducibility precision under conditions where
  results obtained
• by same analytical procedure
• on identical sample
• in different laboratories, different operators,
  different equipment
• Reproducibility established by interlaboratory
  study (standardisation of an analytical
  procedure)
• Intermediate precision
• Repeatability Reproducibility

38
Evaluation of Precision

• 10 samples for each conc.under r,R, within lab R
• Standard Deviation
• Determination in pairs under r,R, within lab R
• Std. Dev. between two single determinations
• a-b, the difference between the values, d, the
  number of pairs

sr SR SRw
39
Repeatability (r) and within-lab reproducibility
(Rw)
ANOVA table for a single factor balanced design
with 3 replicate samples on the same day.
repeatability (Sr²) and within-lab
reproducibility variances (SRw²) Sr²
Srepl² SRw² Sr² Sdays² The Srepl²and
Sdays² can be obtained from mean squares as
(nrepl 3) Srepl² MSrepl Sdays² (MSdays
MSrepl) / 3
40
Repeatability and reproducibility

• The value of 2.8?
• Variance of difference between 2 replicate
  measurements is 2s²
• Confidence interval at 95 level on the
  difference is 0 1.96 v2 s ? 1.96 x 1.41 sr
  2.8 sr
• ? 95 probability that difference between
  duplicate determinations will not exceed 2.8 sr
• r limit of the repeatability r 2.8 sr
• R limit of the reproducibility R 2.8 SR

41
Precision criteria 2002/657/CE
42
Horwitz RSDR() 2(1-0.5logC)
43
Determination of Trueness

• Using Certified Reference Materials
• Using RM or In-house materials
• Using Reference methods
• Single sample
• Many samples
• Via Interlaboratory study

44
Trueness, extraction yield (recovery) and
apparent recovery

• Trueness means the closeness of agreement between
  the average value obtained from a large series of
  test results and an accepted reference value.
  Trueness is usually expressed as bias
• Recovery (extraction yield) yield of a
  preconcentration or extraction stage of an
  analytical process for an analyte divided by
  amount of analyte in the original sample.
• Apparent recovery observed value derived from an
  analytical procedure by means of a calibration
  graph divided by reference value.

45
Trueness criteria 2002/657/CE

• When no such CRMs are available, it is acceptable
  that trueness of measurements is assessed through
  recovery of additions of known amounts of the
  analyte(s) to a blank matrix. Data corrected with
  the mean recovery are only acceptable when they
  fall within the ranges

46
Conclusions

• All methods must be validated (re validation
  might be necessary)
• Validation is fit for intended purpose ltgt
  determining performance characteristics
• accuracy profile
• Acceptance criteria (i.e. Horwitz)
• Complex statistics
• Relation among
• Validation Quality control proficiency
  testing