Validation of a Quantitative Analytical Procedure

1
Validation of a Quantitative Analytical
Procedure Accuracy (total error) profile
Federaal Agentschap voor de Veiligheid van de
Voedselketen

• Dr. Jacques O. DE BEER
• Workshop IPH 27th April 2007
• Scientific Institute of Public Health - Brussels
  (Belgium)

2
Method Validation General Concepts

• Different regulations relating to GLP, GMP, GCP
  (OECD, EU)
• Normative or Regulatory documents (ISO 17025,
  ICH, EMEA, FDA, dir. 2002/657/EG)
• ? both suggest that analytical procedures have
  to comply to certain acceptance criteria.
• This request imposes that these procedures are to
  be validated.
• - Some documents define the validation criteria
• - No proposals on experimental approaches !!
• - Limited to general concepts !!

3
Introduction - Definition

• Method Validation is the confirmation by
  examination and the provision of objective
  evidence that the particular requirements for a
  specific intended use are fulfilled. EN ISO/IEC
  17025 5.4.5.1
• Methods need to be validated or revalidated
• before introduction into routine application
• whenever conditions change for which the method
  has been validated (e.g. Instrument with
  different characteristics)
• whenever the method is modified and modifications
  are outside original scope of the method.

4
European and International regulatory bodies and
their guidelines on different aspects of QA
Body Full name Guidance on
Eurachem Focus for Analytical Chemistry in Europe Method validation
CITAC Cooperation of International Traceability in Analytical Chemistry Proficiency testing Quality Assurance
EA European Cooperation for Accreditation Accreditation
CEN European Committee for Normalization Standardization
IUPAC International Union of Pure Applied Chem. Method validation
ISO International Standardization Organisation Standardisation
AOAC ILAC Association of Official Analytical Chemists International Laboratory Accreditation Cooperat. Internal qual. Control Proficiency testing Accreditation
FDA US Food and Drug Administration Method validation
USP United States Pharmacopoeia Method validation
ICH International Conference on Harmonization Method validation
5
Objectives of an analytical procedure

• Able to quantify as accurately as possible each
  unknown quantity to be determined.
• After analysis the difference between returned
  result x and the unknown true value µT be small
  or lt acceptance limit ?
• -? lt x - µT lt ? ? ?x - µT ? lt ? (eq.1)
• ? depends on objective of analytical procedure
  e.g. 1-2 on bulk, 5 on pharmaceuticals, 15
  for biological samples ? previously defined

6
Objectives of an analytical procedure

• Analytical procedures characterized by (cfr.
  def.)
• true bias dM systematic error (unknown)
• true precision s²M random error measured by a
  standard deviation or variance (unknown)
• Estimates of bias and precision obtained by
  experiments during the validation
• Reliability of these estimates depends on
  adequacy of experiments on known samples (Valid.
  Stds), experimental design, number of
  experiments
• These estimates ? an intermediary but obligatory
  step to evaluate if procedure is likely or not to
  quantify with sufficient accuracy the unknown
  quantities not objectives per se

7
Examples of procedures having the same acceptance
limits l 15
Procedure 1
Procedure 2
Bias 7 RSD 3
Bias 1 RSD 8
Procedure 3
Procedure 4
Bias 0 RSD 20
Bias 7 RSD 12
8
Objectives of an analytical procedure

• Figure 4 different (hypothetical) methods giving
  the distribution of 95 of the measures
• Each method has a true bias dM , a true precision
  s²M , a common acceptance limit ? ( 15 ?
  bioanalytical procedure)
• Procedure 3 negligible bias (0) unsatisfactory
  precision (20 CV) too many measures beyond /-
  15 of the true value does not fulfill objective
• Procedure 4 bias (7) precision (12)
  important proportion outside acceptance limits
  does not fulfill objective but both lt 15
  required by Washington Conf.
• Procedures 1 and 2 fulfill (valid) at least 95
  of results inside acceptance limits

9
Objectives of an analytical procedure

• Procedure 1 presents a bias ( 7), but is very
  precise (3 CV)
• Procedure 2 presents a negligible bias ( 1),
  but is less precise (8 CV)
• FIRST CONCLUSION
• Differences between these two procedures dont
  matter since results are never too far from true
  values of the sample to quantify.
• Quality of results is far more important than
  the intrinsic characteristic properties of
  procedure in terms of bias or precision.

10
Objectives of an analytical procedure

• To develop a procedure without bias and error ?
  considerable cost not acceptable strategy
• Analyst has to take minimal risks, compatible
  with the analytical objectives (within reasonable
  time!!)
• Set up acceptable maximum proportion of
  measurements that might be outside acceptance
  limits (?)
• e.g. 5 or 20 of measurements outside (?) as
  maximum risk.
• inside triangles (next fig.) ? space of
  acceptable procedures characterized by true
  bias dM and a true precision s²M
• Acceptable procedures 95, 80, 66 of
  measurements within 15 limits (recommendations
  Washington Conference) ? proportion depends on
  objectives!!!

11
measurements within 15 bias-precision limits
Proc.3
20
(0,20)
15
Proc.4
True precision ()
66
(7,12)
10
(1,8)
80
Proc.2
5
95
(7,3)
-10
-5
0
10
5
0
Proc.1
True bias ()
12
Objectives of an analytical procedure

• Interior triangle area of all analytical
  procedures of which 95 of result X should be
  included within acceptance limits (?), set
  according constraints of analytical domain
• 2 other triangles proportions of 80 and 66 of
  measurements included within ? (accept. limits)
• ? procedure with true bias 0 true precision
  15 only 66 will fall within acceptance limits
  (?)
• ? procedure with true bias 0 true precision
  8 95 will fall within acceptance limits (?)

13
Objectives of an analytical procedure

• Figure procedures 1 and 2 located inside region
  of acceptance
• this region guarantees that at least resp. 95
  and 80 of the results are within acceptance
  limits (?)
• for the same risk of the measurements outside
  acceptance limits, procedures 3 and 4 not
  considered as valid
• for more important risk, procedures 3 and 4 could
  be valid.

14
Objectives of an analytical procedure

• FURTHER CONCLUSION
• Procedure qualified as acceptable if
• it guarantees that the difference between
  every sample measurement (x) and its true value
  (µT) is inside the predefined acceptance limits
  ( l)
• In equation P(?x - µT ? lt l) ? b (eq.
  2)
• b proportion of measurements inside acceptance
  limits
• l acceptance limit, fixed a priori according
  objectives of the method
• Expected proportion of measurements falling
  outside the acceptance limits ? risk of an
  analytical procedure

15
Objective of the validation

• What ?
• to give to the laboratories as well to the
  regulatory bodies guarantees that every single
  measurement performed in routine is close enough
  to the unknown true value of the sample ?x -
  µT ? lt acceptable limit l
• Objective of validation not simply to obtain
  estimates of bias and precision it is to
  evaluate these guarantees and risks
• These estimates of bias and precision are
  required to evaluate risks

16
Objective of the validation

• With respect to this objective, 2 basic notions
  should be considered
• close enough (eq. 1) meaning that routine
  measure will be less than the acceptance limit ?
  from its unknown true value
• guaranteed, (eq. 2) meaning that it is very
  likely that analysis result will be close enough
  to the true unknown value.

17
Objective of the validation

• decision tools are needed giving guarantees
  that future measurements are reasonably inside
  acceptance limits ?

18
Decision rules

• Current position with respect to the decision
  rules used in the phase of validation ? most of
  them based on use of the null hypothesis
• H0 bias 0 ? H0 relative bias 0 ? H0
  recovery 100
• Bias x - µT
• Relative bias 100 (x - µT)/µT
• Recovery 100 x/µT
• A procedure wrongly declared adequate when the
  95 C.I. of the average bias includes 0
• Test inadequate in validation context of
  analytical procedures because decision based on
  computation of rejection criterion of Student
  t-test

19
Test based on H0 bias 0
20

(0,20)
Proc.3
PROCEDURES VALID
15
(7,12)
Proc.4
10
True precision ()
(1,8)
Proc.2
5
(7,3)
Proc.1
NOT VALID
0
-10
-5
10
5
-15
15
0
True bias ()
20
Decision rules

• According to the decision rule based on the null
  hypothesis H0 in fig. procedures 2, 3 and 4 are
  valid and procedure 1 is rejected
• But procedure 1 shows reduced bias ( 7) and a
  small RSD (3) ? outside triangle rejected !!
• procedure 3 has high RSD (20), procedure 4 has
  bias of 7 and RSD of 12 ? accepted !!
• ? bad precision ? large C.I. ? contains 0 as
  bias value ? method accepted
• ? good precision ? small C.I. ? may not contain 0
  as bias value ? method rejected
• null hypothesis H0 inadequate in analyt.
  validation

21
Test based on acceptance limits ( 15)
20

(0,20)
Proc.3
PROCEDURES NOT VALID
15
Proc.4
(7,12)
10
True precision ()
(1,8)
Proc.2
5
ß 80
(7,3)
0
-10
-5
10
5
-15
15
0
Proc.1
True bias ()
22
Decision rules

• According to the decision rule based on use of
  acceptance limits ? triangle in fig. with
  acceptible valid procedures
• Triangle in fig. corresponds to procedures with
  measurement proportion inside acceptance limits
  (?) a priori chosen proportion (e.g. 80) as
  given by equation P(?x - µT ? lt l) ? b (eq. 2)
• ? more sensible decision rule procedures with
  good precision ? accepted
• bad precision ? rejected
• Biased procedure ? small variance acceptable !!
• Procedure with higher variance ? needs small bias
  

23
Decision rules Accuracy profile

• easy and visual decision rule use of the
  accuracy profile within the acceptance limits (
  l)
• Accuracy profile constructed from the
  ß-expectation intervals on the expected
  measurements
• - allows to decide on capability of analytical
  procedure to give results inside l
• - describes dosage interval (range) in which the
  procedure is able to quantify with known accuracy
  and a fixed risk at the end of the validation
• e.g. risk of 5 ? guarantee that 95/100 future
  measurements will be included in acceptance
  limits, fixed according requirements (1-2 on
  bulk, 5 on pharmaceut., 15 in bioanalysis)

24
Decision rules

• Accuracy profile by concentration level (C1, C2,
  ...) obtained by computing ß-expectation
  tolerance interval ? allows evaluating the
  proportion of expected measurements inside
  acceptance limits
• This interval is obtained from available
  validated estimates of the bias and precision of
  the procedure (by concentration level)
• This interval of measurements expected within
  level b ( proportion of measurements inside l)
  has b-expectation confidence limits

25
Decision rules

• If for each concentration level j ß-expectation
  tolerance interval are included within acceptance
  limits ? method accepted!
• Tolerance interval calculation
• - what matters is the guarantee of the results,
  expected in the future by the same analytical
  procedure in routine
• - estimation of µj, s²B,j, s²W,j at every conc.
  j are used to estimate the expected proportion of
  observations within the predifined acceptance
  limits -l,l, i.e.
• Eµ,s P?x - µT ? lt ldM, sM ? b

26
Calculation of ß-expectation tolerance interval

• estimated bias (mean added
  concentrations minus mean calculated
  concentrations)
• j conc. level
• these statistical parameters (trueness,
  within/between precision) might be calculated for
  each concentration level from validation
  standards.

27
Calculation of ß-expectation tolerance interval

• Calculation of the interval in which a proportion
  ß of all samples with a certain real
  concentration is observed (method of Mee) ß
  expectation tolerance interval

ISO 5725-2 calculation of within and between
variance
28
Calculation of ß-expectation tolerance interval

• n degrees of freedom (Satterthwaite)
• p number of series (days)
• n number of replicates per series

Qt ß quantile of the Students t-distribution
with ? degrees of freedom
29
Calculation of ß-expectation tolerance interval

• interval representing in the region containing
  ß of analysis results for a certain
  concentration level j

after rearrangement
30
Calculation of ß-expectation tolerance interval

• Interval consists of two terms
• bias /- coefficient of variation for
  intermediate precision expression of method
  accuracy
• method is accurate for this concentration level
  if obtained tolerance interval is included within
  acceptance limits -?,?

31
Accuracy profile
bias ()
l
mean relat. bias
0
acceptance limits
concentration
bias limits of confidence
- l
C1
C2
C3
C4
LLQ
ULQ
RANGE
dosage interval
32
Decision rules

• Estimates of bias and variance are essential to
  compute evaluation of the expected proportion of
  measurements within acceptance limits
• Accuracy profile obtained by connecting the lower
  or upper limits of confidence (cfr. fig)
• If a subsection (concentration range) falls
  outside the acceptance limits ? new limits of
  quantification be defined and a new dosage
  interval (Upper and Lower Limits of
  Quantification)

33
Decision rules (conclusion)

• Accuracy profile represents limits ULQ and LLQ
  in agreement with definition of criterion
• LLQ smallest quantity of the substance that
  can be measured with defined accuracy
• Accuracy profile as single decision tool
• Allows reconciling the objectives of the
  procedure and those of the validation
• Allows to visually grasp the capacity of the
  procedure to fulfill its analytical objective

34
Validation Protocols Life Cycle

• Validation has to be considered as an element
  intervening after the development of a new
  analytical procedure
• Objective of procedure to be used in routine
• Usage in routine must be coupled with a quality
  control (QC) of which the 2 objectives are
• the validity of the found results on the unknown
  samples
• the assessment of the continuity of the
  performances of the procedure at the time of its
  exploitation

35
Protocols in validation phase

• Main objectives in validation phase
• demonstrate specificity/selectivity
• validate the response function (or calibration
  model used in routine)
• estimate precision (repeatability and
  intermediate precision), trueness, accuracy
• validate the quantitation limits, validate the
  range (dosage interval) cfr. accuracy profile!
• assess linearity of the analytical procedure
  (results directly proportional to concentration
  in the sample cfr. definitions)

36
Protocols in validation phase

• ?preparation of calibration standards (CS) with
  fixed number of concentration levels and
  repetitions by level
• ?preparation of the validation standards (VS) in
  the matrix are independent samples
• VS prepared and treated independenly as future
  samples ? essential for good estimation of
  between-series variance.
• To estimate intermediate precision, VS analyzed
  on different days, equipment and by different
  operators.
• Validation phase is ultimate stage before
  exploitation ? allows to estimate procedures
  performances in the expected experimental
  conditions
• ? allows to check procedures capability to
  quantify unknown sample

37
Protocols in validation phase

• Question whether or not presence of a matrix
  effect.
• If no matrix effect, question is which
  concentration levels will be used for calibration
  ? apply described validation protocols (V1 and
  V2)
• Evidence of matrix effect apply protocol V5
• In case of doubt apply protocols V3 and V4
  according to calibration levels (cfr.Table)
• Which types of standards (CS and VS),
  concentration levels?
• VS prepared in matrix and independent must
  similate future samples

38
Choise of number of CS and VS depending on
selected protocol
standards conc. levels Protocol (no matrix, doubt, matrix) Protocol (no matrix, doubt, matrix) Protocol (no matrix, doubt, matrix) Protocol (no matrix, doubt, matrix) Protocol (no matrix, doubt, matrix)
standards conc. levels V1 V2 V3 V4 V5
CS. calibration out matrix Low Mid High 2 2() 2 2(-) 2 2 2() 2 2(-) 2
CS. calibration in matrix Low Mid High Addit. 2 2() 2 2(-) 2 2 2(-) 2 2()
VS. validation in matrix Low Mid High 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
Minimum number of series Minimum number of series 3 3 3 3 3
Total number of experiments Total number of experiments 33 45 (39) 39 63 (51) 45 (39)
39
Description of protocol V1
Series 1
Series 2
Series 3
R
Calibration standards
Validation standards
(1)
(1)
Additional validation standards (linearity ICH)
(1)
Conc
40
Description of protocol V2
Series 1
Series 2
Series 3
R
Calibration standards
)
(
(
)
)
(
Validation standards
(1)
(1)
Additional validation standards (linearity ICH)
(1)
Conc
41
Description of protocol V3
Series 1
Series 2
Series 3
Calibration Standards without matrix
R
Calibration Standards within matrix
Validation standards
(1)
(1)
Additional validation standards (linearity ICH)
(1)
Conc
42
Possible concentation levels by type of procedure
(e.g. 6 comparative procedures)

1. Determination of single chemical substance
   reference available or determination of active
   ingredient in a pharmaceutical speciality
   (matrix)
2. Determination of available synthesis impurity in
   an active substance or pharmaceutical speciality
   (matrix) at concentration levels gt LOQ
3. Determination of available synthesis impurity in
   an active substance or pharmaceutical speciality
   (matrix) around impurity limit (impurity limit gt
   LOQ)
4. Simultaneous determination of chemical substance
   and one of its non-available impurities in this
   substance or pharmaceutical speciality (use
   substance as tracer to allowed maximum
   concentration of impurity)

43
Possible concentation levels by type of procedure
(e.g. 6 comparative procedures)

• Determination of active substance for measuring
  dissolution kinetics for a dry dosage form
  (matrix)
• Determination of active ingredient and its
  metabolites in plasma (drugs), drug residues, ...
  
• WHICH CONCENTRATION LEVELS ?
• ? cfr. TABLE

44
Examples of possible concentration levels by type
of procedure
Procedure 1 2 3 4 5 6
Calibration standards Calibration standards Calibration standards Calibration standards Calibration standards Calibration standards Calibration standards
Low Mid High addition 100 (120) LOQ (½ Cmax) (Cmax) 80 LA 100 LA (120 LA) LOQ/LA (50) 120 Cmin (50) 120 LOQ ½ Cmax Cmax X
Validation standards Validation standards Validation standards Validation standards Validation standards Validation standards Validation standards
Low Mid High 80 100 120 LOQ ½ Cmax Cmax 80 LA 100 LA 120 LA LOQ/LA 50 120 Cmin (50) 120 LOQ ½ Cmax Cmax
LA admitted limit Cmax max. conc. Cmin
min. conc.
45
Protocols in validation phase

• Identify relationship between response Y and
  concentration X using calibration standards
  (response function).
• Regression models are fitted, accuracy profiles
  calculated, one model selected ? decision about
  validity of the procedure of interest.
• Model depends on ? procedure type
  (pharmaceutical, bio-analytical, immuno-assay)
  ? fixed method objectives
• Linear regression (origin or not) envisaged.
• Mathematical transformations applied on X and Y
• Quadratic regression may be useful

46
Protocols in validation phase

• Back-calculation of estimated VS concentrations
  by series by ? calibration curve equations
• For each concentration level ? estimation of
  trueness and precision
• ? calculation of limits for accuracy cfr. CIj
  (bias) (include large proportion of results)
• ? accuracy profile for each fitted model
• Accuracy profile ? visual decision tool to
  evaluate capability of the method ? if not within
  pre-fixed acceptance limits
• - restrict dosis range ? new limits of
  quantification
• - extend acceptance limits (possible??)

47
ACCURACY PROFILES with same VALIDATION PROTOCOL
(0.01 5.0 ng/ml)
A
B
15
15
quadratic regression
Bias ()
Bias ()
weighed linear regression
-15
-15
C
D
15
15
Bias ()
Bias ()
-15
linear regression
-15
linear regression throug 0
linear regression on log transformed data
E
F
linear regression on square root transformed data
15
15
Bias ()
Bias ()
-15
-15
Concentration
Concentration
1
2
3
4
5
0
0
1
2
3
4
5
48
Protocols in validation phase

• Figure Accuracy profiles for validation of
  dosing procedure of chemical substance in
  biological matrix.
• Protocol V5 applied some concentration levels
• Essentially low levels ? good estimation of LOQ
• 2 of 6 response functions (A quadratic regress.
  B weighed regression) answer objective
  acceptance limits 15
• ? accuracy profile allows to decide about method
  capability
• Quantifiable dosing range with known accuracy
  0.01 5.0 ng/ml at risk ? 5

49
CONCLUSIONS

• Lack of generalisation between different
  validation protocols ? harmonized approach
• Proposal to review objectives of the validation
  according to objectives of the analytical
  procedure
• Distinction between diagnosis rules and decision
  rules
• Objectives of validation not simply to obtain
  estimates of bias and precision but also
• To evaluate risks or confidences that any single
  measurement is close enough to unknown true value
• Trueness, precision, linearity, ..., no longer
  sufficient to make these guarantees.

50
CONCLUSIONS

• Adapted decision tool ? accuracy profile of the
  analytical procedure, based on
• ?-expectation tolerance interval at each
  concentration level
• concept of total error (bias standard
  deviation)
• Allows to bring together objectives of the
  procedure and those of validation
• Allows to visually grasp the capacity of the
  procedure ? to fulfil its objectives
• ? to control risk associated with its
  use in routine

51
References

• C. Hartmann et al., An analysis of the Washington
  Conference Report on bioanalytical method
  validation
• J. Pharm. Biomed. Anal., 12(11) (1994) 1337-1343
• Ph. Hubert et al., The SFSTP guide on the
  validation of chromatographic methods for drug
  bioanalysis from the Washington Conference to
  the laboratory.
• Anal. Chim. Acta, 391 (1999) 135-148
• P. Chiap et al., Validation of an automated
  method for the liquid chromatographic
  determination of atenolol in plasma application
  of a new validation protocol.
• Anal. Chim. Acta, 391 (1999) 227-238

52
References

• B. Boulanger et al., An analysis of the SFSTP
  guide on validation of chromatographic
  bioanalytical methods progress and limitations.
• J. Pharm. Biomed. Anal., 32 (2003) 753-765
• Ph. Hubert et al., Validation des procédures
  analytiques quantitatives. Harmonisation des
  démarches.
• STP Pharma Pratiques, 13(3) (2003) 101-138
• Ph. Hubert et al., Harmonization of strategies
  for the validation of quantitative analytical
  procedures. A SFSTP proposal part I
• J. Pharm. Biomed. Anal., 36 (2004) 579-586

53
References

• Ph. Hubert et al., Validation des procédures
  analytiques quantitatives. Harmonisation des
  démarches. Partie II - Statstiques
• STP Pharma Pratiques, 16(1) (2006) 28 58
• Ph. Hubert et al., Validation des procédures
  analytiques quantitatives. Harmonisation des
  démarches. Partie III Exemples dapplication
• STP Pharma Pratiques, 16(2) (2006) 87 121
• M. Feinberg et al., New advances in method
  validation and measurement uncertainty aimed at
  improving the quality of chemical data
• Anal. Bioanal. Chem 380 (2004) 502-514
• M. Feinberg et al., A global approach to method
  validation and measurement uncertainty
• Accred. Qual. Assur 11 (2006) 3-9