Validation: concept,

1
Validation concept, considerations

• Beni Kaufman

2
Will be presenting

• Review
• The concept
• Validation Components and their measurement
• experimental design of PCR validation
• Process vs. Modular validation

3
References

• Guidance for Industry Bioanalytical Method
  Validation.
• U.S. Department of Health and Human Services,
  Food and Drug Administration (FDA), Center for
  Drug Evaluation and Research (CDER), Center for
  Veterinary Medicine (CVM) May 2001
• PCR Validation Performance Characteristics
• Analytical Environmental Immunochemical
  Consortium (AEIC) Biotech Consensus Paper S.
  Charlton, R. Giroux, D. Hondred, C. Lipton, K.
  Worden
• Validation of Analytical procedures Methodology,
  International Conference on Harmonization of
  Technical Requirements for the Registration of
  Pharmaceuticals for Human Use, 1996

4
Warning

• Politically sensitive material!
• Politically!!!
• sensitive material!!
• Discuses components avoid criteria!

5
Validation Components
6
Selectivity (Specificity)

• The ability of the analytical method to
  differentiate (and quantify) the analyte in the
  presence of other components in the sample (to
  amplify only the Sequence of interest.)
• Selectivity may be affected by
• Interference
• Cross amplification of non target sequences
  (function of, Primer design)
• Matrix effects
• Background signal (Sybr green)
• Quality quantity of DNA
• Reaction conditions (master-mix, thermocycling
  profile)

7
Selectivity (Cont.)

• Assessed by
• Fragment length analysis (right size amplicon)
• Electrophoresis gel analysis
• CE

8

Selectivity (Cont.)
Dissociation Curve
do not use r2774, 02-08-2006, 15Hr 58Min.mxp

• Assessed by
• Melting curve analysis

9
Precision

• The closeness of agreement (degree of scatter)
  between a series of measurements obtained from
  multiple sampling of the same homogeneous sample
  under the prescribed conditions.
• Variation among rep.s within an assay
• Same as Repeatability
• Measured by parameters of variation, mostly CV

10
Precision parameters
11
Accuracy/Trueness

• The closeness of mean test results to the true
  value of the analyte.
• Qualitative assay
• Measured by error rate
• false positive False positives/ of
  negatives
• false negative False negatives/ of positives

12
Accuracy/Trueness (cont.)

• Quantitative assay
• The mean recovery at several points across the
  quantitative range
• Recovery 100 (observed/actual)
• (also,
• the deviation of the mean from the true value)

13
Accuracy/Trueness measured
14
Linearity Range

• Linearity The ability of the assay (within a
  given range) to obtain test results which are
  directly proportional to the concentration/amount
  of the analyte
• Range The interval between the upper lower
  concentrations of an analyte for which the assay
  has suitable levels of precision, accuracy
  linearity.

15
Linearity Range (cont.)

• Linearity and Range can be evaluated
  simultaneously
• Demonstrated on a dilution series (transgene
  genomic DNA/null genomic DNA) across a relevant
  range of concentrations
• The Range is established by confirming acceptable
  degrees of linearity, accuracy, precision,
  within or at the extremes of a specified range.

16
Linearity evaluated

• Linearity is evaluated by a plot of signals as a
  function of analyte concentration linear
  regression analysis.

17
(No Transcript)
18
Sensitivity

• Two concepts of sensitivity
• Change in response per amount of reactant -gt
  dose-response curve
• In PCR the dose response is derived from the
  amplification efficiency - We optimize the assay
  for a maximal dose response (100 amp.
  Efficiency)
• Therefore,
• dose-response is reflected in the standard curve.
  Its captured in the Linearity component
• is the basis for quantification
• The source of our resolution power

19
Sensitivity (cont.)

• Limit of detection (LOD), The minimum amount of
  target analyte that can be detected with a given
  level of confidence
• Applies to QL QT PCR
• Limit of quantification (LOQ), The lowest amount
  of target analyte that can be quantified with
  acceptable levels of precision and accuracy.
• Applies only to QT PCR

20
Determining LOD LOQ

• Spiking series
• Decreasing amounts of transgenic seed are mixed
  in with conventional seed to create a series of
  seed pools with varying proportion of transgenes.
• Seed pools are ground to flour
• DNA isolated from flour and used for PCR
  targeting the corresponding target sequence.

21
Sensitivity (cont.)

• The LOD will be lowest spike detected with an
  acceptable confidence level.
• The LOQ will be the lowest spike that can be
  differentiated from zero with an acceptable
  confidence level

22
Ruggedness

• The effectiveness of an analytical process in
  face of small environmental/operating conditions,
  such as
• Different analysts
• Different equipment
• Different labs
• Effectiveness is measured as changes in the
  precision or accuracy.

23
Ruggedness (cont.)

• Effectiveness is measured as changes in the
  precision or accuracy
• For qualitative PCR evaluated by the changes in
  error rate and LOD
• For quantitative PCR evaluated by HORRAT
• Where the Relative Standard Deviation of
  Reproducibility (RSDr) is given as
• RSDr 2(1-0.5lnC) 2C-0.1505
• (C concentration or quantity) And
• HORRAT RSDr(observed)/RSDr(expected)
• HORRAT is expected to be close to 1
• Horwitz, W. (1995) Protocol for the design,
  conduct and interpretation of method performance
  studies, Pure and Appl. Chem, 67331-343

24
Ruggedness measured
25
Robustness

• Describes the reliability of an analysis with
  respect to variations in method parameters.
• Measured by experimentally defining the critical
  range of
• Template concentration
• Primer concentration
• Mg2 Concentration
• Thermocycling temperature range
• Usually part of the assay optimization, prior to
  the validation process.

26
Cartoon Break
27
Seems to be a tedious process!

• It
• Is !!!

28
But,

• the right
• experimental design
• Can take away some of the edge
• For example

29
QT PCR Validation design

• Experiment
• Series of conventional seed pools fortified with
  transgenic seed at a decreasing ratio.
• (For example from 2 to 0.01 at -0.5X
  increments).
• Highest level serves as positive control
• Negative control
• Five reps per level
• Isolate, quantify, normalize, PCR (IQNP)
• All in all 8 spike levels x 5 replicates 40
  amplifications
• Repeat 3 times, 3 different instruments,
  different analysts,
• (3 different dates (?) Astrological effect)

30
QT PCR Validation design

• Analyze
• Selectivity all amplifications yielded the right
  size amplicon (on gel, or by Tm)
• Precision Calculate CV among reps within plates
• Accuracy Calculate mean recovery within plates
• Linearity use samples as standards create
  standard curve- test linearity
• Range based on results of Precision, Accuracy,
  Linearity define range.
• LOD Identify the lowest detected spike with an
  accepted confidence limit
• LOQ Identify lowest spike that its confidence
  interval does not overlap zero.
• Ruggedness HORRAT, or alternatively, ANOVA
  between plates, runs, annalists.

31
QL PCR Validation design

• Experiment
• Series of conventional seed pools fortified with
  transgenic seed at a decreasing ratio.
• ( from 2 to 0.01 at -0.5X increments).
• Highest level serves as positive control
• Negative control
• Five reps per level
• IQNP
• Analyze
• Selectivity all amplifications yielded the right
  size amplicon (On gel or by Tm)
• LOD The lowest spike level to yield
  amplification tentative LOD

32
QL PCR Validation design

• Experiment
• Two plates, each plate, half null, and half
  spiked at tentative LOD.
• Isolate, quantify, normalize,
• PCR the two plates on different instruments,
  different analysts, etc
• Analyze
• Accuracy Calculate positive and negative error
  rate.
• Confirm LOD if false negative lt defined
  criteria (5?)
• Ruggedness compare error rates between
  plates/instruments/analysts

33

• That wasnt that bad wasnt it?
• ?

34
Not only PCR!

• The testing process is made of a number of
  consecutive steps, all can be validated, some
  have to be validated
• Sampling
• Sub-sampling
• DNA Isolation
• DNA Quantification
• DNA Normalization
• PCR
• Post-PCR
• Data Analysis

35
Modular Validation

• The recognition that many of the applications
  steps, in the testing process require independent
  validation of their function
• For better efficiency
• Brought about the idea of Modular Validation
• A. Holst-Jensen, J-AOAC, 1995

36
Modular Validation

• Validate each step (module).
• Once, validated, different modules can be
  combined in to a process that no longer require
  validation
• DNA is DNA!?
• IT IS NOT.

37
Whole Process Validation

• Particle size
• DNA isolation efficiency
• Instrument error
• Matrix effect
• Standards
• All affect the out come of the testing process,
  therefore, the validation is of the whole process
  and only in the context of the given matrix,
  instrumentation, standards

38

• You cant mix match
• Any deviationwill require
• VALIDATION.

39
(No Transcript)