Statistical Aspects to Setting Specifications on Biological Products

1
Statistical Aspects to Setting Specifications on
Biological Products

• Robert Capen, Ph.D.
• Associate Director, Scientific Staff
• Department of Vaccine Biometrics Research
• Merck Co., Inc.

2
Outline

• Introduction
• Action Limits
• Release Limits
• Final Thoughts
• Suggested Readings

3
Introduction

• A specification is a list of tests, references to
  analytical procedures, and appropriate acceptance
  criteria with numerical limits, ranges, or other
  criteria for the tests described, which
  establishes the set of criteria to which a drug
  substance or drug product or materials at other
  stages of their manufacture should conform to be
  considered acceptable for its intended use.
• FR, 63, No. 110, 6/98, Notices

4
Introduction

• Acceptance Criteria Numerical limits, ranges, or
  other suitable measures for acceptance which the
  drug substance or drug product or materials at
  other stages of their manufacture should meet to
  conform with the specification of the results of
  analytical procedures.
• FR, 63, No. 110, 6/98, Notices

5
Introduction

• Expiry limits represent a quality commitment over
  the entire shelf life of the product. They are a
  commitment by the manufacturer to its customers
  that the measured characteristic (e.g., potency)
  will fall within the prescribed limits, whether
  the product is measured on the day of its
  manufacture or on the day it expires.
• Dillard (2002)

6
Introduction

• Release limits are generated by the manufacturer
  to ensure compliance to the expiry limits. They
  apply only at the time of manufacture. They are
  means to adjust for the uncertainties caused by
  product instability and measurement variation so
  that, with a high degree of confidence, any batch
  that meets its release requirements will also
  conform to its expiry requirements over the shelf
  life of the product.
• Dillard (2002)

7
Introduction

• An action limit is an internal (in-house) value
  used to assess the consistency of the process at
  less critical steps. These limits are the
  responsibility of the manufacturer.
• FR, 63, No. 110, 6/98, Notices

8
Introduction

• Control or Tolerance Limits provide the
  boundaries between which results generated under
  normal operating conditions can be expected to
  fall. They reflect the range of common cause or
  chance variation expected in the data when the
  manufacturing process and analytical method are
  operating in a consistent (i.e., predictable)
  manner.

9
Introduction

• An Out-of-Trend (OOT) result is a stability
  result that does not follow the expected trend,
  either in comparison with other stability batches
  or with respect to results collected during a
  stability study.
• PhRMA CMC Statistics and Stability Expert Teams,
• Identification of Out-of-Trend Stability Results,
• Pharmaceutical Technology, April 2003

10
Introduction

• The Producers Risk (PR) is the risk associated
  in claiming that the characteristic being tested
  is OOT (or OOS) due to chance variation alone.
• Investigating a good lot
• The Consumers Risk (CR) is the risk associated
  in claiming that the characteristic being tested
  meets its acceptance criterion due to chance
  variation alone.
• Releasing a bad lot

11
Introduction

• Justification of the Specification
• Linked to the manufacturing process
• Account for instability in the drug substance or
  drug product
• Linked to preclinical and clinical studies
• Linked to analytical procedures
• FR, 64, No. 159, 8/99, Notices

12
Introduction
Expiry Limit(s)
Consumer Driven
Acceptance Limit(s) at Release
Consumers Risk OOS
Action Limits at Release
Producers Risk OOT
In-Process Action Limits
13
Action Limits

• Statistically derived as tolerance limits
• Assesses consistency of manufacturing process NOT
  quality of product
• Must account for pertinent sources of
  manufacturing and analytical variability
• Beware of multiplicity

14
Action Limits
Univariate sample from a normal distribution
Two-Sided Action Limits
15
Action Limits

• Choice of k
• Naïve k 1.96, 2.576, 3 corresponding to
  approximately a 5, 1 and 0.3 PR
• Tolerance factors kk(a, p, n)
• 1 a Confidence level
• 1 p Coverage level
• n Sample size

16
Action Limits
Derivation of k
17
Action Limits

• Exact values for k have to be obtained through
  numerical integration
• Odeh and Owen (1980)
• Partially reproduced in Hahn and Meeker (1991)
• Approximate values for k also exist and are
  simple to program into Excel
• Wald and Wolfowitz (1946)
• Gardiner and Hull (1966)

18
Action Limits
Comparing the Naïve Interval (with an Assumed PR
of 5) to a Tolerance Interval that Limits the PR
to no More than 5 with 99 Confidence (ie, a
99/95 Interval)
For n 25, the tolerance interval is derived as
approximately
19
Action Limits
One-Way Treatment Structure in a CRD
Assumptions
20
Action Limits
Two-Sided Action Limits
Derivation of ?
21
Action Limits
The Desired ? is the Solution to
Beckman and Tietjen (1989)
22
Action Limits

• s2 is an estimate of
  and can, for the balanced case (ni n for
  all i), be expressed as

The corresponding degrees of freedom can be
found using satterthwaites (1946) approximation
where the notation x refers to the greatest
integer x
23
Action Limits

• s2 has an approximate chi-squared distribution
  with f degrees of freedom

24
Action Limits

• For the unbalanced case, a reasonable estimator
  of is

where is the harmonic mean of the ni and
MSTH is, essentially, the unweighted variance of
the within-group averages
25
Action Limits
Thomas and Hultquist (1978)
26
Action Limits

• Thomas and Hultquist (1978) show
• that . This leads to
• the reasonable estimate for var (y) of

Burdick and Graybill (1992)
27
Action Limits

• The corresponding df are given by

28
Action Limits

• For more complicated (balanced) design structures
  consult Beckman and Tietjen (1989).
• In practice, for either balanced or unbalanced
  data, a reasonable approximate approach involves
  estimating var (y) through a variance-component
  analysis and choosing for ? a value of 3 or 4.

29
Release Limits

• As mentioned previously release limits are
  in-house acceptance criteria derived so that if a
  lot passes these criteria it will satisfy its
  shelf-life criteria with high confidence.

30
Release Limits
Simple Case No Degradation
Assay standard deviation
Potency
Number of replicates at release
L
Statistical multiplier corresponding to a CR of
100? of passing a non-efficacious lot.
Time
31
Release Limits

• Is there anything missing?
• A term for lot-to-lot variability?
• No. The decision about the disposition of a
  particular lot is specific to that lot.
• The basic unit for decisions on quality (release,
  reject, retest, recall) is always the lot.
• A term accounting for variation in the slopes?
• Possibly.

32
Release Limits
Contrived Example
Generally, the slopes are near 0 but occasionally
we observe a significant slope. The variability
among the slopes may be sufficiently large so
that it should be accounted for in the derivation
of the release limit.
Potency
Time
33
Release Limits

• Why not just pass if the potency at time 0 is gt
  L instead of gt R?
• Does not account for assay variability. The
  consumer is not protected against the release of
  a non-efficacious lot.
• A time 0 potency result between L and R could
  just be assay variability working in our favor.
  To be sure need to have time 0 result gt R.
• For a lot that is put on stability, is the value
  derived for R appropriate at other time points?
  Or is looking at data on a point-by-point basis
  the right thing to be doing to begin with?

Fairweather, et al (to be published)
34
Release Limits
Linear Degradation
Degradation over Shelf Life of Product
Release Limit (R)
Potency
L
Shelf Life Limit
0
Shelf Life T
Time
35
Release Limits

• The degradation over the shelf life of the
  product is simply -bT where b is the estimated
  average slope
• May need to include multiple degradation steps
• Frozen/lyophilized storage to refrigerated liquid
• Sealing/packaging/inspection at RT
• Handling at physicians clinic
• Marketing needs RT stability claim
• Both assay and manufacturing sources of
  variability must be accounted for

36
Release Limits
Allen, Dukes and Gerger (1991) (without the
sslope term)
Accounts for lot-to-lot variation in the slopes,
imprecision in the estimate of the average
slope and assay variation at time of release only.
Wei (1998)
Accounts for lot-to-lot variation in the slopes
and assay variation at every time point. Assumes
all quantities known.
37
Release Limits

• The extra multiplier (z0.05) is a result of how
  Wei formulated the problem.
• k depends on T and the number of replicates at
  each time point.
• If n replicates are obtained only at time 0 and
  time T, then T2k 2/n

38
Release Limits
Models
ADG
None Provided. Pieced-Meal Together.
Wei
Linear Mixed-Effects Model.
Starting Model for Wei
Random intercepts and slopes
i Batch j Time Point k Replicate
Random assay variability
39
Release Limits
Wei conditions on the observe potency at
release (time 0). The resulting conditional
model becomes
The release limit, R, based on the conditional
model keeps the chance of failing the shelf-life
limit for each released lot under a prescribed
tolerance.
40
Release Limits
ADGs Piece-wise Approach Equivalent to
ADG Fixed intercept, slope
Modified ADG Fixed intercept, random slope
41
Release Limits
Release Limits
ADG
Wei
42
Release Limits
Properties
The release limit, R, generated from both Weis
and ADGs models satisfy the following
property Given that the observed potency at time
0 is gt R, then
P(f(estimated mean potency) at time T lt L) lt ?
Wei uses the 95 lower confidence limit as the
estimate of mean potency at expiry while ADG use
the estimated potency itself.
43
Release Limits
Comparing the Two Approaches
Consider what happens to the release limits when
there is no degradation (b ? 0, sb ? 0) and the
variability among the slopes is negligible
(sslope ? 0). Assume the lot is only tested at
release (time 0) and all terms are known.
Wei
ADG
44
Release Limits

• Weis approach appears to be too conservative.
  It requires the 95 lower confidence limit to be
  above R to have high confidence that the true
  potency is above L.
• ADGs approach agrees with the no-degradation
  derivation discussed earlier.

45
Final Thoughts
Putting it All Together...Ideally
Instability Assay Variability
Assay Variability
Process/Assay Capability
As the manufacturing process and assay improve,
the action limits move close to the target and
the release limits move close to the expiry limits
46
Final Thoughts

• For highly variable assays/processes, the release
  limits may be so large as to be unrealistic
• Inside the action limits
• At a level that would require the manufacture to
  target an unobtainable potency level
• Replication is the short-term solution to this
  problem.
• Long-termprocess has to improve

47
Final Thoughts
What Happens Between the Action and Release
Limits?
Action Limits Derived by Statistical Means
Potency
Potency
Release Limit Derived from Expiry Limit, Product
Degradation and Assay Variability
Scenario 2
Scenario 1
48
Final Thoughts
Fundamental Difference Between Pharmaceutical
Manufacturer and Regulatory Body
Manufacturer
Reg. Agency
Scenario 1
Scenario 2
49
Final Thoughts
Lastly, the Statisticians Credo Give Us More!

• Statistically derived limits are only as good as
  the data that were used to generate them. Plan
  to re-evaluate them, especially if set early in
  the development of a new product.

50
Suggested Readings
Non-Statistical

• ICH Guideline Q6A Specifications Test
  Procedures and Acceptance Criteria for New Drug
  Substances and New Drug Products Chemical
  Substances
• ICH Guideline Q6B Specifications Test
  Procedures and Acceptance Criteria for
  Biotechnological/Biological Products
• Federal Register, Vol. 63, No. 110, June 1998,
  Notices (note this is essentially the same as
  Q6B)
• Federal Register, Vol. 64, No. 159, August 1999,
  Notices
• Boddy, A.W., et al, (1995), An Approach for
  Widening the Bioequivalence Acceptance Limits in
  the Case of Highly Variable Drugs, Pharm. Res.,
  Vol. 12, No. 12.
• Hayakawa, T. (1997), Global Perspective on
  Specifications for Biotechnology Products -
  Perspective from Japan, Development of
  Specifications for Biotechnology Pharmaceutical
  Products. Dev. Biol. Stand. Vol 91, Brown, F.
  and Fernandez, J. (eds).

51
Suggested Readings

• Baffi, R. (1997), The Role of Assay Validation in
  Specification Development, Development of
  Specifications for Biotechnology Pharmaceutical
  Products. Dev. Biol. Stand. Vol 91, Brown, F.
  and Fernandez, J. (eds).
• Geigert, J. (1997), Appropriate Specifications at
  the IND Stage, Development of Specifications for
  Biotechnology Pharmaceutical Products. Dev.
  Biol. Stand. Vol 91, Brown, F. and Fernandez, J.
  (eds).
• PhRMA CMC Statistics and Stability Expert Teams
  (2003), Identification of Out-of-Trend Stability
  Results A Review of the Potential Regulatory
  Issue and Various Approaches, Pharmaceutical
  Technology, April, Pages 38 52.
• Dillard, R.F. (2002), Statistical Approaches to
  Specification Setting with Application to
  Bioassay, The Design and Analysis of Potency
  Assays for Biotechnology Products (Brown, F. and
  Mire-Sluis, A., eds), Developments in
  Biologicals, Basel Karger, vol 107, pages
  117-127.

52
Suggested Readings
Statistical

• Hahn, G. and Meeker, W. (1991), Statistical
  Intervals A Guide for practitioners, John Wiley
  Sons, Inc., New York, N.Y.
• Wald, A. and Wolfowitz, J. (1946), Tolerance
  Limits for a Normal Distribution, Annals of
  Mathematical Statistics, 17.
• Gardiner, D. and Hull, N. (1966), An
  Approximation to Two-Sided Tolerance Limits for
  Normal Populations, Technometrics, Vol. 8, No. 1.
• Owen, D. B. (1968), A Survey of Properties and
  Applications of the Noncentral t-Distribution,
  Technometrics, Vol. 10, No. 3.
• Odeh, R. E. and Owen, D. B. (1980), Tables for
  Normal Tolerance Limits, Sampling Plans and
  Screening, New York Marcel Dekker, Inc.
• Beckman, R. and Tietjen, G. (1989), Two-Sided
  Tolerance Limits for Balanced Random-Effects
  ANOVA Models, Technometrics, Vol. 31, No. 2.

53
Suggested Readings

• Thomas, J. and Hultquist, R. (1978), Interval
  Estimation for the Unbalanced Case of the One-Way
  Random Effects Model, Annals of Statistics, 3.
• Burdick, R. and Graybill, F. (1992), Confidence
  Intervals on Variance Components, Statistics
  textbooks and monographs, Vol. 127., Edited by
  Owen, D.B., Marcel Dekker, Inc., New York, NY.
• Faulkenberry, G. and Weeks, D. (1968), Sample
  Size Determination for Tolerance Limits,
  Technometrics, Vol. 12.
• Quesenberry, C. (1993), The Effect of Sample Size
  on Estimated Limits for X and X Control Charts,
  Journal of Qual. Tech., Vol. 25, No. 4.
• Burr, I. (1976), Statistical Quality Control
  Methods, Vol. 16 of the series Statistics
  textbooks and monographs, Owen, D. (ed)., Marcel
  Dekker, New York, N.Y.

54
Suggested Readings

• Satterthwaite, F.E. (1946) An Approximate
  Distribution of Estimates of Variance
  Components, Biometrics Bulletin, 2, 110-114.
• Mandel, J. (1984) Fitting Straight Lines When
  Both Variables are Subject to Error, J. of
  Quality Technology, vol. 16, no. 1, 1 14.
• Allen, Paul V., Dukes, Gary R., and Gerger, Mark
  E. (1991), Determination of Release Limits A
  General Methodology, Pharm. Res., 8(9), 1210 -
  1213.
• Wei, Greg C. G. (1998), Simple Methods for
  Determination of the Release Limits for Drug
  Products, J. of Biopharmaceutical Stat., 8(1),
  103-114.
• Graybill, F.A. (1976) Theory and Application of
  the Linear Model, North Situate MA Duxbury Press

55
Back-up Slides
56
Action Limits

• To illustrate the difference between the naïve
  interval and a tolerance interval, a simulation
  study was performed as follows
• 10000 independent samples of size n 5 and 25
  were drawn from a normal population with mean
  100 and standard deviation 10. (5000 samples
  of size n 120.)
• The naïve interval was
  calculated for each sample.
• A 95/95 tolerance interval
  was calculated for each sample for n 5 the
  95/95 tolerance interval
  
  was calculated for
  each sample for n 25.

57
Action Limits
58
Action Limits
59
Action Limits
60
Action Limits
61
Action Limits
Further Statistics from the Simulation Studies
62
Action Limits

• Normality is critical assumption
• Distribution-free tolerance intervals can be used
  if assumption violated
• Based on order statistics
• Generally require quite large sample sizes to
  achieve typical coverage and confidence levels
• Hahn and Meeker (1991)

63
Action Limits

• When one analytical method is to be replaced by
  another, it is often necessary to translate the
  existing action limits into action limits for the
  new method.
• Often there is a linear relationship between the
  two methods. Taking this into account is
  necessary in the derivation of the new action
  limits.

64
Action Limits

• In many cases, the linear relationship has to be
  estimated using errors-in-variables regression.
• Mandel (1984)
• When the x method has small measurement error
  relative to the y method or when Berksons
  condition holds, then usual linear regression
  techniques can be used.

65
Action Limits
Prediction Line Upper Tolerance Bound
a bx0 gs UALNM
a bx0
New Method
x0 LALCM
Current Method
66
Action Limits
Simple Linear Regression Model
Assumptions
67
Action Limits
Upper Action Limit at the Point x0
Graybill (1976)
68
Action Limits

• When significant measurement error variability
  exists in both methods, a reasonable approach
  would be to use the estimates of a, ß and se from
  the errors-in-variables regression in the
  derivation given on the previous slide.