The Mean versus Individual Values: A Key to Developing a Risk Based Quality System

1
The Mean versus Individual ValuesA Key to
Developing a Risk Based Quality System

• Tim Schofield
• GlaxoSmithKline
• 2009 Non-Clinical Biostatistics Conference
• Harvard School of Public Health, Boston, MA
• October 23, 2009

2
Outline

• Goals and development of a quality system
• Hurdles to achieving an effective quality system
• Ambiguities in stability evaluation
• Method validation and transfer
• Definition of design space

3
Goals and development of a quality system

• Outline
• Goals and development of a quality system
• Hurdles to achieving an effective quality system
• Ambiguities in stability evaluation
• Method validation and transfer
• Definition of design space

• Goal to help assure safe and effective therapies
  and vaccines for patients
• Achieved through strategic product development
  and appropriate bridging of manufacturing to
  development experience
• Development and maintenance of key analytical
  methods
• Coordinated product development
• Preclinical and clinical studies that reveal the
  boundaries of critical quality attributes which
  forecast patient safety and efficacy
• Establishment of appropriate specifications
  (explicit protection) and control limits
  (implicit protection)
• Vigilance to important product characteristics
  including stability
• Requires a vision towards information management
  throughout the product lifecycle

4
Hurdles to achieving an effective quality system

• Outline
• Goals and development of a quality system
• Hurdles to achieving an effective quality system
• Ambiguities in stability evaluation
• Method validation and transfer
• Definition of design space

• Historical modes of development and operation are
  difficult to change in an aggressive competitive
  environment
• In the interest of efficiency over effectiveness,
  many departments operate in silos
• Basic research, nonclinical development, and
  clinical development
• Process research, formulation development, and
  analytical development
• Regulatory guidelines and expectations engender a
  disincentive for collecting valuable information
• USP General Notices
• FDA OOS Guidance

5
Hurdles to achieving an effective quality system
(cont.)
It may be appropriate to specify in the test
method that the average of these multiple assays
is considered one test and represents one
reportable result. ? Reportable result the
value that is held to the specification (AC) In
cases where a series of assay results (to produce
a single reportable result) are required by the
test procedure and some of the individual results
are OOS, some are within specification, and all
are within the known variability of the method,
the passing results are no more likely to
represent the true value than the OOS results.
For this reason, a firm should err on the side of
caution and treat the reportable average of these
values as an OOS result, even if the average is
within specification. (emphasis mine) --
Guidance for Industry Investigating out of
specification (OOS) test results for
pharmaceutical production (2006)
6
Hurdles to achieving an effective quality system
(cont.)

• A quality system paradigm The quality system
  should help assure that product in the market
  conforms to the attributes of materials tested
  during clinical development
• Individual doses/measurements are not linked to
  clinical outcomes

Quality Control
Development
Bioassay
Clinical
Clinical
Quality Attribute
Bioassay
Quality Attribute
95 Efficacy
6
7
Ambiguities in stability evaluation

• Outline
• Goals and development of a quality system
• Hurdles to achieving an effective quality system
• Ambiguities in stability evaluation
• Method validation and transfer
• Definition of design space

• Shelf-life determination versus stability
  monitoring
• ICH shelf-life determination aims at the mean
  product profile

• . . . however, FDA inspectors cite stability OOS
  results during post licensure studies
• Ex., a batch which yields a 24-month shelf-life
  pre-licensure would have a 30 chance of
  yielding a stability OOS if tested post licensure

7
8
Ambiguities in stability evaluation (cont.)

• Shelf-life determination versus stability
  monitoring (cont.)
• One approach to mitigating risk of a post
  licensure stability OOS establish shelf-life
  based on protecting individual stability
  measurements
• Historical practice in some pharma companies
• WCBP Strategy Forum on Stability solution to
  post licensure stability OOS was offered as a
  late breaking presentation
• An option offered in the approach being developed
  by the PQRI Stability Shelf Life Working Group

9
Ambiguities in stability evaluation (cont.)

• Shelf-life determination versus stability
  monitoring (cont.)
• Solutions which warrant individual stability
  measurements do not contribute to the quality of
  pharmaceuticals or vaccines
• Controlling stability measurements protects the
  stability unit rather than the customer
• Under some strategies this would force a company
  to set excessively wide specifications which
  might increase customer risk
• Holding individual stability measurements to
  specifications creates a disincentive for
  collecting data
• Contrary to QbD which advocates for data
  collection to facilitate process understanding
  and control

10
Ambiguities in stabilityevaluation (cont.)

• Shelf-life determination versus stability
  monitoring (cont.)
• More valuable to work on stability study design
  and approaches which appropriately model product
  stability
• Mixed effects modeling rather than the ICH
  poolability strategy
• The ICH approach creates a disincentive for
  increasing the number of batches (increased power
  to detect a difference in slopes/intercepts)
• Fixed power approach by Ruberg Stegeman (1991)
• A random batches approach can be instituted with
  more development batches and/or as part of
  continuous development throughout the product
  lifecycle
• Bayesian approaches can be utilized to leverage
  prior knowledge
• Incorporates appropriate modeling of intra- and
  inter-assay variability

11
Method validation and transfer

• Outline
• Goals and development of a quality system
• Hurdles to achieving an effective quality system
• Ambiguities in stability evaluation
• Method validation and transfer
• Definition of design space

• The classical approach to method validation and
  transfer has been to assess performance
  characteristics of the method
• Mean accuracy and intermediate precision
• An equivalence approach has been proposed to
  demonstrate conformance to acceptance criteria
• Transfer - H1 lab1-lab2 lt ??
• USP lt1033gt - H1 Relative bias lt ?
• Studies are not sufficiently powered to apply
  this approach to intermediate precision
• USP lt1033gt proposes using variance component
  analysis to establish assay format
• Like stability, method performance can be
  re-evaluated after adequate experience has been
  achieved

USP lt1033gt Bioassay Validation
12
Method validation and transfer (cont.)

• The classical approach ignores the combined
  impact of bias and variability
• Boulanger et.al. suggest the use of a
  ?-expectation tolerance interval to warrant
  future results

If the ?-expectation tolerance interval falls
within the acceptance interval (?T?? ?), then the
probability that any future result will fall
within the acceptance interval is ??.
Boulanger, B., 2009 FDA/Industry Statistics
Workshop
13
Method validation and transfer (cont.)

• Comparison of equivalence versus tolerance
  approaches
• For a fixed ?/? and study size there is a
  significantly greater risk of failing the study
  using the ?-expectation TI approach versus the
  equivalence approach
• Transfer of Methods Supporting Biologics and
  Vaccines (Liu, R.,et.al. 2009)
• However the criteria on individual results
  cannot be the same as the criteria on a mean of
  many results. (Boulanger, B., et.al. 2009)
• ? would have to be larger than ? to harmonize the
  risks but this impacts the customer risk of
  receiving a bad result (under ?)

• An equivalence approach might be appropriate to
  CMC methods, while tolerance approach should be
  utilized for bioanalytical methods

14

• Outline
• Goals and development of a quality system
• Hurdles to achieving an effective quality system
• Ambiguities in stability evaluation
• Method validation and transfer
• Definition of design space

Definition of design space

• Design Space the design space is the
  established range of process parameters that has
  been demonstrated to provide assurance of
  quality.
• - emphasis mine -
• Formal Experimental Design a structured,
  organized method for determining the relationship
  between factors (Xs) affecting a process and the
  output of that process (Y). Also known as Design
  of Experiments.

ICH Q8(R2), Pharmaceutical Development
14
15
Definition of design space (cont.)

• ICH Q8(R2) shows design space as the intersection
  of a specification with the response surface
  associating process factors with a quality
  attribute

Spec

• This is the mean region where the mathematical
  model formulated from a series of experimental
  runs meets specs
• How do we assure quality using this approach?
• What is the experimental unit

Design space example due to Greg Stockdale (2009)
16
Definition of design space (cont.)

• Peterson, et.al. (2009) have suggested a Bayesian
  multivariate approach to definition of design
  space to assure quality
• From this design space can be defined as the
  region with suitable probability (reliability) of
  meeting specification

Reliability Heat Map
17
Definition of design space (cont.)

• What do we mean by meeting specification
• What is the experimental unit?

• The process average?
• No, because of batch-to-batch variability
• The batch average?
• Yes, because specifications relate to batchs
• An individual dosage unit?
• No, because specifications dont relate to
  individual dosage units
• An individual assay result?
• No, because the measurement process is not linked
  to patient outcome

Process
Batch 1
Batch 2
Batch 3
YAssay
18
Definition of design space (cont.)

• Regulatory challenge
• The process design space is a part of the NDA/BLA
• Should describe the batch process and the impact
  of changes on overall process variability
• Should be flexible to changes in the measurement
  system which impacts measurement variability
• Technology changes
• Assay format changes
• Measurement variability should be the subject of
  the process control strategy rather than the
  development of design space
• Specification development

19
Summary

• Many areas of CMC development and commercial QC
  are impacted by the issue of whether decisions
  are made from individual measurements or from
  averages
• Agreeing to averages opens up opportunities for
  strategic design and analysis of key development
  experiments, as well as important quality
  initiatives (QbD)
• Statisticians provide not only design and
  analysis expertise, but also strategic thinking
  regarding rational goals in ensuring quality of
  pharmaceutical products

20
References

• Guidance for Industry Investigating
  Out-of-Specification (OOS) Test Results for
  Pharmaceutical Production, U.S. Department of
  Health and Human Services, Food and Drug
  Administration, Center for Drug Evaluation and
  Research (CDER), October 2006, Pharmaceutical
  CGMPs
• Ruberg, S.J., Stegeman, J.W., 1991. Pooling data
  for stability studies testing the equality of
  batch degradation slopes. Biometrics 47,
  10591069.
• Bruno Boulanger, Eric Rozet, Francois Moonen,
  Serge Rudaz, Philippe Hubert, A risk-based
  analysis of the AAPS conference report on
  quantitative bioanalytical methods validation and
  implementation, Journal of Chromatography B, 877
  (2009) 22352243
• Rong Liu, Timothy L. Schofield, and Jason J.Z.
  Liao, Transfer of Methods Supporting Biologics
  and Vaccines, accepted by Statistics in
  Biopharmaceutical Research, 2009.
• USP lt1033gt Bioassay Validation, Pharmacopeial
  Forum, March/April, 2009.
• ICH Q8(R2), Pharmaceutical Development, Current
  Step 4 version, August 2009
• Peterson, J. J., A Posterior Predictive Approach
  to Multiple Response Surface Optimization,
  Journal of Quality Technology, 2004, 36, 139-153.
  
• Peterson, J. J. A Bayesian Approach to the ICH Q8
  Definition of Design Space, Journal of
  Biopharmaceutical Statistics, 2008, 18, 959-975.
• Stockdale, G. and Cheng, A. , Finding Design
  Space and a Reliable Operating Region using a
  Multivariate Bayesian Approach with Experimental
  Design, Quality Technology and Quantitative
  Management, 2009, (online at the QTQM web site).

21
Acknowledgements

• The PQRI Stability Shelf Life Working Group
• Stan Altan (JJ)
• Bruno Boulanger (UCB)
• Jinglin Zhong (FDA)
• John Peterson (GSK)
• Seth Clark (Merck)