Claude Beigel, PhD.

1
Practical session metabolites Part II goodness
of fit and decision making

• Claude Beigel, PhD.
• Exposure Assessment Senior Scientist
• Research Triangle Park, USA

2
Recommended Tools for Assessing Goodness of Fit

• Same recommended approach as for parent substance
• Combination of visual assessment and statistical
  tests
• Visual assessment, although bringing some level
  of subjectivity, is necessary to discern between
  normal data variability (scattering) and
  systematic model deviation, this is not done by
  the statistical tests

3
Recommended Tools for Assessing Goodness of
FitVisual Assessment

• Visual check of model description of measured
  data and distribution of residuals (plot of
  residuals, Predicted - Observed)
• Systematic deviation indicates kinetic model may
  not be appropriate (unless deviation can be
  attributed to experimental artifacts)

Example 8.2 of report, SFO-SFO fit
4
Recommended Tools for Assessing Goodness of
FitVisual Assessment

• Residuals should be randomly distributed on
  vertical axis

Example 8.2 of report, SFO-FOMC fit
5
Recommended Tools for Assessing Goodness of
FitStatistical Indices

• Chi2 (?2) statistical test
• Minimum error percentage to pass ?2 test at a 5
  significance level
• Needs to be performed for each substance
  individually, to avoid that good fit of the main
  substances (parent and/or major metabolite)
  overshadows goodness of fit of more minor
  substances (weighting issue)
• Calculated from fitted versus observed substance
  data (use of average values for replicates is
  recommended)
• Degrees of freedom for the substance defined as
  number of substance data points used in ?2 test
  minus number of estimated parameters for the
  substance
• Do not count replicates if averages used,
  excludes data points set to 0 (metabolite at
  time-0) or not counted (ltLOD/LOQ)
• Metabolite parameters defined as metabolite
  formation fraction and degradation rate
  parameters (dependent of kinetic model used)

6
Recommended Tools for Assessing Goodness of
FitStatistical Indices

• One-sided t-test for evaluating uncertainty of
  rate constant parameters
• To determine whether rate is significantly
  different from 0
• If p lt 0.05, parameter is considered
  significantly different than zero
• If p between 0.05 and 0.1, weight of evidence
  should be considered
• Especially important for metabolites that do not
  show a clear decline
• Because parameters in parent metabolite fits
  (formation and degradation parameters) can be
  highly correlated, the t-test is performed at
  final step (all parameters fitted together)
• Degrees of freedom defined as number of data
  points (including replicates) minus number of
  fitted parameters

7
Recommended Tools for Assessing Goodness of
FitData Handling / Methodology

• Basic data handling
• Paste ModelMaker output (integration table) in
  Excel spreadsheet
• Extract fitted values corresponding to measured
  times for each substance
• Average replicates if necessary
• (an automated Excel spreadsheet may be created
  for that purpose, but not available yet)
• Minimum ?2 error for metabolites may be
  calculated using Parent degradation kinetics.xls
  file
• Paste measured Vs. fitted values in Chi2 all
  models worksheet, update number of parameters
  cell and click calculate

8
Recommended Tools for Assessing Goodness of
FitData Handling / Methodology

• Residuals may be plotted in Parent degradation
  kinetics.xls file
• Valid only for 1- or 2-replicate data sets (if
  more, needs to be done manually)
• Paste measured Vs. fitted values (all replicates)
  in SFO no-reps or SFO 2-reps worksheet
• t-test for rate constant parameters may be
  performed using provided t-test.xls file
• For each rate constant parameter, enter parameter
  estimate and standard error, number of data
  points and number of parameters estimated

9
Hands-on Example 1

• Exercise 1
• Open ModelMaker file for example 1
• From result table, extract fitted value for each
  sampling time, write down in output tables for
  parent, metabolite1 and metabolite2
• Enter values in Metabolitesexamplesoutput.xls,
  averages are calculated automatically

10
Hands-on Example 1

• Visual assessment
• Check ModelMaker plot of fit and answer following
  questions for each substance
• Does fitted line adequately describe data, are
  there obvious over- or under-predictions
  (including day-0)
• Plot residuals for each substance and answer
  following questions
• Do residuals show distinct pattern, are most of
  the points above or below 0-line, what is the
  magnitude?
• Statistical indices
• Calculate minimum ?2 error percentage for each
  substance
• Perform t-test for all rate constant parameters
  (parent and metabolites) and record P-value and
  conclusion

11
Hands-on Example 1Visual Assessment
12
Hands-on Example 1Statistical Indices
13
Hands-on Example 2, Parent FOMC

• Exercise 2
• Open ModelMaker file for example 2, parent FOMC
• From result table, extract fitted value for each
  sampling time, write down in output tables for
  parent and metabolite
• Enter values in Metabolitesexamplesoutput.xls,
  averages are calculated automatically
• Perform visual assessment and calculate
  statistical indices

14
Hands-on Example 2, parent FOMCVisual Assessment

15
Hands-on Example 2, parent FOMCStatistical
Indices
16
Hands-on Example 2, parent DFOP

• Exercise 3
• Open ModelMaker file for example 2, parent DFOP
• From result table, extract fitted value for each
  sampling time, write down in output tables for
  parent and metabolite
• Enter values in Metabolitesexamplesoutput.xls,
  averages are calculated automatically
• Perform visual assessment and calculate
  statistical indices

17
Hands-on Example 2, parent DFOPVisual Assessment

18
Hands-on Example 2, parent DFOPStatistical
Indices
19
Hands-on Example 2, Metabolite Decline

• Exercise 4
• Open ModelMaker file for example 2, metabolite
  decline
• From result table, extract fitted value for each
  sampling time, write down in output tables for
  parent and metabolite
• Enter values in Metabolitesexamplesoutput.xls,
  averages are calculated automatically
• Perform visual assessment and calculate
  statistical indices

20
Hands-on Example 2, Metabolite DeclineVisual
Statistical Assessment
21
Decision MakingTrigger Endpoints

• Based on visual assessment and statistical
  indices, is SFO model acceptable for the
  metabolite (in combination with best-fit model
  for parent)?
• Yes ? use SFO DT50/90 endpoints
• No and clear decline of metabolite, use FOMC
  model for metabolite (in combination with
  best-fit model for parent)
• If FOMC acceptable based on visual assessment and
  statistical indices, use FOMC DT50/90 endpoints
• If not, model decline of metabolite with best-fit
  model and use decline DT50/90 as conservative
  endpoints
• No and no apparent decline of metabolite
• Assess relevance of study with regard to
  metabolite
• Check other studies
• Study with metabolite may be needed

22
Decision MakingModeling Endpoints

• Based on visual assessment and statistical
  indices, is SFO model acceptable for the
  metabolite (in combination with appropriate model
  for parent)?
• Yes ? use modeling endpoints for metabolite
• No and clear decline
• If formation fraction estimate is reliable, use
  with decline rate constant as conservative
  endpoints
• If not, use formation fraction of 1 with decline
  rate constant as conservative endpoints
• If metabolite biphasic, use appropriate
  higher-Tier approach (e.g. DFOP, PEARL)
• If terminal metabolite and biphasic, use FOMC
  DT90/3.32 as half-life
• No and no apparent decline of metabolite
• Assess relevance of study with regard to
  metabolite
• Check other studies
• Study with metabolite may be needed

23
Hands-on Examples

• Determine appropriate trigger and modeling
  endpoints for Example 1 metabolites 1 and 2 and
  Example 2 metabolite