Influenza Neuraminidase Inhibitor IC50 Data: Calculation, Interpretation and Statistical Analyses

1
Influenza Neuraminidase Inhibitor IC50
DataCalculation, Interpretation and Statistical
Analyses
2
Presentation Outline

• Determining IC50 values
• Curve-fitting methods
• Sources of variation
• Identifying IC50 outliers
• Determining cut-offs/thresholds
• Outlier values versus resistant viruses
• Monitoring trends over time in IC50 data

3
Abbreviations

• NA Neuraminidase
• NI Neuraminidase Inhibitor
• RFU Relative Fluorescence Units
• RLU Relative Luminescence Units
• VC Virus Control

4
Determining NI IC50 Values

• IC50 The concentration of NI which reduces NA
  activity by 50 of the virus-control, or upper
  asymptote
• Estimated by Measuring the NA activity (RFU or
  RLU) of an isolate against a range of dilutions
  of the drug, as well as without drug
  (virus-control)

5
Determining IC50 Values Calculation Options

• Curve fitting-Statistical software
• Graph Pad Prism (www.graphpad.com)
• 595
• Grafit (www.erithacus.com/grafit/)
• 400-500
• Jaspr (Developed by CDC)
• Contact CDC for suitability and further details
• Point-to-Point calculation
• Excel templates (Created by Health Protection
  Agency, UK)

6
IC50 Calculation Curve-Fitting (Graph-pad)
Upper Asymptote
Isolate Mean IC50(nM)
A/Eng/683/2007 1.2
A/Eng/684/2007 586
A/Eng/685/2007 1.1
A/Eng/686/2007 1.5
Non linear regression analysis Sigmoidal dose
response curve
7
IC50 Calculation Curve-Fitting (Grafit)
Isolate Mean IC50(nM)
A/Eng/683/2007 1.1
A/Eng/684/2007 620
A/Eng/685/2007 1.1
A/Eng/686/2007 1.5
8
IC50 Calculation Point to Point
Isolate Mean IC50(nM)
A/Eng/683/2007 1.1
A/Eng/684/2007 608.5
A/Eng/685/2007 1.0
A/Eng/686/2007 1.4
9
IC50 Calculation Comparison of IC50 values from
different calculation methods
Isolate GraphPad Grafit Point to Point
A/Eng/683/2007 1.2 1.1 1.1
A/Eng/684/2007 586 620 608.5
A/Eng/685/2007 1.1 1.1 1.0
A/Eng/686/2007 1.5 1.5 1.4
10
Determining IC50 Values Sources of Variation

• Method of calculation
• Using point-to-point or curve fitting software
• Choice of curve fitting software used
• Intra-assay variation
• Difference between 2 or more replicates
• Inter-assay variation
• Difference in calculated value for a given
  isolate in multiple assays

11
IC50 Calculation Comparison of Point to Point
versus curve-fitting
Isolate Point to Point GraphPad Ratio
292R 0.69 0.69 1.00
292K gt4000 8551 -
119V 40.3 40.7 1.01
A/Lisbon/22/2007 0.59 0.56 1.05
A/Latvia/685/2007 0.68 0.69 1.01
A/Denmark/1/2007 0.43 0.41 1.05
A/Denmark/2/2007 0.54 0.51 1.06
Curve Fitting WILL give an IC50 value by
extrapolating the curve when drug dilutions do
not reach a true end point This does not
necessarily give an accurate IC50 value
12
IC50 Calculation Troubleshooting

• Important to examine curves carefully to ensure
  IC50 is valid

Low VC technical error
Poor curve fit
Drug titration error
Poor curve fit
13
Comments Choice of IC50 Calculation Method

• Choice of IC50 calculation method will make no
  more than about 5 difference to IC50 value, for
  most samples
• Must be clear exactly how the curve fitting and
  calculation of IC50 is working
• Is IC50 based on 50 of RFU/RLU of VC or 50 of
  fitted upper asymptote.
• Regardless of method used, careful examination of
  the curve produced is required to identify
  technical issues.
• See presentation on validation and
  troubleshooting of IC50 testing methods

14
Analysis of intra-assay variation
Isolate Replicate 1 Replicate 2 Ratio
292R 0.75 0.63 1.19
292K 3769 gt4000 -
119V 42.0 38.6 1.09
A/Lisbon/22/2007 0.57 0.61 1.07
A/Latvia/685/2007 0.70 0.66 1.06
A/Denmark/1/2007 0.40 0.46 1.15
A/Denmark/2/2007 0.58 0.49 1.18
15
Comments Intra-assay Variation

• Variation between replicates in the same assay
  can be 15-20.
• This variation is greater than that seen with
  changes to curve-fitting method
• Using replicates and taking the average reduces
  this effect.
• A large difference between replicates (e.g. gt30)
  of a given virus indicates a technical issue
• In these instances repeat testing should be
  performed

16
Analysis of Inter-assay Variation
Introduction of new drug batch
17
Analysis of Inter-assay Variation
Isolate Assay 1 Assay 2 Ratio
274H Oseltamivir 0.42 0.36 1.17
274H Zanamivir 0.17 0.22 1.29
274Y Oseltamivir 300 303 1.01
274Y Zanamivir 0.31 0.37 1.19
119V Oseltamivir 43.7 34.0 1.29
119V Zanamivir 1.41 1.32 1.07
152K Oseltamivir 920 1316 1.43
152K Zanamivir 239 3.51 1.47
18
Comments Inter-assay Variation

• Variation in IC50 values for a virus in multiple
  assays can be 50.
• Control viruses should be included in every assay
  to identify technical issues.
• Control viruses should be validated, and have a
  defined range between which the IC50 is valid.
• Assays in which the control virus IC50 falls
  outside the accepted range should be reaped in
  their entirety.

19
Conclusions Determining IC50 Values

• Several methods for IC50 calculation available at
  a range of price and sophistication
• Variation due to choice of IC50 calculation
  method is minimal (5-10) in comparison with
  intra-assay (20) and inter-assay (50)
  variation.
• Choice of curve-fitting method should be made
  based on individual laboratory circumstances
• All variation can be minimised using appropriate
  assay controls (reference/control viruses,
  validation of curves generated)
• Consistency in methodology used (statistical and
  laboratory) is important for long term analysis
  (time trends)

20
Identifying IC50 outliers

• Aim identify isolates with higher (or lower)
  than expected IC50 values (outliers)
• First determine the normal range of IC50 values
• Each
• Various statistical methods may be used
• Critical to ensure that any outliers do not
  unduly affect the cut-off/threshold
• Outlier does not equal resistant
• Identifies isolates that may be worth further
  investigation (retesting/sequencing)

21
Identifying IC50 outliers Commonly Used
Statistical Methods

• SMAD
• Robust estimate of the standard deviation based
  on the median absolute deviation from the median
• Box and Whisker plots
• Graphical representation of the 5 number summary
  of the data (the sample minimum, the lower or
  first quartile, the median, the upper or third
  quartile, the sample maximum)
• Both methods require a minimum dataset to perform
  robust analyses (gt20)
• Cut offs can be calculated mid-season, once a
  reasonable number of samples has been tested, to
  monitor outliers
• At the end of the season, cut offs can be updated
  and a retrospective analysis of all season data
  performed.

22
Using SMAD Analysis

• Create a scatter plot of all data
• Useful to see the spread and trend of the data
• Log transform the data
• Calculate a robust estimate of the standard
  deviation based on the median absolute deviation
  from the median using log10 data
• Templates for this analyses are available from
  HPA, UK
• Major outliers all those with values more than
  3SD above the median
• Minor Outliers all those more than 1.65SD above
  the median

23
Using SMAD Analysis Example Data
Early-Mid Season Estimate Early-Mid Season Estimate
Median 0.97
Robust SD 1.27
Minor Outlier (1.65SD) 1.45
Outlier (3SD) 1.99
Post Season Estimate Post Season Estimate
Median 1.1
Robust SD 1.25
Minor Outlier (1.65SD) 1.56
Outlier (3SD) 2.11
24
Using Box and Whisker Analysis

• This analysis can be performed in Graphpad Prism,
  with the box and whisker plots drawn
  automatically
• Calculations can be done in excel, but drawing
  the box and whisker plots is more complicated
• A template for plotting the graphs is available
  from Adam Meijer
• Log transform the data
• Calculate the median, upper quartile, lower
  quartile, interquartile range, upper minor and
  major fences, and lower major and minor fences
• Mild outliers lie between the minor and major
  fences
• Extreme outliers lie outside the major fence

25
Box and Whisker Plots Principle
Excel Formulae Upper quartile (Q3)
(QUARTILE(B2B150,3) Lower quartile )Q1)
(QUARTILE(B2B150,1) IQR Q3-Q1
Mild outlier
Extreme outlier
Equivalent values in SMAD analysis
26
Using Box and Whisker Analysis Example Data
Excel Output
Graphpad Prism Output
Box and Whisker Box and Whisker
Median 1.08
Mild outlier lower fence (1.5IQR) 0.55
Extreme outlier lower fence (3IQR) 0.34
Mild outlier higher fence (1.5IQR) 1.95
Extreme outlier higher fence (3IQR) 3.14
27
Do we need to log-transform?

• Most results from dilution assays produce
  geometric results so likely to be sensible
• Sometimes data are skewed. (e.g. lower quartile
  much closer to median than upper quartile)
• Important to log transform as robust methods
  assume data are normal once outliers are removed.

28
Using Log10 versus non logged Data
Non Log Data Non Log Data
Median 0.72
Robust SD 0.50
Minor Outlier (1.65SD) 1.55
Outlier (3SD) 2.23
Log10 Data Log10 Data
Median 0.72
Robust SD 0.50
Minor Outlier (1.65SD) 1.55
Outlier (3SD) 2.23
29
Impact of Excess Numbers of Outliers

• If the data have a large number of outliers, both
  SMAD and BW struggle to determine sensible cut
  offs.
• As resistant virus is very clearly different,
  these values can be removed prior to analysis to
  allow sensible calculations of cut offs for the
  remaining data.
• Below, data is shown for H1N1 in 2007/8, when
  sensitive and resistant virus co-circulated.
• Cut offs calculated do not accurately apply to
  the sensitive IC50 data

Sensitive and Resistant Isolates
Resistant Isolates removed
30
Conclusions Identifying Outliers

• Determining cut offs/thresholds identifies those
  isolates with IC50 values higher than the normal
  range
• Cut offs/thresholds need to be subtype specific
• Season specific cut offs are useful, if enough
  data is generated in one season, but data from
  multiple seasons can be merged to perform a more
  reliable analyses
• Box and whisker plots and SMAD analyses generate
  slightly different cut offs
• Q31.5xIQR (mild outlier cut off) is equivalent
  to 2.7SD from SMAD
• Q33xICR is equivalent to 4.7SD from SMAD.
• Cut offs calculated by box and whisker analyses
  are higher than those from SMAD analyses.
• Choice depends on individual laboratory
  preference
• Box and whisker plots present the data well
• Both methods minimise the impact of outlier
  values on the analyses, but both will fail once
  too many outliers are present
• Data begins to have two populations

31
Monitoring Trends Over Time

• The normal range of IC50 values for a particular
  subtype can change over time
• This could be seen by an increase in the number
  of outliers, or by changes in the median
• Simple to monitor, using the methods already
  described for identifying outliers
• scatter plots/box-whisker
• Other statistical methods can be used to further
  analyse data from several seasons

32
Trends in IC50 Data Scatter Plot
33
Trends in IC50 Data Box and Whisker
34
Summary

• Good use of statistical methods can help
  interpret the IC50 results and ensure assay
  results are reliable.
• Analyses of data not only identifies individual
  outliers, but allows continuous monitoring of
  trends
• Retrospective analyses of multiple seasons of
  data can identify changes in viral
  characteristics and susceptibilities
• Do not use statistics without first looking at
  the data by scatter plot to find obvious
  deviations which require an adapted statistical
  approach
• Challenge is to find explanations for trends