Incorporating direct and indirect evidence using Bayesian methods

1
Incorporating direct and indirect evidence using
Bayesian methods

• An application to a NICE assessment of 2nd-line
  chemotherapy for ovarian cancer

2
The NICE process

• Systematic review of existing clinical-effectivene
  ss and cost-effectiveness evidence
• Includes an estimate of the cost-effectiveness
  for each technology considered
• From a decision-making perspective it is
  necessary to simultaneously compare all relevant
  comparators

3
Decision analysis approach

• Synthesise relevant data on effectiveness,
  resource use and value parameters
• Link this evidence to policy-relevant
  decision-making
• Decision-analytic models play a key role in NICE
  appraisal process

4
The Case Study

• Part of a Technology Assessment Report (TAR) for
  NICE
• 2 previous TARs separately examined topotecan and
  PLDH as 2nd-line chemotherapy for ovarian cancer
• Update was commissioned in part because previous
  assessments did not provide a simultaneous direct
  comparison of topotecan, PLDH and paclitaxel

5
Focus for this presentation

• The synthesis of clinical trial data on the
  relative effectiveness topotecan, paclitaxel and
  PLDH in the overall patient population with
  ovarian cancer
• This process is necessary to inform the adoption
  decision
• In this example a decision-analytic model was
  used to combine the effectiveness data with
  evidence on costs and patient preferences

6
Brief overview of model
7
Decision-analytic model - survival

• Model calculated overall survival as sum of two
  distinct periods
• Progression-free period
• Period from progression to death (equal to
  overall survival minus progression-free survival)
• These periods were quality-adjusted using utility
  weights to calculate QALYs

8
Decision-analytic model - costs

• Included the costs of
• Pre-medication
• The study drugs
• Drug administration
• Monitoring
• Adverse events
• Objective of model estimate lifetime costs and
  QALYs for topotecan, paclitaxel and PLDH

9
The Data
10
Clinical effectiveness data

• Systematic review identified 9 RCTs
• 4 excluded as comparator groups unlicensed
• Unlicensed comparator in each was uniquely
  represented so did not provide indirect evidence
  about treatments of interest
• 2 of remaining 5 included patients from
  restricted patient population
• Final 3 included patients from overall population
• Focus on these 3 studies

11
Focus - 3 trials in overall population

• PLDH versus topotecan
• Topotecan versus paclitaxel
• PLDH versus paclitaxel
• 3 different pair-wise comparisons of 3 treatments
  of interest
• No common comparator among all 3 trials
• No simultaneous direct comparison of all relevant
  comparators

12
Available clinical trial data

• Survival data were presented as
• median weeks overall survival and
  progression-free survival (PFS)
• hazard ratios between treatments
• Differences in available data
• Trial C provided data on overall survival but not
  PFS
• Trial C had shorter follow-up than trials A and B

13
Approaches to simultaneous comparison of
treatment effects - 1

• Base on common comparator
• Apply hazard ratios reported in trials A and B to
  common baseline (topotecan)
• Exclude information provided by trial C
• Possible to select alternative treatment as
  common comparator and use any 2 of the 3 trials
  in this approach
• Result will differ according to choice of common
  comparator
• Choice of comparator ultimately based on
  judgement of analyst

14
Approaches to simultaneous comparison of
treatment effects - 2

• Possible to incorporate all the evidence on
  treatment effects simultaneously in a mixed
  treatment comparison (MTC) model
• Combines direct and indirect evidence
• Does not rely on common comparator, but does
  require complete network
• i.e. each study have a treatment in common with
  at least 1 other study
• Choice of included trials based on judgement of
  analyst

15
Mixed treatment comparison

• Extends assumptions in simple meta-analysis to
  include principal of transitivity
• Had paclitaxel been included in trial A or PLDH
  included in trial B, the observed relative
  treatment effect of paclitaxel vs. PLDH would
  have been the same as that observed in trial C
• ?Pac_PLDH ?Pac_Top ?Top_PLDH
• where ? is the log hazard ratio

16
The 2 approaches

• Make direct comparison of all 3 treatments using
  a common comparator, topotecan
• exclude trials that do not include this common
  comparator
• Use MTC model to combine evidence from all trials
• assume relative hazard exchangeable between trials

17
Comparing the 2 approaches

• Both approaches were implemented in the Bayesian
  inference software program WinBUGS
• Log hazard ratios assumed to be normally
  distributed about a true underlying effect size,
  ?, according to precision (1/variance), ?2
  observed in the clinical trials
• Log(HRPac_Top) N(?Pac_Top, ?2Pac_Top)

18
Mean survival

• Hazard ratios from both approaches were applied
  to a baseline absolute hazard, ?
• Topotecan selected as baseline regimen
• The calculated absolute hazards for each
  treatment could then be converted to mean
  survival by taking the inverse of the hazard, 1/?

19
Data extraction
20
Examining the data

• Inconsistent evidence
• Trial A topotecan gt paclitaxel
• Trial B PLDH gt topotecan
• Trial C paclitaxel gt PLDH
• Possible reasons
• Random chance small size of trial C
• Relative treatment effect varies with length of
  follow-up
• i.e. survival curves for paclitaxel and PLDH
  expected to cross had trial C been followed up
  for same period as A and B

21
Selecting the evidence

• When using either approach to making a
  simultaneous direct comparison the analyst must
  select the trials to include
• Common comparator approach
• Assumes that trial C provides no information on
  comparison of paclitaxel and PLDH
• MTC model
• Assumes that hazard ratio independent of
  differing length of follow-up

22
WinBUGS code overall survival
23
Results

• First column unadjusted from trial data
• Indirect comparison of Pac vs. PLDH outside of
  95CI from trial C (e0.288 1.33)
• Second column reflects combined evidence from all
  3 trials
• Marginally smaller 95 credible intervals

24
Effect on CEA
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Discussion common comparator

• Simultaneous comparison of all relevant
  alternatives necessary for decision-making
• Problems with traditional meta-analysis when no
  common comparator between all relevant trials
• Separate pair-wise comparisons may be
  inconsistent
• Result can differ according to choice of common
  comparator
• Strong assumption that excluded trials provide no
  information

29
Discussion MTC model

• MTC model provides analytic framework to
  incorporate evidence where there exists both
  direct head-to-head evidence and indirect
  evidence
• Validity of result dependent on underlying
  assumptions about pooling
• Inappropriate synthesis may introduce bias
• Requires complete network
• Implications for search strategies

30
MTC model for meta-analysis

• MTC model can be implemented largely based on
  same assumptions as standard meta-analysis
• Advantages over approaches that rely on common
  comparator increase as the network of trial
  evidence becomes more complex

31
Complexity of network
32
Bayesian approach

• In this example we used uninformative priors
• Approach can incorporate prior information where
  available
• E.g. expert opinion on likelihood that hazard
  ratio would change with extended follow-up in
  trial C
• Not necessary to use Bayesian approach
• Although use of uninformative priors will give
  same answer as non-Bayesian

33
Beyond the decision

• NICE and other decision-makers may make
  recommendations about further research
• This can be informed by value of information
  analysis
• Failure to incorporate all available evidence may
  lead to erroneous recommendations
• Incorporation of all evidence can change point
  estimates as well as level of uncertainty about
  effectiveness
• Effect on decision uncertainty unclear

34
Beyond the data?

• Available data limited
• Trials report multiple outcomes
• Trial C only reported overall survival
• Additional information on overall survival may be
  suggestive of effects on other outcomes like PFS
• Should we attempt to reflect this in the model?