Estimation and Adjustment of Bias in Randomised Evidence Using Mixed Treatment Comparison Meta-analysis

1
Estimation and Adjustment of Bias in Randomised
Evidence Using Mixed Treatment Comparison
Meta-analysis

• Sofia Dias, NJ Welton, AE Ades
• with Valeria Marinho, Georgia Salanti, Julian
  Higgins
• Avon RSS, May 2010

Department of Community Based Medicine
2
Overview

• Motivation
• Treatment networks and MTC
• Adjusting for Bias in Mixed Treatment Comparisons
  Meta-analysis (MTC)
• The MTC model
• Example Fluoride dataset
• Probability of bias model
• Results and Conclusions

3
Mixed Treatment Comparisons

• Often more than two treatments for a given
  condition
• Network of trials comparing different
  interventions for a condition
• Direct and indirect evidence available on
  treatment effects
• Because of the network structure, there is enough
  information to estimate and adjust for bias
  within the network
• For bias adjustment, there is no need to rely on
  exchangeability assumption between meta-analyses
  in different fields

4
Example The Fluoride Data

• 6 different interventions for preventing dental
  caries in children and adolescents
• No Treatment
• Placebo
• Fluoride in Toothpaste
• Fluoride in Rinse
• Fluoride in Gel
• Fluoride in Varnish
• From 6 Cochrane Reviews

Active Treatments
Marinho et al., 2002 2003 2004 (Cochrane
Library)
5
Network and Number of trials

• 130 trials
• eight 3-arm trials
• one 4-arm trial
• 150 pairwise comparisons

6
Introduction to MTC

• 1. Six treatments 1,2,3,4,5,6
• 2. Take treatment 1 (No Treatment) as reference
• 3. Then the treatment effects d1k of all other
  treatments relative to 1 are the basic parameters
• 4. Given them priors
• d1,2, d1,3,, d1,6 N(0,1002)

7
Functional parameters in MTC

• The remaining contrasts are functional
  parameters d2,3 d1,3 d1,2
• d2,4 d1,4 d1,2
• d4,6 d1,6 d1,4
• d5,6 d1,6 d1,5
• Any information on functional parameters tells us
  indirectly about basic parameters
• Either FE or RE model satisfying these conditions

8
Notation

• Data
• i 1,,130 study index
• k 1, 2, 3,,6 treatment index
• rik number of caries occurring in trial i,
  treatment k, during the trial follow-up period
• Eik exposure time in arm k of trial i
• (in person years)

9
Fluoride Poisson MTC RE model
i 1,,130
Exposure time in person years
rate at which events occur in arm k of trial i
Priors
10
MTC results LHR relative to No Treatment
Residual deviance is 278.6 (270 data points)
11
Posterior mean of residual deviances for each
point
12
Check how evidence is combined in the network

• Poor fit can indicate inconsistency in the
  network
• For each pair, separate direct evidence from
  indirect evidence implied by the rest of the
  network
• Can see how evidence is combined in the network
  to give overall MTC estimate
• Helpful to locate pairs of comparisons where
  there may be problems

Dias et al., Stats in Med. 2010
13
LHR for Placebo v Toothpaste
Bayesian p-value 0.32
14
LHR for Placebo v Varnish
Bayesian p-value 0.04
15
LHR for Rinse v Varnish
Bayesian p-value 0.02
16
Bias Models

• But we have additional information on the risk of
  bias of all included studies

17
Treatments Treatments Treatments Treatments Treatments Treatments No of studies Allocation concealment Allocation concealment Allocation concealment Blinding Blinding Blinding
NT P T R G V No of studies adequate unclear inadequate Double Single ?
1 0 1 0 0 1 0
4 1 3 0 3 1 0
3 0 3 0 1 0 2
1 0 1 0 1 0 0
3 0 2 1 0 2 1
9 0 5 4 0 6 3
4 0 3 1 0 3 1
61 8 46 7 61 0 0
25 2 20 3 22 0 3
9 0 6 3 9 0 0
3 0 3 0 3 0 0
1 0 1 0 1 0 0
1 0 0 1 0 1 0
4 0 3 1 2 2 0
1 0 1 0 0 1 0
Total Total Total 130 11 98 21 103 17 10
18
MTC RE model with bias
Priors
19
MTC Bias Model

• Assume non-zero mean bias, bi b ? 0, in
  comparisons of NT or Pl with Active treatments
• For Active-Active comparisons assume mean bias is
  zero
• Expect bias to increase size of treatment effect
  b lt 0

20
Fluoride Risk of Bias indicators

• Allocation concealment
• Best empirical evidence of bias
• But 98/130 studies unclear
• Only 11/130 studies adequate
• Some comparisons have no adequately concealed
  trials
• Blinding also available to inform risk of bias
  status
• Used Any bias as a composite indicator of bias
  54/130 studies at risk of bias.

21
Probability of Bias Model

• Any study with unclear allocation concealment has
  a probability p of being at risk of bias
• Adequately concealed trials are not at risk of
  bias
• Inadequately concealed trials are at risk of bias
• Use only allocation concealment as bias indicator
• Bias terms identifiable in this rich network

22
Probability of Bias Model
23
Comparing Model Fit
ResDev pD DIC Between trial heterogeneity Between trial heterogeneity
MTC with no bias adjustment 278.6 259.3 537.9 0.22 (0.19, 0.26)
Bias adjustment Bias adjustment Bias adjustment Bias adjustment
AnyBias 277.6 257.9 535.5 0.15 (0.12, 0.18)
Probability of bias 274.6 253.0 527.6 0.12 (0.10, 0.15)
Compare with 270 data points Compare with 270 data points Compare with 270 data points Compare with 270 data points Compare with 270 data points Compare with 270 data points
24
Posterior mean of residual deviances for each
point MTC and Prob of bias models
Study 42 Placebo v Toothpaste (1 of 69
trials) Allocation concealment unclear Study 63
No Treat v Varnish (1 of 4 trials) Allocation
concealment unclear and not double blind Study
102 Placebo v Varnish (1 of 3
trials) Allocation concealment unclear
25
Treatment effects relative to No Treatment
(LHR)Unadjusted MTC (solid) and Probability of
Bias model (dashed)
26
Varnish effects

• Cochrane Review to assess efficacy of Fluoride
  Varnish (Marinho et al, 2004)
• Noted that the small number and poor
  methodological quality of varnish trials might be
  overestimating the true effect of this
  intervention.
• The results of the bias-adjusted analysis support
  this hypothesis.

27
Which treatment is best?
Unadjusted MTC Unadjusted MTC Bias-adjusted MTC Bias-adjusted MTC
Probability Best () Rank Probability Best () Rank
No Treatment 0 6 0 6
Placebo 0 5 0 5
Toothpaste 3.6 2.9 9.3 2.7
Rinse 4.1 2.8 53.8 1.6
Gel 3.7 3.2 12.4 2.9
Varnish 88.5 1.2 24.6 2.8
28
Results Probability of Bias

• Bias
• posterior mean -0.19, CrI (-0.36, -0.02)
• posterior sd 0.40, CrI (0.29, 0.55)
• Trials with unclear allocation concealment are at
  risk of bias with probability p
• Posterior mean of p 0.13
• Model identified 5 trials (with unclear
  allocation concealment) as having a high
  probability of bias

29
Prob of bias for studies with unclear allocation
concealment
o unclear allocation concealment unclear
allocation concealment and single blind ??
unclear allocation concealment and unclear
blinding status
30
Other findings

• Between trial heterogeneity in treatment effects
  reduced in bias-adjusted model
• Model with Active-Active bias was also fitted
  with similar results Active-Active bias had
  posterior mean of zero
• But assumptions on direction of bias
• Assumed bias would favour the newest treatment
  (also the most intensive)

31
Conclusions

• Bias estimation and adjustment possible within
  MTC because there is a degree of redundancy in
  the network
• Assumption that study specific biases are
  exchangeable within the network
• Uses only internal evidence
• Weaker than required from using external evidence
• Ideas extend to multiple bias indicators
• But will need a very rich evidence structure

32
Consequences for Decision Modelling

• Uses only internal evidence
• May be more acceptable to patient groups,
  pharmaceutical industry
• Risk of bias indicator chosen based on empirical
  research
• Results may change if different bias indicators
  chosen
• Again
• Assessment of model fit sensitivity analysis
  crucial if decisions based on these models are to
  have credence

33
References

• Our website http//bristol.ac.uk/cobm/research/mp
  es
• Dias S, Welton NJ, Marinho VCC, Salanti G,
  Higgins JPT and Ades AE (2010) Estimation and
  adjustment of Bias in randomised evidence using
  Mixed Treatment Comparison Meta-analysis. Journal
  of the Royal Statistical Society A, to appear Vol
  173 issue 4 (available online).
• Dias S, Welton NJ, Caldwell DM and Ades AE (2010)
  Checking consistency in mixed treatment
  comparison meta-analysis. Statistics in Medicine,
  29, 945-955.
• Schulz KF, Chalmers I, Hayes RJ and Altman DG
  (1995) Empirical Evidence of Bias. Dimensions of
  Methodological Quality Associated With Estimates
  of Treatment Effects in Controlled Trials. JAMA,
  273, 408-412.