Meta-analysis with missing data: metamiss

1
Meta-analysis with missing data metamiss

• Ian White and Julian HigginsMRC Biostatistics
  Unit, Cambridge, UK
• Stata users group, London
• 10 September 2007

2
Motivation

• Missing outcome data compromise trials
• So they also compromise meta-analyses
• We may want to
• correct for bias due to missing data
• down-weight trials with more missing data
• NB missing data within trials, not missing trials

3
Plan

• Meta-analysis of binary data
• Haloperidol example
• Standard approaches to missing data
• Imputation methods
• IMORs
• Methods that allow for uncertainty
• Demonstration

4
Haloperidol meta-analysis
  Haloperidol Haloperidol Haloperidol Haloperidol Placebo Placebo Placebo Placebo missing 
  r1 f1 m1 n1 r2 f2 m2 n2 missing 
Arvanitis 25 25 2 52 18 33 0 51 2
Beasley 29 18 22 69 20 14 34 68 41
Bechelli 12 17 1 30 2 28 1 31 3
Borison 3 9 0 12 0 12 0 12 0
Chouinard 10 11 0 21 3 19 0 22 0
Durost 11 8 0 19 1 14 0 15 0
Garry 7 18 1 26 4 21 1 26 4
Howard 8 9 0 17 3 10 0 13 0
Marder 19 45 2 66 14 50 2 66 3
Nishikawa 82 1 9 0 10 0 10 0 10 0
Nishikawa 84 11 23 3 37 0 13 0 13 6
Reschke 20 9 0 29 2 9 0 11 0
Selman 17 1 11 29 7 4 18 29 50
Serafetinides 4 10 0 14 0 13 1 14 4
Simpson 2 14 0 16 0 7 1 8 4
Spencer 11 1 0 12 1 11 0 12 0
Vichaiya 9 20 1 30 0 29 1 30 3
rsuccesses ffailures mmissing ntotal
5
Standard approaches to missing data

• Available cases (complete cases) ignore the
  missing data
• assumes MAR missingness is independent of
  outcome given arm
• Assume missingfailure
• implausible, but not too bad for health-related
  behaviours
• Neither assumption is likely to be correct

6
Other ideas

• Sensitivity analyses, e.g. do both
  missingfailure and available cases
• but these could agree by chance
• Explore best / worst cases
• Use reasons for missingness
• Explicit assumptions about informative
  missingness (IM)
• IM missingness is dependent on outcome

7
metamiss.ado

• Processes data on successes, failures and missing
  by arm feeds results to metan
• Available cases analysis (ACA)
• Imputed case analyses (ICA)
• impute as failure ICA-0
• impute as success ICA-1
• best-case ICA-b (missingsuccess in E, failure
  in C)
• worst-case ICA-w
• impute with same probability as in control arm
  ICA-pC
• impute with same probability as in experimental
  arm ICA-pE
• impute with same probability as in own arm ICA-p
  (agrees with ACA)
• impute using IMORs ICA-IMOR (see next slide)

8
More general imputation IMORs

• Measure Informative Missingness using the
  Informative Missing Odds Ratio (IMOR)
• Odds ratio between outcome and missingness
• Cant estimate IMOR from the data, but given any
  value of IMOR, we can analyse the data
• Generalises other ideas e.g.
• ICA-0 uses IMORs 0, 0
• ICA-1 uses IMORs ?, ?
• ICA-b uses IMORs ?, 0
• ICA-p uses IMORs 1, 1
• ICA-pC uses IMORs OR, 1 where OR is odds ratio
  between arm and outcome in available cases

9
Getting standard errors (weighting) right

• Weight 1 treat imputed data as real
• Weight 2 use standard errors from ACA
• Weight 3 scale imputed data to same sample size
  as available cases
• Weight 4 algebraic standard errors
• same as weight 1 for ICA-0, ICA-1, ICA-b, ICA-w
• same as weight 2 for ICA-p
• uses Taylor expansion for ICA-IMOR
• for ICA-pC ICA-pE, we condition on the IMOR (I
  can explain)

10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
Allowing for reasons (ICA-R)

• Specify number of missing individuals in each arm
  to be imputed by each scheme ICA-0, ICA-1,
  ICA-pC, ICA-pE, ICA-p, ICA-IMOR.
• Can take these data from a different outcome
  metamiss scales to missing
• If missing in a particular study, metamiss
  imputes using combined studies

22
(No Transcript)
23
Allowing for uncertainty

• So far we have pretended we really know the IMORs
• This is never really correct
• Now we allow them to be unknown but from a
  user-specified distribution

24
Bayesian approach allowing for uncertain IMORs
(Rubin, 1977)
25
Bayesian analysis

• Elicit prior for dE, dC or use N(0,12) or N(0,22)
• Get posterior distribution by integrating over
  the 2-dimensional distribution of dE, dC.
• metamiss does this fast accurately by
• Standard normal approximation to posterior given
  dE, dC
• Integrate using Gauss-Hermite quadrature.
• Alternatives
• Taylor expansion (inaccurate for large SD of log
  IMOR)
• Full Bayesian Monte Carlo (slow, little gain in
  accuracy)

26
Understanding priors for log IMOR implied prior
for P(success missing) when P(success
observed) 1/2
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
Proposal 4 sensitivity analyses
IMORs Options (e.g.) Sensitive to Works via
fixed equal imor(2 2) Imbalance in missingness Point estimates
fixed opposite imor(2 1/2) Amount of missing data Point estimates
random equal sdlogimor(2) corr(1) Imbalance in missingness Weightings
random uncorrelated sdlogimor(2) corr(0) Amount of missing data Weightings
31
Summary

• Tool for sensitivity analysis
• Requires thought about plausible missing data
  mechanisms
• Would be nice to overlay sensitivity analysis
  with ACA
• Further work includes combining uncertainty with
  reasons
• I also have a program mvmeta for multivariate
  meta-analysis

32
References

• 1st part Higgins JPT, White IR, Wood A.
  Imputation methods for missing outcome data in
  meta-analysis of clinical trials. Clinical
  Trials, submitted.
• 2nd part White IR, Higgins JPT, Wood AM.
  Allowing for uncertainty due to missing data in
  meta-analysis. 1. Two-stage methods. Statistics
  in Medicine, in press.
• Related White IR, Welton NJ, Wood AM, Ades AE,
  Higgins JPT. Allowing for uncertainty due to
  missing data in meta-analysis. 2. Hierarchical
  models. Statistics in Medicine, in press.
• metamiss.ado available from http//www.mrc-bsu.ca
  m.ac.uk/BSUsite/Software/Stata.shtml

33
Extra slides
34
(No Transcript)