Introduction to Costeffectiveness Analysis

1
Introduction to Cost-effectiveness Analysis

• Ming-Yu Fan, PhD
• January 30, 2008

2
Outline

• Increasing interest in C-E analysis
• Incremental Cost-Effectiveness Ratio (ICER)
• Methods for constructing confidence intervals
• A simulation study

3
Increasing interest in CEA

• On PubMed, the keyword cost-effectiveness
• retrieves 49710 citations
• The keyword cost-effectiveness ratio retrieves
• 3193 citations
• 1080 publications using incremental cost-
• effectiveness ratio
• () These numbers were obtained on 1/28/08. On
  1/30/08, they are 49734, 3195, and 1082

4
Measures for cost-effectiveness

• Incremental Cost-Effectiveness Ratio (ICER)
• ICER (C1 - C2) / (E1 - E2)
• (C1, E1) (cost, effect) in the
  intervention/treatment group
• (C2, E2) (cost, effect) in the control/usual
  care group
• Net Health Benefits (NHB)
• NHB E C/?
• ? a rate of substitution of dollars for health
• INHB(?) NHB1(?) - NHB2(?)

5
ICER C-E plane

• ICER (C1 - C2) / (E1 - E2) ?C / ?E

6
ICER cont.

• Best scenario Quadrant IV ? intervention is
  effective and cost-saving
• Worst scenario Quadrant II ? intervention is
  worse than usual care and costs more
• Most common scenario Quadrant I ? intervention
  is more effective than usual care and costs more

7
Methodological challenge

• Notation
• µC E(?C), µE E(?E)
• VC Var(?C), VE Var(?E)
• Cov Covariance(?C, ?E)
• Expected value (mean) of a ratio does not have a
  close form
• E(?C/?E) E(?C) / E(?E) µC/µE
• Variance of a ratio does not have a close form
• Var(?C/?E)
• Both approximations are based on Taylors
  expansion

8
Methodological challenge cont.

• Conventional 95 confidence interval
• (mean - 1.96se), (mean 1.96se)
• Normal distribution or large sample size
• Good estimation of mean and variance
• ICER
• Ratio is heavily skewed
• Only approximated estimations of mean and
  variance are available

9
Alternative confidence intervals

• Bootstrap methods
• Fiellers method
• Many simulation studies have shown that these two
  (especially Fiellers method) yield best results

10
Bootstrap procedure

• Sample the data With Replacement until the same
  sample size is reached
• Derive the statistics on the bootstrap sample
  (e.g. mean, ICER)
• Repeat the procedure for many times (e.g. 1000)
• Construct the confidence interval based on the
  statistics (e.g. using the 1000 ICERs)

11
Bootstrap - example

• Original sample
• DFD depression-free-days
• Intervention
• N 6
• Mean DFD / year 190
• Mean costs / year 8000
• Usual Care
• N 4
• Mean DFD / year 130
• Mean costs / year 7700
• ICER
• (8000-7700)/(190-130)
• 5

12
Bootstrap sample 1

• Bootstrapping IV and UC samples separately
• Intervention
• N 6
• Mean DFD 193
• Mean costs 8083
• Usual Care
• N 4
• Mean DFD 120
• Mean costs 7475
• ICER
• (8083-7475)/(193-120)
• 8.3

13
Bootstrap sample 2

• Bootstrapping IV/UC samples separately
• Intervention
• N 6
• Mean DFD 183
• Mean costs 7833
• Usual Care
• N 4
• Mean DFD 140
• Mean costs 7925
• ICER
• (7833-7925)/(183-140)
• -2.1

14
Bootstrap example cont.

• Total number of possible bootstrap samples
• (66)(44) (46656)(256) 11,943,936
• Total number of unique means/variances
• 46235 16,170
• Repeat the same bootstrapping procedure for 1000
  times, we
• will obtain 1000 ICERs
• These 1000 ICERs provide an approximate
  distribution of the
• estimated ICER
• We can make statistical inferences about the
  estimated ICER
• using this approximate distribution

15
(No Transcript)
16
Bootstrap methods

• Percentile
• Order the 1000 ICERs (from small to large)
• Take the 25th and the 976th ICERs
• The 95 confidence interval ICERb25, ICERb976
• Normal
• Derive the mean (µb) and standard deviation (sb)
  from the
• 1000 ICERs
• µb Si(ICERbi) / 1000,
• (sb)2 Si(ICERbi - µb)2 / (1000-1)
• The 95 confidence interval
• (µb - 1.96sb), (µb 1.96sb)

17
(No Transcript)
18
Bootstrap methods cont.

• Bootstrap-t
• Generate a t-statistic (tb (ICERb ICERs)/seb)
  within each bootstrap sample then follow the
  percentile method using the t-statistics
• Better than the percentile method because the
  distribution of t-statistic is closer to a normal
  distribution than the distribution of ICERb
• If the estimation of s.e. is biased, bootstrap-t
  might have worse coverage rate than
  bootstrap-percentile method
• Bias-corrected accelerated (BCa)
• Instead of (a/2) and (1-a/2) (e.g. 0.025 and
  0.975 for a 0.05), using a1 and a2 to get the
  percentiles
• a1 and a2 are functions of bias-corrections and
  accelerated scalars

19
Katon et al. Arch Gen Psych. 2005
20
More about bootstrap

• Why use bootstrap?
• No need to assume the distribution family of the
  data
• With todays computing power, bootstrap can be
  done
• very easily and quickly
• Good asymptotic properties
• Commonly chosen number of repetitions 1000
• The distribution of the bootstrap samples depends
  on the original sample size, not so much on the
  number of repetitions

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Other bootstrap procedure

• Parametric procedure (e.g. assume a normal
  distribution)
• Weighted bootstrap assigning different weights
  to different observations
• Ex smaller weight for outliers
• Bayesian approach

27
Fiellers method

• Ratio is difficult to model
• Mean and variance do not have a close form
• Distribution is very skewed
• Fieller suggested to transform the ICER into a
  linear variable
• ICER R ?C / ?E
• New statistic S ?C ?ER
• The mean and variance of (?C ?ER) are very
  easy to derive

28
Fiellers method cont.

• Notation
• Mean of (?C ?ER) µs µ?C - µ?ER
• Standard deviation of (?C ?ER) ss
• By Central Limit Theorem
• (?C ?ER) µs / ss is normally distributed
• The 95 confidence interval for the statistic can
  be derived by

29
(No Transcript)
30
Fiellers method cont.

• With simple algebra, the inequality can be
  re-written as
• aR2 bR c 0
• where
• a (?E)2 1.962 s2?E
• b 2 ?E ?C 1.962 s?E?c
• c (?C)2 1.962 s2?c
• The upper and lower limits of the 95 confidence
  interval for
• R (ICER) are the 2 boundaries that satisfy the
  inequality
• The left hand side of the inequality represents a
  parabola

31
Fiellers method - example
32
Fiellers method - complications

• aR2 bR c 0
• 4 scenarios (and their solutions for R)
• (1) a 0, (b2 4ac) 0 ? a close interval
• (2) a lt 0, (b2 4ac) 0 ? an open interval
• (3) a 0, (b2 4ac) lt 0 ? an empty set
• (4) a lt 0, (b2 4ac) lt 0 ? the whole real line

33
(No Transcript)
34
Fiellers method cont.

• Why use Fiellers method
• Several simulation studies have demonstrated that
  Fiellers
• method yield better results than other method
• Easy to compute
• Unique results of the same data (contrast to
  bootstrap)
• When does Fiellers method result in meaningless
• confidence intervals?
• Scenario (3) where a gt 0 and (b2 4ac) lt 0
  never happens
• When a 0, the confidence interval is a closed
  interval

35
Fiellers method cont.

• Whats the interpretation for a 0?
• a (?E)2 1.962 s2?E 0
• ? (?E/s?E )2 1.962
• ? the incremental effect is statistically
  significant
• When the incremental effect is not statistically
• significant, it is not interesting clinical-wise
  to conduct cost-
• effectiveness analysis. Statistically, it is
  appropriate to run analysis
• using ICER and make inferences based on it

36
A simulation study

• 3 distribution families
• (1) Both effect and cost follow a normal
  distribution
• (2) Both effect and cost follow a log-normal
  distribution
• (right skewed)
• (3) Effect ? normal Cost ? log-normal
• For each distribution family, 162 different
  distributions
• Sample size 50, 100, 400
• Correlation coefficient between cost and effect
  -0.5, 0.1, 0.5
• Ratio of the variances (control / intervention)
  1, 3
• ICER 2000, 10000, 50000
• 3 distances between the ICER and the origin
• (relevant to the effect size)

37
Distance between ICER and origin

• A and B have the same ICER

38
(No Transcript)
39
(No Transcript)
40
Problem with ICER

• Cant distinguish ICERs on quadrant I from
  quadrant III
• Quadrant I positive intervention effect, more
  costly
• Quadrant III negative intervention effect, cost
  saving
• Cant distinguish ICERs on quadrant II from
  quadrant IV
• Quadrant II negative intervention effect, more
  costly
• Quadrant IV positive intervention effect, cost
  saving
• Negative ICERs are difficult to interpret

41
Problem with ICER cont.

• Example 3 studies comparing to the same control
  group
• A ?E 10, ?C -1000
• B ?E 10, ?C -500
• C ?E 5, ?C -500
• Pair-wise comparison
• A is better than B (same effect, more
  cost-saving)
• B is better than C (same cost-saving, more
  effective)
• ? A gt B gt C
• ICER
• (A) ICER -100 (B) ICER -50 (C) ICER
  -100
• ? A C gt B

42
Summary

• ICER is an intuitive measure for
  cost-effectiveness analysis
• but is only appropriate when
• The incremental effect is statistically
  significant from 0
• Both incremental effect and incremental cost are
  positive
• The available methods might not be appropriate
  for other
• scenarios
• With large samples, normal method,
  bootstrap-percentile
• method, and Fiellers method are all equally
  good
• For small samples (or extremely skewed costs), my
  
• preference is (1) Fiellers method (2)
  bootstrap-percentile (3)
• normal method

43
(No Transcript)