Statistics for the Interventionist

1
Statistics for the Interventionist

• Gregory J. Dehmer, MD
• Professor of Medicine, Texas AM College of
  Medicine
• Director, Cardiology Division
• Scott White Clinic

Statistics means never having to say your certain
2
Gregory J. Dehmer, MD, FSCAI
I have no relevant financial disclosures to make.
3
Expectations - Hypothesis
p1.0

• You will completely understand statistics at the
  end of this brief talk
• This is inherently boring materialand I cant
  fix that
• Provide some comments about studies and
  statistics that I hope are helpful

p
4
Some Things Are Just Really Bad Ideas
5
Another Questionable Combination

The busy interventional cardiologist
Statistics-in-a- Box SYSTAT has every
statistical procedure you need
Hazardous equipment Dont operate unless you
know what you are doing
6
You Will Be Reviewed

• Many papers now undergo formal statistical review

Statistical Consultants NEJM Circulation Circ
Res JACC etc . . .
7
Evolution of Evidence
Primary Evidence
Secondary Evidence
Randomized controlled trial Observational
studies Uncontrolled trials Descriptive
studies Case reports
Synthesized quantitative Data (meta-analyses) Sy
stematic reviews Summary reviews Opinions of
respected authorities
8
Evolution of Evidence
Primary Evidence

• Issues related to RTCs
• Exclusions
• Missing data
• Power calculations
• Confidence intervals
• Confusing endpoints
• Non-inferiority

Randomized controlled trial Observational
studies Uncontrolled trials Descriptive
studies Case reports
9
RTCs Problem 1 Exclusions

• Exclusion of cases is a major weakness (during
  analysis)
• Most common reason is a desire to ensure that all
  patients are adequately treated
• Awkward to retain a patient in the analysis who
  died during the 1st week of therapy or were
  unwilling to stick with the therapy

Your Out ! !
10
RTCs Problem 1 Exclusions
Exclusions more likely in - the aggressive
treatment arm or the high-risk group
Standard Therapy
R
More aggressive than standard therapy

• Likely to have more non-adherent patients
• Non-adherent patients are a higher risk group
• Exclusion of high-risk patients improved the
  average of the remaining patients
• If exclusions are permitted, the more aggressive
  arm appears artificially better

11
RTCs The Problem With Exclusions
Ideally, data analysis should look forward from
randomization
Good Events
Good Events
R
Bad Events
Side-Effects
Dropouts
These are the data (warts and all) that the
clinician needs to know to assess what will
happen to their patient if this therapy is
selected
12
RTCs The Problem With Exclusions
Data analysis when cases of inadequate treatment
are excluded it like looking backward through
rose-colored glasses
Good Events
Good Events
R
Data Analysis
Bad Events
Side-Effects
Dropouts
Excluding bad events and focusing only on the
good results of the remaining cases may look
impressive, but is not of practical value to
clinicians who need to make prospective therapy
decisions for their patients.
13
RTC Lesson 1 Avoid Exclusions
Check to see if the size of the analyzed groups
are similar
R
Standard therapy
New therapy
n 4932
n 4931
Exclusions
n 4100
n 4932
Beware of potential bias
14
RTCs Problem 2 Missing Data
Missing random data weakens the study, but is not
a serious concern However when data are missing
because of aspects of treatment or disease, major
bias can arise. Patients missing outcomes
observations are more likely those with poor
outcomes
Higher-risk ptsdont tolerate the therapy, drop
outleaving the low-riskpts who naturallyhave
higher EFs
LVEF
of patients 200
120
50
15
RTC Problem 2 - Beware of Missing Data

• Make every effort to have data values at all key
  time points
• Can use imputed values
• Carry previous measure forward
• Inserting a conservative value
• Averaging adjacent values
• Computer models which use similar patients with
  complete information

16
RTC Problem 2 - Beware of Missing Data

• Sensitivity analysis determines the impact of
  the missing data and the imputation method used.
  
• If the results are qualitatively similar, one can
  deduce that the basic study conclusion does not
  depend on the type of imputation used (or the use
  of imputation).

Treatment 2
Treatment 1
Replace missing values with a conservative
imputation
Replace missing values with an anti-conservative
imputation
Analyze and flip-flopimputation strategy
17
RTC Lesson 2 Avoid Missing Data
Rule of ThumbIf the proportion of cases
excluded or with missing data in ? the sizeof
the treatment difference reported,the study is
likely unreliable
Consolidated Standards for Reporting Trials
Lancet. 200135711911194
18
RTCs Problem 3 Power Calculations

• Power calculation
• Determine what is a clinically meaningful
  difference between the two groups. (10)Would
  anyone care if the difference in restenosis was
  2?
• Amount of variation in the measurement of the
  endpoint (standard deviation)

19
RTCs Problem 3 Understanding Power

• One-tailed (sided) test
• Used when previous data, physical limitations or
  common sense tells you that the difference, if
  any, can only go in one direction
• Example Contrast effect on renal function
• Two-tailed (sided) test
• Used when the difference, if any, can go in
  either direction.
• Example Drug effect on serum K

20
RTCs Problem 3 Understanding Power

• Power Calculation
• - Null hypothesis RS (control) - RS
  (treatment) zero (0) If you reject the
  null hypothesis then you are saying there is a
  difference between the two
• ? threshold of significance typically
  0.05 (5) If you reject the null
  hypothesis when it is actually true Type
  I error
• There is a 5 chance that there no
  difference, but your analysis concludes there
  is Probability of a Type I error ?

21
RTCs Problem 3 Understanding Power

• Power Calculation
• ? threshold of significance typically
  0.05 Saying there is a difference when
  there is none
• ? the level you are willing to accept for
  the chance of missing an important
  difference when there really is one (20)
  (Type II error) Accepting the null
  hypothesis when it is, in fact, false
  Power 1 - ? 1 0.20 0.80
  or 80

22
RTC Lesson 3 Know What Power Means
23
RTCs Problem 4 Understanding CIs

• Standard deviation
• Relates to one data set
• Fasting cholesterol of everyone in this room
• Mean (average)
• SD is an expression of how much spread there is
  around the mean value
• Equation for SD

SD mark the limits of scatter
Approximately 68 are within 1 SD Approximately
95 are within 2 SD
24
RTCs Problem 4 Understanding CIs

• Confidence intervals
• Relate to populations (consider this room a
  population)
• Measure cholesterol in a sample of the population
  (n 10)
• How well does the sample mean represent the
  population mean?
• 95 CI tells you that the mean of the population
  has a 95 chance (19 out of 20 times) of being
  within the range of the sample mean

25
RTCs Problem 4 Understanding CIs
26
RTCs Lesson 4 Know Your CIs

• Confidence intervals
• Each sample has a mean and SD
• SEM SD/ vn
• The 95 CI is 1.96 x SEM
• There is only a 5 chance that this range of
  values excludes the true population mean value

Variable
27
RTCs Lesson 5 OR RR Are Not the Same
The PURSUIT Trial
The primary endpoint (composite of death or MI at
30 days) was comparedin patients receiving
eptifibatide vs. placeboEptifibatide group
672 out of 4722 reached the primary
endpoint Placebo group 745 out of 4739 reached
the primary endpoint
Odds Ratio Odds in E 672 / 4050 0.166 Odds in
P 745 / 3994 0.187 Odds ratio 0.166 / 0.187
0.899
Risk Ratio Odds in E 672 / 4722 0.142 Odds in
P 745 / 4739 0.157 Odds ratio 0.142 / 0.157
0.905
There is a separate, but similar mechanism for
calculating CI for ORs and RRs
The PURSUIT Investigators NEJM 1998339436-443
28
RTCs Problem 5 Ratio Confusion
Relationship between ORs and RRs for studies
assessing harm Each line on the graph relates
to a different baseline prevalence, or event rate
in the control group When the prevalence of the
event is low, say 1, the RR is a good
approximation of the OR For example, when the
OR is 10, the RR is 9, an error of 10 We can
use this graph to get a grasp of how misleading
it could be to interpret ORs as if they were RRs.
Relative Risk
Odds Ratio
29
RTCs Problem 5 Ratio Confusion
Relationship between OR and RR for studies which
are assessing benefit Each line on the graph
relates to a different baseline prevalence, or
event rate in the control group When event rates
are very low the approximation is close, but
breaks down as event rates increase For
example, if the event rate is 50 and there is a
20 reduction in the odds, the relative risk
adjustment will be little over 10
Relative Risk
Odds Ratio
30
RTCs Problem 6 Confusing Endpoint

• Superiority trial Designed to test for a
  statistically significant and clinically
  meaningful improvement (or harm) from the use of
  the experimental treatment over the usual care.

Not different
Superior
Superior
Superior
Experimental Treatment Better
Control Treatment Better
31
RTCs Problem 6 Confusing Endpoint

• Equivalence trial Evaluates whether the
  difference in outcome for the experimental
  treatment compared with standard care falls
  within the boundary of a clinically-defined
  minimally important difference (MID)

MID
Clinically and statistically equivalent
Neither clinically nor statistically equivalent
Had these come from a superiority trial they
would be clinically equivalent, but statistically
inferior (2) or superior (1)
Statistically equivalent
Statistically equivalent
Experimental Treatment Better
Control Treatment Better
32
RTCs Problem 6 Confusing Endpoint

• Noninferiority trial Results are evaluated
  assuming that the experimental treatment is not
  worse than the standard treatment by a
  clinically-meaningful amount.

MID
Not inferior
CI too wide for any conclusions
CI does not crossthe MID
CI crosses MIDindicating inferiority of the
experimental Rx
Not inferior
Experimental Treatment Better
Control Treatment Better
33
Resources

• http//www.jr2.ox.ac.uk/bandolier/
• http//www.statsoft.com/textbook/stathome.html
• http//www.bettycjung.net/Statsites.htm
• http//www.tufts.edu/gdallal/bmj.htm
• Link to Br Med J series of papers on statistics
• 2006-2007 Circulation series Statistical Primer
  for Cardiovascular Research
• Motulsky H. Intutitive Statistics. Oxford
  University Press 1995

A statistician is a person who comes to the
rescue of figures that cannot lie for themselves
34
Remember
Statistics are like a bikini. What they reveal
is suggestive, but what they conceal is vital