Epidemiological Malpractice: How Biomedical Experts Systematically Misinform Healthcare Policy

1
Epidemiological MalpracticeHow Biomedical
Experts Systematically Misinform Healthcare
Policy

• Rod Hayward
• Director, VA Center for Practice Mgt Outcomes
  Research
• VA Ann Arbor Healthcare System
• Co-director, Robert Wood Johnson Clinical
  Scholars Program
• Professor of Internal Medicine Health Mgt and
  Policy
• University of Michigan Schools of Medicine
  Public Health
• September 2006

2
Research Directed at Improving the Health Care

Early Evidence
Validating Causality
Interpreting the Pt Outcomes Health Policy
Implications
How to Optimize Care
Health Services Research
Basic Science
Clinical Epidemiology
Clinical Trials
Epidemiology
3
Rising Healthcare Costs

• There is more health-care that is beneficial than
  we can afford
• We cannot have everything- Ready access with
  individual choice- Low coinsurance- Contain
  healthcare costs
• Things are only going to get worse

4
Get Worse??

• 45 Million uninsured
• Even insured Americans often do not receive
  high-priority care
• Under-insurance is rampant
• Insurance premiums increasing dramatically

5
Two Over-Arching Interests

• What medical interventions have the greatest
  potential to improve the publics health?
• How can we create a healthcare system that is
  more efficiency?

6
4 Stories of Epidemiological Malpractice

1. Promoting treatments to the masses when they
   truly benefit only a subset of patients

7
Flatland (Edwin A. Abbott)

• A civilization that experiences the world in
  2-dimensions
• 3-dimensional objects exist but are experienced
  as 2-dimensional objects as they intersect their
  worlds plane
• The possibility of a third dimension was
  difficult to imagine, highly-feared and publicly
  reviled

8
Flatland (cont.)
9
Hayward et al. BMC Res Methods 2006
10
Statistical Power of Traditional Subgroup Analysis
True 5-year NNT
True RRR
5-year CER
Risk Factor
-239
-0.19
2.2
Absent (75)
183
0.13
4.2
Present (25)

• Heterogeneity Test Statistical Power 0.09
• Interaction Effect Statistical Power 0.23

11
Statistical Power of Risk-Stratified
Analysis(Hayward et al. Health Affairs 2005)

• Statistical power 0.83

12
Is There Evidence That This Is Important?

13
Benefit of CEA in Symptomatic Patients with 70
to 99 Carotid Stenosis (Rothwell et al. Lancet
1999)

• European Carotid Surgery Trial

14
Benefit of CEA in Symptomatic Patients with 70
to 99 Carotid Stenosis (Rothwell et al. Lancet
1999)

• European Carotid Surgery Trial

15
Conditions/Treatments In Which Literature
Suggests Baseline Risk Is a Major Predictor of RRR

• Htn
• CAD-CABG
• Hyperlipidemia
• Lung CA Adjuvant Radiation Therapy
• AMI Thrombolytics, PCTA
• CHF Spironolactone
• Carotid endarterectomies
• AAA
• Sepsis

16
(No Transcript)
17
4 Stories of Epidemiological Malpractice

• Promoting treatments to the masses when they
  truly benefit only select patients
• Passing off lousy observational analyses as being
  from a randomized trial

18
NCEP (Circulation 2004)

• some investigators have suggested that
  guidelines can be simplified by merely
  recommending that high-risk patients be treated
  with the doses of statins used in clinical
  trials. In view of NCEP, this suggestion does
  not take advantage of the strong database
  supporting the log-linear relationship between
  LDL levels and CHD risk

19
NCEP (Circulation 2004)

• Recent clinical trials nonetheless have
  documented that for every 1 reduction in LDL-C
  levels, relative risk for major CHD events is
  reduced by approximately 1. HPS data suggest
  that this relationship holds for LDL-C levels
  even below 100 mg/dL.

20
NCEP (Circulation 2004)

• Thus, in terms of absolute risk, an LDL-C of 70
  mg/dL seems preferable for high-risk patients
  compared with a level of 100 mg/dL.

21
Log-linear Effect
22
Statins Are Highly Beneficial

• High CV-risk patients should be on at least
  moderate dose of a statin, but . . .
• Statins have several non-cholesterol mediated
  mechanisms of action (anti-inflammatory and
  anti-thrombotic)

23
But there is tons of evidence that LDL is
critically important?
24
Yes, but there is a lot of counter-evidence as
well

• Much less of an LDL-CV risk association in
  southern Europe
• In Asia and in elderly, a U-shape association has
  been reported
• No association in Framingham for LDL lt 145mg/dl
  if HDL is normal
• No significant benefit of a 40 LDL reduction in
  renal dialysis patients (NEJM 2005)
• Statins decrease stroke significantly

25
The only sub-group analysis of the importance of
LDL response(Lancet 2003)
26
Results of Systematic Review

• There is not a single cohort analysis that meets
  standard epidemiological criteria
• Most cohort analyses did not mention whether they
  had accounted for any potential confounders and
  none controlled for statin exposure, statin
  intolerance/cross-over or adherence.

27
Why Control for Confounders in a Clinical Trial?

• Clinical trials directly examine interventions,
  NOT their mechanism of action
• Protection from confounding only applies to the
  intervention to which subjects were randomized
• Cohort analyses using clinical trial data can
  dramatically increase bias due to self-selection
  (healthy volunteer effects)

28
Healthy Volunteer Effects (HVE)

• Adherence to placebo rx better outcomes
• Recent Victims HRT, B-carotene, Vit E,
  Pre-natal care

29
Those Who Achieve Treatment Goals in Clinical
Trials

• Arm of Randomization (unbiased)
• More tolerant of Rx (hardiness, HVE)
• More adherent to Rx (HVE)
• Cross-over (HVE)

30
Must Control For Medication Exposure

• Treatment Goal Achieved (lt70)
  Event RatesOn Treatment (n 220)
  4
• Not on Treatment (n 30)
  13
• Treatment Goal Not Achieved (gt70)
• On Treatment (n 250)
  4
• Not on Treatment (n 500)
  13

31
Must Control For Medication Exposure

• Treatment Goal Achieved (lt70)
  Event RatesOn Treatment (n 220)
  4
• Not on Treatment (n 30)
  13
• Treatment Goal Not Achieved (gt70)
• On Treatment (n 250)
  4
• Not on Treatment (n 500)
  13

Treatment Goal Achieved
Event Rates Yes (lt 70) (n 250)
5 No (gt
70 (n 750)
10
32
Cohort Analysis Using Clinical Trial Data
? in LDL
? in CRP
Statin Therapy
Decrease in CV-event/mortality
? in nitrotyrosine
Other known Independent Risk Factors
33
A few other examples

1. Blood pressure goals (lt130/80)
2. Expensive hypoglycemic meds are highly
   cost-effective (A1c lt 7)
3. Pre-natal care saves lives and dollars
4. Preventable adverse events cost Billions each
   year

34
4 Stories of Epidemiological Malpractice

• Promoting treatments to the masses when they
  truly benefit only select patients
• Passing off lousy observational data as being
  from a randomized trial
• False dichotomies hiding poor marginal gains

35
Setting Blood Pressure Goals

• The HOT trial demonstrates the benefits of
  lowering BP to at least 82mHg HOT trial

36
HOT Trial Cohort Analyses
37
Whenever There Is Treatment Harm That Is
Independent of Treatment-related RRR
38
HOT Trial Cohort Analyses
39
Aggressive Treatment of Htn (DBP goal 80 vs 90)
CV Mortality (HOT Trial. Lancet 1998)

• Achieved DBP 85mmHg vs 81mmHg

40
Aggressive Treatment of Htn (DBP goal 80 vs 90)
CV Mortality (HOT Trial. Lancet 1998)

• Achieved DBP 85mmHg vs 81mmHg
• Not published. Results estimated.

41
Advantages (for Disease Advocates) of Dichotomies
and Hiding Marginal Gains

• Allows you to advocate for extending more
  treatment to more people by
• Using healthy volunteer effects
• Ignoring possible treatment harms
• Allowing the average benefit to be mainly driven
  by the subset with large deviations from goal

42
Relationship between A1c Microvascular Risk
(Vijan et al 1997)
43
Common Conceptualization of Technical Quality

• Received Didnt
  Receive
• Recommended vs. Recommended
• Care
  Care

Good Quality vs. Poor
Quality
44
Relationship between Receipt of Care Quality

• 0
  100Demand High
  Demand

Quality Value/Dollar
Pt Utilities
45
of Treatment Years Needed to Prevent 1 Yr of
Blindness (Vijan et al. Ann Internal Med 1997)

• A1c 9 7
• Pt Age (Pt Years)
• 45 yrs 40
• 65 yrs 180

46
of Treatment Years Needed to Prevent 1 Yr of
Blindness (Estimates if BP controlled)

• A1c 8 7
• Pt Age (Pt Years)
• 45 yrs gt 400
• 65 yrs gt 6000

47
Some Recent Recommendations

• A1c lt 7
• BP lt 130/80
• LDL lt 70mg/dl
• CRP lt 2
• Exercise 90min/day
• BMI lt 22
• .8 drinks of red wine daily

48
4 Stories of Epidemiological Malpractice

1. Promoting treatments to the masses when they
   truly benefit only select patients
2. Passing off lousy observational data as being
   from a randomized trial
3. False dichotomies hiding poor marginal gains
4. Using measurement error to ones full advantage

49
To Err Is Human

• As many as 98,000 people die each year in US
  hospitals due to medical errors (IOM, 1999)
• Medical errors may be the 5th leading cause of
  death (Washington Post, 1999)
• . . . like 3 jumbo jets fully loaded with
  patients crashing every other day (NY Times,
  1999)
• Therefore, doctors are approximately 9000 times
  more dangerous than gun owners. (Benton County
  News Tribune, 2000)

50
Studies of deaths have all found that 5-10 of
deaths are preventable

• Harvard Medical Practice Study
• Utah/Colorado Study
• VA Mortality Study
• RAND Mortality Study

51
Studies of deaths have all found that 5-10 of
deaths are preventable

• Harvard Medical Practice Study
• Utah/Colorado Study
• VA Mortality Study
• RAND Mortality Study
• Federal statistics report that 44k-100k deaths
  due to medical errors a year . . 8th leading
  cause of death
• Ann Arbor News, Sunday March 12,
  2006

52
Estimating Preventable Deaths (PDs) If
Sensitivity Specificity Are Good A Thought
Experiment

• Assumptions
• The accuracy of 2 of 2 reviewers rating a death
  as preventable is Sensitivity 90
  Specificity 90
• The true rate of preventable deaths is 0.5

53
A Thought Experiment (cont.)(Hayward et al HSR
in press)

• Therefore, out of every 10,000 deaths, on
  average- 50 PDs - 9950 non-PDs
  (true PD rate 0.5)

54
A Thought Experiment (cont.)(Hayward et al HSR
in press)

• Therefore, out of every 10,000 deaths, on
  average- 50 PDs - 9950 non-PDs
  (true PD rate 0.5)- 45 True Positives (TPs)
  50 0.9- 995 False Positives (FPs) 9,950
  (1 - 0.9) (est. PD rate 1040/10000
  10)

55
But what about the 50 gzillion in costs that
result from medical errors???
56
Estimating Associations Between Adverse Events
Costs

• If each day you put a red sticky dot on the ankle
  of a random sample of 2 of patients in the
  hospital,
• A cross-sectional analysis of hospitalizations
  would find that putting a dot on a patients
  ankle is associated with about a 3-fold increase
  in their hospital length of stay.

57
Hayward you !_at_/!ing !Harming 1 patient is
one too many
58
Obsessing on avoiding Friendly Fire can kill
people

• Preventable adverse drug events are rampant
  (SSRIs, pain meds, ACE-Is, beta blockers,
  anticoagulants, etc)
• vs.
• Underuse of the above medications results in many
  preventable deaths much suffering

59
Jacobs Laws
60
Jacobs Laws
1. People care much more about feeling good then
doing good,
61
Jacobs Laws
1. People care much more about feeling good then
doing good, 2. When people believe that they
are doing good, it makes them feel good,
62
Jacobs Laws
1. People care much more about feeling good then
doing good, 2. When people believe that they
are doing good, it makes them feel good, 3.
Therefore, unless its real important, just let
people continue to fool themselves into feeling
good
63
Performance measures are the bomb!

• Effective market signals,
• What you measure tends to improve,
• But they are probably blunt instruments

64
Overall Quality in VA vs. Insured US
Population(Asch et al)

• Adjusted Odds Ratios
• VA Performance Measures 1.5 (1.4, 1.6)
• UnmeasuredConditions 1.0 (0.9, 1.1)

65
ASA High CV Risk Men
20 to 25 RRR for MI CV Death
66
Statins for High CV Risk Men
25 to 40 RRR for Macrovascular Events Death
67
3 BP meds in an attempt to get diabetics BPs
lt130-135/80
25 to 35 RRR for Macrovascular Events Death
(MI, CHF CVA)
50 to 70 RRR for Microvascular Complications
(eyes kidneys)
68
For diabetics Close eyecare surveillance of
known retinopathy and every 3 year screening
60 to 90 RRR of blindness
69
Cost and patient inconvenience of ASA, 3 BP meds
moderate dose statin

• 4 pills a day
• 10-30 a month
• Side-effects rare

• Plavix 90-120 a month
• Crestor 80-110 a month
• Avandia 80-110 a month

70
Headline
NCQA Committee on Performance Measurement adopts
long overdue A1c lt 7 measure

• Epidemiology from experts, NIDDK, industry
• Guidelines from experts, advocacy groups, with
  heavy industry financial support
• Support for performance measures by experts,
  advocacy groups, with heavy industry support
• Studies on quality chasm by health services
  researchers

71
Guideline Development of A1c lt7

• False dichotomies w/o consideration of very low
  marginal benefits or treatment-related costs,
  patient burden or risks.
• Unanimously rejected by Diabetes Alliance
  Technical Advisory Panel
• Heavy campaign by ADA and Industry
• Unanimously adopted by NCQAs CPM without much
  discussion or debate

72
Headline
Almost 60 of Americans have inadequately
controlled blood pressure

• RCTs from experts and industry
• Guidelines from experts, advocacy groups, with
  heavy industry financial support
• Support for performance measures by experts,
  advocacy groups, with heavy industry support
• Studies on quality chasm by health services
  researchers

73
Guideline Development of BP lt 140/90

• Average result combining low and high risk
  patients together
• Terrible cohort studies
• False dichotomies w/o consideration of very low
  marginal benefits or treatment-related costs,
  patient burden or risks.
• Because of measurement error in BP measures, the
  only way to have a good performance measure is to
  push patients way below 140/90 (or cheat)

74
HOT Trial Cohort Analyses
75
Aggressive Treatment of Htn (DBP goal 80 vs 90)
CV Mortality (HOT Trial. Lancet 1998)

• Achieved DBP 85mmHg vs 81mmHg
• Not published. Results estimated.

76
Headlines
Almost 40 of diabetics are at risk for blindness
because they do not receive recommended eye
screening

• Epidemiology from experts, NIE and industry
• Guidelines from experts, advocacy groups, with
  heavy industry and NIE financial support
• Support for performance measures by experts
  and advocacy groups
• Studies on quality chasm by health services
  researchers

77
Guideline Development for annual eye exams

• False dichotomies w/o consideration of very low
  marginal benefits or treatment-related costs,
  patient burden or risks.
• Surrogate quality measure became enshrined as
  the true measure of quality because it is easy
  to measure (optimally timed laser therapy is the
  truly true quality measure).

78
Suboptimal Timing of Retinal Laser
Therapy(Vijan 2000, Hayward 2005)

• 238 DM patients undergoing photocoagulation at
  one of 3 sites
• 40-50 with sub-optimal timing.
• 2/3 of the problem due to inadequate F/U of known
  retinopathy 1/3 due to very poor screening
  (gt3yrs)
• No cases had complications related to going
  1.5-3.0 years between screening examinations.

79
Highlighting Motes While Ignoring Beams

• 50 of recommended care is not received
• (Quality chasm)
• vs.
• 95 of recommended care is unimportant
• (The medical evidence)

80
Who dominates the policy debate on guidelines and
performance measures?

• NIH the scientific experts
• Advocacy groups (AHA, ADA, Americans for fill
  in something unequivocally desirable)
• Philanthropies
• Industry
• Quality safety experts

81
A Root Cause of the Cost-Quality Problem in
Healthcare Financing Delivery

• We are systematically misinforming providers,
  payers and consumers about the benefits of
  treatments by
• 1. Promoting treatments to the masses by
  using average benefits
• 2.Setting extreme treatment goals/guidelines
  using extremely bad science,
• 3. Promoting guidelines and performance
  measures without regard to costs, patient
  burden/risks and how much improving those
  processes will improve patient outcomes.

82
A Fatal Healthcare Policy Flaw?

• The current biomedical structure provides those
  with an inherent bias towards advocating more
  treatment to more individuals disproportionate
  control over information on treatments benefits,
  risks and costs,
• 2. Unless we implement changes to provide better
  balance in biomedical information, all other
  health policy efforts to improve efficiency will
  be severely handicapped

83
Structural Solutions Improvements

1. Require that clinical trials report relative and
   absolute benefits as a function of overall risk
   (when feasible) (CONSORT, FDA, Independent
   Taskforces)
2. Create an independent taskforce or taskforces
   that reviews evidence (akin to USPSTF and NICE)

84
Structural Solutions Improvements

• 3. Require that clinical trials be registered
  and that data be routinely inspected by FDA
  and/or the independent task forces

85
Needed Social/Cultural Changes

1. Need to change our definition of experts
   subspecialized researchers, NIH, professional
   organizations, etc. are often merely special
   interest groups they should be listened to
   carefully but their evidence needs to be
   reviewed by generalist experts.
2. No more dichotomies!!!!!
3. Its a multi-dimensional world get used to it.

86

• Everything should be made as simple as possible
  but not one bit simpler
• 
  Albert Einstein