Information Mastery Determining the Usefulness of Therapy Articles

1
Information MasteryDetermining the Usefulness of
Therapy Articles
2
Case

• You are seeing a 34 y/o woman with recurrent
  migraine headaches 3-4 times per month. All
  attempts to prevent them have had minimal
  success. She heard some vitamin supplement may
  help.

3
Study Methodsto Answer This Question

• Epidemiology Patients taking a vitamin are less
  likely to have migraines
• Pharmacology Drug x affects cerebral vasculature
  in rat brain isolates
• Case report It worked on one patient
• Case-series It worked on a bunch of patients
• Randomized controlled trial 1/2 get drug, 1/2
  placebo. No one knows who til the end who took
  what

4
Determining Relevance

• Read the title and the conclusion of the abstract
• Is it a POEM that will change your practice?
• Read on to validate the study only if all answers
  are YES

5
Validity

• Internal validity How well was the study done?
  Do the results reflect the truth?
• External validity can I apply these results to
  MY patients?

6
Levels of Evidence (LOE)

• Centre for Evidence-Based Medicine, Oxford
• Expert Opinion LOE 5
• Case Series LOE 4
• Case Control LOE 3b
• RCT LOE 1b
• SR with homogeneity 1a
• http//www.cebm.net/levels_of_evidence.asp

7
Determining Validity

• Read the methods section
• Answer questions on lower half of worksheet
• Study design flaws are common, but are they
  fatal?

8
Was it arandomized controlled trial?

• Randomization is the best protection against
  being mislead

9
The value of randomization

• 32 controlled trials of anticoagulation in acute
  MI
• Results by type of study
• Chalmers TC, et al. N Engl J Med 19772971091-6.

10
The value of randomization

• 50 RCTs vs 56 historical control articles
• 79 of historical control studies found benefit,
  as compared with 20 of RCTs
• Overestimate of benefit by historical control
• treatment of cirrhosis 1.8
• Coronary surgery 2.6
• Anticoagulation post-MI 2.2
• DES to prevent miscarriage 33.5
• Sacks H, et al. Am J Med 198272233-40.

11
Was allocation assignment concealed?

• Did investigators know to which group the
  potential subject would be assigned before
  enrolling them?

12
Importance of concealed allocation

• Trials with unconcealed allocation consistently
  overestimate benefit by 40
• Schulz KF, Chalmers I, Hayes RJ, et al. JAMA
  1995273408-12
• Schulz KF, Grimes DA. Lancet 2002359614-18.

13
Was allocation assignment concealed?

• Concealed allocation ? blinding
• Blinding can occur without concealed allocation
• Surfactant in the NICU
• Allocation can be concealed in an unblinded study
• PT vs surgery for knee DJD
• Moseley JB, O'Malley K, Petersen NJ, et al. N
  Engl J Med 2002 34781-8.

14
Conducting a Study
Potential Subjects
Trial starts
Actual Subjects
Randomization Blinding, etc
A
B
15
Take the red or blue pill?
16
Importance of concealed assignment

• Meta-analysis of trials evaluating screening
  mammography
• In studies in which allocation wasnt concealed
• Higher SE status, education level in screened
  group
• Age disparity (average 6 mo older in the
  unscreened group)
• Richer, smarter, younger
• Trials with concealed allocation screening
  harmful!
• No effect or increased mortality
• 20 more mastectomies
• Lancet Jan 8, 2000 Oct 20, 2001

17
(No Transcript)
18
Technical Nitpicking? Could this really make a
difference?

• Cumulative database 500,000 women
• Current policy is based on very small
  differences
• Deaths in unscreened women 902
• Deaths in screened women 837
• Death difference (of 456,349) 65!
• Systematic bias is not random error for which
  meta-analysis can compensate

19

• Mundus Vult Decipi- The world wishes to be
  deceived
• People would rather be deceived than have the
  truth cause anxiety
• Caleb Carr, Killing Time

20

• YOU WANT ANSWERS??!!!
• I WANT THE TRUTH!!
• YOU CANT HANDLE THE TRUTH!!
• Jack Nicholson and Tom Cruise
• A Few Good Men

21
Are the study patients similar to yours?

• Addresses generalizability of results to your
  practice (external validity)
• Examples
• Nonfebrile seizure rate following febrile
  seizures
• DCCT

22
Nonfebrile Seizure Incidence
23
Diabetes Complications and Control Trial

• Patients
• Type 1 diabetes, 13-39 years old
• No Htn, chol, diabetic complications
• Willing to check BS QID, inject insulin 3-4
  times/day
• Monthly visits for 6.5 years
• Twice weekly phone follow-up x 6.5 years
• Bajillion tests over 6.5 years
• Are these patients representative of type 2
  diabetics seen in FP?
• ADA uses these results to support tight glucose
  control in type 2 DM
• N Engl J Med 1993329977-986.

24
Were all the patients properly accounted for at
its conclusion?

• Complete follow-up?
• Intention to treat analysis?
• Patients are analyzed in the groups to which they
  are assigned
• Attempts to reflect real world clinical
  situations in which not all patients are
  compliant
• Watch when they compare only compliers with
  compliers and non-compliers
• Compliant subjects always do better overall

25
Was study double-blinded?

• Did the patients know to which group they were
  assigned?
• Did the treating physician know?
• Did investigators assessing outcomes know
  (triple-blinding up to 7 levels!)?
• Judicial assessor blind allocation concealment
  surgery RCTs
• Schulz KF. Ann Int Med 2002136254-9.

26
Were intervention and control groups similar?

• See Table 1 of most studies
• Randomization is best way to avoid bias, though
  imbalances still can occur (especially if
  allocation was not concealed)
• Small differences sometimes are important

27
Significance of Results?

• Next session

28
SummaryReading Therapy Articles

• Determine Relevance
• Is it a common POEM (longer/better) that will
  require you to change your practice?
• Determine Validity
• Answer questions on lower half of worksheet
• Weigh fatal vs. nonfatal flaws
• Ideal RCT study of patients similar to yours

29

• Effect on Patient-Oriented Outcomes
• Symptoms (drivers license)
• Functioning (visual loss)
• Quality of Life (amputation)
• Lifespan

Valid Patient-Oriented Evidence

• Effect on Disease Markers
• Diabetes (photocoagulation, GFR, NCV)
• Arthritis (x-ray, sed rate)
• Peptic Ulcer (endoscopic ulcers)

Disease-Oriented Evidence
Relevance of Outcome

• Effect on Risk Factors for Disease
• Improvement in markers (blood pressure,
  cholesterol, HBA1C, microalbuminuria)

• Highly Controlled Research
• Randomized Controlled Trials
• Systematic Reviews

• Physiologic Research
• Preliminary Clinical Research
• Case reports
• Observational studies

Uncontrolled Observations Conjecture
Validity of Evidence