EvidenceBased Medicine Critical Appraisal of Harm

1
Evidence-Based MedicineCritical Appraisal of Harm
Department of Medicine - Residency Training
Program Tuesdays, 930 a.m. - 1200 p.m., UW
Health Sciences Library
2
Steps in Practicing EBM

• Convert the need for information into an
  answerable question.
• Track down the best evidence with which to answer
  that question.
• Critically appraise the evidence for its
  validity, impact, and applicability.
• Integrate the evidence with our clinical
  expertise and our patients characteristics and
  values.

3
Review Last Weeks Session
4
Steps in Practicing EBM

• Convert the need for information into an
  answerable question.
• Track down the best evidence with which to answer
  that question.
• Critically appraise the evidence for its
  validity, impact, and applicability.
• Integrate the evidence with our clinical
  expertise and our patients characteristics and
  values.

5
The Answerable Question
6
Good questions are the backbone of practicing
EBM. It takes practice to ask the
well-formulated question.
7
Well-Built Clinical ?s

• Directly relevant to the care of the patient and
  our knowledge deficit.
• Contains the following elements
• the patient or problem being addressed
• the intervention or exposure being considered
• the comparison intervention or exposure, when
  relevant
• the clinical outcomes of interest.

8
Well Formulated ?s

• Focus scarce learning time on evidence directly
  relevant to patients needs and our particular
  knowledge needs.
• Suggest high-yield search strategies.
• Suggest forms that useful answers might take.
• Help us to model life-long learning techniques
  for our colleagues and students.
• Are answerable and, thus, reinforce the
  satisfaction of finding evidence that makes us
  better, faster clinicians.

9
Harm Questions
10
Steps in Practicing EBM

• Convert the need for information into an
  answerable question.
• Track down the best evidence with which to answer
  that question.
• Critically appraise the evidence for its
  validity, impact, and applicability.
• Integrate the evidence with our clinical
  expertise and our patients characteristics and
  values.

11
General Resources
META-SEARCH ENGINES PrimeAnswers TRIP
SUMSearch SYSTEMATIC REVIEWS/META-ANALYSES Cochr
ane Library PubMed Clinical Queries EVIDENCE
GUIDELINES/SUMMARIES AHRQ Evidence Reports
Clinical Evidence AHRQ Preventive Services
CLINICAL RESEARCH CRITIQUES ACP Journal Club
1996- Bandolier 1994- BestBETs CASE
REPORTS/SERIES, PRACTICE GUIDELINES, ETC National
Guideline Clearinghouse PubMed
12
Steps in Practicing EBM

• Convert the need for information into an
  answerable question.
• Track down the best evidence with which to answer
  that question.
• Critically appraise the evidence for its
  validity, impact, and applicability.
• Integrate the evidence with our clinical
  expertise and our patients characteristics and
  values.

13
Strategies for Critical Appraisal of Studies of
Harm
14
Strategies for Critical Appraisal of Studies of
Harm
15
Judging validity with just 4 questions!

• 1. Did investigators assemble clearly defined
  groups of patients similar in all important ways
  other than exposure?
• 2. Were exposures and outcomes measured in the
  same ways in both groups (objective/blinded)?
• 3. Was follow-up sufficiently long and complete
  (5 and 20 rule)?
• 4. Do the results of the harm study fulfill some
  of the tests for causation?

16
Types of Studies(in order of decreasing
likelihood of being valid)

• Systematic reviews are ideal because individual
  RCTs seldom large enough to detect rare adverse
  events with precision - unfortunately, SR are
  uncommon.
• RCTs are difficult to conduct for most studies of
  harm.
• Cohort studies - exposed and unexposed followed
  for development of outcome of interest.
• Case-control studies - cases with outcome of
  interest compared with controls for exposure.
• Cross-sectional studies.
• Case reports.

17
Criteria for Inferring Causality

• Is it clear that the exposure preceded the onset
  of the outcome?
• Is there a dose-response relationship?
• Any positive evidence from a dechallenge-rechallen
  ge study?
• Is the association consistent across studies?
• Does the association have biological plausability?

18
Strategies for Critical Appraisal of Studies of
Harm
19
Judging clinical importance with just 2 questions!

• 1. What is the magnitude of the treatment effect?
• RR (Exposed ER - Unexposed ER)/Unexposed ER
• AR difference Exposed ER - Unexposed ER
• NNH 1/AR difference
• 2. How precise is this estimate of the treatment
  effect?
• 95 CI - range of values within which we can be
  95 sure that the population value lies.

20
Calculating NNT/NNH
1. A randomized trial of new drug Ligatite
reveals that 25 of World Cup skiers who take the
drug for one year have ACL tears whereas 50 of
World Cup skiers who take the placebo for the
year have ACL tears. What is the NNT?
NNT 1/AR reduction 1/(0.50-0.25) 4
2. The study of the drug Ligatite also notes
that 20 of athletes taking the drug develop
clinical depression whereas 10 of athletes
taking the placebo develop depression. What is
the NNH?
NNH 1/AR increase 1/(0.20-0.10) 10
3. An advertisement for a new drug fails to
mention that it increases the relative risk of
myocardial infarction by 50 over 5 years. You
read a valid study describing this finding. What
is the NNH?
Unknown without knowing the event rate in the
control population.
21
The Odds Ratio

• Used as an estimate of the risk ratio if the risk
  of the disease in a population is low.
• Is the principle measure of effect from
  case-control studies (cannot calculate event
  rates). Also used to report effect size in
  meta-analysis.
• Odds of exposure in the disease group divided by
  odds of exposure in non-diseased group.

OR (a/c)/(b/d) ad/cb
22
Converting OR to NNH
Calculator available at http//www.cebm.utoronto.
ca/practise/ca/statscal/orToNnt.htm
23
Avoiding TIV(table induced vertigo)

• ORs greater than 1.5 produce NNH lt 50 across
  most PEERs
• Patient needs to be at risk (non-trivial PEER) in
  order to be concerned.
• for any OR, NNH greatest when PEER0.5
• Consider carefully nature of harm (are your
  patients values disrupted by the intervention
  and its sequelae)

24
Estimating Our Patients Expected Event Rates
(PEER)

• 1. Assign our patient the overall control event
  rate from the study.
• 2. If there is a subgroup of patients in the
  study with similar characteristics assign the
  event rate for that subgroup.
• 3. If a validated clinical predication guide is
  available use it to assign an event rate.
• 4. Look for a different paper that describes the
  prognosis of untreated patients more similar to
  our patient and use its results to assign an
  event rate.

25
Clinical Tools for Estimating PEER
Available at http//hin.nhlbi.nih.gov/atpiii/calc
ulator.asp?usertypeprof09
26
Strategies for Critical Appraisal of Studies of
Harm
27
Applicable to Our Patient?

• 1. Is our patient so different from those in the
  study that its results cannot apply?
• 2. What is our patients risk of benefit and harm
  from agent?
• 3. What are our patients preferences, concerns,
  and expectations from this treatment?
• 4. What alternative treatments are available?

28
Returning to Ligatite
The trial of the drug revealed that 25 of World
Cup skiers who take the drug for 1 year have ACL
tears whereas 50 of skiers who take the placebo
have ACL tears. It also revealed that 20 of
exposed skiers developed depression whereas 10
of unexposed skiers developed depression. Your
patient reads about this in Ski Magazine and asks
you to write a prescription. In discussing the
medication with her you want to provide her with
an estimate of the magnitude of risk reduction
she would realize. Is her NNT 4 and should she
take this medication?
Probably Not 1. Her risk of an ACL tear is
substantially less so you have to re-estimate her
expected event rate. 2. She is unlikely to be
skiing year round so NNT is at least 2 to 3 times
as high. 3. There is risk of developing
depression (NNH 10 over 1 year). 4. Whether or
not to take the drug should take into account the
relative value to the patient of preventing an
ACL tear faced with the probability of developing
depression.
29
Harm Questions