Evidencebased Practice Centers

1
Evidence-based Practice Centers

• Created in 1997 now 13 centers
• Produce
• evidence reports
• systematic reviews
• technology assessments
• rapid reviews
• meta-analyses and cost analyses
• analysis of large databases
• Work with public and private sector
• partners

2
Evidence-based Medicine

• Mark Helfand, MD
• Director
• Oregon Evidence-based Practice Center

3
What is the kind and strength of the evidence you
are relying on to make a recommendation?
The Question
4
What does evidence-based mean?

• A comprehensive, systematic, open minded review
  of all the evidence
• The evidence determines the conclusion, not vice
  versa
• Not, the citation of papers supporting a
  preformed conclusion (and trashing of those that
  dont)
• Not, the use of evidence when it is positive
  but judgement when it isnt

5
Systematic literature reviews

• Are systematic to remove bias in finding and
  reviewing the literature.

6
Systematic literature reviews

• Are systematic to remove bias in finding and
  reviewing the literature.
• Experts may interpret the data (and their own
  experience) differently.

7
How sure are we?Expert estimates of breast
implant rupture rates
0 0.2 0.5 1 1 1 1.5 2 3
3 4 5 5 5 5 5 5 5 5
6 6 6 8 10 10 10 10 13
13 15 15 18 20 20 20 25
25 25 30 30 40 50 50 50
62 70 73 75 75 75 75 80
80 80 80 80 80 100
Source Dr. David Eddy
8
Experts estimates of the effect of colon cancer
screening on chance of dying
Source Dr. David Eddy
9
Experts estimates of probability of acute
retention in men with BPH
Source Dr. David Eddy
10
Systematic literature reviews

• Are systematic to remove bias in finding and
  reviewing the literature.
• Studies with disappointing results may get less
  attention

11
Excludes 5 mg bid group
12
Trial 114
13
Systematic literature reviews

• Are systematic to remove bias in finding and
  reviewing the literature.
• Experts may underplay controversy or select only
  supportive evidence

14
Simpson et al, 2004
15
Simpson et al, 2004
16
(No Transcript)
17
In a double-blind study vs risperidone GEODON
sustained control of positive symptoms at 1 year
1
18
In a double-blind study vs risperidone GEODON
sustained control of positive symptoms at 1 year
1
19
Systematic literature reviews

• Are systematic to remove bias in finding and
  reviewing the literature.
• Experts may underplay controversy or select only
  supportive evidence
• Emphasize the best evidence

20
The best evidence

• Reflects patients concerns
• By addressing health outcomes patients, their
  caregivers, and families care about

21
The best evidence

• Reflects patients concerns
• By addressing health outcomes patients, their
  caregivers, and families care about

• Help you feel similar to other people
• Help you feel less lonely and removed from others
• Help you feel more hopeful and happy
• Allow you to think and express yourself more
  clearly

22
Selecting questions

• Researchers often use their own curiosity or
  research interest as the basis for selecting
  questions.
• They often use standard scales and measures
  instead of seeking a deeper understand of the
  patients well-being and quality of life.

23
Selecting questions

• Our premise is that important questions arise
  from practice, and from life. Experts in
  practice--and patients--select the populations,
  interventions, and outcome measures of interest.

24
The best evidence

• Reflects patients concerns
• By addressing health outcomes patients, their
  caregivers, and families care about
• By using simple measures of benefit and risk

25
Example
26
Why use systematic literature reviews?

• Define the strengths and limits of the evidence.
• Clarify what is based on evidence and what is
  based on other grounds.
• Do not necessarily tell you what to do when the
  evidence is limited. Other factors, such as
  equity, clinical judgment, values, and
  preferences play a role in using the evidence.

27
Rules for linking evidence to recommendations


local judgments and values
Evidence-based decision-making
28
An evidence-based decision process

• Makes use of an independent, systematic review of
  the evidence
• Employs rules for linking evidence to
  recommendations
• Produce explicit, defensible recommendations

29
Oregon ApproachWhat are we after?

• Systematic drug-class reviews should address
  questions that reflect clinicians and patients
  concerns.
• Decision-makers should begin to wrestle with the
  idea of what is good evidence.
• Manufacturers should gain market share if they
  produce good evidence of superiority over other
  drugs in a class.
• Patients, caregivers, payers (and NAMI) should
  demand better evidence about outcomes that matter
  !

30
(No Transcript)
31
Drug Class Review on Atypical Antipsychotics
32
Included Drugs
Clozapine not posted risperidone (1993) not
posted olanzapine (1996) not posted quetiapine
(1997) not posted ziprasidone (2001) posted arip
iprazole (2002) posted
33
Eligible Outcomes
34
Results

• 196 studies included overall
• 33 head-to-head
• 24 placebo-controlled
• 58 active controlled
• 63 observational studies
• 18 systematic reviews
• 427 study publications excluded

35
SchizophreniaHead to Head Trials

• 3 Effectiveness Trials
• 12 month pragmatic trial of olanzapine,
  risperidone or continuing typical AP
• One 12-month switching study of olanzapine
  risperidone
• InterSept trial of clozapine and olanzapine to
  prevent suicidality found clozapine superior
• 30 Efficacy Trials

36
Head to head trials in outpatients
37
Summary Benefits

• Clozapine, olanzapine and risperidone had similar
  efficacy with two exceptions
• Clozapine olanzapine in suicidality/suicide
  prevention
• Olanzapine risperidone in reducing rates of
  relapse
• Aripiprazole, quetiapine, and ziprasidone
  Evidence too limited to say

38
Summary Harms

• Weight gain
• Greater risk for olanzapine than risperidone
• Results mixed in long-term observational studies
• Diabetes mellitus
• Risk greater for olanzapine than risperidone, but
  studies had mixed results
• Risk with clozapine relative to others not clear
• Limited evidence on quetiapine
• Other long-term safety
• No conclusions about comparative safety can be
  made

39
Other harms

• Movement disorders
• Somnolence
• Hyperprolactinemia/sexual dysfunction
• Long QT interval
• Bone marrow problems

40
Outpatient studies

• Better head-to-head comparisons of antipsychotics
  are needed to discern the relative efficacy and
  safety profiles of these compounds.

41
What we can do together

• select and refine questions that puts patients
  and caregivers concerns center stage
• Rely on unbiased reviews to inform patients,
  families, and clinicians
• Promote an evidence-based process, not just
  systematic reviews.
• Promote higher standards for evidence about
  treatments for mental illnesses

42
Observational Studies Long-term Safety

• 48 studies, ? 6 months in duration
• primarily schizophrenia patients
• 8 head-to-head cohort studies
• 10 AAP versus typical AP cohort studies
• 29 descriptive epidemiologic studies
• 1 case-control study
• Death Rates ranged from 0.1 to 3.3 for
  clozapine, quetiapine and risperidone (7
  uncontrolled studies)

43
Criticism

• By adhering to rigorous rules of inclusion, the
  process maximizes the validity of assessing
  proven treatment efficacy (strength), while it
  ignores or discards other germane but less
  statistically rigorous evidence of real-world
  effectiveness and cost-effectiveness (weakness).

44
Our response

• We agree controlled trials ignore important
  aspects of effectiveness

45
Limitations of RCTs

• There arent enough of them.
• They test interventions that may or may not fit
  easily into practice.
• They often dont tell you about important
  subgroups.
• They may not extend for a long time.

46
More limitations of RCTs

• Design features are poorly adapted to the purpose
  of assessing average effectiveness
• Populations
• run-in periods
• Exclusions
• Comparators and comparisons
• Outcome measures
• Followup period
• Feasibility
• Implementation costs
• Maintenance costs

47
Most common problems with head-to-head trials

• Doses of the different drugs arent equivalent.
• Strategies for using the drugs arent realistic.
• Usually, focus on efficacy or harms but not on
  both
• Do not address all important outcomes

48
RCTs harms

• Design features are poorly adapted to the purpose
  of assessing harms
• run-in periods
• exclusions of susceptible people
• Reporting is poor
• unreported
• Selectively reported
• Misleadingly reported
• Lack of severity data

49
Applicability How to bias an efficacy study and
stillget a good-quality rating

• select compliant patients
• dilute the control group interventions
• measure only certain outcomes
• cheat
• selective use of cut-off dates
• what are the norms?

50

• We agree controlled trials ignore important
  aspects of effectiveness
• and agree on what information wed like to have.

51
Quality of the evidenceat 4 levels

• Type of study.
• Quality of each study based on study design.
• Overall quality of the evidence for a key
  question.

52
1. Types of studies

• case reports, case series
• animal studies
• studies of etiology
• prospective cohort studies
• open-label controlled or uncontrolled studies
• randomized trials

53
2. Quality of individual studies

• quality (good, fair, or poor) for each type
  of study design
• Use of random allocation
• Concealed allocation
• Double-blind method
• Exclusions after randomization
• applicability

54
Internal Validity Criteria for RCTs cohort
studies

• Initial assembly of comparable groups
• Maintenance of comparable groups
• Minimal loss to follow-up
• Measurements equal, reliable, valid
• Clear definition of interventions
• All important outcomes considered
• Intention-to-treat analysis

OHSU EPC
55
3. Evidence at each linkage

• Aggregate internal validity Are there any
  studies with good design (for the question) that
  were also well-conducted? Is the best evidence
  of good internal validity?
• Consistency/coherence Do studies conflict in
  their findings? Is there a body of supporting
  evidence so that the best evidence makes sense?

56
3. Rating each link in the AF

• Quality and consistency of studies
• large numbers of patients
• consistent results across studies
• Applicability of studies
• patient populations, interventions, outcomes like
  those of interest to the organization
• real life evidence not just efficacy
• attention to harms

57
Systematic literature reviews

• Define the strengths and limits of the evidence.
• Clarify what is based on evidence and what is
  based on other grounds.
• Do not necessarily tell you what to do when the
  evidence is limited. Other factors, such as
  equity, clinical judgment, values, and
  preferences play a role in using the evidence.

58
What Does it Mean for Decisions to be
Evidence-Based?

• Decisions are based on best evidence
• Best evidence
• Is unbiased
• Is appropriate for decision at hand
• Includes all germane evidence

Luce
59
An evidence-based decision process

• Makes use of an independent, systematic review of
  the evidence
• ?Employs rules for linking evidence to
  recommendations
• ?Produce explicit, defensible recommendations

60
Strength of recommendations
61
Strength of recommendations
62
What is evidence-based medicine?

• Evidence-based medicine is the integration of
  best research evidence with clinical expertise
  and patient values.

David Sackett
63
What is evidence-based medicine?

• Where there is evidence of benefit and value, do
  it
• Where there is evidence of no benefit, harm, or
  poor value, dont do it.
• When there is insufficient evidence to know for
  sure, be conservative

David Eddy
64
(No Transcript)
65
Evidence-based Practice Centers

• Created in 1997 now 13 centers
• Produce
• evidence reports
• systematic reviews
• technology assessments
• rapid reviews
• meta-analyses and cost analyses
• analysis of large databases
• Work with public and private sector
• partners

66
Oregon Evidence-based Practice Center

• USPSTF
• Drug class reviews for states
• Food claims for FDA
• Various other topics
• HBOT for cerebral palsy
• Rehabilitation for traumatic brain injury
• Treating actinic keratoses
• Telemedicine
• VBAC
• Osteoporosis diagnosis and treatment
• Preventing youth violence

67
Oregon Evidence-based Practice Center

• EVIDENCE REPORTS FOR DRUG CLASSES
• http//www.ohsu.edu/drugeffectiveness/reports/
• USPSTF RECOMMENDATIONS
• http//www.ahrq.gov/clinic/uspstfix.htm

68
Criticism 3. EBM hurts minorities and vulnerable
populations

• -- each drug is unique
• -- each patient is unique
• -- doctors should be able to choose any drug for
  any patient

69
Other study designs could be helpful, after the
following questions are answered

• Will our users find them credible enough to use
  them?
• Can it be identified, introduced into the review
  in a systematic way?
• Can we tell a good outcomes study from a poor
  one?
• Can we tell a good economic study from a poor
  one?
• Can users incorporate it into decisions in a
  meaningful way?

70
Most common problems with observational studies
of adverse events

• Incomplete ascertainment
• Few data on severity of the event
• Dont report on efficacy (to examine trade-offs)
• Confounding, bias

71

• Level 1 Would you have this done for yourself
  or for someone else in your immediate family?
• Influenced by ones personal experience with
  the disease and capacity to deal with risk.
• Affects few people.
• Level II What would I recommend to my
  patient/client?
• Physician making a recommendation for his/her
  patients. Influenced by prior experience, but
  the scientific evidence may play a greater role.
• Affects possibly hundreds of people.
• Level III What would I recommend to the nation,
  the world?
• Across-the-board recommendations for a
  population.
• Must be based on rigorous assessment of the
  scientific evidence.
• Affects hundreds of thousands, even millions of
  people.

72
1998First FDA application 2001FDA approval
for schizophrenia2004Approval in acute
maniaAugust, 2004Warning hyperglycemia and
diabetes April, 2005Warning on off-label use
in elderly (olanzapine), Abilify (aripiprazole),
Risperdal (risperidone), and Seroquel
(quetiapine). June, 2005Lilly settles Zyprexa
suits