How to Analyze Therapy in the Medical Literature (part 1)

1
How to Analyze Therapy in the Medical
Literature(part 1)
Akbar Soltani. MD. Tehran University of Medical
Sciences (TUMS) Shariati Hospital www.soltaniebm.c
om
2
  The hierarchy of evidence

• Systematic reviews and meta-analyses
• Randomised controlled trials
• Cohort studies
• Case-control studies
• Cross sectional surveys
• Case reports
• Expert opinion

3
Three Step Guide in Using an Article to Assess
Therapy

• Are the results of the study valid?
• What are the results? What measures of precision
  of effects were reported (CIs, p-values)?
• How can I apply these results to patient care?

4
Randomized control trial design
5
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

6
Benefits of Random Allocation (Randomization)

• 1.Reduces bias in those selected for treatment
• guarantees treatment assignment will not be based
  on patients prognosis
• 2.Prevents confounding
• known and unknown potential confounders are
  evenly distributed

7
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

8
Ensuring Allocation Concealment

• NOT RANDOMIZED
• Date of birth, alternate days, etc

9
Do Not Confuse Allocation Concealment with
Blinding

• Allocation concealment seeks to prevent selection
  bias, protects assignment sequence before and
  until allocation, and can always be successfully
  implemented

10
(No Transcript)
11
Do Not Confuse Allocation Concealment with
Blinding (Contd)

• Blinding seeks to prevent information bias,
  protects sequence after allocation, and cannot
  always be successfully implemented

12
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

13
Placebo effect Trial in patients with chronic
severe itching
No treatment
Trimeprazine tartrate
Cyproheptadine HCL
Treatment vs no treatment for itching
14
Placebo effect Trial in patients with chronic
severe itching
No treatment
Trimeprazine tartrate
Cyproheptadine HCL
Placebo
Treatment vs no treatment vs placebo for itching
Placebo effect - attributable to the expectation
that the treatment will have an effect
15
(No Transcript)
16
Blinding and Reporting

• Usually reduces differential assessment of
  outcomes (information bias)
• Authors should explicitly state who was blinded
  and how.
• Many investigators and readers consider a
  randomized trial as high quality simply because
  it is double-blind, as if double-blinding is
  the sine qua non of an RCT.
• Trials not double-blinded should not
  automatically be deemed inferior trials.

17
Blinding in randomised trials hiding who got
what

• Double blinding prevents bias but is less
  important, than adequate allocation concealment.
• open studies are more likely to favour
  experimental interventions over the controls and
  that studies that are not double-blinded can
  exaggerate effect estimates by 17
• Furthermore, in some trials, blinding cannot be
  successfully implemented, whereas allocation
  concealment can always be successfully
  implemented.

18
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

19
Significance tests for baseline differences
20
Baseline data
Effect of azathioprine on the survival of
patients with primary biliary cirrhosis
Azathioprine Placebo
Mean age 54.7 54.9
Serum bilirubin (?mol/L) 37.2 30.9
Stage I disease 14 12
Stage II disease 44 43
Stage III disease 15 15
Stage IV disease 27 30
Christensen et al. Gastro 1985
21
Baseline data
Effect of azathioprine on the survival of
patients with primary biliary cirrhosis
Azathioprine Placebo
Mean age 54.7 54.9
Serum bilirubin (?mol/L) 37.2 30.9
Stage I disease 14 12
Stage II disease 44 43
Stage III disease 15 15
Stage IV disease 27 30
Christensen et al. Gastro 1985
22
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

23
How complete was the follow up? How many dropouts
were there?

• Conventionally, a 20 drop out rate is regarded
  as acceptable, but this depends on the study
  question.
• Some regard should be paid to why participants
  dropped out, as well as how many.
• It should be noted that the drop out rate may be
  expected to be higher in studies conducted over a
  long period of time.
• A higher drop out rate will normally lead to
  downgrading, rather than rejection of a study.

24
Bias a one-sided inclination of the mind

• Not random 40
• Not double-blind 17
• Duplicate information 20
• Small trials 30
• Poor reporting quality 25

Over-estimation of treatment effect
25
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

26
Intention to treat
Montorri V, Guyatt G. CMAJ 2001 165(10) p1340
27
Intention to treat
Montorri V, Guyatt G. CMAJ 2001 165(10) p1340
28
Intention to treat
High risk?
Montorri V, Guyatt G. CMAJ 2001 165(10) p1340
29
Assess Validity and Applicability to my practice
setting

• 1.Is the study a randomized control trial (RCT)?
• Yes (go on) No (stop)
• 2.Were the patients properly selected for the
  trial and randomized with concealed assignment?
• Yes (go on) No (stop)
• 3.Were patients and study personnel blind to
  treatment?
• Yes (go on) No (pause)
• 4.Were the intervention and control groups
  similar at the start? (Check Table 1 of most
  studies)
• Yes (go on) No (stop)
• 5.Was follow-up complete?
• ii. Were patients analyzed in the groups to which
  they were randomized (intention-to-treat
  analysis)?

30

• Thank you!