Critical Appraisal: Epidemiology 101

1
Critical AppraisalEpidemiology 101

• POS Lecture Series
• April 28, 2004

2
What to Believe?
3

• "A proof is a proof. What kind of a proof? It's a
  proof. A proof is a proof. And when you have a
  good proof, it's because it's proven."

4
Introduction

• Why do I need Critical Appraisal Skills?
• Not all literature accurate
• Conclusions drawn not always possible
• Why the inaccuracies?
• Stupidity
• Publish or perish
• Money
• Being cynical and suspicious is healthy

5
The best defense is to be prepared
6
Introduction

• Types of studies
• Important components of a good randomized trial
• 6 important questions to ask yourself when
  reading a paper

7
Study Types

• Descriptive, Observational, Experimental
• Descriptive series, case report
• Observational groups determined by
  predetermined factor
• Experimental investigator in control of group
  assignments

8
Types of StudiesObservational

• Case-control
• uses
• Advantages and disadvantages
• Cost, good for causation in rare disease
• Recall bias

9
Types of Studies Observational

• Cohort
• Definition
• Advantages and disadvantages
• Prospective
• Cost high
• Esp if disease is rare or time between exposure
  and onset of disease is long

10
Types of StudiesExperimental

• Randomized trial
• Gold Standard
• Advantages and disadvantages

11
Principles of a Good Trial

• Ideas, research question, hypothesis
• Clinical relevance
• Is it possible?
• Time, finances, ethics

12
Principles of a Good Trial

• Literature search
• Background
• Results of other trials
• Convinced it was extensive

13
Principles of a Good Trial

• Patient Selection
• Inclusion and exclusion criteria
• Are they well defined?
• Are they reasonable?
• Are they clinically relevant?
• Do they change the results?

14
Principles of a Good Trial

• Sample size calculation
• Most ortho literature does not mention
• There is SOME science
• Based on primary outcome measurement

15
Sample Size Calculation

• n 2 (?? ??) ? / ? 2
• Z of a (Type one error)
• Usually 0.05 z1.96
• Z of ß (Type II error)
• Usually 0.2 Z1.28

16
Sample Size Calculation

• n 2 (?? ??) ? / ? 2
• ? S.D. of outcome measure
• How do you know??
• Pilot study
• published

17
Sample Size Calculation

• n 2 (?? ??) ? / ? 2
• ? Clinically relevant difference
• This is the variable that can be manipulated
• Depends of risks/cost of treatment

18
Sample Size Calculation

• n 2 (?? ??) ? / ? 2
• Equivalency trial
• Rarely done ?0.05 and sample size increases
• A neg trial that does not address this can not
  conclude no difference in treatments only we
  failed to prove a difference

19
Randomization

• Computer, random number table, coin toss
• Not birthday, MCP
• Block randomization
• Small number, multi-center
• AABB, ABBA, etc
• Potential for bias

20
Blinding

• Always adds weight to a study
• Are the subject and investigators blinded
• Is it feasable or possible?

21
Intervention

• Well defined, particulars discussed

22
Outcome Measurement

• Primary outcome measure
• Secondary outcome measures
• Data dredging

23
Analysis

• Biostats
• Definitely some trust here
• Everyone cant be an expert

24
Relative Risk Reduction(RRR)
Unreamed Reamed
Non-Union Rate .1 .05
RRR (0.1 0.05)/ 0.1 50 If outcome is
rare, this is misleading
25
Absolute Risk Reduction(ARR)
Unreamed Reamed
Non-Union Rate .1 .05
ARR 0.1 0.05 5 Good for rare outcomes
and NNT
26
Number Needed to Treat(NNT)
Unreamed Reamed
Non-Union Rate .1 .05
ARR 0.1 0.05 5 NNT 1/ARR 1/0.05 20
27
Lost to Follow-up

• 20 added to sample size
• Good Investigators very aggressive
• Worse case Analysis

28
Six Questions to Ask before you change your
practice!
29
1. Really Randomized?
30
2. All clinically relevant outcomes Reported?
31
3. Patients similar to your own?
32
4. Was clinical and statistical significant
considered?
33
5. Is the intervention feasible in your practice?
34
6. All patients accounted for?