Epidemiology for the Statistically Challenged Medical Student

1
Epidemiology for the Statistically Challenged
Medical Student

• Heather Murray
• MD, MSc, FRCP(C)
• Department of Emergency Medicine

2
Epidemiology - definition

• the study of the distribution and determinants
  of health related states and events in
  populations and the application of this study to
  the control of health problems.

3
Another definition...

• the science of turning bullshit into airplane
  tickets

4
Lecture overview

• Research Methodology - types of studies and brief
  assessment of quality
• Measurement - use and interpretation of 2x2
  tables and the things that go with them
• Statistical Gobbledygook - a look at the tests
  and numbers that we love to hate

5
(No Transcript)
6
Study types
7
Observational

• Observational studies use available information
  or collect new information
• Descriptive (hypothesis generating)
• Analytical (hypothesis testing with comparator
  groups)

8
Experimental

• Experimental studies test hypotheses via
  investigator manipulation of variable(s)
• Clinical trials (patients)
• Community trials (small populations)
• Meta-analysis (completed clinical trials)

9
ObservationalDescriptive Studies

• Hypothesis generating
• Describe characteristics of disease (outcome) or
  exposure (risk factor) with regards to
• Populations (i.e. demographics, SES, lifestyle)
• Geographic distribution / place
• Frequency variations over time (i.e. seasonal)

10
Types of Descriptive Studies

• Case report or case seriesNew syndrome
  described in gay men with similar histories and
  PCP pneumonia
• Correlational study (population study)Meat
  consumption in different countries vs. colon
  cancer rates
• Cross-sectional study (essentially a
  correlational study of individuals) Low
  beta-carotene levels associated with cancer
  (cause or result?)

11
Advantagesand Disadvantages

• Usually quick and inexpensive
• Use databases which are already available
• Efficient allocation of resources, planning of
  education and promotion programs
• However - always retrospective, no help with
  causality, conclusions are sometimes misleading
  (due to bias or confounding)

12
ObservationalAnalytical Studies

• Primarily hypothesis testing using comparator
  groups
• Goal is to determine if intervention (or
  exposure) affects (or is associated with)
  outcome
• Researcher records intervention and outcome

13
Analytic Studies - Observational

• Case-control why me??
• Individuals are classified based on presence of
  disease
• Then matched with a similar control group and
  their exposure history is compared
• Usually retrospective but can be prospective if
  you collect cases over a period of time
• Example testing association of ER residency
  and prior early childhood head trauma

14
Advantages

• Quick and inexpensive
• May be only method for rare disorders with long
  lag times between exposure and disease
• Fewer subjects required than cross sectional
  studies (more efficient)

15
Disadvantages

• Very susceptible to bias (both exposure and
  outcome have occurred at the time of the study)
• Mainly selection and recall bias
• Potential for confounding
• Can only study one outcome
• Cannot calculate incidence rates for exposed vs.
  unexposed patients

16
A word about confounding

• Confounding factors are related both to the
  exposure and the outcome
• Confounders are not an intermediate step between
  exposure and outcome, though
• i.e. Testing association between ER residency and
  early childhood head traumaconfounder is that
  ERPs tend to drop their kids (who want to be like
  their parents)

17
Analytic Studies - Observational

• Cohort what will happen to me?
• individuals are classified on the basis of
  exposure
• Exposure history is not under researchers
  control
• Can be retrospective or prospective
• example Follow graduating students (medical and
  non-medical) to test hypothesis that residency
  causes decline in social functioning

18
Advantages

• Allows measurement of incidence rates in exposed
  vs. unexposed (how many residents in the cohort
  become unfriendly?)
• Subjects can be matched for possible confounders
  (maybe only the residents with small children
  are grumpy?)
• Can study rare exposures(if high attack rate)
  (maybe just the ER residents are grumpy?)
• Can examine multiple outcomes
• (grumpy, out of shape and broke??)
• Easier and cheaper to administer than an RCT

19
Disadvantages

• Expensive and time-consuming (can take years for
  outcome of interest to occur)
• Lost to f/u serious threat to validity
• Potential for bias (particularly selection bias)
  and contamination
• Blinding of subjects and investigators difficult
• Inefficient for uncommon outcomes

20
Analytic Studies -Experimental

• Clinical trials (controlled vs. randomized)
• researcher manipulates the intervention or
  exposure (independent variable) and records
  effect on outcome of interest (dependent
  variable)
• RCT uses randomization to (hopefully) eliminate
  bias
• example medical students randomized to drug
  epiagra or placebo ? exam success

21
RCT - components

• Selection of target population
• Subject recruitment and enrollment
• Randomization
• Measurement of baseline characteristics
• Treatment / Intervention
• Follow-up / Data collection on outcome
• Data management
• Statistical Analysis

22
Assessment of Quality JAMA Users Guide to
Medical Literature (Therapy) 1993270(21)2598

• Primary Guides for Validity
• Was assignment randomised?
• Definition of randomised assignment each
  patient has an equal chance either study arm
• Were all enrolled patients accounted for at trial
  conclusion?
• Complete follow-up?
• Intention-to-treat analysis?

23
Assessment of Quality 2

• Secondary Guides for Validity
• Were patients, clinicians and study personnel
  (data analyzers) blinded?
• Were the groups similar at the start of the
  trial?
• Aside from experimental therapy, were the groups
  treated equally? (minimize co-interventions)

24
Community Trial

• Type of clinical trial where the object of
  randomization is a small population
• Example randomizing physician practices to
  additional NP care or usual MD care
• Used in situations where clusters of individuals
  share too many characteristics to produce
  unbiased assessments

25
Meta-analysis

• Structured and systematic integration of
  information from different studies of a problem
• Systematic review ? meta-analysis
• Each trial is like a patient in a clinical trial

26
Meta-analysis- components

• Question selection
• Search for relevant studies
• Selection of studies (inclusion criteria)
• Quality appraisal
• Assessment of heterogeneity
• Data collection (missing information collected)
• Statistical Analysis (sensitivity analysis)

27
Meta-analysis Quality Assessment

• JAMA Users Guide to Medical Literature
  (Overview) 1994272(17)1367
• Primary Guides to Validity
• Was the clinical question focused?
• Exposure, outcome, patient, control
• Were the study inclusion criteria appropriate?

28
Meta-analysis Quality Assessment

• Secondary Guides to Validity
• Is it likely that important articles were
  missed?
• Was the quality of included studies assessed?
• Example Jadad score 0-5
• Were these assessments reproducible?
• Were the results similar from study to study?

29
Summary - Will this be on the exam?

• Should be able to describe and give examples of
  different study types
• Summarize 2 or 3 advantages and disadvantages of
  each type
• List the features of a well performed clinical
  trial and/or meta-analysis

30
(No Transcript)
31
Diagnostic tests

• More fun than an anal fissure in a vinegar
  bath

32
The Dreaded 2 X 2 Table...

• The Truth
• disease present disease absent
• a b
• test
• - c d
• a true positive b false positive
• c false negative d true negative

33

• Sensitivity measure of a tests ability to
• correctly identify patients with disease a/
  (ac)
• Specificity measure of a tests ability to
• correctly identify patients without disease
  d/(bd)

34
Sensitivity and Specificity

• Generally stable measures of a tests ability
  to assess probability of disease
• Either measure alone does not tell you
  probability of a positive or negative test being
  correct

35

• Positive Predictive Value
• the proportion of patients with a positive
• test who have the disease a/(ab)
• Negative Predictive Value
• the proportion of patients with a negative
• test who do not have the disease d/(cd)

36
Positive and Negative Predictive Values

• Tell you probability of a positive or negative
  test being correct
• Unstable measures that vary greatly with
  pretest probability of a target disorder

37
Clinical example

• A patient comes to the ER with chest pain - is
  this an MI??
• You have a blood test...
• - will be positive and correctly identify 85 of
  persons with heart attacks (sens 85)
• - will be positive in 25 of persons who do not
  have heart attacks (spec 75)

38
Estimation of Pre-test Probability

• A 56 yo man complains of a heaviness in the
  middle of his chest, feels awful, looks awful and
  is SOBPTP 80
• A 20 yo woman has pain over her right ribs and it
  hurts to move.
• PTP 5

39
Pre-test Probability 80

• cardiac pain non-cardiac
• 680 50
• test
• - 120 150
• 800 200

40
Predictive Values

• PPV with 80 prevalence 93
• NPV 55
• sensitivity 85
• specificity 75

41
Pre-test Probability 5

• cardiac pain non-cardiac
• 43 238
• test
• - 7 712
• 50 950

42
Predictive Values

• PPV with 5 prevalence 15
• NPV 99
• sensitivity 85
• specificity 75

43
Enter the Likelihood Ratio

• Based on both the sensitivity and specificity of
  a test
• A ratio not a proportion/percentage
• Allows the calculation of post-test probability
  based upon pretest probability and results of
  test (Fagan nomogram)

44
Positive Likelihood Ratio

• Sensitivity1 - Specificity

45
Negative Likelihood Ratio

• Specificity 1 - Sensitivity

46
Using the LR

• For a test with a sensitivity of 85 and a
  specificity of 75
• the LR() 3.4 and the LR(-) 0.2
• Interpretation
• test 3.4 times more likely in patients with
  disease
• - test 0.2 times more likely in patients with
  disease (or 5 times less likelythe inverse)

47
One last example

• V/Q scan
• High probability LR 18.3
• Intermediate probability LR 1.2
• Low probability LR 0.36
• Normal / near normal LR 0.10

48

• Fagan nomogram

49
How to use the LR.

• LRs of gt10 or lt0.1 generate large and often
  conclusive changes from pre to post test
  probability
• LRs of 1-2 and 0.5 to 1 alter probability to a
  small (and rarely important) degree

50
Summary - Diagnostic Tests

• 2x2 table disease at the top!
• Be able to define and calculate
• Sensitivity and Specificity
• NPV and PPV
• LR () and LR (-)
• Understand the effect of prevalence (Pretest
  Probability) on these calculations

51
(No Transcript)
52
Incidence vs. Prevalence

• A prevalence is the proportion of a group
  possessing a clinical condition at a certain
  point in time (i.e. the prevalence of boredom in
  the room right now is)
• An incidence is the proportion of a group
  initially free of the condition that develop it
  over a given period (i.e. the incidence of
  boredom since the last slide featuring a cartoon
  is)

53
Why the difference?

• Incidence and prevalence give us different
  information
• Prevalence what proportion of people have a
  condition?
• Incidence at what rate do new cases of this
  condition arise over time?

54
Statistical Gobbledygook (but what does it mean??)

• Measures of association (RR, AR, OR, NNT)
• Hypothesis testing and the letter p
• Alpha, beta, power and error
• Confidence intervals

55
Measures of Association

• disease
• -
• a b
• exposure
• - c d
• RR a / (ab) OR ad/bc
• c / (cd)

56
Relative Risk a/(ab) c/(cd)

• Relative Risk is a measure of association used in
  cohort studies or clinical trials
• Basically the event rate in exposed over
  unexposed patients(or treated over untreated
  patients)
• RR gt 1.0 increased risk of event
• RR lt 1.0 decreased risk of event

57
Odds Ratio ad/bc

• Used in case control studies where incidence
  rates are not available
• Can approximate the RR in diseases where the
  incidence is rare (lt5)
• Why? In rare diseases, ab b and cd d
• ? a/(ab) a/b/c/d ad/bc c/(cd)

58
Absolute and Relative Risk Reduction

• Used in looking at effects from therapeutic
  trials
• The difference in risk of outcome (expressed as a
  proportion or as an absolute) from patients
  receiving one therapy versus the other

59
When the treatment reduces bad events...

• EER Experimental event rate
• CER Control event rate
• ARR EER - CER
• RRR EER - CER / CER
• NNT 1 / ARR

60
PGY-5 ER (n200)Epiagra vs. Placebo

• Exam Failure Exam Pass
• Epiagra 8 92 100
• Placebo 28 72 100
• 36 164 200

61

• X members of the placebo group failing ER exam
  (28/100 or 28)
• Y members of the Epiagra group failing ER exam
  (8/100 or 8)
• RR (8/100) / (28/100) 0.29
• i.e. Epiagra protective against failure

62

• EER members of the Epiagra group failing ER
  exam (8/100 or 8)
• CER members of the placebo group failing ER
  exam (28/100 or 28)
• ARR 0.08-0.28 0.2 (20)
• RRR 0.2 / 0. 28 0.71 (71)
• NNT 5

63
Hypothesis testing

• Null hypothesis the true difference between the
  control and experimental treatments is zero
• Observed differences between treatment groups may
  be due to true difference, or due to chance
• p value likelihood of observed result occurring
  due to chance (0.05 cutoff)

64
Statistical Error

• Type I error - Concludes there is a difference
  when none exists (false positive)
• by convention accepted as 5 (? 0.05)
• Type II error - Treatment difference exists but
  is not recognized (false negative)
• by convention accepted as 20 (? 0.20)
• Power of a study to detect a true difference 1
  - ? (usually 80)

65
(No Transcript)
66
Multiple Comparisons

• If you torture the data long enough it will
  confess
• More comparisons more likely to find something
  significant
• Statistical adjustments should be made for
  multiple comparisons

67
Sample Size Calculations

• Elements required in a sample size calculation
• ?
• ?
• expected event rate in the control group
• clinically important difference between groups

68
Confidence intervals

• Easiest to understand when we think about a mean
  and the distribution around that mean

69
What is a CI?

• Confidence interval gives an idea of the
  precision of the statistical estimate
• Or, how close the observed mean is to the
  population mean
• So, the 95 confidence interval maps out a number
  range 95 likely to include the true sample mean

70
How do you calculate it?

• The confidence interval is calculated using the
  statistical value (mean, correlation, proportion)
  sample size and the standard error (estimated
  using the SD) of the sample
• Can be calculated for rates, means, proportions
  or nearly any other statistic you can think of...

71
So What?

• The key here is overlapping confidence intervals
  (translates into pgt0.05)
• Exam fail rate controls 20 (95CI 12-28)
• Exam fail rate Epiagra 10 (95CI 4-16)
• Or a confidence interval of the difference
  between means or proportions that crosses zero
  (also pgt0.05)
• Epiagra improvement rate 10(95CI -2 - 12)

72
Interpreting Negative Trials

• Examining the confidence interval helps assess
  whether there is a possibility of error in the
  conclusions
• Negative trial with wide 95CI may have type II
  error (look at upper boundary)
• Positive trial with wide 95CI may have type I
  error (look at lower boundary)

73
Summary

• Need to be able to calculate measures of
  association for the results of a simple trial
  (RR, OR, ARR, RRR and NNT)
• Must understand concept of hypothesis testing
• Describe the types of error in clinical trials
  and the elements for sample size calculation
• Understand the concept of 95CI

74
(No Transcript)
75
(No Transcript)