People and Measurements

1
People and MeasurementsThe Nuts Bolts of
Research Optimizing Subjects Variables and
Introduction to Kaiser Division of Research

• Alan S. Go, M.D.
• Division of Research, Kaiser Permanente of
  Northern California
• Depts. of Epidemiology, Biostatistics, and
  Medicine, UCSF
• August 11, 2009

2
Todays Objectives

• Brief Introduction to Research in Kaiser
  Permanente of Northern California
• Gain a better understanding of the Kaiser
  Division of Research, population, and databases
• Selecting the People
• Develop systematic approach to optimize subject
  selection
• Choosing the Measurements
• Understand the implications of exposure outcome
  variable/measurement choices
• Application to a Real Research Question
  The ATRIA Study

3
AHA Cardiovascular Outcomes Research Center
Fellowship Opportunity
http//www.americanheart.org/presenter.jhtml?ident
ifier9713

• Two-year fellowship sponsored through the AHA
  Pharamceutical Roundtable focused on training the
  next generation of outcomes researchers
• Kaiser Permanente-Stanford University AHA CV
  Outcomes Research Center
• Contact Alan S. Go, MD (alan.s.go_at_kp.org)

4
Kaiser Division of Research Population

• Regional research division focused on
  epidemiology and health care effectiveness
  research
• Kaiser Northern California population
• gt3.2 million members (gt2 million adults) 52
  women

Race/Ethncity Overall, Men, Women,
Hispanic 12.3 12.6 12.0

Native Amer. 1.7 1.5 1.9
Asian 17.0 17.8 16.3
Hawaiian/PI 0.6 0.6 0.6
Black/AA 6.3 5.2 7.3
White/Eur. 61.8 61.5 62.1
Other 12.6 13.4 11.9
2005 Kaiser Permanente Members Health Survey (N.
Gordon)
5
Kaiser Administrative Databases

• Demographic membership characteristics
• Unique lifetime medical record number to track
  information across all major databases
• Age gender and race/ethnicity
• Membership, drug benefit, and insurance status
• Physician identifiers characteristics
• Clinic and medical center characteristics

6
Kaiser Clinical Databases/Registries

• Inpatient diagnoses/procedures
• Ambulatory diagnoses/procedures
• Outpatient pharmacy prescriptions
• Inpatient and outpatient laboratory tests
• Pathology findings
• Selected Kaiser disease registries
• Chronic kidney disease, heart failure, diabetes
  mellitus, GDM, cancer, HIV/AIDS, PCOS, etc.
• At end of 2008, regional EMR based on Epic

7
Subjects and Variables The Nuts and Bolts of the
Research Question

• After deciding a great research question,
  figuring out WHO you want to study and WHAT you
  want to measure are the next key steps

8
Selecting Your Subjects
9
Optimizing Subject Selection A Delicate
Balancing Act
Feasibility Accessibility Cost Time/Efficiency
Generalizability Accuracy Diversity Adequate Size
10
Subject Selection The Nitty Gritty

• Explicitly Define Inclusion Criteria
• Demographic features (e.g., age, gender, race)
• Clinical criteria
• Geographic/administrative characteristics
• Sampling time frame
• Explicitly Define Exclusion Criteria
• Minimum number needed to be feasible with
  acceptable generalizability to target population

11
Subject Sampling TechniquesHow to Get the
People? (1)

• Convenience Samples
• True convenience (e.g., 25 clinic patients I know
  well)
• Consecutive (e.g., next 100 patients undergoing
  liposuction)
• Probability Samples
• Simple random (e.g., using random number table)
• Stratified or weighted random (e.g., by gender)
• Cluster (e.g., by clinic or neighborhood)

12
Subject RecruitmentHow to Get the People? (2)

• Successful Recruitment Generally Means
• ? response, generalizable sample, adequate size,
  completed on time (or early!)
• For database only studiesNot usually a big
  problem
• For hands-on studies (e.g., surveys, cohorts,
  trials)
• Expect that it will be harder than you think!
• Use reasonable inclusion/exclusion criteria
• Acceptable subject burden/potential benefits
• Efforts to minimize subject non-response

13

• Applying These Principles to Answer My Research
  Question
• What is the association between use of the blood
  thinner, warfarin, and the risk of ischemic
  stroke bleeding in patients with atrial
  fibrillation treated in a usual clinical care
  setting?

14
Warfarin for Stroke Prevention in AF

• Atrial fibrillation (AF) is most common
  clinically significant arrhythmia1 and ? stroke
  risk 5-fold2,3
• RCTs in selected nonvalvular AF (NVAF) patients
  showed warfarin ? stroke by 68 but ? bleeding3
• Aspirin much less effective (RRR 20)
• Warfarin recommended for most NVAF patients, but
  concerns about whether trial results can be
  applied to the real world

1 Go AS et al. JAMA. 20012852370-75. 2 Wolf PA
et al. Stroke 199122983-88. 3 Atrial
Fibrillation Investigators. Arch Intern Med
19941541449-57
15
AnTicoagulation and Risk Factors In Atrial
Fibrillation The ATRIA Study
16
ATRIA Study
Atrial Fibrillation
Warfarin ? TE/Bleeds
17
ATRIA Study Subjects
Ambulatory adults with diagnosed nonvalvular AF
in Kaiser No.Calif.
All adults with nonvalvular AF in U.S.
18
ATRIA Study Inclusion Criteria

• Sampling Frame Goal Identify all ambulatory
  adults with diagnosed chronic nonvalvular AF
• Inclusion criteria
• Demography gt18 years, M/F, all race/ethnicities
• Clinical Criteria Diagnosed AF from outpatient
  ECG databases (?1 outpatient AF dx ?1 ECG with
  AF or ?2 outpatient AF dx only)
• Geography/Administrative Received care in Kaiser
  Permanente of Northern California
• Time Period AF diagnosis found in 1996-1997

19
ATRIA Study Exclusion Criteria

• Exclusion criteria
• No health plan membership
• Transient perioperative atrial fibrillation
• Concomitant hyperthyroidism
• Diagnosed valvular heart disease
• No outpatient care during 12 months after index
  date
• No drug benefit surrounding index date

20
ATRIA Cohort Assembly
Suspected AF
13,559 Ambulatory Adults with Diagnosed Chronic
Nonvalvular AF and Known Warfarin Status
Validation studies suggest 87 of cohort
w/ECG-confirmed AF
21
ATRIA Baseline Characteristics

• Mean age SD 71 12 yr
• Women 43
• Previous stroke 9
• Previous heart failure 29
• Hypertension 50
• Diabetes mellitus 18
• Previous coronary disease 28

The ATRIA cohort is older, has more women, and
greater comorbid burden than RCT
populations?likely generalizable to AF patients
in typical practice
22
Making the MeasurementsImplications for
Exposure Outcome Variable Choices
23
The most elegant design of a clinical study will
not overcome the damage caused by unreliable or
imprecise measurement.

• J.L. Fleiss (1986)

Fleiss, JL. The design and analysis of clinical
experiments. pp. 1-5. 1986. John Wiley and Sons,
New York.
24
Accuracy must be balanced against practical
considerations, and that method chosen which will
provide the maximal accuracy within the bounds of
the investigators resources and other practical
limitations.

• J.H. Abramson (1984)

Abramson, JH. Survey methods in community
medicine (3rd Ed.), p. 121. 1984. Churchill
Livingstone, Edinburgh.
25
Planning the MeasurementsRelationship of Key
Exposures
Predictor
Outcome
Often generally categorized as exposures
26
Additional Exposure Considerations

• Dose Issues
• Cumulative exposure
• Exposure rate
• Time Issues
• Start of exposure
• When it ended
• Exposure distribution

• Alcohol Use
• Total of drinks
• Drinks/day
• Date of first Anchor Steam
• Date of last margarita
• Daily vs. binge drinking

27
General Variable Types

• Continuous
• Quantitative intervals with typical ranking
• Examples
• Cholesterol level
• Number of drinks
• Day supply of drug
• Waist size
• Time

• Categorical
• Dichotomous (yes/no) (e.g., death, diabetes)
• Nominal (no order) (e.g., ethnicity,
  occupation)
• Ordinal (ordered rank) (e.g., NYHA HF Class
  I-IV)

28
Typical Data Sources
Goal choose the source that gives data closest
to the gold standard while being feasible to
collect

• Survey/questionnaire
• Interviews
• Diaries
• Direct observation
• Environmental measurements

• Databases/registries
• Medical records
• Physiologic measures
• Biomarkers (e.g., DNA, sera)
• Imaging tests
• Pathology

29
General Measurement Goals

• You get the same result when measured
  repeatedlywithin the same subject, between
  subjects, and over time?maximize PRECISION
• It represents what its really supposed to be?
  maximize ACCURACY/VALIDITY high sensitivity
  specificity

30
The Measurement Spectrum

• After deciding the exposure/outcome of interest,
  measurement includes
• Written instructions for applying the method for
  measuring the variable
• Doing the measurement method itself
• Spelling out collected data for analysis
• Implementing quality control procedures
  throughout (i.e., making sure you get what you
  meant to get)

31
Improving Precision and Accuracy of Variables
Reducing Bias

• Standardize methods
• Pretest, pretest, pretest
• Refine/automate instrument
• Train evaluate staff
• Timely editing, coding correcting of forms
• Multiple measurements

• Use or validate against gold standard
• Less obtrusive measures
• For outcomes, blinding to exposure status
• Institute quality control measures during data
  collection, processing, and analysis

32

• Applying These Principles to Answer My Research
  Question
• What is the association between use of warfarin
  and the risk of ischemic stroke bleeding in
  patients with atrial fibrillation treated in
  usual clinical care?

33
ATRIA Study Measurements
Ambulatory adults with diagnosed NVAF in KPNC
All adults with NVAF in U.S.
- Longitudinal warfarin use - Hospitalized
ischemic stroke or other systemic
embolism - Hospitalized bleeding event
Warfarin ? TE/Bleeds
34
Planning ATRIA Measurements
-Demographic features -Stroke risk
factors -Warfarin contraindications
Predictor
Outcome
(?)
35
Exposure Example Warfarin

• Warfarin use (main predictor)
• Baseline warfarin useAt least one of the
  following within 3 months of index AF dx date
• ?1 filled Rx for warfarin in pharmacy database
• Coumadin therapy in outpatient db (ICD-9
  V58.61)
• gt1 outpatient INR measurement in lab database
• Longitudinal warfarin usetime-dependent
  exposure based on warfarin Rx and INR tests
• Validation study of method for baseline use
• Chart review of random sample of users
  non-users 96 raw agreement (?0.92)

36
Outcome Example Ischemic Stroke

• Ischemic stroke (main outcome)
• Identification method searched databases
• Primary discharge ICD-9 codes for possible acute
  ischemic stroke (e.g., 433.x, 434.x, 436.0) found
  in hospital discharge and billing claims
  databases
• Validation method reviewed medical records
• Obtain Kaiser/non-Kaiser hospital records
• 3-physician review (/- Neurology consultant)
• Unable to blind warfarin status at time of event
• Valid stroke required documented acute
  neurological deficit lasting gt24 hours not due to
  other etiology

37
What Did We Find?

• Among 13,559 adults with nonvalvular atrial
  fibrillation, longitudinal use of warfarin
  therapy was associated with
• 49 adjusted decrease in risk of ischemic stroke
• Modest absolute increase in risk of intracranial
  hemorrhage (0.51 vs. 0.33 per 100 person-years)
• Net benefit of warfarin greatest among patients
  at the highest risk for ischemic stroke

Go AS, Hylek EM, Chang Y, et al. Anticoagulation
therapy for stroke prevention in atrial
fibrillation how well do randomized trials
translate into clinical practice? JAMA 2003
2902685-92.
38
Questions?

• Alan S. Go, M.D.
• Director, Comprehensive Clinical Research Unit
• Assistant Director for Clinical Research
• Division of Research
• Kaiser Permanente of Northern California
• 2000 Broadway St, Oakland CA 94612
• Tel [phone]/Email Alan.S.Go_at_kp.org