Follow-up and Compliance: Outline

1
Follow-up and Compliance Outline

• Follow-up
• Contents, frequency
• Importance of complete follow-up
• Compliance/adherence
• Importance
• How to measure and maximize
• Reported follow-up and compliance
• CONSORT guidelines
• ITT and other issues in analysis
• Definitions
• Different flavors
• More on data analysis, if time

2
Follow-up in RCTs

• What happens after randomization
• Carefully lay out procedures to be followed
• Describe on forms and in Operations Manual
• First reaction do everything on everyone at
  every visit
• e.g. labs or MRIs at all visits
• But great opportunities for efficiencies and cost
  savings
• More parsimonious visit may increase adherence
• Ask the following
• Do only at some visits?
• Do only on a subset?
• Dont do at all

3
Follow up Styles

• Intensity of follow up depends on study needs
  (and budget)
• Maximal
• In patient GCRC-type study
• Phase I and (sometimes phase II) studies
• Lots of measurements done frequently
• Minimalist
• Few follow up visits and measurements
• Advantages and disadvantages of each

4
More parsimonious measurementsRCT of Zoledronic
acid vs. placebo (osteoporosis)

• 7765 participants
• Primary outcomes vertebral fracture, hip
  fracture
• Secondary outcomes bone density, markers of
  bone metabolism
• Markers of bone metabolism
• Done on subset (convenient) of about 500
  participants (7 of total)
• Only at some visits (every 6 months)
• Cost savings vs. all participants at all visits
  gt95

Black et. al , NEJM, 5/07
5
Zoledronic acid and Mean Serum ß-CTX
1.0
0.9
Annual dose
0.8
0.7
0.6
Mean Serum ß-CTX (ng/mL)
0.5
0.4
0.3
0.2
0.1
0.0
0
6
12
18
24
30
36
Months
ZOL n 257 236 200 136 191 190 174
PBO n 260 248 214 156 196 197 170
Black DM, et al. NEJM. May 5, 2007
6
Large and Simple Trials

• Get a whole lot of people
• Randomize, do as few follow-up measurements as
  possible
• Can be difficult to carry out in practice
• Examples of how to streamline follow-up
• Physicians Health study Randomize to aspirin
  or placebo, mail out drugs, follow-up by mail
• Use data collected for other purposes for
  follow-up/endpoints
• Population mortality
• Medical systems (Medicare, Kaiser in U.S. Govt.
  health in Europe)
• Internet follow-up direct from participants

7
Large and Simple Trials Vitamin D

• 2700 men and women from general medical practice
• Oral vitamin D (100,000 units) given 4 x per year
  (oral) for prevention of fractures in UK
• Sent oral D (and placebo) via mail
• Ascertained fractures via post (and National
  Health Service data base)
• Mortality via National statistics
• Found decrease in fracture risk (possibly
  mortality)

Effect of four monthly oral vitamin D3
(cholecalciferol) supplementation on fractures
and mortality in men and women living in the
community randomised double blind controlled
trial.BMJ. 2003 Mar 1 326(7387) 469.
8
Large and Simple Trials Vitamin D

• OBJECTIVE To determine the effect of four
  monthly vitamin D supplementation on the rate of
  fractures in men and women aged 65 years and over
  living in the community.
• DESIGN Randomised double blind controlled trial
  of 100 000 IU oral vitamin D3 (cholecalciferol)
  supplementation or matching placebo every four
  months over five years.
• SETTING AND PARTICIPANTS 2686 people (2037 men
  and 649 women) aged 65-85 years living in the
  general community, recruited from the British
  doctors register and a general practice register
  in Suffolk.
• MAIN OUTCOME MEASURES Fracture incidence and
  total mortality by cause.
• RESULTS After five years 268 men and women had
  incident fractures, of whom 147 had fractures in
  common osteoporotic sites (hip, wrist or forearm,
  or vertebrae). Relative risks in the vitamin D
  group compared with the placebo group were 0.78
  (95 confidence interval 0.61 to 0.99, P0.04)
  for any first fracture and 0.67 (0.48 to 0.93,
  P0.02) for first hip, wrist or forearm, or
  vertebral fracture. 471 participants died. The
  relative risk for total mortality in the vitamin
  D group compared with the placebo group was 0.88
  (0.74 to 1.06, P0.18). Findings were consistent
  in men and women and in doctors and the general
  practice population.
• CONCLUSION Four monthly supplementation with 100
  000 IU oral vitamin D may prevent fractures
  without adverse effects in men and women living
  in the general community.

9
Vitamin D Summary of 5 Yr. Results
VIT D PBO RR (p) (n1345) (n1341) A
ny fracture 8.8 11.1 0.8 (.04) Hip, wrist,
vert. 4.5 6.5 0.67 (.02) Mortality 16.7 18.4
0.88 (.18)
CONCLUSION Four monthly supplementation with 100
000 IU oral vitamin D may prevent fractures
without adverse effects in men and women living
in the general community
BMJ. 2003 Mar 1 326(7387) 469.
10
Large and Simple Trials

• Efficiencies
• No selection criteria (other than age)
• No follow-up clinic visits or measurements
• No follow-up labs or measurements
• Outcomes from self-assessed mail-in
• Study very efficient (VERY) and very
  generalizable
• Cost of Vitamin D trial about 500,000.
• Cost of WHI about 1,000,000,000
• Often not feasible but should be considered

11
To be added in fture

• Could put in slide about later vitamin d studies
  lack of compliance.

12
Compliance or adherence

• Trial is meaningless unless participants adhere
  to interventions
• Think in very precise terms about meaning
• Term drop out often used but is ambiguous
• Several aspects
• 1. Adherence to medications/interventions
• 2. Adherence to visit schedules/reporting
  (missing main outcome)
• 3. Adherence to protocol (eg. Admitting only
  eligibles)
• Lack of adherence leads to
• Bias in either efficacy or safety
• Decreased power
• Uninterpretable results

13
Attempt to have complete information for primary
outcome

• Important to get primary outcome from all those
  randomized
• Techniques to assure complete follow up for
  primary outcome
• Have all participants come to final close out
  visit (even if missed other visits)
• Contact by telephone
• Use surrogate contact
• Use external data sources (National Death Index,
  Medicare, Kaiser, etc).

14
Missing data for primary endpoint (may be result
of loss to follow up or ?)

• Can lead to bias especially when LTFU varies by
  treatment
• Eg. Treated drop out more than placebo
• Randomization no longer valid
• Other forms of bias may exist in those lost
• Older, younger, sicker
• Even random loss to follow up could have impact
• Loss of power
• Other forms of bias

15
Potential effect of incomplete visit follow-up on
results in clinical trials

• - Fracture Intervention Trial (alendronate vs.
  placebo)
• - X-rays obtained at baseline, 2 years, 3 years
• - Vertebral fractures defined from changes in
  radiographs
• - FU radiographs on 97 of participants _at_ year 3
• Time (yrs) Relative risk (CI)
• BL to 2 0.34 (66 reduction)
• BL to 3 0.49 ( 51 reduction)

16
Effect of Incomplete Follow-up Virtual
Experiment

• FIT I Follow-up x-rays on 97 of surviving
  participants at year 3
• What if follow-up less complete?
• Randomly lose 50 between year 2 and 3

17
Use of Survival Analysis for X-Rays in FIT I
Virtual Experiment

• Time (yrs) Relative risk
• 2 0.34
• 3 0.49
• 3 (50 LTFU) 0.37
• LTFU Lost to follow-up

18
Effect of High Rate of Incomplete Follow-up on
Results

• If early results differ from later results, could
  create bias when comparing one study to another
• Even a random (therefore unbiased) loss to
  follow-up can affect results
• Clearly most losses to follow-up are not unbiased
  so more reasons for concern
• Loss of randomization
• Bias
• Less credibility of results

19
Potential impact of missing data for primary
outcome
Hollis and Campbell, BMJ 2004
20
Effect of High Rate of Loss to Follow-up on
Results

• More generally, people lost to follow-up may be
  different than those who remain
• Could be differential in two treatment groups
• Due to treatment (e.g. estrogen)
• Impact of loss to follow up
• Groups may no longer be comparable
• Loss advantage of randomization
• Could bias results

21
Measuring adherence

• Medication-taking
• Just ask! (self report)
• Pill counts
• Biochemical assays for some drugs (blinding
  issues)
• High tech pill bottles
• Can count number of times bottle actually opened
• Visit schedule
• N missed visits
• Visits within schedule
• etc.

22
Hi Tech approach to Medication Adherence
23
Adherence goals

• Ideal all participants continue to take
  medication (perfectly) throughout the trial and
  attend all follow-up visits until the very end
• Why might participants stop medication?
• Side effects (real or perceived)
• Complex regimens
• Want to take a true active medication
• New info on old medication
• New competing medication
• Want to stop active medication
• New info on old medication (e.g, ERT increases BC
  risk)

24
Some Examples of Bad Adherence Days

• Womens Health Initiative
• After first year results seen by DSMB, letter
  sent to all participants observed a small
  increase in cardiovascular disease among ppts on
  HRT
• Many stopped medications
• PROOF trial (effect of Nasal Calcitonin on
  osteoporosis)
• 1994 to 1999
• 1997 Alendronate approved with significant
  marketing and excellent results

25
Effect of Stopping Medication Classical
interpretation

• Placebos start active medicationgtbecome more
  like actives
• Actives stop active medication and start
  inactivegtbecome more like placebo
• Two groups become more similar
• Treatment effect is underestimated/conservative
• Comforting
• Classical interpretation may not hold
• Example patients stop study meds to take a
  medication that is better than active study
  medication

26
Strategies to enhance adherence Design

• Pre-randomization
• 2 or more screening visits
• Exclude those who will move or non-adherent
• Consider run-in period
• Trial run of drug/treatment
• Typically 2-4 weeks, usually of placebo (not
  always)
• Value controversial
• Intervention less frequent/more convenient
  dosing
• Study visits
• Close enough to maintain contact/not too close
• Study measurements
• Painless, interesting, useful
• Give patients results (when possible)

27
Strategies to enhance adherence implementation

• Educate participants
• Importance of science and their on-going
  participation
• Expectations for participation in study
• Warm and fuzzy stuff
• Participants to feel appreciated
• Staff in clinic spend enough time
• Sensitive to ppts. scheduling needs
• Ease of logistics/transportation to clinics
• Ex. Van for transportation encourage group
  solidarity
• Reimbursement for travel
• Parties/events with all participants

28
Strategies to enhance compliance

• Gifts (e.g. gift cards for moderate amounts)
• Information, Newsletters, other
• Emphasize early follow-up
• Most drop outs occur in early study period
• FIT (4 years total) 2/3 of drop outs occurred in
  first year, most of those in first 6 months

29
Adherence to Follow-up visits

• Goal visits all on time (within window)
• Set appointments flexibly
• Reminders prior to appt.
• Give study calendar
• Listen to concerns/problems

30
Study Management Methods to Enhance Adherence

• Monitor adherence during study
• Important to assess as go along
• Spot systematic or individual problems (and make
  corrections)
• Can manage via study website

31
Example of Visit Adherence Monitoring

• Horizon study
• IV dosing of zoledronic acid
• 1 dose per year
• Endpoint hip and vertebral fractures
• 7400 women in 23 countries, 269 sites
• Adherence to visit schedule monitored via IVRS
• Interactive Voice Response System (telephone
  punch 1, punch 2) system
• Reports on web used by local monitors

32
Adherence to medication is not the same as
adherence to visit schedule

• Drop out is very vague term
• Can have perfect visit adherence (come to all
  visits on time) but--
• Not take a single study med pill
• Take only 60 of pills
• If miss visits or stop coming to visits, then
  generally dont take study medication
• Exceptions do occur..could take study medications
  but not attend follow-up visits

33
Follow-up visits for those who have stopped study
medications?

• Practice varies dramatically across studies
• Option 1 Stop follow-up as soon as drug stops
• Option 2 Continue to collect follow-up info
• Advantages of each
• ??

34
Follow-up visits for those who have stopped study
medications?

• Practice varies dramatically across studies
• Option 1 Stop follow-up as soon as drug stops
• Option 2 Continue to collect follow-up info
  (highly recommended)

• Saves money
• Wont see long term negative effects

• Can do formal intention to treat analysis
• Can do all sorts of subsets and per protocol
  analyses
• Increase costs

35
Intention to Treat Analysis (ITT)

• ITT coined by AB Hill in textbook on Stat (1961)
• One of the main Commandments of RCT bible
• Original definition All subjects will be
  analyzed according to the treatment group they
  were originally intended by the randomization
  process
• generally interpreted as including all patients,
  regardless of whether they satisfied entry
  criteria, treatment was actually received or
  withdrawal or deviation from protocol-- Hollis
  BMJ article

36
True Meaning of ITT is Ambiguous

• Most rigorous interpretation includes the
  following principles (Hollis, discussion)
• All patients included in analysis even if they
  didnt start intervention
• Patients who were randomized but did not meet
  inclusion criterion should be included in ITT
• All patients included regardless of compliance
• Ascertainment for primary outcome on all
  randomized

37
Two Purposes of ITT (from Hollis)

• 1. Maintain validity of original randomization
• Groups can only differ by chance
• Exclusion of some subjects post-randomization
  (e.g. didnt take any pills) could create bias
• Medical/surgery example in Hollis (table 1)
• 2. Makes clinical trial more like real-world
  situation
• Clinicians in real world have been known to
  deviate from protocols
• Especially important for pragmatic trials (e.g.
  behavioral interventions)
• (Probably a bit naive to compare any trial to
  the real clinical world)

38
ITT is considered sacred but.
39
Beware of we did an ITT analysis

• Generally considered sacred, almost god-like
  virtue
• The term ITT used differently in different
  studies
• Probably does mean that all randomized (or most)
  were included in analysis
• Examples of differences in meanings
• In practice, ITT does NOT always mean that
  participants were followed beyond stopping study
  medications
• Very common to do Modified ITT (MITT) only
  include those who received at least one treatment
  (pill or ?)

40
How are ITT Principles Applied in Real Trials?

• Hollis (1999) evaluated application of ITT in 249
  trials in BMJ, JAMA, Lancet, NEJM
• About 50 (119 trials) reported using ITT
• Found that most trials with (ITT claimed)
  violated one or more of the 4 ITT principles

41
Per protocol analyses as alternatives to ITT

• If all ppts. are followed regardless of adherence
  to medications, several types of options
• Various flavors of per protocol
• Include only those patients who took all study
  medications and completed all visits
• Include all patients but only for the time that
  they remained on study medications
• If obtain complete follow-up on all ppts., can
  run several different types of analyses and any
  discrepancies could be informative.

42
Per protocol analyses as alternatives to ITT
Hollis and Campbell, BMJ 2004
43
Per protocol analyses as alternatives to ITT
Hollis and Campbell, BMJ 2004
44
Per protocol analyses as alternatives to ITT
Hollis and Campbell, BMJ 2004
45
Per protocol analyses as alternatives to ITT
Hollis and Campbell, BMJ 2004
46
Analysis based on post-randomization variables

• Per-protocol limits analysis to adherers
• Per-protocol is one example of analysis which
  stratifies based on post-randomization experience
• Other example stratify based on attained value
  of surrogate
• In Zoledronic acid trial, stratify fracture
  results based on attained increase in BMD at one
  year. VERY biased.
• More generally, subgroup analyses by post-rand.
  factors are biased, sometimes extremely
  biased--BEWARE

47
Can analysis be restricted to those who adhere?
Coronary Drug Project (CDP, NEJM 1980)

• 5 year mortality
• Overall Adherence
• gt 80 (2/3) lt 80 (1/3)
• Clofibrate (n1065) 18 15 25

48
Can analysis be restricted to those who adhere?
Coronary Drug Project (CDP, NEJM 1980)

• 5 year mortality
• Overall Adherence
• gt 80 (2/3) lt 80 (1/3)
• Clofibrate (n1065) 18 15 25
• Placebo (n2695) 19 15 28
• Lessons
• Unknown/unmeasured confounders associated with
  compliance
• Differ in placebo and active groups
• Be wary of post-randomization groupings

49
Completion of follow-up for all participants
regardless of adherence?

• Yes, very important
• All included, according to original randomization
• Follow-up completed on all ppts.
• Events are included even when they occurred after
  medications stopped
• Most rigorous approach
• Generally conservative estimate of treatment
  effect
• Per protocol or as-treated analyses are possible
  as secondary analyses

50
Problems with ITT/full follow-up approach

• ITT/full follow-up not holy grail
• Does not estimate full biologic efficacy of
  drug/intervention
• Advising individual patients may depend on
  efficacy
• Utility underestimated
• May be anti-conservative for adverse effects
• per-protocol may be preferred

51
CONSORT Guidelines for Reporting Results of Trials

• http//www.consort-statement.org/
• The CONSORT statement is an important research
  tool that takes an evidence-based approach to
  improve the quality of reports of randomized
  trials. The statement is available in several
  languages and has been endorsed by prominent
  medical journals such as The Lancet, Annals of
  Internal Medicine, and the Journal of the
  American Medical Association
• Includes guidelines for reporting trials and the
  Consort Flowchart

52
CONSORT checklist
53
Example of CONSORT Diagram Trial of Zoledronic
acid vs. placebo Black, NEJM, 5/07
54
Another example of CONSORT Diagram Gaziano, Vit.
C and E for Prostate CA JAMA, 1/09
55
Summary of ITT from Hollis in BMJ

• The Brits they dont know how to cook nor run a
  monarchy but.

• They sure do talk gooder than us Americans

56
ITT Key Messages (Hollis)

• ITT gives a pragmatic estimate of benefit of
  treatment policy rather benefit in patients who
  receive treatment exactly as planned
• Full application of ITT possible only when
  complete outcome available on all randomized
• Many trials that claim ITT varied in handling of
  missing data, deviations from protocol, etc.
• ITT often inadequately described and applied

57
Recommendations for ITT (Hollis)

• ITT best regarded as complete trial strategy
  (design, conduct, analysis) and not simply
  analysis
• Design
• Justify in advance any inclusion which if
  violated merit exclusion from ITT
• Conduct
• Minimise missing response on primary outcome
• Follow up subjects who withdraw from treatment
• Analysis
• Include all subjects according to randomization
• Investigate potential effects of missing response

58
Recommendations for ITT

• Reporting
• Specify how ITT used (explicitly describe
  handling of deviations from randomization and
  missing responses)
• Report deviations and missingness
• Discuss potential effects of missing reponse
• Base conclusions on ITT analyses

59
Follow-up and Adherence summary

• Best trial
• All participants remain on medication
• All participants are followed for primary outcome
  until end of study
• Pre-planned analysis and handling of deviations
  from protocol
• Design and implemention of study can help move
  toward that goal
• Important to report details about adherence and
  how it could affect results

60
(More) Data Analysis Issues in Clinical Trials

• Overview of simple data analysis for clinical
  trials
• Data analysis for non-standard designs
• Cross over
• Cluster randomization
• Factorial designs
• Special topics in data analysis in RCTs (FFD
  page 300-309) (today and next week)
• Subgroups (Wang, et al, assigned reading)
• Adjustment for baseline covariables
• Multiple endpoints
• Efficacy
• Safety (an interesting example)

61
Adjusted analysis in a randomized trial

• - What if important prognostic variables
  (confounders) are maldistributed by chance alone?
• eg. Trial of MI placebos older than treated
• Adjust for age?
• - Controversial issue
• If you adjust for enough variables, you will
  eventually change the results. High potential
  for hanky-panky.

62
Adjusted analysis in a randomized trial

• Potential solutions
• If a specific variable is highly prognostic, then
  use stratified blocking to guarantee balance
• Perform analysis unadjusted and then adjusted
• Pre-specify condition under which adjustment will
  be done for a small set of prognostic variables
• - eg. If age, BP or ldl are maldistributed
  (plt.05), then adjust for that/those variable(s)
  only.

63
Multiple efficacy endpoints

• Often many ways to slice the outcome pie
• Different endpoints or subgroups of endpoints
• SELECT all Ca, prostate Ca, prostate CA
  mortality
• Multiple comparisons problems
• Some solutions
• Very explicit predefinition of endpoints
• Limit number of endpoints
• Formal p-value adjustment
• FDA single endpoint only

64
Multiple Endpoints Making a Mountain Out of a
Molehill

• Multiple Outcomes of Raloxifene Evaluation (MORE)
  trial
• Main outcome vertebral fractures
• Secondary outcome non-vertebral fractures
• Main osteoporotic subtypes hip, wrist
• Overall, no effect of raloxifene on NV fractures
• Looked at 14 subtypes of fractures
• One significant ankle. Wanted to title paper
  Raloxifene reduces ankle fractures

65
WHI Invasive Breast Cancer
3
2
1
years 1 2 3
4 5 6
7
66
Drug Safety AssessmentA Multiple Comparisons
Nightmare

• Adverse experiences (anything bad that happens
  to a patient) are collected in regulatory trials
  as open text and then categorized
• Many categories (1000 or more)
• Most have very few events
• Some prespecified ones to be taken more seriously
• But what about surprises?
• Vioxx and heart disease
• How to control for spurious findings?
• P-values almost meaningless

67
HORIZON trial NEJM 5/07

• Trial of Zoledronic acid (a bisphosphonate) used
  IV annually vs. placebo
• About 8000 patients worldwide
• Safety evaluations from previous studies of same
  drug suggest possible renal effects,
  osteonecrosis of jaw, oversuppression of bone
  remodeling
• Focused safety evaluations built into trial
• Also looked at preferred terms for individual
  medical conditions (perhaps 1000 such terms)

68
Atrial fibrillation in published trials of
bisphosphonatesSerious adverse events (SAEs)

• TX PBO
• Trials (Tx) N () N () RR (95 CI) P-value
• HORIZON-PFT3 (Zol) 50 (1.3) 20 (0.5) 2.3
  (1.4-3.9) lt 0.001 (5/07)
• FIT (Alendronate)1 47 (1.5) 31 (1.0) 1.5
  (0.97-2.4) 0.07 (5/07)
• Other Alendronate2 8 (0.2) 7 (0.2)
• HORIZON-RFT4 (Zol) 30 (2.9) 28 (2.7) 1.1
  (0.6-1.8)
• Risedronate5 (5 mg) 29 (0.6) 24 (0.5) 0.49
• (2.5 mg) 24 (0.5) 24 (0.5)

1. Cummings et al letter. N Engl J Med.
20073561895-6 2. Statement by Merck, May 3,
2007 3. Black et al. N Engl J Med.
20073561809-22 4. Lyles, et. Al. NEJM 9/07 5.
Karam et al letter. N Engl J Med. In press
69
Atrial fibrillation in published trials of
bisphosphonatesFDA review (11/15/08)
70
Atrial fibrillation in published trials of
bisphosphonatesFDA review (11/15/08)
71
Slicing and Dicing a Continuous Outcome Variable

• A continuous variable can be analyzed as a
  comparison of two means (generally preferred)
• Or as dichotomized value
• Diastolic Blood Pressure
• Could compare proportions gt 90 mm Hg, gt 100 mm
  Hg
• Could look at variety of dichotomization points
• Nice example on page 309 of FFD
• Specify any potential dichotomizations apriori