Chapter 10 Data Monitoring, Monitoring Committee Function

1
Chapter 10Data Monitoring, Monitoring Committee
Function Statistical Methods
2
Some References

• Texts/Chapters
• 1. Friedman, Furberg DeMets (1998) 3rd
  edition, Fundamentals of Clinical Trials,
  Springer-Verlag, NY, NY
• 2. Pocock (1983) Clinical Trials, Wiley.
• 3. Ellenberg S, Fleming T and DeMets D Data
  Monitoring Committees in Clinical Trials A
  Practical Perspective. John Wiley Sons, Ltd.,
  West Sussex, England, 2002.
• 4. Jennison C and Turnbull B (2000) Group
  Sequential Methods with Application to Cinical
  Trials. Chapman Hall, NY.
• 5. DeMets DL (1998) Data and Safety Monitoring
  Boards. In Encyclopedia of Biostatistics.
  John Wiley and Sons, West Sussex, England, Vol.
  2, pp. 1067-71.
• 6. DeMets and Lan. The alpha spending function
  approach to interim data analysis. In, Recent
  Advances in Clinical Trials Design and
  Analysis. Kluwer Academic Publishers, Boston,
  MA, 1995.

3
Some References

• Review Papers
• 1. Greenberg ReportOrganization, review, and
  administration of cooperative studies.
  Controlled Clinical Trials 9137-148, 1988.
• 2. DeMets and Lan (1994) Interim analyses
  The alpha spending function approach.
  Statistics in Medicine, 13(13/14)1341-52, 1994.
• 3. Lan and Wittes. The B-value A tool for
  monitoring data. Biometrics 44579-585, 1988.
• 4. Task Force of the Working Group on
  Arrhythmias of the European Society
  of Cardiology The early termination of clinical
  trials causes, consequences, and control.
  Circulation 89(6)2892-2907, 1994.
• 5. Fleming and DeMets Monitoring of clinical
  trials issues and recommendations. Controlled
  Clin Trials 14183-97, 1993.
• 6. DeMets, Ellenberg, Fleming, Childress, et al
  The Data and Safety Monitoring Board and AIDS
  clinical trials. Controlled Clin Trials
  16408-21, 1995.
• 7. Armstrong and Furberg Clinical trial data
  and safety monitoring boards The search for a
  constitution. Circulation 1, Sess6, 1994.

4
Data MonitoringRationale

• 1. Ethical
• 2. Scientific
• 3. Economic

5
A Brief History

• A 40-year history
• Greenberg Report (1967)
• Coronary Drug Project (1968)
• NIH Experience and Guidelines
• Industry and ICH Guidelines
• Department of Health Human Services Policy
  (Shalala, 2000)

6
Greenberg Report Recommendations

• Develop a mechanism to terminate early if
• Question already answered
• Trial cant achieve its goals
• Unusual circumstances
• Hypothesis no longer relevant
• Sponsor decision to terminate should be based on
  advice of external committee

7
Coronary Drug Project (CDP)

• References
• Design (Circulation, 1973)
• Monitoring Experience (CCT, 1981)
• Major Outcome (JAMA 1970, 1972, 1973, 1975)
• Tested several lipid lowering drugs in post MI
  patients
• Multicenter study
• Mortality as primary outcome
• Began recruitment in 1965

8
Coronary Drug Project

• First trial to benefit from Greenberg Report
• Policy Advisory Board
• Senior Investigators, External Experts, NIH
• Initially reviewed interim data
• Data Coordinating and Statistical Center
• Safety Monitoring Committee formed (1968), after
  trial was underway

9
Early NHLBI CT Model
Funding Agency

Policy Advisory Board
Data and Safety Monitoring Board
Steering Committee
Central Lab(s)
Multiple Clinics
Working Committees
Data Coordinating Center Data Management Statistic
al Analysis
10
NHLBI CT Model

Funding Agency
Data Monitoring Committee
Steering Committee
Central Lab(s)
Coordinating Data Center
Clinics
Working Committees
11
NIH DMC Activity

• Ref Statistics in Medicine (1993)
• CDP (Coronary Drug Project) became model for
  National Heart, Lung, and Blood Institute (NHLBI)
• heart, lung, blood disease trials
• National Eye Institute (NEI) (1972)
• Diabetic Retinopathy Study
• National Institute Diabetes, Digestive and Kidney
  (NIDDK)
• Diabetes Complication and Control Trial (1980)
• National Cancer Institute (NCI)
• Prevention Trials, Cooperative Group Therapeutic
  Trials
• National Institute Allergy and Infectious Disease
  (NIAID)
• AIDS Clinical Trial Group (ACTG) (1986)

12
Industry/FDA/ICH

• Industry sponsorship of RCTs expanded
  dramatically since 1990 in several disease areas
  (e.g. cardiology, cancer, AIDS)
• Industry use of DMCs growing as well
• FDA 1989 guidelines very brief mention of data
  monitoring and DMCs
• International Conference on Harmonization (ICH)
• ICH/E9
• Section 4.5 Interim Analyses
• Section 4.6 Independent DMCs
• ICH/E6

13
Independent DMCsWhen are they Needed?

• Department of Health and Human Services Policy
• Shalala (NEJM, 2000) All NIH FDA trials must
  have a monitoring plan, for some a DMC may be
  required
• NIH policy (1998)
• all sponsored trials must have a monitoring
  system
• safety, efficacy and validity
• DMC for Phase III trials
• FDA guidelines (Nov 2001)

14
Need for Independent DMCs

• Phase I Trials (dose)
• Monitoring usually at local level
• Phase II Trials (activity)
• Most monitoring at local level
• Some randomized, blinded, multicenter Phase II
  trials may need IDMC
• Phase III IV (effectiveness, risk, benefit)
• Most frequent user of IDMC
• Structure of monitoring depends on risk (e.g.
  Phase I-IV)

15
Data Monitoring Committee

• FDA suggests a need for an
• Independent DSMB for
• Pivotal Phase IIIs
• Mortality or irreversible
• morbidity outcome

16
Industry-Modified NIH Model
Pharmaceutical Industry Sponsor
Steering Committee
Regulatory Agencies

Independent Data Monitoring Committee (IDMC)
Central Units (Labs, )
Data Management Center (Sponsor or CRO)
Statistical Analysis Center
Clinical Centers
Institutional Review Board
Patients
17
DMC Relationshipsand Responsibilities

• Patients
• Study Investigators
• Sponsor
• Local IRBs
• Regulatory Agencies

18
Early Administrative AnalysisDMC and Executive
Committee

• 1. Recruitment/Entry Criteria
• 2. Baseline Comparisons
• 3. Design Assumptions
• a. Control only
• b. Combined groups

19
Design Modifications

• 1. Entry Criteria
• 2. Treatment Dose
• 3. Sample Size Adjustment
• 4. Frequency of Measurements

20
DMC Data ReviewInterim Analysis

• 1. Recruitment
• 2. Baseline Variables
• -Eligibility
• -Comparability
• 3. Outcome Measures
• -Primary
• -Secondary
• 4. Toxicity/Adverse Effects
• 5. Compliance
• 6. Specified Subgroups

21
DMC Recommendations

• 1. Continue Trial / Protocol Unmodified
• 2. Modify Protocol
• 3. Terminate Trial

22
Reasons for Early Termination

• 1. Serious toxicity
• 2. Established benefit
• 3. Futility or no trend of interest
• 4. Design, logistical issues too serious to fix

23
DMC Decision Making Process Complex (1)

• Recruitment Goals
• Baseline risk and comparability
• Compliance
• Primary and secondary outcomes
• Safety

24
DMC Decision Making Process Complex (2)

• Internal consistency
• External consistency
• Benefit/Risk
• Current vs future patients
• Clinical/Public impact
• Statistical issues

25
DMC Decision Making Role

• DMC makes recommendations, not final decisions
• Independent review provides basis for DMC
  recommendations
• DMC makes recommendations to
• Executive Committee who recommends to sponsor,
  or
• Sponsor
• DMC may, if requested, debrief Executive
  Committee and/or sponsor
• Rarely are DMC recommendations rejected

26
DMC Meeting Format

• Open Session
• Progress, blinded data
• Sponsor, Executive Committee, DMC, SAC
• Closed Session
• Unblinded data
• DMC, SAC
• Sponsor Rep? (Not recommended)
• Executive Session
• DMC only
• Debriefing Session
• DMC Chair, Sponsor Rep, Executive Committee Rep

27
DMC Relationships

• Regulatory Agencies (e.g. FDA)
• Could perhaps brief DMC about specific concerns
  at Open Session
• Should not participate in DMC Closed Sessions
• Should be briefed about DMC recommendations/decisi
  ons ASAP following Executive Committee

28
DMC Membership

• Monitoring is complex decision process and
  requires a variety of expertise
• Needed expertise
• Clinical
• Basic science
• Clinical trial methodology
• Biostatistics
• Epidemiology
• Medical ethics
• Helpful expertise
• Regulatory
• Some experience essential

29
DMC Confidentiality

• In general, interim data must remain confidential
• DMC may rarely release specific/limited interim
  data (e.g. safety issue)
• Members must not share interim data with anyone
  outside DMC
• Leaks can affect
• Patient Recruitment
• Protocol Compliance
• Outcome Assessment
• Trial Support

30
DMC Liability

• Recent events (eg Cox-IIs, Vioxx) have raised the
  potential for litigation (??)(Vioxx or COX-IIs
  (painkillers) can raises the risk of heart
  attack, stroke and death and were withdrawn from
  the market)
• Members have been gotten a subpoena (?? )
• DMC Charters (??) for industry trials now often
  cover indemnification clauses (????)
• No indemnification yet for NIH trials

31
DMC Needs On-LineData Management and Analysis

• DMC reluctant to make decisions on old data
• Minimize data delay and event verification
• Be prepared from start
• Focus on key variables, not complete case reports
  (delays can be problematic)

32
Levels of Independence

• Totally Inhouse Coordinating Center
• Internal DM, Internal SAC, External DMC
• Internal DM, External SAC, External DMC
• External DM(e.g. CRO), External SAC, External DMC

33
DMC Summary

• NIH Clinical Trial Model - long history of
  success
• Adaptation for industry can be made
• SC, DMC, SAC or DM are critical components
• Independence of DMC essential
• Best way to achieve this goal is for external SAC
  and external DMC

34
Data Monitoring Process

• 1. DMC and the decision process
• 2. A brief introduction to statistical
  monitoring methods
• a. Group Sequential
• b. Stochastic Curtailment
• 3. Examples
• Ref BHAT, DeMets et al. Controlled
  Clin Trials,1984

35
Decision Factors

• 1. Comparability
• 2. Bias
• 3. Compliance
• 4. Main effect vs. Potential side effects
• 5. Internal Consistency
• a. Outcome measures
• b. Subgroups
• c. Centers
• 6. External Consistency
• 7. Impact
• 8. Statistical Issues/Repeated Testing

36
Beta-blocker Heart Attack Trial (BHAT)

• Preliminary Report. JAMA 2462073-2074, 1981
• Final Report. JAMA 2471707-1714, 1982
• Design Features
• Mortality Outcome 3,837 patients
• Randomized Men and women
• Double-blind 30-69 years of age
• Placebo-controlled 5-21 days post-M.I.
• Extended follow-up Propranolol-180 or 240 mg/day

37
BHATAccumulating Survival Data

• Date Data Monitoring
• Committee Meeting Propranolol Placebo Z(log
  rank)
• May 1979 22/860 34/848 1.68
• Oct 1979 29/1080 48/1080 2.24
• March 1980 50/1490 76/1486 2.37
• Oct 1980 74/1846 103/1841 2.30
• April 1981 106/1916 141/1921 2.34
• Oct 1981 135/1916 183/1921 2.82
• June 1982
• Data Monitoring Committee recommended
  termination

38
Beta-Blocker Heart Attack Trial October 1,
1981LIFE-TABLE CUMULATIVE MORALITY CURVES
39
Beta-Blocker Heart Attack TrialBaseline
Comparisons

• Propranolol Placebo
• (N1,916) (N1,921)
• Average Age (yrs.) 55.2 55.4
• Male () 83.8 85.2
• White () 89.3 88.4
• Systolic B.P. 112.3 111.7
• Diastolic B.P. 72.6 72.3
• Heart rate 76.2 75.7
• Cholesterol 212.7 213.6
• Current smoker () 57.3 56.8

40
Beta-Blocker Heart Attack TrialTotal
Mortality(Average 24-Month Follow-Up)

• Propranolol Placebo
• Age 30-59 5.9 7.1
• 60-69 9.6 14.4
• Sex Male 7.0 9.3
• Female 7.1 10.9
• Race White 6.7 9.0
• Black 11.0 15.2

41
Beta-Blocker Heart Attack TrialTotal
Mortality(Average 24-Month Follow-Up)

• Propranolol Placebo
• Risk Group I 13.5 16.9
• Risk Group II 7.8 11.4
• Risk Group III 5.2 7.1

42
DMC Interim Analysis

• Ethical, scientific and financial reasons
• Repeated analysis of accumulating data causes a
  statistical problem

43
Data Monitoring
44
Classical Sequential Analysis

• Observations are taken sequentially
• After each observation
• Decide whether to stop sampling (one group is
  significantly better, or worse, than the other)
• Or take another observation
• Originally developed by Wald (1947)
• Applied to the clinical trial by Armitage (1975)

45
Why Sequential Analysis? (Armitage, 1975)

• Data reduction
• Estimation with desired precision
• Medical ethics

46
Repeated Significance Tests

• Assume X 1 , X 2 , ? N(?, 1)
• Let S n X 1 ? X n
• N is the maximum sample size
• Testing H 0 ? 0 vs H A ? ? 0
• Nominal significance level is 0.05

47
Repeated Significance Tests

• For each n ? N , we assess if Sn ? 1.96? n
• Stop sampling and reject H0 at the first n ? N ,
  if any, such that Sn ? 1.96? n
• Otherwise, stop sampling at N and do not reject
  H0

48
Probability of Type I Error

• ?N P Sn ? 1.96? n for some n ? N H0
• By the law of the iterated logarithm, eventually
  reject H0 when in fact it is true
• ?N might be large for some N

49
The Type I Error Probability when the Maximum
Number of Observations is N
N aN
1 0.050
2 0.083
3 0.107
4 0.126
5 0.142
10 0.193
15 0.225
20 0.248
25 0.266
30 0.280
50
The Required Critical Values and Nominal Level
Giving a Type I Error Probability 0.05 for
Various Values of N
N Critical Value Nominal Level
1 1.96 0.050
5 2.42 0.015
10 2.56 0.010
15 2.64 0.008
20 2.68 0.007
50 2.80 0.005
100 2.88 0.004
200 2.96 0.003
51
Group Sequential Procedures

• Repeated significance tests after every
  observation are not easy to conduct
• Apply the significance test at longer intervals
• Compute summary statistic at each interim
  analyses, based on additional group of new
  subjects (events)
• Compare statistic to a conservative critical
  value such that a0.05 overall

52
Group Sequential Procedures

• Boundaries
• Haybittle-Peto (1971,1976)
• Pocock (1977)
• OBrien-Fleming (1979)
• Lan-DeMets (1983)
• Slud-Wei (1982)

53
(No Transcript)
54
Group Sequential Boundaries
55
Pocock's boundary
N ? 0.05 ? 0.01
1 1.96 2.58
2 2.18 2.77
3 2.29 2.87
4 2.36 2.94
5 2.41 2.99
56
Lan-DeMets Procedure

• Criticism of classical group
• Sequential procedure
• Number of interim analyses must be specified in
  advance
• Equal increments

57
Lan-DeMets Procedure

• Specify a(t) spending function
• a(t) defines rate at which Type I error is
  spent where t is the proportion of information
  accumulated by calendar time tc
• 0 ? t ? 1
• a(t) increasing,
• a(0) 0
• a(1) a

58
Lan DeMets Procedure

• The function ? is arbitrary
• Examples
• where z?/2 is denoted such that ?(z?/2) 1-
  ,
• and ?2(t) ? log1 (e - 1)t

59
Information and Calendar Time
t proportion of information accumulated by tc
Example Immediate Response X1,X2,...,Xn,...,XN Y
1,Y2,...,Yn,...,YN tc t 2n / (2N) n /
N
60
Information and Calendar Time
Example Failure time (e.g., logrank)
61
Lan DeMets Procedure

• Assume X1 , X2 , . . . N(? , 1)
• Testing H0 ? ? 0 vs H1 ? gt 0
• Let Zi be the accumulated test statistic at
  calander time i at which the information time is
  ti .
• Find boundary values Ci such that
• P ( Z1 ? C1 ) ?( t1 ),
• P ( Z1 lt C1 , Z2 ? C2 ) ?( t2 ) - ?( t1 ), .
  . . .

62
Boundary Crossing Probability
E.g., K 5, ? 0.025 Upper
Boundary C1 C2 C3 C4 C5 Pocock (2.41,
2.41, 2.41, 2.41, 2.41) OBF (4.56, 3.23, 2.63,
2.28, 2.04) Pocock OBF 1. P Z1 gt C1
0.0079 (0.000) 2. P Z1 gt C1 or Z2 gt
C2 0.0079 0.0059 0.0138 (0.0006)
3. P Z1 gt C1 or Z2 gt C2 or Z3 gt C3
0.0138 0.0045 0.0183 (0.0045) 4. P
Z1 gt C1, ..., Z4 gt C4 0.0183 0.0036
0.0219 (0.0128) 5. P Z1 gt C1, ..., Z5 gt C5
0.0219 0.0031 0.0250 (0.0250)
63
(No Transcript)
64
(t2) - ? (t1)
65
Examples of ?(t)

• Approximates
• 1. OBF
• 2. ?2 (t) ? ln 1 (e - 1)t Pocock
• 3. ?3 (t) ?t
• Comparison of Boundaries (? .025, N 5)
• Values C1 C2 C3 C4 C5
• 1. OBF 4.56 3.23 2.63 2.28 2.04
• ?1(t) 4.90 3.35 2.68 2.29 2.03
• 2. Pocock 2.41 2.41 2.41 2.41 2.41
• ?2(t) 2.44 2.43 2.41 2.40 2.38
• 3. ?3(t) 2.58 2.49 2.41 2.34 2.28

66
BHAT GSB
67
Cardiac Arrhythmia Suppression Trial (CAST)

• Ref NEJM 321(6)406-12, 1989
• Cardiac arrhythmias associated with increased
  risk of sudden death
• New class of drugs (eg, encainide, flecanide)
  suppressed arrhythmias
• CAST designed to test effect on sudden death

68
CAST GSB

• ? spending function approach
• ?(t) ½ ? t t lt 1
• ? t 1
• for benefit ? 0.025
• Used symmetric ? 0.025 boundary for harm

69
CAST Interim DataSudden Death
Time Placebo Drug LogRank ZL
ZU 9/01/88 5/576 22/571 -2.82 -3.18 3.01 3/30/89 9
/725 33/730 -3.22 -3.04 2.71 Initially expected
100 events/arm
70
CAST Sequential Boundaries
71
Stochastic Curtailed Sampling

• During study, whether the current trend in the
  data can lead to the acceptance or rejection of
  H0 ?
• Group sequential methods focus on existing data
• Curtailed sampling in addition considers the data
  which have not yet been observed
• Lan, Simon and Halperin (1982)

72
Example

• H0 ? 0.5 (Prob(Heads)) vs. HA ? ? 0.5
• Flip coin 400 times
• Stotal number of heads
• Reject H0 if z ? 1.96, where
• or when S - 200 ? 20
• After 350 coin flips and 220 heads, we know
  for sure we will reject H0 .

73
Stochastic Curtailing

• Let Z(T)statistic at end of trial
• Z(t)current value at time t
• Rrejection region
• P Z(T) ? R H0 ?
• P Z(T) ? HA ?
• or P Z(T) ? R HA 1 - ?

74
Stochastic Curtailing

• Lan, Simon, Halperin (1982)
• reject when P Z(T) ? R H0 , Z(t) ?0
• shows very positive trend
• accept when P Z(T) ? HA , Z(t) ?A
• shows negative trend
• P Type I error ? ? / ?0
• P Type II error ? ? / ?A

75
Example

• Population 1 X N(mx , s2 )
• Population 2 Y N(my , s2 )
• H0 mx my
• HA mx gt my
• For design
• HA mx - my 0.1, s2 1, ? 0.05, 1- ? 0.8
  
• Need 1250 subjects per group
• Reject if Z ? 1.645

76
Example (continued)

• During Study

No Z
I 250 0.113 1.26
II 500 0.125 1.98
III 750 0.122 2.26
IV 1000 0.12 2.68
V 1250 ? ?
77
Example (continued)

• Conditional Probability
• D 0.12 P ? 0.999
• D 0.03 P ? 0.98
• D 0.00 P ? 0.95

78
B-ValueA Method for Computing Conditional
PowerLan Wittes (1988) Biometrics

• Let t n/N (or d/D)
• Z(t) current standardized statistic
• Now Z(1) B(1) and
• ( observed remaining)

79
Visual Aid
H0 ? 0 HA e.g. ?
B(1)
B(t)
80
Conditional Power

• PZ(1) ? Z? Z(t), ?)

81
Conditional Power
1. Survival D total events 2. Binomial N
total sample size
82
Conditional Power (2)
3. Means N total sample size
83
Example BHAT

• Expected Deaths D 398
• Observed Deaths 183 Placebo d 318
• 135 Propranolol D d 80 398
• Observed logrank Zd 2.82 t 318/398 .80
• Compute Conditional Power under H0
• 1 - ? - 1.25
• 0.89