II-1

1
Part II Basic Statistical Concepts in Randomized
Clinical Trials

• Sylvan B. Green, M.D.
• Arizona Cancer Center
• University of Arizona
• Tucson AZ
• SCT Pre-Conference Workshop
• Fundamentals of Clinical Trials

2
Outline

• ? Rationale for randomized trials
• ? Blinding
• ? Intention to treat
• ? Significance testing Confidence intervals
• ? Sample Size

3
Randomized Trials

• Important
• in evaluating interventions for the prevention,
  diagnosis, and treatment of disease
• Ethical
• in the presence of uncertainty
• Robust
• large trials recommended to increase reliability
• Applicable to studies of efficacy and of
  effectiveness
• Can answer more than one question at a
  time (factorial trials and other designs)
• In some situations, can randomize entire
  groups (e.g., communities, medical practices)

4
Statistical Issues

• In designing any clinical study, we have to keep
  in mind two issues related to participant
  (patient) heterogeneity
• the effect of chance
• the effect of bias (whether conscious or
  unconscious)
• These are addressed by
• having adequate numbers of participants in the
  study
• using randomization for intervention (treatment)
  assignment

5
Purposes of Comparative Randomized Trials

• To find out which (if any) of two or more
  interventions is more effective.
• To convince others of the results.

6
Observational (non-randomized) Studies

• Are they useful?
• Epidemiologic investigations (etiology)
• Drug development Phase I and II trials
• Medical databases
• may provide information on patterns of care,
  cost, and both clinician patient preferences
• analyses of such data may generate important
  hypotheses to be tested in future trials
• Should they be recommended for comparing
  alternative interventions?

7
Problems with Non-randomized Controls

• Effect of unmeasured or unknown prognostic
  factors
• Differential patient selection due to
  requirements for consent
• Bias in treatment assignment(unlike
  epidemiologic research concerning cause)
• Defining "time zero"
• Possible time trends in
• patient population disease characteristics
• diagnostic methods supportive care

8
(No Transcript)
9
Simpsons Paradox
10
Why Randomize?

• Bias (conscious or unconscious) is avoided
• Predictive factors (known and unknown) tend to be
  balanced between intervention comparison groups
• Randomization provides a valid basis for
  statistical tests of significance
• Having a concurrent comparison group controls for
  time trends
• Results are more likely to be convincing

11
Beta Carotene and Cancer - 1

• Alpha-Tocopherol Beta-Carotene Cancer Prevention
  Study
• Ref The ATBC Cancer Prevention Study Group. N
  Engl J Med 1994 330 1029-1035
• METHODS. Randomized, double-blind,
  placebo-controlled primary prevention trial
  29,133 male smokers from southwestern Finland.
• RESULTS. Unexpectedly, a higher incidence of lung
  cancer among the men who received beta carotene
  (change in incidence, 18 percent 95
  confidence interval, 3 to 36 percent).
• CONCLUSIONS. No reduction in the incidence of
  lung cancer among male smokers after 5-8 years of
  alpha-tocopherol or beta carotene. In fact, this
  trial raises the possibility that these
  supplements may actually have harmful as well as
  beneficial effects.

12
Beta Carotene and Cancer - 2

• Beta Carotene and Retinol Efficacy Trial
• Ref Omenn GS, Goodman GE, Thornquist MD, et
  al. N Engl J Med 1996 334 1150-1155
• METHODS. Multicenter, randomized, double-blind,
  placebo-controlled primary prevention
  trial18,314 smokers, former smokers, and
  workers exposed to asbestos.
• RESULTS. Compared with the placebo group, the
  treatment group had relative risk of lung cancer
  1.28 (95 confidence interval, 1.04 to 1.57
  P0.02) and relative risk of death from lung
  cancer 1.46 (95 confidence interval, 1.07 to
  2.00)
• The trial was stopped 21 months earlier than
  planned.
• CONCLUSIONS. Beta carotene plus vitamin A (4 yrs
  average) had no benefit and may have had an
  adverse effect in smokers and workers exposed to
  asbestos.

13
Beta Carotene and Cancer - 3

• Physicians' Health Study
• Ref Hennekens CH, Buring JE, Manson JE, et al.
  N Engl J Med 1996 334 1145-1149
• METHODS. Randomized, double-blind,
  placebo-controlled trial 22,071 male physicians.
• CONCLUSIONS. In this trial among healthy men, 12
  years of supplementation with beta carotene
  produced neither benefit nor harm in terms of the
  incidence of malignant neoplasms, cardiovascular
  disease, or death from all causes.

14
Large Simple Trials

• More use of randomized trials is needed to
  address areas of uncertainty in medicine
• Given patient heterogeneity and the play of
  chance, large numbers of patients are needed to
  provide reliable estimates of the effect of
  treatment
• Realistic effects are relatively modest in size
  (but still potentially of great public health
  importance)
• Simplicity of trials permits larger numbers of
  patients with lesser expenditure of resources
• simplified eligibility criteria
• focus data collection on important endpoints

15

• "There is simply no serious scientific
  alternative to the generation of large-scale
  randomized evidence. If trials can be vastly
  simplified, as has already been achieved in a few
  major diseases, and thereby made vastly larger,
  then they have a central role to play in the
  development of rational criteria for the planning
  of health care throughout the world."
• Peto R, Collins R, Gray R.
• J Clin Epidemiol 1995 48 23-40

16
Randomized Trials as a Desirable Option

• Important to obtain unbiased comparisons of
  interventions
• Large trials (adequate sample size) for reliable
  inferences
• Randomized trials can present to participants the
  best choice for state-of-the-art
  intervention(consider current uncertainty about
  efficacy toxicity)
• Increased knowledge of trials can benefit
  participants and science

17
Comparative Randomized Trials Nature of Control

• Depends on the purpose of the trial. For
  example
• Investigate a new experimental intervention
  versus nothing (or versus placebo)
• Compare one drug or regimen with another
• a new experimental intervention versus a
  "standard" intervention
• compare two alternative commonly-used
  interventions with each other
• compare one modality versus another
• Study the result of adding an additional agent to
  a standard regimen
• Compare different doses or intensities of an
  intervention
• Investigate the effect of adjuvant therapy versus
  treating only at recurrence

18
Randomized Trials Nature of Intervention

• a drug (or drug regimen)
• a surgical procedure
• a medical device
• a therapeutic modality (radiation, biologic
  therapy, etc)
• a micronutrient
• a diet
• a behavioral intervention (education)
• a clinical approach to diagnosis, symptom
  management, palliative care, etc.
• The common denominator there is a choice between
  two alternative approaches uncertain which is
  preferable
• This uncertainty involves the balance of
  potential beneficial as well as possible adverse
  outcomes

19
Design of Intervention Studies

• Objectives
• Outcome measure - often recommended single
  primary outcome measure, and limited number of
  secondary outcome measures
• quantity on continuous scale
• dichotomous (binary) outcome
• non-response versus response
• recurrence versus no-recurrence
• incident case of disease versus disease-free
• dead versus alive (in a specified time after
  randomization)
• failure versus success
• time-to-event outcome
• time to failure
• disease-free survival
• overall survival

20
Masking (Blinding)

• Classical double--blind study neither the
  participants nor the study personnel know the
  result of the randomized assignment
• placebo, double dummy, masked vials
• blinding may not be possible
• surgical versus medical intervention
• one intervention has obvious side-effect
• Outcome assessed by masked observer
• Masked review and evaluation of outcome by
  central expert panel
• Likewise with initial eligibility

21
Avoiding Bias in Assessment

• Interesting example Prostate Cancer Prevention
  Trial
• Randomized trial of finasteride versus
  placeboOutcome biopsy-proven prostate cancer
• Question how to use PSA monitoring while
  avoiding differential biopsy rates between
  treatment groups resulting from an effect of the
  intervention on PSA
• Suggestion use blinded PSA determinations
  reported simply as "elevated" or "not-elevated,"
  categorized using group-specific threshold values
• Reference Feigl P, Blumenstein B, Thompson I, et
  al. Design of the Prostate Cancer Prevention
  Trial (PCPT). Control Clin Trials 1995 16150-63

22
Randomized Trials "Intent-to-Treat" Analyses

• Include all individuals randomized, counted in
  the group to which they were randomized
  (regardless of what occurs subsequently)
• The first analysis of any randomized trialthe
  analysis supported by the randomizationmaintains
  comparability (in expectation)
• Provides a test of the "policy" ("strategy",
  "intention") embarked upon at the time of
  randomization
• Note Plan adequate sample size to account for
  non-compliance

23
"Intent-to-Treat" Analyses-2

• Obtain realistic estimate of the intervention
  effect
• Issue What is the goal?
• "biologic efficacy" full compliance
• versus
• "pragmatic efficacy" intent-to-treat
• Notes
• "biologic efficacy" may be unattainable (in
  presence of intolerance or toxicity) danger of
  false optimism
• estimation of "biologic efficacy" may not be
  straightforward danger of bias

24
"Missing" Patients

• 1. Exclusions (never randomized)
• gt No bias in randomized comparison
• gt Do influence interpretation and
  generalizations
• 2. Withdrawals (deliberately omitted from
  analysis)
• gt Severe bias may arise
• gt Withdrawals may be acceptable if based on
  eligibility criteria determined at baseline and
  not affected by events subsequent to
  randomization
• 3. Losses to follow-up (missing outcome data)
• gt Bias may arise if the loss is related to the
  intervention and the outcome

25
Notes on Missing Patients

• Treatment dropouts do not necessarily have
  missing outcome data
• we should design trials ( informed consent
  processes) so that treatment modifications
  and/or dropout do not lead to off-study
• such patients should still be followed for
  outcome
• Patients who need (or want) to modify their
  therapy may be prognostically different from
  those who are maintained on the therapy initially
  assigned (and this may vary by treatment group)

26
Incomplete compliance / Treatment dropouts

• Severe bias may arise if deliberately omitted
  from analysis
• comparing compliers in both groups may be
  biasedas treated analysis may even be worse
• gt lose the comparability provided by
  randomization

27
"Intent-to-Treat" Analyses - Caveat

• In equivalence trials, excessive noncompliance
  may lead to apparent equivalence which does not
  reflect reality- here, intent-to-treat analysis
  does not have the usual advantage of
  "conservatism"

28
Statistical Significance

• Deciding (from the observed data) that two
  intervention strategies differed, when in fact
  they were equivalent, false-positive result
• Statistical tests quantify the probability of
  such false-positive results
• e.g., an observed difference with p 0.01 the
  probability of obtaining a difference this
  extreme (or more so) by chance alone is less
  than or equal to 1
• When evaluating evidence, it is informative to
• present the specific p-value
• calculate a confidence interval
• quantifies uncertainty of estimated intervention
  effect
• indicates range of values within which we think
  the true intervention effect lies

29
USA TODAY - April 17 2000
30
Decisions in Hypothesis Testing
31
Decisions in Hypothesis Testing
32
Decisions in Hypothesis Testing
33
Analogy

• Jury Trial (criminal law)

• Clinical Trial (statistical testing)

Acknowledgement to Susan Hilsenbeck
34
Hypothesis Testing

• Using randomization to assign interventions
  provides a valid basis for statistical tests of
  significance
• the process of randomization makes it
  possible to ascribe a probability distribution to
  the difference in outcome between groups under
  the null hypothesis
• For hypothesis testing, we want the probability
  that an estimate of the mean difference would be
  as large or larger than the observed estimate, by
  chance alone
• This can be implemented directly with
  randomization-based inference

35
Randomization Tests (Permutation Tests)

• We calculate the outcome for each of the ways
  (permutations) that the intervention assignments
  could have occurred during the randomization
• Paired data (N pairs) 2N
• Unpaired data (N per group) 2NCN
• The rank of the observed outcome among all
  possible outcomes provides the one-tailed
  significance level
• (e.g., if it is in the top 1, then it is
  significant at the 0.01 level)
• This is based on the randomization distribution
  (avoids other statistical assumptions)

36
Design of Intervention Studies

• Determine the sample size trial duration (HOW
  MANY)
• A key part of any trial is to determine the
  sample size and trial duration. When designing a
  randomized trial, one must plan for an adequate
  number of participants (sample size), so as to
  have the desired power to detect a particular
  intervention effect.

37
Sample Size - 1

• DEPENDS ON
• Significance level
• type I error ? probability of rejecting the
  null hypothesis (H0) when in fact H0 is true
• Power
• type II error ? probability of accepting the
  null hypothesis (H0) when in fact H0 is false
• 1 ? power to detect an alternative hypothesis
  Ha
• Estimate of the difference (treatment effect) one
  wishes to detect
• specify?Ha
• Variance of the outcome measure

38
Sample Size - 2

• For survival (time-to-event) data, consider
• hazard rate for events
• accrual rate and duration
• duration of follow-up
• losses to follow-up(Note losses can both
  decrease power and introduce bias)

39
Data Analyses

• Descriptive Statistics
• location (mean, median)
• variability
• frequency
• Hypothesis Testing
• Estimation and Confidence Intervals
• Exploratory Data Analysis
• Adjusted Analyses
• summary measure from stratified analysis
• regression model
• Overviews (Meta analyses)

40
Three situations

• Group A Group B difference
• Predictive covariate
• Males 120 (n50) 130 (n50) 10
• Females 110 (n50) 120 (n50) 10
• Mean 115 125 10
• Confounding
• Males 120 (n70) 130 (n50) 10
• Females 110 (n30) 120 (n50) 10
• Mean 117 125 8
• Interaction (effect modification)
• Males 120 (n50) 140 (n50) 20
• Females 110 (n50) 120 (n50) 10
• Mean 115 130 15

41
Conclusions

• Randomized trials are important to obtain
  unbiased comparisons of interventions
• Use of both unadjusted and adjusted analyses
• Consider prognostic importance of covariates
  selected for adjustment
• Large trials (adequate sample size) for reliable
  inferences
• when prior reason to suspect important
  interaction, trial large enough to investigate
  subgroups (adequate power to test the
  interaction)
• otherwise, focus on primary question(s)can
  explore data for subgroup interactions, but
  interpret cautiously (may suggest hypothesis for
  future study)