Assessment of NeuroAIDS in Africa

1
AssessmentofNeuroAIDS in Africa

• Statistical Issues and Design
• Scott Evans, Ph.D.
• Harvard University

2

• A statistician is a person that is good with
  numbers but that lacks the personality to become
  an accountant.

3
Outline

• Design Issues
• Variation
• Selection bias
• Threats to scientific integrity
• Drop-out
• Noncompliance
• Some design options to consider
• Futility
• Adaptive designs and sample-size re-estimation
• Quiz

4
Design Issues for Trials in Africa

• Attributes
• Lots of patients
• Adequately power studies
• Fast accrual rates once initiated
• Lots of variation
• Implies less precision and power
• E.g., neuropsych tests
• Noise bigger than signal
• Decreases ability to detect treatment effects

5
A5199 Enrollment Viral Load
6
A5199 Enrollment Gender
7
A5199 Enrollment CD4
8
A5199 Enrollment Neuropsych (Medians)
9
Variability in Africa

• Neuropsych results
• Uganda (Ned Sacktor) vs. Ethiopia (David Clifford)

10
Minimizing Variability

• Goal of clinical trial design
• How?
• Standardization of methods and definitions
• Via training
• A5199 Diagnoses
• Objective endpoints (vs. subjective)
• Quantification and reading activities done
  centrally as much as possible

11
Design Strategies to Reduce Variation

• Eliminate Inter-patient variation by matching,
  pairing, cross-overs
  
• Using matching, pairing, or cross-overs
• Reduces sample size

12
Example Comparing Response Rates

• Study alpha0.05, power80

13
Ramifications of High Variation

• Stratified analyses (if lucky)
• May be limited to subgroup analysis
• Which studies are not powered to do

14
Concerns for Trials in Africa

• Selection Bias?
• A5175
• International sites enrolling patients with
  higher CD4 than domestic (US) sites
  
• Generalizability?
• Threats to trial integrity
• Due to cultural differences and availability of
  resources affecting
  
• Drop-out ? missing data
• Compliance issues

15
Futility Analyses

• 8 of new medicinal compounds entering Phase I
  trials, eventually reach the market
  
• Futility low likelihood of statistical or
  clinical significance if a trial continues to
  planned completion

16
Futility Analysis

• Ethical
• helps protect human subjects by guarding against
  unnecessary exposure to potentially harmful
  treatments
  
• Cost and resource efficiency
• No ? cost associated with futility analyses

17
Predicted Intervals (PIs)

• Predict CI at final analysis conditional upon
• Observed data, and
• Assumptions regarding data yet to be observed
• Reasonable assumptions include
• Observed trend continues
• HA is true (or various alternatives are true)
• H0 is true
• Best and worst case scenarios (often useful for
  binary data))

18
Example NARC 009/ ACTG A5180

• Randomized, double-blind study of Prosaptide for
  the treatment of HIV-associated neuropathic pain
  
• 5 arms
• 2, 4, 8, 16 mg, placebo (PBO)
• Objective examine efficacy and safety after 6
  weeks of treatment
  
• Primary endpoint 6-week change from baseline of
  weekly average of random daily prompts (Gracely
  pain scale) using an electronic diary
  
• Design 390 subjects (78/arm)
• Sized such that the width of the 95 CI for
  difference between any active arm and PBO was
  less than 0.24 (assuming SD of changes0.35)

19
Example NARC 009/ ACTG A5180

• Interim analysis conducted after primary endpoint
  data on 167 subjects obtained.
  
• Table 1
• Mean changes in pain (with CIs)
• Negative changes decreases in pain
• CIs and PIs for between-group differences
• (active minus PBO)

20
Example NARC 009/ ACTG A5180
21
Example NARC 009/ ACTG A5180

• Each PI straddles 0 (except 8mg which favors
  PBO)
  
• Statistical significance unlikely
• Compare width of CI to PI
• No substantial increase in precision with
  continued enrollment
  
• Required changes in yet-to-be-accrued subjects
  for the final CI to exclude 0, are inconsistent
  (larger than) observed changes (not contained in
  CI for mean change)
• NARC DSMB recommended trial termination

22
Adaptive Designs

• Sample size re-estimation
• Seamless Phase II/III designs
• Dynamic randomization

23
Sample-Size Re-estimation

• N (Total Budget / Cost per patient)?
• Hopefully not!
• Power on precision (width of confidence
  interval)
  
• More informative than hypothesis testing
• Estimates of effect (clinical significance)
• P-values only provide statistical significance
• Sample size calculations require assumptions
• Variability
• a clinically relevant difference
• placebo effect, etc.

24
Sample Size Re-estimation

• Checking assumptions and re-calculating sample
  size may be a good idea in trials in Africa since
  assumptions may likely be based on data from
  trials in the US (which may not be valid)
• Virus differences, population differences
• A5199 N129
• 66 female (ACTG 90 male)
• 83 black (ACTG 25 black)
• Median age 32 (ACTG 42)
• Internal pilot studies
• Must be done carefully

25
Seamless Phase II/III Designs

• Duration of drug development is generally
  shorter
  
• No hiatus for setting up phase III
• Uniformly inferior (statistically) to standard
  sequential methods (wrt power)
  
• May still be attractive for other reasons (time
  efficiency)

26
Adaptive Designs

• Screening studies for multiple potential
  treatments (Stage 1)
  
• Selecting promising treatment(s) for stage 2
• Dynamic randomization
• Adapt the probability of randomization to various
  groups based on the success of the treatments in
  the trial to date
  
• Modified play the winner

27
A5199 Entry Viral Load
28
Normative Data

• Norms for neuropsych tests
• Not necessary for tests of statistical
  significance
  
• Can control for confounders in modeling
• Randomization ensures valid tests regardless
• Necessary to assess clinical significance
• Necessary to assess individual impairment
• Need better grasp of clinical relevance of
  neuropsych test scores

29
Neuropsych Endpoints

• Are composites (e.g., NPZ3) interpretable
  (clinically)?
  
• Do we understand the clinical relevance of
  individual components?
  
• Composites are nice for sample size calculation.
  Does analysis need to be multivariate?
  

30

• Use statisticians as strategists
• Not just technicians

31
Quiz Question 1

• Assume 10 of all tested treatments are truly
  effective. A treatment is selected for testing
  and a trial is designed with 90 power and
  alpha0.05. The trial result is positive. What
  is the probability that the treatment is truly
  effective?
• A. 95
• B. 90
• C. 75
• D. 67
• E. 50

32
Quiz Question 2

• A low p-value
• Represents a deficiency in a urinalysis
• Implies important clinical significance
• Means the paper will get published
• Is P(dataH0)
• Is P(hypothesis being true)