To Prevent Selection Bias

1
To Prevent Selection Bias
Minimal Balance is Sufficient

• Wenle Zhao, PhD
• Medical University of South Carolina, Charleston,
  SC, 29425, USA
• Society for Clinical Trials 36th Annual Meeting
• Arlington, VA, USA - May 17-20, 2015

2
Contents

1. Where does Selection Bias Come From?
2. How to prevent selection bias?
3. How to avoid random serious imbalance?

3
The Worst Thing in the World of Clinical Trials
Funding?
Recruitment?
A Completed Trial with Suspicious Selection Bias.
4
Defense Measurements against Selection Bias
Subject Enrollment
Treatment Allocation
Outcome Assessment
Real-time Subject Randomization
Treatment Masking
Random Allocation
Allocation Concealment
The only reliable protection left against
selection bias.
5
Allocation Randomness
Target allocation ratio
Random variable U(0,1)
To balance treatment distribution Permuted Block
Randomization Biased Coin, Urn Design
To balance baseline covariate Stratified
Randomization Minimization
Allocation Randomness
Complete Randomization
Permuted Block Randomization
Minimization
6
Predictability Defeats Concealment Masking
Deterministic Assignment
100
87
Proportion of DA
50
33
25
20
0
B 2
B 4
B 6
B 8
Minimization
Permuted Block Randomization
7
Evidence of Selection Bias in Randomized Trials
1. Heparin for myocardial infarction
E
2. University Group Diabetes Program
E
3. Talc and mustine for pleural effusions
P
4. Tonsillectomy for recurrent throat infection
in children
P
5. Oxytocin and amniotomy for induction of labor
P
6. Western Washington Intracoronary Streptokinese
Trial
P
7. RSV immune globulin in infants and young
children
E
8. A trial to assess episiotomy
E
9. Canadian National Breast Cancer Screening Study
Selection bias evidence identified
P
E
10. Surgical Trial
P
11. Lifestyle Heart Trial
P
12. Coronary Artery Surgery Trial
E
13. Etanercept for children with juvenile
rheumatoid arthritis
P
14. Edinburgh Randomized Trial of Breast Cancer
Screening
E
15. Captopril Prevention Project
P
16. Göteborg (Swedish) Mammography Trial
P
17. HIP Mammography Trial
E
18. Hypertension Detection and Follow-up Program
E
19. Randomized Trial to prevent vertical
transmission of HIV-1
P
20. Effectiveness trial of diagnostic test
P
21. S African trial of high-dose chemotherapy for
metastatic breast cancer
E
22. Randomized study of a culturally sensitive
AIDS education program
P
23. Runaway Youth Study
P
24. Cluster randomized trial of palliative care
P
Suspicious election bias due to p-value lt 0.05
P
25. Randomized trial of methadone with or without
heroin
P
26. Randomized NINDS trial of tissue plasminogen
activator for acute ischemic stroke
P
27. Norwegian Timolol Trial
P
28. Laparoscopic versus open appendectomy
P
29. The Losartan Intervention for Endpoint
reduction in Hypertension Study
P
30. The Heart Outcomes Prevention Evaluation
Study
P
8
Protect Trials Against Selection Bias
Selection bias will result small
p-values Complete randomization may (5 chance)
see a p-value lt 0.05
P lt 0.05 ?
9
The Logic

• Real-time complete randomization
• Eliminates selection bias due to allocation
  predictability
• Eliminates selection bias due to allocation
  concealment failure
• Totally eliminates selection bias

• Without selection bias, complete randomization
  may still have
• Imbalance in treatment distribution
• ? Power loss is trivial
• Imbalance is baseline covariate distribution
• ? Adjustment, not balancing, is the solution
• Serious baseline covariate imbalance with p-value
  lt 0.05
• ? 5 chance for any covariate
• ? 60 chance for at least one in 10 covariates
• ? Suspicion of selection bias
• ? Trouble in trial result interpretation

10
Options We Have

• Stratified Restricted Randomization
• Permuted Block Randomization
• Biased Coin Design - Efron
• Urn Design - Wei
• Big Stick Design Soares Wu
• Maximal Procedure Berger et al.
• Block Urn Design Zhao Weng
• Unnecessarily tighten control imbalances.
• Disabled when number of strata getting large.
• Minimization
• Most assignments are deterministic.
• Dynamic Hierarchy Balancing
• Hierarchy order is hard to justify.

11
Minimal Sufficient Balance Procedure
Subject ready for randomization
T-test for continuous var. ?2 test for
categorical var.
N
Any serious imbalance?
Any p-value lt 0.2?
Y
Current assignment can effectively reduce
imbalances?
Y
N
Complete randomization
Biased coin assignment
The proportion depends on p-value threshold and
biased coin probability
Next subject
12
Example NINDS rt-PA Stroke Study
p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP
Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810
Serious imbalances found in 2 of the 11 baseline
covariates.
13
Example NINDS rt-PA Stroke Study
NINDS rt-PA Stroke study data. Simulations
5000

• Baseline covariates
• Severity (NIHSS)
• Age
• Onset to treat
• Glucose
• Center

14
Example NINDS rt-PA Stroke Study
Balance 11 baseline covariates
Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario. Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled NINDS rt-PA data, Imbalance control limit p-Value 0.3. ? 0.65, simulation 1000/scenario.
p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP
Low 2.5 boundary 0.226 0.259 0.237 0.262 0.244 0.262 0.214 0.245 0.239 0.252 0.246
Low 5 boundary 0.276 0.295 0.279 0.288 0.280 0.292 0.277 0.288 0.278 0.281 0.287
Low 10 boundary 0.309 0.341 0.316 0.323 0.315 0.326 0.309 0.322 0.319 0.322 0.330
Median 0.605 0.631 0.626 0.624 0.624 0.638 0.609 0.634 0.626 0.638 0.610
Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810
15
Summary

• Complete randomization eliminates selection bias
  due to allocation predictability.

• Real-time randomization eliminates selection bias
  due to allocation concealment failures.

• Minimization method has the highest proportion of
  deterministic assignments, and therefore is
  vulnerable to selection bias.

• Power loss due to treatment imbalance is trivial.
  

• Justification, not balancing, is the solution for
  covariate confounding effects.

• Using Minimal Sufficient Balancing to prevent
  random serious imbalances, while maintaining a
  high level of allocation randomness.

16
Thank You! Contact me at zhaow_at_musc.edu
17
Some of my works on Randomization

• Zhao W, Ciolino J, Palesch Y. Step-forward
  randomization in multicenter emergency treatment
  clinical trials. Acad Emerg Med. 2010
  Jun17(6)659-65. doi 10.1111/j.1553-2712.2010.00
  746.x. PMID 20624149

.

• Zhao W, Weng Y, Wu Q, Palesch Y. Quantitative
  comparison of randomization designs in sequential
  clinical trials based on treatment balance and
  allocation randomness. Pharm Stat. 2012
  Jan-Feb11(1)39-48. doi 10.1002/pst.493. PMID
  21544929

• Zhao W, Weng Y. Block urn design - a new
  randomization algorithm for sequential trials
  with two or more treatments and balanced or
  unbalanced allocation. Contemp Clin Trials. 2011
  Nov32(6)953-61. doi 10.1016/j.cct.2011.08.004.
  PMID 21893215

• Zhao W, Hill MD, Palesch Y. Minimal sufficient
  balance--a new strategy to balance baseline
  covariates and preserve randomness of treatment
  allocation. Stat Methods Med Res. 2012 Jan 26.
  Epub ahead of print PMID 22287602

• Zhao W. Selection bias, allocation concealment
  and randomization design in clinical trials.
  Contemp Clin Trials. 2013 Sep36(1)263-5. doi
  10.1016/j.cct.2013.07.005. Epub 2013 Jul 19. No
  abstract available. PMID 23871796

• Zhao W. A better alternative to stratified
  permuted block design for subject randomization
  in clinical trials. Stat Med. 2014 Dec
  3033(30)5239-48. doi 10.1002/sim.6266. PMID
  25043719

• Zhao W, Durkalski V. Managing competing demands
  in the implementation of response-adaptive
  randomization in a large multicenter phase III
  acute stroke trial. Stat Med. 2014 Oct
  1533(23)4043-52. doi 10.1002/sim.6213. Epub
  2014 May 22. PMID 24849843

• Zhao W, Mu Y, Tayama D, Yeatts SD. Comparison of
  statistical and operational properties of subject
  randomization procedures for large multicenter
  clinical trial treating medical emergencies.
  Contemp Clin Trials. 2015 Mar41211-8. doi
  10.1016/j.cct.2015.01.013. Epub 2015 Jan 29.
  PMID 25638754