Three Statistical Issues

1
Three Statistical Issues

• (1) Observational Study
• (2) Multiple Comparisons
• (3) Censoring Definitions

2
Non-Randomized Study

• Benefit of Randomized Study Characteristics
  that may be associated with outcome tend to be
  balanced across treatment groups
• Limitation of Observational Study
  Characteristics that may be associated with
  outcomes may not be balanced.
• Example Sicker patients could tend to be
  assigned to one of the treatments more often than
  the others.
• As a result, crucial to compare the groups as
  thoroughly as possible with respect to
  covariates.
• Where balance does not exist, analyses should be
  adjusted for imbalances.

3
Examples from Paper
4
How to adjust?

• Regression Methods
• Cox proportional hazards model (survival
  analysis)
• Linear regression (continuous outcomes)
• Logistic regression (binary outcomes, e.g.
  response rate)
• Other regression methods (Generalized Linear
  Models)

5
Multiple Comparisons

• Not just comparing treatment 1 versus treatment
  2.
• Three comparisons
• treatment 1 vs 2
• treatment 1 vs 3
• treatment 2 vs 3
• Recall type 1 and type 2 errors..

6
Type 1 Error

• The probability of concluding there is a
  difference in treatments when there truly is NO
  difference.
• Also called the alpha level of the test.
• Determines what is a significant p-value.
• Generally set to 0.05
• So, 5 of the time we will say there IS a
  difference when there really is not.

7
Type II error

• The probability of concluding there is NO
  difference in treatments when there truly is a
  difference.
• Also, called the beta level
• Also, Power 1 - beta (so power is concluding
  that there is a difference when there truly is a
  difference)
• Generally, we like to have beta is less than 20.
• So, 20 of the time we will say there is no
  difference when there really is a difference

8
Back to Type 1 Error

• If we are making one comparison, we have a 5
  chance of making the error of concluding that
  there is a difference when there really is no
  difference.
• What if we have three comparisons?
• 3x5 15 chance of finding some difference
  between treatments if no treatment differences
  exist.

9
Solutions

• Some say.
• Note to take into consideration when drawing
  conclusions
• Easy way to handle it
• Other places (e.g. Harvard).
• Bonferroni and other corrections
• Essentially dividing p-value among tests so that
  the sum of the type I errors is 0.05
• Example
• For three comparisons, allow Type 1 error for
  each comparison of 5/3 1.33.
• So only conclude that there is a significant
  difference between two treatments if pvalue is
  less than 0.013.

10
Another Solution

• ANOVA type tests
• Ho no differences between treatments
• Ha at least one treatment is different than some
  other
• Get just one pvalue.
• Tells you that there is some difference, but no
  information about which is different, or how many
  are different.

11
Example

• var1 var2 var3
  var4 var5
• 1. 5 6 7
  8 9
• 2. 6 7 8
  9 10
• 3. 7 8 9
  10 11
• 4. 8 9 10
  11 12
• 5. 9 10 11
  12 13
• 6. 10 11 12
  13 14
• anova var group
• Number of obs
  30 R-squared 0.4068
• Root MSE
  1.87083 Adj R-squared 0.3119
• Source Partial SS df
  MS F Prob gt F
• -----------------------------------
  ----------------------------
• Model 60.00 4
  15.00 4.29 0.0089
• group 60.00 4
  15.00 4.29 0.0089

pvalue
12
Pvalues for Individual T-tests 10 comparisons