Selecting Among 2-Sample Tests

1
Selecting Among 2-Sample Tests

• Selection among tests is often based on the
  underlying distributions of the two populations.
  We often consider the ones below. Here x is a
  vector of values of the variable
• uniform (density is dunif(x, min, max) where min
  and max are the endpoints of the uniform density.
• normal (density is dnorm(mean,sd)
• exponential (density is dexp(rate), where rate is
  the parameter of the exponential (so 1/rate
  mean of the exponential distribution)
• Cauchy (density is dcauchy(location,scale) where
  location and scale are its parameters
• Laplace (density is (1/2b)(exp(-abs(x-a)/b) where
  -infltxltinf, and bgt0 is the scale parameter
  and a is the location parameter)
• Write R code that will plot all these densities
  on the same axis for comparison, especially
  comparison of the tails of the distributions.
  Fat tailed underlying densities often call for
  nonparametric methods

2

• Assume Xi from treatment 1 and Yj from treatment
  2 and that they have cdf's given by
• Also assume the cdf's are continuous (no ties).
  The hypothesis tested is
• If the cdfs are normal with unknown but equal
  variances, then the t-test is the best.
  Departure from normality has little effect on
  Type I error, because of the Central Limit
  Theorem, but there are problems with power if
  they are non-normal.
• See Table 2.9.1 for a comparison of the Wilcoxon
  rank-sum test versus the t-test. Note that for
  small samples and "light-talied" distributions
  (those without much chance of outliers), the
  t-test is probably better. Otherwise, use
  Wilcoxon
• Another way of comparing these two tests is via
  relative efficiency (and ARE, asymptotic relative
  efficiency). See the definition on p. 62
  basically, test 1 is more efficient than test 2
  if it requires a smaller sample size to achieve
  the same power.

3

• The ARE of the Wilcoxon test versus the t-test is
  gt .864 but can be arbitrarily large (see Table
  2.9.2). Even for normal cdfs, the ARE of
  Wilcoxon vs. t-test is .955
• The permutation test vs. the Wilcoxon test and
  the t-test is discussed in 2.9.4. There are a
  couple things to note here
• little is gained by doing more than 1600 randomly
  sampled permutations
• see Table 2.9.3 for comparison with t-test
• see Table 2.9.4 for comparison with Wilcoxon test