APPENDIX B SOME BASIC TESTS IN STATISTICS

1
APPENDIX B SOME BASIC TESTS IN STATISTICS
Slides for Introduction to Stochastic Search and
Optimization (ISSO) by J. C. Spall

• Organization of appendix in ISSO
• Standard one-sample test
• P-values
• Confidence intervals
• Basic two-sample tests
• Matched pairs t-test
• Unmatched pairs t-test with identical variances
• Unmatched pairs t-test with nonidentical
  variances
• Other approaches to testing
• One- and two-sample tests important in stochastic
  search, optimization, and Monte Carlo simulation

2
The Standard One-Sample Test

• One set of data Xi for testing on ? ? E(Xi)
• Famous test statistics
• z and t have a N(0, 1) and t-distribution,
  respectively
• t-statistic useful in small samples both z and t
  often used with non-normal samples
• Large values of z or t indicate rejection of
  null hypothesis that ? is some chosen value
  (commonly ? 0)

3
P-Values

• P-value Probability that future experiment would
  have value of test statistic at least as extreme
  as that observed in the current experiment
• Provides info. beyond binary accept/reject null
  hypothesis
• Useful as indicator of strength of rejection
• Example If z 2.15, P-value is 0.016 based on
  null hypothesis that ? ? 0
• Fairly strong evidence that ? gt 0

4
Two-Sample Tests

• Two sets of data Xi and Yi for testing ?X
  ?Y
• E.g., Xi and Yi represent simulation outputs
  under two scenarios
• Generic test statistic form
• where (?) denotes appropriate variance estimate
• Three basic categories of tests affecting (?)
• matched pairs
• unmatched pairs identical variances (
  )
• unmatched pairs non-identical variances (
  )
• Large values of t indicate rejection of null
  hypothesis that ?X ?Y