Statistical tests for replicated experiments

1
Statistical tests for replicated experiments

• Normal probability plots are a less formal
  diagnostic tool for detecting effects
• F-tests and t-tests provide a statistical test of
  factor effects

2
Statistical tests for replicated experiments

• Statistical tests are possible for unreplicated
  designs (unreplicated pilot studies are essential
  tools in sample size calculations)
• We will first focus on statistical tests for
  replicated designs

3
Statistical tests for replicated
experiments--Example

• Response--Pulse rate of subject
• Factors
• Treatment (Energy Drink, Placebo)
• Setting (Moderate, Difficult)
• Machine (Stair climber, Recumbent bike)

4
Statistical tests for replicated
experiments--Example
5
Statistical tests for replicated
experiments--Example
6
Statistical tests for replicated
experiments--Example
7
Statistical tests for replicated experiments

• Effect sizes depend on the measurement scale
• Statistical tests are based on standardized
  effects
• To compute standardized effects, start with an
  estimate of experimental error

8
Statistical tests for replicated experiments

• Experimental error can be summarized by the
  square root of the variance of the background
  noise (the standard deviation)
• The experimental error measures variation in a
  single observation

9
Statistical tests for replicated experiments

• The variance is best estimated by the Mean Square
  for Pure Error (MSPE)

10
Statistical tests for replicated
experiments--Example

• The standard deviation for each run is 3 beats
  per minute

11
Statistical tests for replicated experiments

• While the standard deviation for a single
  response is the square root of MSPE, the standard
  deviation of an effect (its standard error) is

12
Statistical tests for replicated experiments

• We divide an effect in a k-factor experiment with
  n replications (e.g., A) by its standard error to
  compute a t-test statistic

13
Statistical tests for replicated experiments

• Test statistics for other effects are computed
  similarly
• U-do-it Calculate the T-statistics of all
  effects for the Exercise data

14
Statistical tests for replicated experiments

• When an effect is negligible, T has a
  t-distribution
• The shape of the t-distribution curve depends on
  the number of replicates (degrees of
  freedom2k(n-1))
• The t-distribution curves have slightly more
  spread than the bell-shaped (normal) curve

15
Statistical tests for replicated experiments--t
curve for 3-factor design
16
Statistics tests for replicated experiments

• If T is larger than the 99.5th or 97.5th
  percentile of the t distribution, an effect is
  significant
• These percentiles are commonly found in textbooks
  (but please use a computer package instead)

17
Statistical tests for replicated experiments--t
critical value for 3-factor design (n4)
18
Statistical tests for replicated experiments

• Sometimes, twice the area to the right of T is
  reported as a p-value. Small p-values suggest
  that a standardized effect is distinguishable
  from background noise
• You definitely need a computer to compute
  p-values--in the following example, the p-value
  for the M effect is 2.122.244

19
Statistical test for replicated
experiments--Example
20
Statistical tests for replicated
experiments--Example

• U-do-it Compute p-values for the remaining
  effects. Which effects are significant? Are
  these the same effects that the probability plot
  detected?

21
Statistical tests for replicated experiments

• F tests for individual effects are equivalent to
  t-tests
• F tests allow several comparisons to be tested
  simultaneously
• The t-test can be used to help in computing the
  number of replications needed in a factorial
  experiment

22
Statistical tests for replicated experiments

• Hypothesis testing can be extended to combine
  estimates of error from both pure error and
  negligible effects
• Negligible effects can be selected a priori or
  from effects plots
• Degrees of freedom for t-tests and F-tests should
  be adjusted accordingly

23
Statistical tests for replicated experiments

• Error estimated from negiglible terms (Lack of
  Fit) is similar to MSPE
• Source DF SS MS
• Residual Error 28 300.38 10.73
• Lack of Fit 4 45.38 11.34
• Pure Error 24 255.00 10.63