New Results about Randomization and SplitPlotting

1
New Results about Randomization and Split-Plotting

• by
• James M. Lucas
• 2003 Quality Productivity Research Conference
• Yorktown Heights, New York
• May 21-23, 2003

2
Contact Information

• James M. Lucas
• J. M. Lucas and Associates
• 5120 New Kent Road
• Wilmington, DE 19808
• [phone]
• JamesM.Lucas_at_worldnet.att.net

3
Research Team

• Huey Ju
• Jeetu Ganju
• Frank Anbari
• Peter Goos

• Malcolm Hazel
• Derek Webb
• John Borkowski

4
PRELIMINARIES

• How do you run Experiments?

5
QUESTIONS

• How many of you are involved with running
  experiments?
• How many of you randomize to guard against
  trends or other unexpected events?
• If the same level of a factor such as temperature
  is required on successive runs, how many of you
  set that factor to a neutral level and reset it?

6
ADDITIONAL QUESTIONS

• How many of you have conducted experiments on the
  same process on which you have implemented a
  Quality Control Procedure?
• What did you find?

7
COMPARING RESIDUAL STANDARD DEVIATION FROM AN
EXPERIMENT WITHRESIDUAL STANDARD DEVIATION FROM
AN IN-CONTROL PROCES
MY OBSERVATIONS
EXPERIMENTAL STANDARD DEVIATION IS LARGER.
1.5X TO 3X IS COMMON.
8

• HOW SHOULD EXPERIMENTS BE CONDUCTED?
• COMPLETE RANDOMIZATION
• (and the completely randomized design)
• RANDOMIZED NOT RESET
• (Also Called Random Run Order (RRO) Experiments)
• (Often Achieved When Complete Randomization is
  Assumed)
• SPLIT PLOT BLOCKING
• (Especially When There are Hard-to-Change
  Factors)

9
Randomized Not Reset (RNR) Experiments

• A large fraction (perhaps a large majority) of
  industrial experiments are Randomized not Reset
  (RNR) experiments
• Properties of RNR experiments and a discussion of
  how experiments should be conducted
• Lk Factorial Experiments with Hard-to-Change and
  Easy-to-Change Factors Ju and Lucas, 2002, JQT
  34, 411-421 studies one H-T-C factor and uses
  Random Run Order (RRO) rather than RNR
• Factorial Experiments when Factor Levels Are Not
  Necessarily Reset Webb, Lucas and Borkowski,
  2003, JQT, to appear studies gt1 HTC Factor

10

• RNR EXPERIMENTS
• (Random Run Order Without Resetting
  Factors)
• OFTEN USED BY EXPERIMENTERS
• NEVER EXPLICITLY RECOMMENDED
• ADVANTAGES
• Often achieves successful results
• Can be cost-effective
• DISADVANTAGES
• Often can not be detected after experiment
• is conducted (Ganju and Lucas 99)
• Biased tests of hypothesis (Ganju and Lucas 97,
  02)
• Can often be improved upon
• Can miss significant control factors

11
Results for Experiments with Hard-to-Change and
Easy-to-Change Factors

• One H-T-C or E-T-C Factor use split-plot
  blocking
• Two H-T-C Factors may split-plot
• Three or more H-T-C Factors consider RNR or
  Low Cost Options
• Consider Diccons Rule Design for the H-T-C
  Factor

12
New Results

• Joint work with Peter Goos
• Builds on the Kiefer-Wolfowitz Equivalence
  Theorem
• Implications about Computer generated designs
  (especially when there are Hard-to-Change Factors)

13
Kiefer-Wolfowitz Equivalence Theorem

• ? is the design probability measure
• M(?) XX/n (kxk matrix for a n point design)
• d(x, ?) x(M(?))-1x (normalized variance)
• So called Approximate Theory
• The following are equivalent
• ? maximizes det M(?)
• ? minimizes d(x, ?)
• Max (d(x, ?) k

14
Very Important Theorem

• Helps find Optimum Designs
• Basis for much computer aided design work
• Justifies using XX Criterion
• Shows Classical Designs are great
• Which Response Surface Design is Best
  Technometrics (1976) 16, 411-417
• Computer generated designs not needed for
  standard situations

15
Optimality Criteria

• Determinant (D-optimality)
• Maximize XX
• D-efficiency XX/n/ XX/n1/k where X
  is an optimum n point design
• Global (G-optimality)
• Minimize the maximum variance
• G-efficiency k/Max d(x, ?)
• G-efficiency lt D-efficiency
• No bad designs with high G-efficiency

16
Computer Generated Design Arrays

• Different criteria give different n point
  designs
• Do not pick a single n
• Some n values may achieve an excellent design
• Check other criteria (especially G-)
• Lucas (1978) Discussion of D-Optimal Fractions
  of Three Level Factorial Designs
• Borkowski (2003) Using A Genetic Algorithm to
  Generate Small Exact Response Surface Designs

17
Equivalence Theorem does not hold for Split-Plot
Experiments

• D- and G- criteria converge to different designs
• Example r reps of a 23 Factorial (linear terms
  model)
• Optimum design depends on d ?w2/?2 where ?w is
  the whole-plot and ? is the split-plot error
• For large values of d
• D-optimal design has 4 r blocks with I A BC
• G-optimal design has 8r 2 blocks (Number of
  observations minus number of split-plot terms)

18
Computer Generated Split-Plot Experiments

• Useful Research
• Recent publications
• Trinca and Gilmour (2001) Multi-stratum Response
  Surface Designs Technometrics 43 25-33
• Goos and Vandebroek (2001) Optimal Split-Plot
  Designs JQT 33 436-450
• Goos and Vandebroek (2003) Outperforming
  Completely Randomized Designs JQT to appear
• All use XX Criterion

19
RELATED SPLIT-PLOT FINDINGS SUPER EFFICIENT
EXPERIMENTS (With One or Two Hard-to-Change
Factor) SPLIT PLOT BLOCKING GIVES HIGHER
PRECISION AND LOWER COSTS THAN COMPLETELY
RANDOMIZED EXPERIMENTS
20
Design Precision Calculating Maximum Variance

• Simplifications for 2k factorials
• Sum Variances of individual terms
• Whole plot terms
• ?w2/ number blocks ?2/ 2k
• Split plot terms
• ?2/2k
• Completely randomized design has variance
• k(?w2 ?2)/ 2k
• Blocking Observation to achieve Super Efficiency

21
26-1 with one or two Hard-to-Change Factors

• Main Effects plus interaction Model
• 22 Terms (1 6 15)
• Use Resolution V, not VI with IABCDE
• Use four blocks IABCFABCFBCDEADEFDEF
• Nest Factor B within each A block giving a
  split-split-plot with 8 Blocks
• B2AB2CF2ACF2CDE2ABDEF2BDEF2
• I and A have variance ?02/32 ?12/4 ?22 /8
• B, AB and CF have ?02/32 ?22 /8
• Other terms have variance ?02/32
• G-efficiency 22(?02?12?22)/(22?0216?1220?22
  ) gt1.0
• Drop ?22 terms for one h-t-c factor results

22
Observations

• Does not use Maximum Resolution or Minimum
  Abberation
• Similar results for most 2k factorials

23
Super Efficient Experiments are not always Optimal

• 26-1 Main effects plus 2FI model
• G-optimum design has 12 blocks when d gets large

24
Conclusions

• Showed K-W Equivalence theorem does not hold for
  Split-Plot Experiments
• Discussed Implications
• Exciting research area
• Much more to do