Statistics and Experimental Design

1
Statistics and Experimental Design

• Shirley Coleman
• Industrial Statistics Research Unit

2
Outline of Talk

• Purpose of Stats and Experimental Design
• History and Applications
• Skill set needed
• Examples
• Importance of planning
• Subtleties
• Summary

3
Purpose of Statistics and Experimental Design

• Investigate
• Make objective decisions
• Experiment efficiently
• Ensure reproducibility
• Build a model
• Predict
• Monitor
• .

4
History of experimental design

• Agriculture
• http//www.rothamsted.ac.uk/
• Oldest agricultural research station
• Rothamsted, Park Grass, 1856
• Cockle Park, Palace Leas, 1897

5
Long term grass experiments
Palace Leas, Cockle Park, Northumberland 1897
Park Grass, Rothamsted Herts, 1856
6
Palace Leas, Cockle Park
7
Palace Leas

• Experiment on effect of fertilisers on hay yield
• Can also look at root structure, species,
• Meteorological station in next field from 1898
• 14 plots of land
• 8 for an experiment with N,P and K
• 6 additional (some abandoned in WWII)
• Results were apparent in1898
• Fabulous set of data

8
Palace Leas plots
9
Palace Leas plots
10
Rothampsted, Park Grass
11
(No Transcript)
12
Importance of publishing!
COLEMAN, S.Y., SHIEL, R.S EVANS, D.A. (1987)
The effect of weather and nutrition on the yield
of hay from Palace Leas meadow hay plots, at
Cockle Park Experimental Farm, over the period
from 1897 to 1980. Grass and Forage Science 42,
353-358.
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
History and Applications

• Agriculture
• Given industrial slant by G.Taguchi
• (b 1924, published 1951, in NE 2000)
• Used in manufacturing
• Gradually used in
• Business, service, health,
• Finance, marketing,
• Bespoke nomenclature

17
Skill set needed

• Logical thinking
• Attention to detail
• Presentation skills
• Analytical tools
• Knowledge of where to go next
• Optimisation, RSA, simulation

18
Examples

• Pressure, temperature, pointer setting,
• haul off speed, welding current,
• granule size, nozzle width,.
• N, P, K
• Beer experiment

19

• Beer experiment
• (What affects frothing when pouring beer?)

20
(No Transcript)
21
Management Methodology

• Six Sigma
• Define
• Measure
• Analyse
• Improve
• Control
• Lean
• PDCA

22
Lean Six Sigma

• Lean focuses on removing complexity
• Six Sigma focuses on process improvement
• Lean Six Sigma attempts to combine the best of
  both
• Lean involves less statistics and is very popular
  in
• some applications, such as healthcare.

23

• Define problem
• QI tools, brain-storming, team roles
• Identifying factors and levels
• Measurement issues
• Decide what, how, when, who and where to measure
• Analyse
• Use current knowledge,
• Set experimental design and pilot
• Improve
• Run experiment and analyse
• Control
• Recommend method for best or least froth
• Look for other opportunities to use what has been
  learnt

24
Team roles

• Secretary
• Waiter
• Pourer
• Measurer
• Observers

25
Factors

• Materials (beer type, temperature of bottles)
• Machines (glass shape)
• Man (steadiness)
• Milieu (pressure, humidity, temperature)
• Method (angle, speed, height, time opened)
• Measures (volume, height)

26
Experimental design

• More information out requires more data in
• However, statistically designed experiments
• help reduce the number of trials with least
• reduction in information
• Eg up to 7 factors can be tested in 8 trials
• Taguchi uses Plackett-Burman designs
• Eg up to 11 factors can be tested in 12 trials

27
Saturated L8 design
28
Taguchi saturated L8 design
29
Identifying factors and levels

• 3 factors each at 2 levels
• Factors -
• Beer type Belgian French
• Glass type Flat Rounded
• Glass angle Upright Tilted
• Response Froth height (mm)

30
Orthogonal array
31
Experimental trials

• Eg Belgian beer poured into a flat
• bottomed glass without tilting
  
• Eg French beer poured into a round
• bottomed glass without tilting

32
Randomisation

• Reduces false replication and bias
• Eg drug trial
• first group of mice have treatment and
• second have placebo,
• if mice are selected by grabbing from cage,
• fittest are caught last and placebo can
• appear to be better than the treatment

33
Team roles

• Secretary Read trial and register result
• Waiter Pick the right glass and bottle
• Pourer Pour the beer into the glass
• Measurer Use the ruler to read off the froth
• Observers Note that procedures are followed

34
Main effects plot
35
Factors and factor levels

• Factors -
• Beer type Belgian French
• Glass type Flat Rounded
• Glass angle Upright Tilted
• Response Froth height (mm)

36
Interaction plot glassangle
37
ANOVA table for beer
38
Graphical analysis of significance
39
(No Transcript)
40
Context and implications

• Know which glasses to buy
• Know how to pour
• Adapt to alternative requirements, eg no froth
• Apply methodology in other contexts

41
Other examples

• Human Resources
• Training requirements
• Explore effects of commands given
• Accounts
• Explore effect of timing and nature
• of reminders
• Preferences
• Explore trade-offs
• Conjoint analysis

42
Conjoint Analysis

• Customers CONsider JOINTly and give their
  opinions,
• trading off factors to reach a desired end.
• For example, to design a conference, consider
• In University or hotel
• 2 days or 3 days
• Evening speaker or not
• 20 minute or longer talks
• .
• Present the different options, eg in a
  questionnaire and analyse results
• Helps determine what people value in different
  product features
• or service attributes.

43
Format of questions (ENBIS)

• How successful do you think the following
• conference would be?
• An informal, applied conference held in a
• conference suite that has an evening session,
• it mostly features presentations involving
• industrialists
• 1 not very successful 5 very successful

44
Results - ENBIS

• ANOVA with categorical responses gave
• Applied vs theoretical and
• Industrialists vs academics
• as important factors

45
Results - ENBIS

• Taguchi style analysis for the variability of
• responses showed
• significantly greater variation in views about
• applied talks than for theoretical talks
• significantly greater variation for
• workshops than for presentations

46
Online Conjoint Analysis

• Used in automobile feature testing to find the
  features
• consumers are willing to give up in order to get
  something
• they value more.
• Outcomes help guide new product design, old
  product
• redesign or repositioning decisions.
• Used in travel industry to determine how much
  consumers
• are willing to pay for a ticket in order to get
  more leg room.

Building Voice of Customer into Product
Development
Source Siegel (2004)
47
Other examples

• Kansei Engineering
• Incorporates emotion into design
• Orthogonal array for products
• Semantic scales for emotion
• Sample of customers
• Highly developed in Japan
• KENSYS 2003-6 in Europe

48
(No Transcript)
49
Analysis relates emotions to design

• Model
• Response is happiness
• Design factors are colour, style, heel, etc
• Aim is to advise designers
• Which shoes will give the desired emotional
• response
• How to develop a balanced portfolio
• Similar results for logistic regression or ANOVA

50
Design a waiting room

• Design factors
• Sofa or chair
• Lighting soft or bright
• Service desk
• Windows
• Other factors
• Waiting time max 30 minutes
• ..

51

• Please rate where you feel
• the image fits on each of the
• following semantic scales.

1
2
3
4
5
Comfortable
Uncomfortable
At Ease
Uneasy
Efficient
Inefficient
Trustworthy
Untrustworthy
Calm
Stimulating
Boring
Interesting
52
Design Rules
Comfortable
Efficient
Seating (sofa) Windows (yes) Lighting
(soft) Service desk (yes)
Seating (chair) Service desk (yes) Max Waiting
Time (30min)
53
Key Drivers of Satisfaction

• Ease of UseNavigation, clarity,
  fresh/relevant content, etc.
• Graphic StyleColour, layout, print size,
  type, no. of photographs,
• graphics and animation.
• Perceived channel advantage Price, Speed,
  etc.
• Privacy and Security Brand, reputation,
  appearance of the site (more
• imp than security logos appearing on website)
• Fulfilment and ReliabilityTimeliness of
  service, availability, breadth and depth of
• products/services, responsiveness/access
  (availability of service
• personnel, multiple communication channels) and
  personalisation.

Source Baur, Schmidt Hammersmith, 2006
54
Website design process
User Survey Conjoint Analysis
Kansei Design for Emotional Appeal
Content Navigation SPC
CLIENT
Response and refinement
Client requirements and goals
Launch
Approval
Web Design for Sticky Relationship
The Brief
Trial Pages
The Prototype
The Prototype
Final design, testing and coding
Strategic planning, engineering
Style book, training, quality tests
Design content and marketing
Building Voice of Customer into Design by
courtesy of A.Parulekar
WEB DESIGN FIRM
55
Statistics

• Quantitative management
• Software
• Graphical
• Tabular
• Comparative tests, ANOVA
• Statistical models
• Regression
• Logistic regression
• Multivariate analysis

56
Tutorials, eg MINITAB

• These easy step-by-step tutorials introduce you
  to the
• Minitab environment and provide a quick overview
  of
• some of Minitab's most important features. The
  tutorials
• are designed to explain the fundamentals of using
  Minitab
• how to use the menus and dialog boxes, how to
  manage
• and manipulate data and files, how to produce
  graphs, .
• Session One Graphing Data
• Session Two Entering and Exploring Data
• Session Three Analyzing Data
• Session Four Assessing Quality
• Session Five Designing an Experiment

57
Data exploration, eg SAS JMP
58
Importance of planning

• Collecting all the data that is needed
• Getting the right people involved
• Getting the right materials in place
• Ensuring time for input from others
• Poke yoka

59
Subtleties include

• Measurement issues
• Sample sizes
• Use continuous variables where possible
• Robustness
• Bias and confounding, especially time trends
• Piloting

60
Summary

• All of the above
• Thanks