Managerial Data

1
Methods for Handling Low Frequency Managerial Data
Jim Stuart Manager, Applied Statistics Eastman
Chemical Company jestuart_at_eastman.com
Kevin White Senior Statistician Voridian, a
Division of Eastman Chemical Company kwhite_at_eastma
n.com
2
Statistical Thinking...

• A habitual way of looking at work that
• recognizes all activities as PROCESSES,
• recognizes that all processes have VARIABILITY,
• uses DATA to understand variation, and to drive
  effective DECISION MAKING.

3
Outline

• Process Thinking Principles
• 8 Lessons for Visualizing Variability
• Databased Decision Making
• Control Charts
• Special Cause Rules
• Change Point Analysis

4
Process Thinking

• Managerial Data Should
• Summarize performance on what is key to business
  success.
• Provide history of how the business has
  performed.
• Help predict the future.
• Provide the foundation for improvement. (Gap
  Identification)
• Provide a signal for reinforcement of
  accomplishments.
• Serve as a means for holding the gains.

5
Principles for Selecting Measures

• Sufficient number to adequately cover all the
  important facets of the business. If it isnt
  important to the business, dont track it.
• Each measure should impact at least one
  stakeholder including suppliers, publics,
  investors, customers, or employees (SPICE).
• Needs to be an appropriate mix of leading and
  lagging measures.
• Lend themselves to charts that are easy to read
  and interpret.
• Measures should be analyzed for appropriateness
  if the situation or strategy changes.
• Well-charted measures can provide a mechanism
  for concise
  communication with stakeholders.

6
Indicators of Performance
Outputs
Inputs
Customers
Process
Products or Services
1
2
Gauge
Gauge
Customer Satisfaction Customer Dissatisfaction
Product and Service Quality
Process Quality/ Reliability (Leading Indicators
of Product and Service Quality)
Quality of Inputs Supplier Quality
Financial/Cost
People
Health, Safety Environmental
7
His project is 10 over budget
Good News? Bad News? No Earthly Idea?
8
Statistical Thinking Lesson 1 It Depends

• Was the budget set at the best current estimate
  or was it a guaranteed not to exceed number?
• What are the implications of financial planning
  if everyone uses guaranteed not to exceed
  numbers?
• What would you suspect if a particular project
  manager finished every project exactly on
  schedule?

9
Statistical Thinking Lesson 2 Variation
HappensAt Least It Should
10
Statistical Thinking Lesson 3 Show Data in
Time Order
11
Statistical Thinking Lesson 4 Beware Your Axes
The selection of the scale of your vertical axis
can have a profound effect on the interpretation
by the audienceparticularly if it is not their
data.
Daily Sales in Thousands
Daily Sales in Thousands
12
Statistical Thinking Lesson 5 Dont
Over-Summarize
Collect and display data at sufficient frequency
to understand the variation, and beware the
trappings of bar-charts!
Opportunity Seeking Improvement Motivating (Lets
fix the dips!)
Management Review And Presentation (Im OK,
youre OK)
13
Im OK, Youre OK Slide
Summary presentations utilizing averages, ranges
or histograms should not mislead the user into
taking action that would not have been taken if
presented as a time series.
14
Statistical Thinking Lesson 6 Display History
to Provide Context
Plot sufficient history to visualize trends
relative to the variation
15
Statistical Thinking Lesson 7 Provide
Comparisons to Enable Gap Analysis
Relevant comparisons should be placed in the
appropriate locations on the graph
16
Statistical Thinking Lesson 8 Use Moving
Averages with Caution

• Helps visualize trends through the noise.
• Length should cover expected cycles. Annual is
  most common.
• Tend to be sluggish.
• Can generate the appearance of cycles or shifts
  which are not truly present.
• Cannot use run rules to signal special causes
• Control limits for moving averages can be
  calculated, but prefer not to place them on the
  graph itself.

17
The Headlines Scream - Great News!
18
What We All Imagine cause this is what
newspaper graphs look like
1995 1996 1997
1998
19
Reality
20
Statistical Thinking...

• A habitual way of looking at work that
• recognizes all activities as PROCESSES,
• recognizes that all processes have VARIABILITY,
• uses DATA to understand variation, and to drive
  effective DECISION MAKING.

P
P
21
Databased Decision Making

• Managers are routinely faced with interpreting
  their metrics and making a real-time decision as
  to whether the latest data point tells them to do
  something.

• Good graphical depiction goes a long way
• Seasoned managers can see signals through the
  noise
• Statistics can take the subjectivity out of such
  decisions
• One size does not fit all

22
Control Charts - The Two Mistakes

• The False Alarm - Interpreting noise as a signal
• The Missed Alarm - Failure to detect a signal

23
Control Charts in Data Rich Environments

• Control limits set at 3 standard errors
• Approximate 0.3 risk of a false alarm
• The risk of the missed alarm is often overlooked
• In parts manufacturing, greater sensitivity can
  be obtained by giving consideration to the
  selection of the rational subgroup

24
The Run of 8 Rule

• Sometimes 7 and sometimes 9
• Also provides low risk of false alarms
• Used with 3 standard error limits, sensitivity is
  improved
• Takes 8 points to initiate signal

Many other rules are also often used in the data
rich environment for greater sensitivity but the
tradeoff is a higher false alarm rate.
25
Average Run Lengths for Typical Data Rich
Environments
26
Average Run Lengths for Typical Data Rich
Environments(Reduced Scale)
27
Why These Rules Work for Data Rich Environments

• High false alarm rates would lead to wasted time
  doing investigation and possibly excessive
  process adjustments.
• Poor sensitivity is often an acceptable trade-off
  because for a lower false alarm rate
• And the next point is never far behind

28
Why Managerial Data Is Different

• The Obvious - less frequent data
• Detection of large process shifts is not as
  important
• Actions taken are different
• Improve mindset, not maintain

29
Traditional Rules Applied to Low Frequency
Managerial Data

• A shift of 1.5 standard errors takes eight points
  on average to detect
• This is little comfort if dealing with monthly
  managerial data

30
The Individuals Chart

• An excellent all-purpose tool
• Very robust - low false alarms for virtually any
  data distribution (typically lt 1)
• A single option for managers will get more use
• But, dont forget the poor sensitivity

31
Sensitivity for Managerial Data

• Data is usually individual observations (cannot
  subgroup)
• With traditional special cause rules, there is no
  control over risk of the missed alarm
• User can control the width of the control limits
• User can employ some modified run rules

These modifications do come with a higher false
alarm rate!
32
Alternative Special Cause Rule Sets

• A - Control limits set at 2.5 std. errors from
  the centerline.
• B - Control limits set at 2.5 std. errors from
  the centerline plus two points past 1.5
  standard errors.
• C - Control limits set at 2.0 std. errors from
  the centerline.
• D - Control limits set at 2.0 std. errors from
  the centerline plus a run of 6.
• E - Control limits set at 2.0 std. errors from
  the centerline plus three points past 1.0
  standard errors.
• F - Runs of 6 consecutive points on one side of
  the centerline.

33
Why Alternative Rules?

• Greater sensitivity is desired with an acceptable
  number of false alarms
• Whats acceptable? It depends (See Lesson 1)!
• The data frequency
• Time to do investigation
• Importance of detecting quickly
• Magnitude of change deemed important

34
Average Run Lengths for Alternative Rules - Chart
1
35
Average Run Lengths for Alternative Rules - Chart
2
36
Average False Alarms Per Year
37
Situational Recommendations
38
Change Point Analysis

• The general principle is Monte Carlo simulation
• Advantages include
• Very easy to use
• Detects mean and variation changes
• Excellent graphics

39
Change Point Analysis

• Confidence Levels for the probability a change is
  real
• Confidence Levels for when the change occurred.
• Handles any type of data
• More sensitive than control charts
• Not confused by outliers

40
Change Point AnalysisExample Graph
41
Change Point AnalysisExample Table
42
Change Point AnalysisSerial Dependency
Change Point Analysis of Accounts Receivable
440000
Accounts Receivable
330000
220000
Jan-1990
Dec-1990
Nov-1991
Oct-1992
Sep-1993
Aug-1994
Jul-1995
Jun-1996
Month/Yr
43
Conclusions

• Process thinking and careful selection of
  measures can help keep managers focused
• Appropriate plotting is 90 of the battle
• Traditional control charts may not be optimal
• Alternative special cause rule sets should be
  considered
• Change point analysis may be the closest thing to
  an all-purpose tool for managers.

44
REFERENCES   1) Stuart J, White K, Methods for
Handling Low Frequency Managerial Data, 2002 ASQ
Annual Quality Congress Proceedings 2) Balestrac
ci D, Data Sanity Statistical Thinking Applied
to Everyday Data, ASQ Statistics Division Special
Publication, Summer, 1998 (Available through ASQ,
www.asqstatdiv.org)   3) Britz G, Emerling D,
Hare L, Hoerl R, Shade J, Statistical Thinking,
ASQ Statistics Division Special Publication,
Summer, 1996 (Available through ASQ,
www.asqstatdiv.org)   4) Leitnaker, MG, Using the
Power of Statistical Thinking, ASQ Statistics
Division Special Publication, Summer 2000
(Available through ASQ, www.asqstatdiv.org)   5) W
heeler DJ, Understanding Variation The Key to
Managing Chaos. Knoxville, TN SPC Press, Inc.
1986. (www.spcpress.com)   6) Taylor WA,
"Change-Point Analysis A Powerful New Tool For
Detecting Changes," WEB www.variation.com/cpa/tec
h/changepoint.html, 2000   7) Wheeler DJ,
Building Continual Improvement, Knoxville, TN
SPC Press, Inc., 1998. (www.spcpress.com)