Metrics and the Cost of Quality

1
Metrics and the Cost of Quality
Douglas M. Stewart, Ph.D. The Anderson Schools of
Management University of New Mexico
2
Why Metrics!

• A strategy without metrics is just a wish. And
  metrics that are not aligned with strategic
  objectives are a waste of time.
• Emery Powell
• Be careful what you measure -- you might just get
  it.
• If you dont keep score, youre only practicing.
• Tom Malone
• You get what you inspect, not what you expect.

3
The Real Bottom Line Mission and Measures

• One of the most powerful management disciplines,
  the one that more than any other keeps people
  focused and pulling in the same direction, is to
  make an organizations purposes tangible.
  Managers do this by translating the
  organizations mission what it, particularly,
  exists to do into a set of goals and
  performance measures that make success concrete
  for everyone. This is the real bottom line for
  every organization whether its a business or a
  school or a hospital. Its executives must answer
  the question, Given our mission, how is our
  performance going to be defined?
• Magretta Stone, Management. 2002, p. 129

4
Why Metrics Now!

• The never satisfied customer.
• Managing the total supply chain.
• Shrinking product life cycles.
• More (not necessarily better) data.
• Profit margin squeezes.
• Presence of an increasing number of alternatives.

5
So What

• Surviving in this new environment means working
  with
• Less lead time
• Less inventory
• Less cost
• More reliance on the supply chain
• Surviving means having better metrics.

6
What are Metrics?

• A verifiable measure stated in either
  quantitative (e.g., 95 inventory accuracy) or
  qualitative (e.g., as evaluated by our
  customer,we are providing above average service)
  terms. A metric is intended to close the gap
  between value, strategy, and specific activities.
• Metrics
• Measure, direct, teach.

7
Metrics The Firms Instrument Cluster
8
Metrics Categories

• Metrics
• Individual measures.
• Metrics Set
• Number of metrics used to evaluate the
  performance of an individual or unit.
• Metrics System
• System for integrating, coordinating and
  alignment metrics, objectives, and plans.

9
TYPES OF METRICS

• Individual vs. Systems
• Operational vs. Financial
• Utilization vs. Internal Rate of Return (IRR)
• Traditional focus - financial
• Critical issue
• Operational improvements do not always result in
  financial results
• Output vs. Process
• Internal to the firm vs. supply chain metrics

10
Types of Metrics

• Product/Output metrics
• Measure results, not the performance of the
  processes that gave rise to this performance.
• Process metrics
• Pertains to the performance of the process
• Typical measures -- cost, time, quality, agility
  (the ability to adapt), capacity, variability

11
Output vs. Process Metrics

• You have to drive to Roswell. You would like to
  leave by no later than noon. You need to arrive
  at no later than 300 pm.
• Output metric - 3 PM arrival time met
• Process metrics
• departure time - 100 PM
• average speed - 85 mph

12
Importance of Process Metrics

• Experience is a wonderful thing. It enables you
  to recognize a mistake when you make it again -
  and again - and again. Process metrics enable
  you to recognize a potential mistake before you
  make it.

13
Key Point

• Output metrics - critical to top management
• Process metrics - critical to those working in
  execution

14
Metrics Process

• I - Intent (what you want to achieve and why?)
• D - Deployment (how do you want to achieve this
  goal?)
• A - Assignment (who will be held accountable?)
• S - Standard (how will you measure achievement?)
• Issues of form, source, feasibility,
  appropriateness.
• E - Evaluation and Reward

15
I.D.A.S.E. Process in Detail

• Intent
• Desired outcome
• What is it that we are trying to measure?
• Why is it that we are measuring this?
• How does this process/outcome support our
  objectives at the functional level? At the
  corporate level?
• What would happen if we were to fail on this
  dimension?
• Who are the champions for this outcome?

16
I.D.A.S.E. Process in Detail

• Deployment
• How are we going to achieve this outcome?
• What type of data do we need to effectively and
  accurately implement this metric?
• Do we have this data? If not, what proxy data
  can we use? Impact of the proxy data?
• Will this be implemented as a predictive or
  outcome metric?
• Impact of the metric on the metric set?
• Is this metric similar to any others currently
  in use?

17
I.D.A.S.E. Process in Detail

• Assignment
• Recognize that there are two different people
  involved in a metric
• Those who monitor the outcomes
• Those who manage the processes that create the
  outcomes
• There are different behaviors
• Scanners -- large number of metrics-- looking for
  gaps
• Searchers - deal with fewer metrics -- interested
  in problem solving

18
I.D.A.S.E. Process in Detail

• Standard
• The basis of comparison
• Defines what is acceptable (and unacceptable)
  behavior
• Different types of metrics
• Can be generated from the benchmarking process
• Can and should be dynamic

19
Standards

• Absolute
• EPA
• Few in number
• Relative
• Relative to our performance
• Past performance
• Best possible future performance
• Pros and cons

20
Standards - 2

• Relative
• Relative to best within group
• Need to develop appropriate group
• Best within the group teach others
• Worse within the group learn from others
• Group membership must be viewed as dynamic
• Relative to a benchmark
• Best in firm
• Best in industry
• Best in class

21
FACCritical Traits of Standards

• F - Feasible
• A - Appropriate
• C - Comparable

22
An Example of Metrics in Action

• OEE
• Operational Equipment Effectiveness
• Availability Performance Quality
• Note what we are measuring
• Quality
• Uptime
• Cost
• Note what we are not measuring
• On-time delivery

23
Metrics Set

• The set of measures that we use to monitor or
  evaluate the performance of a specific person or
  function or activity.
• Metrics set brings together two elements
• Set of metrics
• Load
• Capacity to interpret or act on the measures
• Capacity
• Metrics set recognizes that the process is
  capacitated.

24
Metrics Set

• Ideal number of measures
• 5 /- 3
• If we overload the manager, then we get
• Frustration
• Confusion
• Satisficing
• Negotiation
• Cherry picking

25
Metrics System

• Metrics System
• Total system responsible for coordinating and
  aligning metrics
• Alignment
• Top to bottom
• Coordination
• Across functions
• Why are Alignment and Coordination so important?
• The lessons of the telephone game.

26
Alignment and Coordination

• Alignment occurs first
• Coordination follows
• Alignment involves a translation problem
• Restating higher order objectives into supporting
  activities for the lower levels
• As we go from top to bottom, objectives narrow
  and level of details increase
• Challenges
• Translation gets messed up
• Multiple bosses

27
Attaining Alignment

• Approaches
• Bill of Material Approach
• Balanced Scorecard Approach
• Theory of Constraints Approach
• Strategic Profit Impact Model

28
Approaches for Managing the Metrics System

• Bill of Material approach
• Explosion of metrics from the top down
• Converting each higher level outcome into a
  series of lower level outcomes that must be
  achieved to ensure that the higher level outcome
  is met.
• Problems
• Balanced Scorecard
• Approach
• Problems

29
Balanced Scorecard Approach
30
Approaches for Managing the Metrics System

• Theory of Constraints Approach
• Must look at the entire system and identify the
  constraint
• Now measure the performance of the constraint
• The constraint drives the performance of the
• Firm
• Supply chain
• Constraint must always be working
• Measure this

31
Approaches for Managing the Metrics System

• Theory of Constraints - continued
• Elements
• Profit (P)
• Operating Expenses (E)
• Throughput Time (T)
• Inventory (I)

32
Strategic Profit Model
33
Recent Findings

• Metrics are a source of fear
• Different groups perceive metrics differently
• Resolution strategies
• Negotiation (group optimization)
• Satisfaction of minimum targets (order
  qualifiers)
• Decompose the problem into sub-problems (silo
  management)
• Treat everything as important (cope as you can)
• Find new metrics we can agree upon
• Ignore most and focus on what I can do well
• Seek divine guidance

34
Recent Findings

• Metrics must be managed as a system
• Metrics for motivation and those for process
  control are different
• Global vs. local
• Metrics explosion
• Ceteribus Paribus

35
Metrics Framework
36
Traits of Effective Metrics

• Focused on the critical few, not trivial many
• Linked to value
• Measure directly
• Appropriate display
• Appropriate frequency of updating
• Acceptable time lags between generation of metric
  and report
• Appropriate standard used

37
The Cost of Quality Metrics

• Types of Quality Costs
• Prevention
• Appraisal
• Internal Failure
• External Failure

38
Classifying Quality Costs
39
1-10-100 Rule

• Theory Says
• Prevention 1
• Inspect and Prevent 10
• Given to Customer 100

40
1-10-100 Rule

• Practice Shows
• IBM Rochester A400 Server
• 1-13-92 (hours work required)
• Corning Glass Glass pressings
• 1-80-400 ( cost)
• Texaco Refinery
• 1-10-100 ( cost)
• Cost of Customer Defections
• Companies can boost profits 25 85 by
  decreasing defections 5

41
Cost at Points in the Process
42
Use of a Cost Basis

• Why?
• Allow comparison across different periods
• Typical Bases
• Internal failure as a percent of total production
  costs
• External failure as an average percent of net
  sales
• Procurement appraisal costs as a percent of total
  purchased material costs
• Operations appraisal costs as a percent of total
  production costs
• Total quality costs as a percent of production
  costs

43
Quality Cost Basis Example
44
Quality Cost Basis Solution
45
Use of the System

• Categorizing costs
• Estimating true external failure costs
• Application of cost bases
• Tracking improvement
• ROQ