Measurement

1
Measurement
2
Measurement

• Who has been involved in
• Taking measures?
• Determining what is to be measured?
• Evaluating measures to make decisions?
• What are some of the obstacles in taking
  measures?
• What are some of the challenges in taking
  measures?

3
Transform Enterprise Methodology
Develop Vision and Plan
Plan for change
Create Desired Culture
Improve and Integrate Processes

• A Passion for
• Better
• Faster
• Cheaper

Develop Technology Solutions
4
Improve Integrate Processes
Understand and Improve the Product (2)
Understand the Customer (1)
Understand and Improve the Process (3)
Design Implement Effective Controls (4)
5
Understand and Improve the Process
Bound Process and Identify Relationships (1)
Document and Analyze Process (2)
Implement Improved Process (4)
Design Improved Process (3)
6
Understand and Improve the Process -
Implementation Methodology

• 1. Identify all major processes - List all the
  major processes the organization performs to
  serve the customer (what do we do as a
  department, organization, etc)
• 2. Identify opportunities - identify those
  processes that provide the most "bang for the
  buck" or the greatest chance of producing
  successful results that will benefit the company
• 3. Bound the process Determine a starting and
  ending point of the process (use Process
  Identification Worksheet)
• 4. Identify relationships Determine how the
  process interacts with other processes inside and
  outside the organization. (Perform SIPOC
  (Suppliers, Inputs, Process, Outputs, Customers)
  analysis)
• 5. Document the process - Develop a flowchart of
  the process, define all activities and identify
  resources required to perform the process
• 6. Determine the current condition of the process
  - Collect data about the process. Determine what
  data to collect, how to collect it, when, etc

7
Understand and Improve the Process -
Implementation Methodology

• 7. Stabilize the process Make adjustments to
  the process such that it produces consistent
  results
• 8. Determine what customers want from the process
  - Ask customers what they want from the process
  to guide process improvement goals and strategies
• 9. Identify problems with current process -
  Prepare cause and effect diagrams to reveal root
  causes
• 10. Design a new process - Make changes to the
  process based on what customers want and problems
  that were identified
• 11. Implement the new process - Prepare an action
  plan for testing and implementing the new process
• 12. Determine the impact of the changes on the
  process - collect data about the new process and
  continuously improve the process

8
Agenda

• Why do we measure?
• What should we measure?
• How can we get started?

9
Why Do We Measure?

• To control?
• To budget?
• To gain visibility?
• To comply?
• To satisfy someone else's needs?
• To solve problems?
• To improve?
• It's what we've always done?

10
Why Do We Measure?

• To change or prevent change
• To monitor progress against plans
• To make decisions and take actions

11
Measurement as an Improvement Device
Measurement
Improvement
Evaluation
Control

• The most important role of measurement is
  improvement.

12
Obstacles to Establishing Measurements

• Performance measurement is threatening
• Benefit not worth the investment
• Management's focus on a single indicator
• Labor productivity over emphasized
• Precision considered essential
• Subjective measures believed to be sloppy
• Standards interpreted as performance ceilings

13
Obstacles to Establishing Measurements

• Performance measurement threatening
• Benefit not worth the investment
• Management's focus on a single indicator
• Labor productivity over emphasized
• Precision considered essential
• Subjective measures believed to be sloppy
• Standards interpreted as performance ceilings

14
Agenda

• Why do we measure?
• What should we measure?
• How can we get started?

15
What Should We Measure?

• Let the system decide
• Let management decide
• Let the industrial engineer decide
• Let a consultant decide
• Let the competition decide
• Let the customer decide
• The easy answers

16
Performance Measurement Criteria

• Directly related to business plans and processes
• Primarily non-financial
• Vary between locations
• Change with requirements
• Simple and easy to use
• Fast feedback
• Foster improvement

17
Performance Criteria
If the organization is
It will be
If it supports
It will be
Effective
Innovation
Efficient
Productive
Profitable
QWL
Quality
Achieving Balance
18
Effectiveness

• Accomplishment of the right things,
• on time, within the requirements specified

Value-Adding Transformation
Customers
Inputs
Suppliers
Outputs
Effectiveness
19
Efficiency

• Resources expected to be consumed divided by
  resources actually consumed

Value-Adding Transformation
Customers
Inputs
Suppliers
Outputs
Efficiency
20
Quality

• ". . . a predictable degree of uniformity and
  dependability, at a low cost, suited to the
  market."
• W. Edward Deming (1986)
• ". . . fitness for use, as judged by the user."
• Joseph Juran (1989)
• ". . . conformance to requirements"
• Phillip Crosby (1979)
• ". . . loss imparted to the society"
• Genich Taguchi

21
Quality

• A measure of the extent to which a product or
  experience adds value to society

Defined by the customer!
22
Quality
Defined by the customer!
Value-Adding Transformation
Customers
Inputs
Suppliers
Outputs
Quality
Quality
Quality
Quality
Quality
23
Productivity

• Measures of output divided by measures of input
  for a given period of time

Value-Adding Transformation
Customers
Inputs
Suppliers
Outputs
Productivity
24
Innovation
The creative process of change
Value-Adding Transformation
Customers
Inputs
Suppliers
Outputs
Innovation
25
Quality of Worklife

• The way people respond to the socio-technical
  aspects of the system

Value-Adding Transformation
Customers
Inputs
Suppliers
Outputs
QWL
26
Profitability
Relationship between revenues and costs
Value-Adding Transformation
Suppliers
Outputs
Outcomes
Inputs
Customers
Profitability
27
Performance Criteria
If the organization is
It will be
If it supports
It will be
Effective
Innovation
Efficient
Productive
Profitable
QWL
Quality
Achieving Balance
28
Agenda

• Why do we measure?
• What should we measure?
• How can we get started?

29
Getting Started
Organize for Improvement
Develop Measures
Establish Measurement System
Collect Data Required
Validate the Results
Link to Improvement
30
Organize for Improvement

• Form measurement development team
• Define the target system
• Create suitable environment
• Acceptance on part of users
• Availability of time for development
• Maturity (ability and willingness) of users

31
Develop Measures

• Conduct an audience analysis
• Select the measures based on what is needed to
  make a decision
• Audit the measures
• Break down the measures

32
Audience Analysis

• Who needs the information?
• Why do they need it?
• What information is required?
• How frequently must it be collected?

33
Audience Analysis
Other Audiences
Who Manages
What is used to Manage
What is Managed
34
Select Measures

• Existing measures
• Performance criteria
• Strategic driving forces
• Business objectives
• Organizational system

35
Guidelines For Selection

• Related to customer value
• Developed by people who use them
• Simple and understandable
• Few in number
• Visible
• Show whether process is working
• Linked to strategy

36
Meaningful Measures

• Fully document the process
• Identify and understand customer value
• Involve team in developing measurements
• Relate process characteristics to customer value
• Are simple ways to measure process
• Communicate the measurements to all members

37
Audit Measures
Measures
Customer Retention
Process Capability
Process Definition
Processes Defined
On-Time Delivery
Product Cost
Turnover
Cycle Time
Quotes
Audit
Waste
WIP
ROI
Factors
Strategic Goals
Strategic Goals
Process Integration
Capable work force
Market Presence
Performance Criteria
Effectiveness
Efficiency
Quality
Productivity
QWL
Innovation
Profitability
38
Breakdown the Measures

• Develop a count for the measure
• Identify indicator relative to measure
• Use proxy when direct measurement impossible
• Operationalize the definitions of measures
  identified

39
Failures of Measurements

• Measuring "A" while hoping for "B"
• Focusing on control of excess
• Measuring to find poor performers

40
Establish Measurement System

• Capturing
• Storing
• Retrieving
• Processing
• Portraying

41
Collect Data Required

• Identify data sources
• Eliminate infeasible measures
• Devise forms and logs
• Check accountability
• Begin collecting

42
Validate The Results

• Is the output accurate?
• Is the format correct?
• Is the information useful?
• Is the information timely?

43
Link to Improvement

• Use control charts
• Promote cooperation
• Guide to problem-solving
• Link to reward system
• Establish visible systems

44
Summary

• Measurements drive decisions and actions
• Measure what's important - not what's easy
• Acceptance of measurement process is essential
• Adopt an experimental approach to measurement
• Measurement does not drive performance
  improvement. The driver must be the business
  plan!

45
Performance Measurement

• "Simple, visible measures of what's important
  should mark every square foot of every department
  in every operation. Place special emphasis on
  developing measures associated with revenue
  generation, supplanting the current system bias
  toward cost containment. Every manager should
  track no more than three to five variables which
  capture the essence of the business."

Tom Peters Thriving on Chaos
46
Measurement Concepts

• "The essence of management is that one cannot
  manage that which one cannot measure."

Scott Sink Planning and Measurement in Your
Organization of the Future
"If you can't measure it, you can't fix it."
Ed Northern Interturbine Dallas
47
Chapter 8

• Performance Measurement and Strategic Information
  Management

48
Information Management

• If you dont measure results, you cant tell
  success from failure
• If you cant see success, you cant reward it
  and if you cant reward success, you are probably
  rewarding failure
• If you cant recognize failure, you cant correct
  it

49
Florida Power and Light Example

• FPL thought that lightning was the principle
  cause of service interruptions, but did not
  collect data to verify the situation
• After collecting data, they discovered that
• They had service interruptions without lightning
• Many poles did not have a sufficient groundings
• Lessons learned?
• Many companies assume they know the cause of
  their problems based on their experience, which
  can be misleading without supporting data

50
The Strategic Value of Information

• To lead the entire organization in a particular
  direction, that is, to drive strategies and
  organizational change
• To manage the resources needed to travel in this
  direction by evaluating the effectiveness of
  action plans
• To operate the processes that make the
  organization work and continuously improve

51
Process Flow
Measurement
Data
Analysis
Information
52
Use of Information and Analysis
Measurement supports executive performance review
and daily operations and decision making.
53
Benefits of Information Management

• Understand customers and customer satisfaction
• Provide feedback to workers
• Establish a basis for reward/recognition
• Assess progress and the need for corrective
  action
• Reduce costs through better planning

54
Empirical Survey Results

• Measurement-management companies are more likely
  to
• be in top third of industry financially
• complete organizational changes successfully
• reach clear agreement on strategy
• enjoy favorable cooperation and teamwork
• have more employee empowerment
• have a greater willingness to take risks

55
Example Federal Express

• We measure everything. Thenwe prioritize what
  processes are key to the company.
• Most data collection systems are automated,
  making it fast and easy.
• Seeks internal measures that are predictors for
  external measures.

56
Example Ritz-Carlton

• We only measure what we must. But, we make sure
  that what we measure is important to our
  customers.
• 50 marketing and financial data 50
  quality-related productivity data.
• Cost of quality is top priority. Are
  improvements important to customers, providing a
  good return, and done quickly?

57
Leading Practices

• Develop a set of performance indicators that
  reflect customer requirements and key business
  drivers
• Collecting data that no one uses or wants wastes
  resources
• Use comparative information and data to improve
  overall performance and competitive position
• Involve everyone in measurement activities and
  ensure that information is widely visible

58
Leading Practices

• Ensure that data are reliable and accessible to
  all who need them
• Provide rapid access to data and information to
  all employees who need the information
• The quality of a decision is often a function of
  the information available within the time frame
  in which the decision must be made.
• In my experiences, a lack of information is a
  prime reason for decisions that are
• Right - but too late
• Wrong - lack of data

59
Leading Practices

• Providing rapid access to information can be
  expensive, but what is the cost of not having the
  right info when it is needed?

60
Leading Practices

• Use sound analytical methods to conduct analyses
  and use the results to support strategic planning
  and daily decision making
• Data can provide an indication of important
  cause-and-effect relationships
• Continually refine information sources and their
  uses within the organization

61
Balanced Scorecard

• Financial perspective
• Internal perspective
• Customer perspective
• Innovation and learning perspective

Leading measures
Lagging measures
62
Causal relationships Among Categories of IBM
Rochesters Balanced Scorecard
Products and Channels
Customer Satisfaction and Loyalty
Financial and Market share

• People and Skills

Product and service quality
63
Key Types of Business Performance Measures

• Customer satisfaction measures
• Financial and market performance measures
• Human resource measures
• Supplier and partner performance measures
• Company-specific measures

64
Example Wainwright Industries

• Safety
• Internal customer satisfaction
• External customer satisfaction
• Six sigma quality (manufacturing defects)
• Business performance

65
Common Quality Measures

• Nonconformities (defects) per unit
• Errors per opportunity
• Defects per million opportunities (dpmo)

66
Importance of Comparative Data

• Comparative data industry averages, best
  competitor performance, world-class benchmarks
• Helps recognize the need for improvement
• Provides motivation to seek improvement

67
Designing Effective Performance Measurement
systems

• How will the measures support
• Senior executive performance review
• Organizational planning to address the overall
  health of the organization
• Daily operations and decision making
• To answer these questions, the organization must
• Align its measurement system to its vision and
  strategy
• Select meaningful process-level measurements

68
Fundamental Mistakes

• Not measuring key characteristics critical to
  company performance or customer satisfaction
  thus - organization fails to meet customer
  expectations
• Taking irrelevant or inappropriate measurements
  thus - directs attention to areas that are not
  important to customers
• Using measures that have been around a long time
  and no one knows why they are being used thus
  do not take measures that are not used to make
  decisions

69
Linking Measures to Strategy

• An organization first needs to understand its
• Competitive environment
• Market requirements
• Internal capabilities
• Once these areas are understood, the organization
  must
• Identify key business drivers (key success
  factors) they are things that separate an
  organization from its competition and defines
  strengths to exploit or weaknesses to correct.

70
Linking Measures to Strategy

• What are some examples of KBD?
• For MBNA speed of service
• What should they measure?
• Time to process customer address changes
• Percentage of times phones are picked up within
  two rings
• The time take to transfer calls from the
  switchboard

71
Linkages to Strategy
Key business drivers (key success factors)
Strategies and action plans
Measures and indicators
72
Process-Level Measurements

• Does the measurement support our mission?
• Will the measurement be used to manage change
  that is, actionable?
• Is it important to our customers?
• Is it effective in measuring performance?
• Is it effective in forecasting results?
• Is it easy to understand and simple?
• Are the data easy/cost-efficient to collect?
• Does the measurements have validity, integrity,
  and timeliness?
• Does the measure have an owner?

73
Process-Level Measurements

• Good measures and indicators are actionable
  they provide the basis for decisions at the level
  at which they are applied
• Consider the Pizza Ordering and Filling Process
  on pp.466-467
• Note figure 8.6 p.468

74
Creating Effective Performance Measures

• Identify all customers and their requirements and
  expectations
• Define work processes
• Define value-adding activities and process
  outputs
• Develop measures for each key process
• Evaluate measures for their usefulness

75
The Cost of Quality (COQ)

• COQ the cost of avoiding poor quality, or
  incurred as a result of poor quality
• Translates defects, errors, etc. into the
  language of management
• Workers speak the language of things which
  includes units, defects, ect
• Helps management evaluate the relative importance
  of quality problems and thus identifying
  opportunities for cost reduction
• Aid in budgeting and cost control
• Serve as a scoreboard to evaluate success in
  achieving quality objectives

76
Quality Cost Classification

• Prevention
• Appraisal
• Internal failure
• External failure

77
Quality Cost Classification Prevention

• Quality planning costs
• Process control costs
• Information systems
• Training and general management costs

78
Quality Cost Classification Appraisal Costs

• Test and inspection costs
• Instrumentation maintenance costs
• Process measurement and control costs

79
Quality Cost Classification Internal Failure
Costs

• Scrap and rework costs
• Costs of corrective action
• Downgrading costs
• Process failures

80
Quality Cost Classification External Failure
Costs

• Costs due to customer complaints and returns
• Product recall costs and warranty claims
• Product liability costs

81
Quality Cost Classification

• 60-90 of total quality costs are the results of
  internal and external failure
• Increase in prevention usually generates larger
  savings in all other categories

82
Quality Cost Classification Prevention Example

• 500 cost of replacing a poor-quality component
  in the field
• 50 cost of replacement after assembly
• 5 Replacement during assembly
• .50 cost of changing the design to avoid the
  problem

83
Quality Cost Management Tools

• Cost indexes
• Pareto analysis
• Sampling and work measurement
• Activity-based costing

84
Cost Indexes

• Cost indexes a ratio of the current value to a
  base period
• Index numbers increase managers understanding of
  the data particularly how conditions in one
  period compare with those in other periods

85
Cost Indexes
86
Return on Quality (ROQ)

• ROQ measure of revenue gains against costs
  associated with quality efforts
• Principles
• Quality is an investment
• Quality efforts must be made financially
  accountable
• It is possible to spend too much on quality
• Not all quality expenditures are equally valid

87
Managing Data and Information

• Validity Does the indicator measure what it
  says it does?
• Reliability How well does an indicator
  consistently measure the true value of the
  characteristic?
• Accessibility Do the right people have access
  to the data?

88
Analysis

• Statistical summaries and charts
• Trends over time
• Comparisons with key benchmarks
• Aggregate summaries and indexes
• Cause-and-effect linkages and correlations
  (interlinking)
• Data mining

Basic
Advanced
89
Interlinking

• Quantitative modeling of cause and effect
  relationships between external and internal
  performance criteria

90
Information and Analysis in the Baldrige Award
Criteria

• The Information and Analysis Category examines
  an organizations information management and
  performance measurement systems and how the
  organization analyzes performance data and
  information.
• 4.1 Measurement and Analysis of Organizational
  Performance
• a. Performance Measurement
• b. Performance Analysis
• 4.2 Information Management
• a. Data Availability
• b. Hardware and Software Quality

91
Measurement Discussion Questions

• Why is measurement difficult?