The Trials and Tribulations of BenchMarking

1
The Trials and Tribulations of BenchMarking

• Joanne Kossuth, Olin College of Engineering
• 10/25/2006

2
Meaning

• What does BenchMarking mean to you?
• Do you get that queasy feeling?
• Do you get an empowered feeling?
• Do you hate the word?
• Do you use it as a tool or do it because you have
  to?

3
Definition

• The process of setting benchmarks, which means
  identifying accurate historical data against with
  a data set can be compared now and in the future.
  In some cases the term has come to mean goals,
  which confuses participants in dialogue about
  indicators.indicators.top10by2010.org/glossary.cf
  m

4
Definition

• A continuous process of measurement of products,
  services and work processes, against those
  recognised as leaders.thequalityportal.com/glossa
  ry/b.htm

5
Definition

• The process of re-estimating statistics as more
  complete information becomes available. Estimates
  are usually calculated using only a sample of the
  universe (total count). Therefore, benchmarking
  allows for correction of estimating errors. New
  benchmarking levels are introduced on an annual
  basis.www.vec.virginia.gov/vecportal/lbrmkt/gloss
  ary.cfm

6
(No Transcript)
7
The Top Missteps

• Mistaking participating in a survey for
  benchmarking
• Assuming there are pre-existing benchmarks for
  everything you want to measure
• What about customer service satisfaction and
  delivery of service?

8
Missteps, Cont.

• The process versus the system
• Benchmarking ? research
• Misalignment of benchmarking and business process
• Measuring the unmeasurable
• Missing a baseline?
• Researching partners and executing agreements on
  conduct

9
The Top Benchmarkers

• Bank of America
• Xerox
• TRW Automotive
• U.S. Department of Veterans Affairs
• U.S. Army

10
Top Things to Benchmark

• Information technology systems
• Customer service satisfaction
• Human Resources (HR)
• Employee benefits compensation programs
• Employee development/training
• Process improvement management
• Facility management

11
The Olin Example

• A consultant was hired to look at the college
  from an administrative standpoint only
• The consultant was to make suggestions for
  consolidation, cost cutting and enhanced fiscal
  responsibility
• A concurrent strategic advisory initiative
  internal to the college was also in progress

12
The Olin Example

• The college spent a lot of time and effort
  producing documentation of services
• The consultants provided report with various
  benchmarks without defining the source of the
  benchmarks
• Olin was over on virtually every benchmark in
  terms of quality
• The report was to be presented to the Trustees as
  well as the entire community

13
The Olin Example

• The college was expected to take action on the
  items identified by the consultants
• Does this sound like a familiar scenario?
• What do you think were the pros and cons of this
  process?
• What data would you have wanted to collect and
  understand?

14
Resources

• Resources for benchmarking
• Gartner
• Forrester
• IDC
• ECAR
• Institutional Information
• Consortial Surveys
• Anectodal Stories..and

15
Pros and Cons

• Consider the source corporate, non-profit or
  combination
• Consider the funding mechanism
• Consider the agenda
• Consider the collector of the information
• Consider the questions (validated or not?)

16
Effective Tools

• What have you found to be effective tools?
• Do you use TQM, 6-Sigma or other methodologies?

17
Resources

• http//management.about.com/gi/dynamic/offsite.htm
  ?zi1/XJsdnmanagementzuhttp3A2F2Fwww.benchm
  arkingplus.com.au2Fmistakes.htm
• Benchmarking PLUS, Level 6, 443 Little Collins
  Street, Melbourne, VIC 3000, Australia
• http//www.benchnet.com/bppf2003e.cfm

18
Resources

• http//govinfo.library.unt.edu/npr/initiati/benchm
  k/index.htm
• http//www.apqc.org/portal/apqc/sitejsessionid1V
  B1PGBRTDNTNQFIAJICFEQ

19
Contact Information

• Joanne Kossuth
• Associate Vice President Development and CIO
• 781-292-2431
• Joanne.kossuth_at_olin.edu