The Human Side of Metrics

1
The Human Side of Metrics
Metricon 1.0 Vancouver, Canada August 1, 2006
Dennis Opacki, CISSP QDSP dopacki_at_covestic.com Cov
estic, Inc.
2
How do organizations use metrics?

• Most purposes cited reduce to (Hauser Katz
  1998)
• Metrics enable firms to take stock of where they
  are, and help them plan for the future.
• Metrics provide estimates of future performance.
• Managers use metrics to allocate assets and
  select strategies.
• Metrics form the basis for bonuses and
  promotions.
• Even measures derived from automated systems
  reflect and influence human behavior.

3
Common metrics pitfalls

• Goal displacement (Kerr, 1975)
• Means becomes the ends in themselves.
• People seek rewards to the exclusion of all
  non-rewarded behavior.
• Unintended consequences
• Metrics affect decisions even if those decisions
  inadvertently sacrifice long-term benefits.
  (Hauser Katz, 1998)
• Broken reward systems encourage behavior
  organizations are trying to discourage. (Kerr,
  1975)
• Passive or active resistance
• Staff will find ways to avoid metrics they
  oppose. (Neal, 2006)
• Zone of indifference a person can and will
  accept a communication as authoritative only
  when at the time of his decision, he believes it
  to be compatible with his personal interests as a
  whole. (Kerr, 1975)

4
Factors contributing to failure

• Evolutionary psychology (Economist, 2005)
• Humans are hard-wired not for logic, but for
  detecting injustice.
• People will even forego their own self-interests
  to punish those behaving unfairly.
• Social psychology (Neal, 2006)
• Locus of control
• Internal Sense of self-reliance master of
  own destiny
• External Sense of helplessness external
  parties are in control
• Fundamental attribution errors incorrectly
  assuming bad behavior is irrational or malicious
• Diffusion of responsibility bystander effect
• Everyone believes someone else is better suited
  to act.
• This can occur if responsible parties lacks
  authority to affect change.
• Reciprocity norm Whats in it for me?

5
Factors contributing to failure (cont.)

• Behavioral Economics (Kahneman, 2003)
• Intuition vs. reason people dont think very
  hard
• Intuition uses heuristics, prototypes and
  predictions fast and parallel.
• Reason is slow and single-threaded.
• Natural attributes enhance intuition size,
  color, valence.
• Good/bad can be substituted for any attribute.
• Default options have a natural advantage.
• Prospect theory
• People anticipate emotions associated with
  changes in state.
• People are willing to ignore scope when presented
  with definite solutions to emotional problems.
• Framing
• How we present situations greatly influences
  outcomes.
• Multiple frames increase the chance of specific
  outcomes.

6
Improving chances for success

• Select a few good metrics
• Focus on attributes and scales that people gauge
  intuitively - good/bad, like/dislike, percentile.
• Use a small set of metrics to keep time pressure
  from driving cognitive processes from reason to
  intuition. (Kahneman, 2003)
• Survey employees to understand which behaviors
  are actually being rewarded. (Hauser Katz,
  1998)
• Express metrics in dollars research shows this
  improves peoples ability to assess probability.
  (Kahneman, 2003)
• Use clear metric names and descriptions to
  increase accessibility of metrics traits.
  (Kahneman, 2003)
• Present them well
• Dont ignore entertainment value. (Pijpers
  Montfort, 2006)
• Drill-down is key to getting middle-management
  buy-in. (Gartner, 2005)
• Slicing and dicing bells and whistles
• Use common visualizations to improve mental
  prototyping.
• Give bad news first - peak/end rule (Kahneman,
  2003)

7
Works Cited

• 1 Buytendijk, F. and Gassman, B. (2005).
  Management Update Just Give Me a CPM Dashboard.
  Gartner whitepaper.
• 2 Hauser, J. R. and Katz, G. M. (1998)
  Metrics You are what you measure! European
  Management Journal, 16(5).
• 3 Kerr, Steven (1975).On the Folly of Rewarding
  A, While Hoping for B. Academy of Management
  Journal. 18(4).
• 4 Kahneman, Daniel (2003). Maps of Bounded
  Rationality Psychology for Behavioral Economics.
  The American Economic Review. 93(5).
• 5 Neal, Russ. (2006). Social Psychology
  Variables that Contribute to Resistance to
  Security Assessment Findings. Information Systems
  Security, 15(1).
• 6 No Author Cited, (2005). Survey The concrete
  savannah. The Economist, 377(8458).
• 7 Pjipers, G.G.M. and van Montfort, K. (2006).
  An Investigation of Factors that Influence Senior
  Executives to Accept Innovations in Information
  Technology. International Journal of Management.
  23(1).