HPI Case Studies

1
ASTD 2009 International Conference and
ExpositionWashington, D.C.
M121 Evaluating Human Performance Improvement
A New Approach

• Learning Objectives
• Use cutting-edge approaches to evaluate and
  measure performance improvement solutions to
  ensure the results your clients/organization
  expect.
• Apply the HPI evaluation model to your own HPI
  projects.

Donald J. Ford, Ph.D., C.P.T., President Training
Education Management LLC 1874 Pacific Coast
Highway Redondo Beach, CA 90277 1-310-316-2240 in
fo_at_trainingeducationmanagement.com www.trainingedu
cationmanagement.com
2
Biography

• Donald J. Ford, Ph.D., C.P.T., is president and
  owner of Training Education Management LLC, a
  training and performance improvement consulting
  firm in Redondo Beach, California, USA.
• His consulting clients include ASTD, Toyota,
  Nissan, Rockwell International, U.S. Marines
  Corps, Malaysia Institute of Training and
  Development, Egyptian Central Bank and Sempra
  Energy among others. He has worked in the field
  of human resource development for 23 years,
  including training management positions at Sempra
  Energy, Magnavox, Allied-Signal and Texas
  Instruments.
• He holds a B.A. and M.A. in history and a Ph.D.
  in education, all from UCLA.
• He has published 35 articles and four books on
  topics in training, education and business. His
  books include In Action Designing Training
  Programs (ASTD, 1996), and Bottom-Line Training
  Performance-based Results (TEM, 2005).

3
Kirkpatrick Evaluation Model
Training Environment

• Level 1 Reactions
• Learner
• Client

• Level 2
• Learning
• Learner
• Organization

Learning Event
Work Environment

• Level 4
• Results
• Performance
• Strategic

• Level 3
• Job Behavior
• Learner
• Organization

• Level 5 (Phillips)
• ROI
• Financial

4
Typical Evaluation Points
Analyze
Design/ Develop
Implement
Evaluate

• Evaluation Design
• Tests
• Surveys
• Formative Evaluation

• Skills Transfer
• Performance
• Business
• Results
• Data Analysis
• Evaluation Reports

• Baseline Data
• Learners Performance
• Skill Gaps
• Business Measures

• Reactions
• Pre-Test
• Post-Test

5
Two Types of Evaluation

• Formative
• Improve the process the program
• Focus on process
• Present time
• Measures
• Acceptance
• Satisfaction
• Quality

• Summative
• Sum up the impact, Go-No Go
• Focus on results
• Past time
• Measures
• Performance change
• Business results
• ROI

6
HPI Evaluation Model
HPI Process
Business Analysis
Intervention Implementation
Performance Analysis
Cause Analysis
Intervention Selection
Evaluation
Business Goals
Performance Gaps
Root Causes
Project Management
HPI Outputs
Change Management
Solutions
Evaluation Outputs
Business Measures
Performance Measures
Evaluation Design
Performance and ROI
Analysis Evaluation
Customer Satisfaction
Evaluation Baseline
Summative Evaluation
Formative Evaluation
Evaluation Process
7
Action Plan Evaluation Blueprint

• Project Details (Name, Client, Goals)
• Basic Evaluation Criteria (When, Where, What,
  Who, Why, How)
• Evaluation Design (Model, Data Collection
  Analysis, Instruments)
• Evaluation Resources (People, Materials,
  Budget)
• Evaluation Schedule (Work Breakdown, Timeline,
  Responsibility)
• Evaluation Reporting (Format, Content,
  Messenger)

8
Bottom-Line Impact of HPI/Training
Strategic Growth
Revenues
Cost Savings
Cost Avoidance
Change
9
HPI Example Performance Management at a
Commercial Real Estate Company

• Profile
• 40 year old company, privately held, manages
  shopping malls, office buildings, apartments in
  Los Angeles
• US 26 million revenues
• 150 employees
• Business Problem
• Increasing competition in real estate management
  industry
• Slowing growth and shrinking profit margins

10
HPI Project Goals

• Business Goals
• Increase revenues and profit margins by 5
• Reduce turnover expenses by 20
• Performance Goals
• Increase average employee productivity by 5
• Increase retention by 20

11
HPI MethodologyInterrupted Time Series Design
I
M2
M5
M1
M4
M6
M3
M1- 6 measures at regular intervals I
intervention

• Uses
• Turnover rate pre and post performance management
  intervention
• Productivity pre and post performance management
  intervention
• Profit margin pre and post performance management
  intervention

12
HPI Methodology

• Business Analysis
• Interviews with Chairman of the Board, CEO, HR
  Director, Controller, Director of Development and
  Operations
• Business-HR Strategy Alignment with HR Director

13
HPI Methodology

• Performance Analysis
• Gilbert 6 Box Behavioral Engineering Model
• Manager/Supervisor Interviews
• Employee Focus Groups
• Document Analysis Performance Appraisal files,
  disciplinary files, exit interviews

14
HPI Cause Analysis using Gilbert 6 Box Model

• Organization Factors
• Information Performance Salary Review Policy
• Structure/ Performance Review Process
  Process/ (including forms)Resources Time is
  limited, managers are busy
• Incentives Rewards and consequences for
  Employees and Managers based on
  performance

15
HPI Cause Analysis Individual Factors

• Motives Lack of motivation by managers
  some employees to complete performance
  reviews improve performance
• Capacity Mental capacity and aptitude some
  employees unsuited for current job
• Knowledge Managers don't know how to manage
  performance and conduct reviews.
  Employees don't know how to set Objectives.

16
HPI Intervention Selection

• Root Causes
• Poor performance appraisal system (biased)
• Limited financial incentives (4 raises for
  everyone)
• Ineffective performance management (management
  skills)

• Interventions
• Redesign performance appraisals based on goals
• Design merit-based compensation (pay for
  performance 0-8)
• Train managers on effective performance
  management, employees on goal setting

17
HPI Implementation

• Timeline
• The project began in July and concluded by
  November 30, 2007 (5 months).
• Phase I Performance Analysis
• July August 2007
• Phase II Performance Appraisal/Compensation
  Redesign
• August - October 2007
• Phase III Training Design and Delivery
• October - November 2007

18
HPI Change Management

• Leadership Involvement
• CEO-led communication plan, spoke at every
  training class and public forum
• Employee Involvement
• Key Performers invited to participate in
  designing new system, input helped shape the
  intervention and build employee support
• Rewards and Consequences (Carrot Stick)
• Those who embraced new system were recognized and
  rewarded
• Those who opposed new system suffered
  consequences (loss of raise or position)

19
HPI Evaluation Design ModelThree Phases

• PHASE I Baseline Measures - 2007
• Business Goals
• Revenues/ Profits Annual Profit and Loss
  Statement
• Turnover Replacement Cost per Employee
• Performance Goals
• Retention Turnover rate, quantity of voluntary
  quits
• Productivity Annual Revenues per Employee

20
HPI Evaluation Model Phase II Formative
Evaluation

• Benchmarking
• Industry trade associations and third party HR
  market research on Compensation in financial
  services and Performance Appraisal systems
• Peer Review
• Extensive internal review of all proposed changes
  to the CEO level

21
HPI Evaluation Model Phase II Formative
Evaluation

• Pilot Test
• Pay for Performance piloted in New Business
  Development first
• Performance Appraisal system piloted in HR
• Participant Feedback
• Improvements to new Compensation system based on
  employee input after rollout (adjust pay raises,
  recognition program)

22
HPI Evaluation Model Phase III Summative
Evaluation 2008

• Performance Goals
• Turnover reduced 20 in 1 year (from 25 to 20).
• Productivity increased 8. (Revenues/Employee
  grew from 166K to 179K)
• Business Goals
• Turnover Savings5 less hires times 10K/hire
  50,000/year
• Net Earnings (Profits)Grew from 11 to 14 of
  Revenues(26.4 million times 3)
  805,500/year
• TOTAL Benefits 850,500/year

23
HPI ROI
24
Conclusions

• HPI requires more comprehensive evaluation model
• Formative and summative both important
• Difficult to isolate cause and effect on global
  financial measures such as profit

25
HPI Process
Intervention Implementation
Intervention Implementation
Performance Analysis
Cause Analysis
Intervention Selection
Performance Analysis
Cause Analysis
Intervention Selection
Business Analysis
Business Analysis
Evaluation
Evaluation
Business Goals
Performance Gaps
Root Causes
Project Management
HPI Outputs
Change Management
Solutions
Evaluation Outputs
Performance Measures
Evaluation Design
Performance and ROI
Business Measures
Root Cause Evaluation
Customer Satisfaction
Evaluation Baseline
Summative Evaluation
Formative Evaluation
Evaluation Process

• Summative Measures
• Pilot test
• Cost
• Schedule
• Stakeholder satisfaction
• Change management
• Performance results
• Business results
• ROI

• Business Measures
• Cost savings
• Cost avoidance
• Revenues
• Profits
• Customers
• Markets
• Strategic growth

• Performance Measures
• Productivity
• Retention
• Job satisfaction
• Competencies/skills
• Quality
• Quantity
• Speed

• Formative Measures
• Peer review
• Expert review
• Customer acceptance
• Stakeholder feedback
• Quality
• Benchmarking