Title: Training Evaluation
1Training Evaluation
2Evaluation of Training
- Since huge sums of money are spent on training
and development, how far the programme has been
useful must be determined. - The question of what to evaluate is crucial to
evaluation strategy. The answers depend on the
type of training programme, the organization and
the purposes of evaluation.
3Objectives of evaluation
- To determine success in accomplishing Programme
Objectives - To identify the strengths and weaknesses in the
training process - To compare the costs to the benefits of a
Training Programme - To decide who should participate in future
programmes - To test the clarity and validity of tests, cases
and exercises
4- 6. To identify which participants were the most
successful with the programme - 7. To reinforce major points made to the
participant - 8. To gather data to assist in marketing future
programmes - 9. To determine if the programme was the
appropriate solution for the specific need - 10. To establish a database that can assist
management in making decisions -
5Criteria of Evaluation
- Training validity- did the trainees learn during
training? - Transfer validity- what has been learnt in the
training, has it been transferred on the job? - Intra-organizational validity- is performance of
the new group of trainees, consistent with the
performance of the original training group? - Inter- organizational validity- can a training
program validated in one organization can be used
successfully in another organization?
6(No Transcript)
7- Feedback It helps in giving feedback to the
candidates by defining the objectives and linking
it to learning outcomes.Research It helps in
ascertaining the relationship between acquired
knowledge, transfer of knowledge at the work
place, and training.
8- Control It helps in controlling the training
program because if the training is not effective,
then it can be dealt with accordingly.Power
games At times, the top management (higher
authoritative employee) uses the evaluative data
to manipulate it for their own benefits.
Intervention It helps in determining that
whether the actual outcomes are aligned with the
expected outcomes
9(No Transcript)
10- Before Training
- The learner's skills and knowledge are assessed
before the training program. - During the start of training, candidates
generally perceive it as a waste of resources
because at most of the times candidates are
unaware of the objectives and learning outcomes
of the program. - Once aware, they are asked to give their
opinions on the methods used and whether those
methods confirm to the candidates preferences and
learning style.
11- During Training It is the phase at which
instruction is started. This phase usually
consist of short tests at regular intervals
12- After Training It is the phase when learners
skills and knowledge are assessed again to
measure the effectiveness of the training. - This phase is designed to determine whether
training has had the desired effect at individual
department and organizational levels. - There are various evaluation techniques for this
phase.
13- The various methods of training evaluation are
- Observation
- Questionnaire
- Interview
- Self diaries
- Self recording of specific incidents
14The Four Levels
- Reaction
- Learning
- Behavior
- Results
15All about Kirkpatrick
- In 1959, Kirkpatrick wrote four articles
describing the four levels for evaluating
training programs. He was working on his
dissertation for a Ph.D. when he came up with the
idea of defining evaluation. - Evaluation, as according to Kirkpatrick, seems
to have multiple meanings to training and
developmental professionals. Some think
evaluation is a change in behavior, or the
determination of the final results. -
16KirkPatricks four levels of evaluation of
training impact
- Level
- Reaction
- Learning
- Behaviour
- Results
- Questions
- Were the employees pleased with the programme?
- What did the participants learn in the programme?
- Did the participants change their behaviour based
on what was learned? - Did the change in behaviour positively affect the
organization?
17- The Four Levels represent a sequence of ways to
evaluate (training) programs.As you move from
one level to the next, the process becomes more
difficult and time-consuming, but it also
provides more valuable information.
18- Reaction is the measuring of the reaction of
the participants in the training program. - what participants thought of the program,
including materials, instructors, facilities,
methodology, content etc.
19- Learning is the change in the participants
attitudes, or an increase in knowledge, or
greater skills received, as a result of the
participation of the program. - learning evaluation is concerned with measuring
the extent to which principles, facts, techniques
and skills have been acquired
20- Behavior how much transfer of knowledge, skills,
and attitude occurs after the training. - behavioral change is measured to determine the
extent to which principles, facts, techniques,
and skills have been acquired.
21- Results is the most important and difficult of
all - determining final results after training - evaluation of results involves monitoring
organizational improvement such as cost savings,
work output changes, and quality changes.
22Kaufmans five levels of evaluation of training
impact
- Some researchers, recognising some shortcomings
of Kirkpatricks four-level approach, have
attempted to modify and add to this basic
framework. - Kaufman has expanded the definition of level 1
and added a fifth level addressing societal
issues.
23Level Evaluation focus
5. Societal outcomes Societal and client responsiveness, consequences and payoffs
4. Organizational output Organizational contributions and payoffs
3. Application Individual and small group (products) utilization within the organization
2. Acquisition Individual and small group mastery and competency
1b. Reaction Methods, means, and process acceptability and efficiency
1a. Enabling Availability and quality of human, financial, and physical resources input
24- At level 1, the factor of the concept of enabling
addressees the availability of various resource
inputs necessary for a successful intervention. - Level 5 is the evaluation of societal and client
responsiveness. This moves evaluation beyond the
organization, and examines the extent to which
the performance improvement programme has
enhanced society and the environment surrounding
the organization.
25The CIRO Approach
- Developed by War, Bird, and Racham, is a unique
way to classify evaluation process. - Four general categories of evaluation are
described, which form the letters CIRO - 1. Context evaluation
- 2. Input evaluation
- 3. Reaction evaluation
- 4. Outcome evaluation
26Context evaluation
- Context evaluation involves
- collecting information about a performance
deficiency, - assessing the information to establish training
needs and, - on the basis of those findings, setting
objectives. - Context evaluation involves obtaining and using
information about the current operational
situation (or context) to determine training
needs and objectives.
27- Three types of objectives may be evaluated-
- Ultimate objectives (the particular deficiency in
the organization that the programme will
eliminate or overcome) - Intermediate objectives (the changes in
employees work behavior that will be necessary
for the ultimate objectives to be attained) - Immediate objectives (the new knowledge, skills,
or attitudes that employees must acquire to
change their behavior and reach the intermediate
objective)
28Input Evaluation
- Input Evaluation involves
- obtaining and using information about possible
training resources to choose between alternative
inputs to training. - It involves analysing the resources available
(both internal and external) and determining how
they can be deployed so that there is a maximum
chance of achieving the desired objectives.
29- Input evaluation refers to the process of
collecting evidence and using it to decide on the
training methods
30Reaction Evaluation
- Reaction evaluation involves
- obtaining and using information about
participants reactions to improve the HRD
process. - The distinguishing feature of this type of
evaluation is that it relies on the subjective
input of the participants. - Their views can prove extremely helpful when
collected and used in a systematic manner.
31Outcome Evaluation
- Outcome Evaluation
- involves obtaining and using information about
the results or outcomes of training, and is
usually regarded as the most important part of
evaluation. - If outcome evaluation is to be successful, it
requires careful preparation before the programme
begins.
32- Stages of outcome evaluation-
- Defining trend objectives
- Selecting or constructing some measures of those
objectives - Making the measurements at the appropriate time
- Assessing the results and using them to improve
later programmes.
33- A successful training programme produces some
initial change in a participant, an immediate
outcome, which is reflected in changes of
knowledge, skills, or attitudes. These changes
can be measured during or at the end of the
programme.
34The Philips Five Level ROI Framework
- The ROI process adds a fifth level to the four
levels of evaluation developed by Kirkpatrick.
35Level Brief Description
1. Reaction and planned action
2. Learning
3. Job applications
4. Business results
5. Return on investment
361.Reaction and planned action
- Programme participants satisfaction is measured,
along with a listing of how they plan to apply
what they have learned. - This level of evaluation is important as a
participant satisfaction measure, a favorable
reaction does not ensure that participants have
learned new skills or knowledge.
372. Learning
- Measurements focus on what participants learned
during the programme using tests, skill
practices, role plays, simulations, group
evaluations, and other assessment tools. - A learning check is helpful to ensure that
participants have absorbed the material and know
how to use it.
383. Job application
- A variety of follow up methods are used to
determine if participants apply what they learned
on the job. Level 3 evaluations are important to
gauge the success of the programmes application,
it still does not guarantee that there will be a
positive impact in the organization.
394. Business results
- The measurement focuses on actual results
programme participants achieve as they
successfully apply the programme material.
Typical level 4 measures include output, quality,
costs, time, and customer satisfaction.
405. Return on Investment
- Comparing the monetary benefits from the
programme with its costs. It is usually presented
as a percent or cost/benefit ratio. - Very few organizations actually conduct
evaluations at ROI level, perhaps because ROI
evaluation is often characterised as a difficult
and expensive process.
41(No Transcript)
42Steps to calculate return on investment (ROI),
- Identify outcome(s) (e.g., quality, accidents)
- Place a value on the outcome(s)
- Determine the change in performance after
eliminating other potential influences on
training results. - Obtain an annual amount of benefits (operational
results) from training by comparing results after
training to results before training.
43Steps to calculate return on investment (ROI),
(continued)
- Determine training costs (direct costs indirect
costs development costs overhead costs
compensation for trainees) - Calculate the total savings by subtracting the
training costs from benefits (operational
results) - Calculate the ROI by dividing benefits
(operational results) by costs. - The ROI gives you an estimate of the rupee return
expected from each rupee invested in training.
44- Although business results and ROI are desired, it
is important to evaluate the other levels. A
chain of impact should occur through the levels
as the skills and knowledge learned (level2) are
applied on the job (level3) to produce business
results (level4).
45Outcomes Used in Evaluating Training Programs
Cognitive Outcomes
Skill-Based Outcomes
Affective Outcomes
Results
Return on Investment
46Outcomes Used in Evaluating Training Programs
(continued)
- Cognitive Outcomes
- Determine the degree to which trainees are
familiar with the principles, facts, techniques,
procedures, or processes emphasized in the
training program. - Measure what knowledge trainees learned in the
program. - Skill-Based Outcomes
- Assess the level of technical or motor skills.
- Include acquisition or learning of skills and use
of skills on the job.
47Outcomes Used in Evaluating Training Programs
(continued)
- Affective Outcomes
- Include attitudes and motivation.
- Trainees perceptions of the program including
the facilities, trainers, and content. - Results
- Determine the training programs payoff for the
company.
48Outcomes Used in Evaluating Training Programs
(continued)
- Return on Investment (ROI)
- Comparing the trainings monetary benefits with
the cost of the training. - Direct costs
- Indirect costs
- Benefits
49How do you know if your outcomes are good?
- Good training outcomes need to be
- Relevant
- Reliable
- Discriminate
- Practical
50Importance of Training Cost Information
- To understand total expenditures for training,
including direct and indirect costs. - To compare costs of alternative training
programs. - To evaluate the proportion of money spent on
training development, administration, and
evaluation as well as to compare money spent on
training for different groups of employees. - To control costs.