Training Evaluation - PowerPoint PPT Presentation

About This Presentation
Title:

Training Evaluation

Description:

Training Evaluation – PowerPoint PPT presentation

Number of Views:618
Avg rating:3.0/5.0
Slides: 51
Provided by: word1299
Category:

less

Transcript and Presenter's Notes

Title: Training Evaluation


1
Training Evaluation
2
Evaluation of Training
  • Since huge sums of money are spent on training
    and development, how far the programme has been
    useful must be determined.
  • The question of what to evaluate is crucial to
    evaluation strategy. The answers depend on the
    type of training programme, the organization and
    the purposes of evaluation.

3
Objectives of evaluation
  1. To determine success in accomplishing Programme
    Objectives
  2. To identify the strengths and weaknesses in the
    training process
  3. To compare the costs to the benefits of a
    Training Programme
  4. To decide who should participate in future
    programmes
  5. To test the clarity and validity of tests, cases
    and exercises

4
  • 6. To identify which participants were the most
    successful with the programme
  • 7. To reinforce major points made to the
    participant
  • 8. To gather data to assist in marketing future
    programmes
  • 9. To determine if the programme was the
    appropriate solution for the specific need
  • 10. To establish a database that can assist
    management in making decisions

5
Criteria of Evaluation
  1. Training validity- did the trainees learn during
    training?
  2. Transfer validity- what has been learnt in the
    training, has it been transferred on the job?
  3. Intra-organizational validity- is performance of
    the new group of trainees, consistent with the
    performance of the original training group?
  4. Inter- organizational validity- can a training
    program validated in one organization can be used
    successfully in another organization?

6
(No Transcript)
7
  • Feedback It helps in giving feedback to the
    candidates by defining the objectives and linking
    it to learning outcomes.Research It helps in
    ascertaining the relationship between acquired
    knowledge, transfer of knowledge at the work
    place, and training.

8
  • Control It helps in controlling the training
    program because if the training is not effective,
    then it can be dealt with accordingly.Power
    games At times, the top management (higher
    authoritative employee) uses the evaluative data
    to manipulate it for their own benefits.
    Intervention It helps in determining that
    whether the actual outcomes are aligned with the
    expected outcomes

9
(No Transcript)
10
  • Before Training
  • The learner's skills and knowledge are assessed
    before the training program.
  • During the start of training, candidates
    generally perceive it as a waste of resources
    because at most of the times candidates are
    unaware of the objectives and learning outcomes
    of the program.
  • Once aware, they are asked to give their
    opinions on the methods used and whether those
    methods confirm to the candidates preferences and
    learning style.

11
  • During Training It is the phase at which
    instruction is started. This phase usually
    consist of short tests at regular intervals

12
  • After Training It is the phase when learners
    skills and knowledge are assessed again to
    measure the effectiveness of the training.
  • This phase is designed to determine whether
    training has had the desired effect at individual
    department and organizational levels.
  • There are various evaluation techniques for this
    phase.

13
  • The various methods of training evaluation are
  • Observation
  • Questionnaire
  • Interview
  • Self diaries
  • Self recording of specific incidents

14
The Four Levels
  • Reaction
  • Learning
  • Behavior
  • Results

15
All about Kirkpatrick
  • In 1959, Kirkpatrick wrote four articles
    describing the four levels for evaluating
    training programs. He was working on his
    dissertation for a Ph.D. when he came up with the
    idea of defining evaluation.
  • Evaluation, as according to Kirkpatrick, seems
    to have multiple meanings to training and
    developmental professionals. Some think
    evaluation is a change in behavior, or the
    determination of the final results.

16
KirkPatricks four levels of evaluation of
training impact
  • Level
  • Reaction
  • Learning
  • Behaviour
  • Results
  • Questions
  • Were the employees pleased with the programme?
  • What did the participants learn in the programme?
  • Did the participants change their behaviour based
    on what was learned?
  • Did the change in behaviour positively affect the
    organization?

17
  • The Four Levels represent a sequence of ways to
    evaluate (training) programs.As you move from
    one level to the next, the process becomes more
    difficult and time-consuming, but it also
    provides more valuable information.

18
  • Reaction is the measuring of the reaction of
    the participants in the training program.
  • what participants thought of the program,
    including materials, instructors, facilities,
    methodology, content etc.

19
  • Learning is the change in the participants
    attitudes, or an increase in knowledge, or
    greater skills received, as a result of the
    participation of the program.
  • learning evaluation is concerned with measuring
    the extent to which principles, facts, techniques
    and skills have been acquired

20
  • Behavior how much transfer of knowledge, skills,
    and attitude occurs after the training.
  • behavioral change is measured to determine the
    extent to which principles, facts, techniques,
    and skills have been acquired.

21
  • Results is the most important and difficult of
    all - determining final results after training
  • evaluation of results involves monitoring
    organizational improvement such as cost savings,
    work output changes, and quality changes.

22
Kaufmans five levels of evaluation of training
impact
  • Some researchers, recognising some shortcomings
    of Kirkpatricks four-level approach, have
    attempted to modify and add to this basic
    framework.
  • Kaufman has expanded the definition of level 1
    and added a fifth level addressing societal
    issues.

23
Level Evaluation focus
5. Societal outcomes Societal and client responsiveness, consequences and payoffs
4. Organizational output Organizational contributions and payoffs
3. Application Individual and small group (products) utilization within the organization
2. Acquisition Individual and small group mastery and competency
1b. Reaction Methods, means, and process acceptability and efficiency
1a. Enabling Availability and quality of human, financial, and physical resources input
24
  • At level 1, the factor of the concept of enabling
    addressees the availability of various resource
    inputs necessary for a successful intervention.
  • Level 5 is the evaluation of societal and client
    responsiveness. This moves evaluation beyond the
    organization, and examines the extent to which
    the performance improvement programme has
    enhanced society and the environment surrounding
    the organization.

25
The CIRO Approach
  • Developed by War, Bird, and Racham, is a unique
    way to classify evaluation process.
  • Four general categories of evaluation are
    described, which form the letters CIRO
  • 1. Context evaluation
  • 2. Input evaluation
  • 3. Reaction evaluation
  • 4. Outcome evaluation

26
Context evaluation
  • Context evaluation involves
  • collecting information about a performance
    deficiency,
  • assessing the information to establish training
    needs and,
  • on the basis of those findings, setting
    objectives.
  • Context evaluation involves obtaining and using
    information about the current operational
    situation (or context) to determine training
    needs and objectives.

27
  • Three types of objectives may be evaluated-
  • Ultimate objectives (the particular deficiency in
    the organization that the programme will
    eliminate or overcome)
  • Intermediate objectives (the changes in
    employees work behavior that will be necessary
    for the ultimate objectives to be attained)
  • Immediate objectives (the new knowledge, skills,
    or attitudes that employees must acquire to
    change their behavior and reach the intermediate
    objective)

28
Input Evaluation
  • Input Evaluation involves
  • obtaining and using information about possible
    training resources to choose between alternative
    inputs to training.
  • It involves analysing the resources available
    (both internal and external) and determining how
    they can be deployed so that there is a maximum
    chance of achieving the desired objectives.

29
  • Input evaluation refers to the process of
    collecting evidence and using it to decide on the
    training methods

30
Reaction Evaluation
  • Reaction evaluation involves
  • obtaining and using information about
    participants reactions to improve the HRD
    process.
  • The distinguishing feature of this type of
    evaluation is that it relies on the subjective
    input of the participants.
  • Their views can prove extremely helpful when
    collected and used in a systematic manner.

31
Outcome Evaluation
  • Outcome Evaluation
  • involves obtaining and using information about
    the results or outcomes of training, and is
    usually regarded as the most important part of
    evaluation.
  • If outcome evaluation is to be successful, it
    requires careful preparation before the programme
    begins.

32
  • Stages of outcome evaluation-
  • Defining trend objectives
  • Selecting or constructing some measures of those
    objectives
  • Making the measurements at the appropriate time
  • Assessing the results and using them to improve
    later programmes.

33
  • A successful training programme produces some
    initial change in a participant, an immediate
    outcome, which is reflected in changes of
    knowledge, skills, or attitudes. These changes
    can be measured during or at the end of the
    programme.

34
The Philips Five Level ROI Framework
  • The ROI process adds a fifth level to the four
    levels of evaluation developed by Kirkpatrick.

35
Level Brief Description
1. Reaction and planned action
2. Learning
3. Job applications
4. Business results
5. Return on investment
36
1.Reaction and planned action
  • Programme participants satisfaction is measured,
    along with a listing of how they plan to apply
    what they have learned.
  • This level of evaluation is important as a
    participant satisfaction measure, a favorable
    reaction does not ensure that participants have
    learned new skills or knowledge.

37
2. Learning
  • Measurements focus on what participants learned
    during the programme using tests, skill
    practices, role plays, simulations, group
    evaluations, and other assessment tools.
  • A learning check is helpful to ensure that
    participants have absorbed the material and know
    how to use it.

38
3. Job application
  • A variety of follow up methods are used to
    determine if participants apply what they learned
    on the job. Level 3 evaluations are important to
    gauge the success of the programmes application,
    it still does not guarantee that there will be a
    positive impact in the organization.

39
4. Business results
  • The measurement focuses on actual results
    programme participants achieve as they
    successfully apply the programme material.
    Typical level 4 measures include output, quality,
    costs, time, and customer satisfaction.

40
5. Return on Investment
  • Comparing the monetary benefits from the
    programme with its costs. It is usually presented
    as a percent or cost/benefit ratio.
  • Very few organizations actually conduct
    evaluations at ROI level, perhaps because ROI
    evaluation is often characterised as a difficult
    and expensive process.

41
(No Transcript)
42
Steps to calculate return on investment (ROI),
  1. Identify outcome(s) (e.g., quality, accidents)
  2. Place a value on the outcome(s)
  3. Determine the change in performance after
    eliminating other potential influences on
    training results.
  4. Obtain an annual amount of benefits (operational
    results) from training by comparing results after
    training to results before training.

43
Steps to calculate return on investment (ROI),
(continued)
  • Determine training costs (direct costs indirect
    costs development costs overhead costs
    compensation for trainees)
  • Calculate the total savings by subtracting the
    training costs from benefits (operational
    results)
  • Calculate the ROI by dividing benefits
    (operational results) by costs.
  • The ROI gives you an estimate of the rupee return
    expected from each rupee invested in training.

44
  • Although business results and ROI are desired, it
    is important to evaluate the other levels. A
    chain of impact should occur through the levels
    as the skills and knowledge learned (level2) are
    applied on the job (level3) to produce business
    results (level4).

45
Outcomes Used in Evaluating Training Programs
Cognitive Outcomes
Skill-Based Outcomes
Affective Outcomes
Results
Return on Investment
46
Outcomes Used in Evaluating Training Programs
(continued)
  • Cognitive Outcomes
  • Determine the degree to which trainees are
    familiar with the principles, facts, techniques,
    procedures, or processes emphasized in the
    training program.
  • Measure what knowledge trainees learned in the
    program.
  • Skill-Based Outcomes
  • Assess the level of technical or motor skills.
  • Include acquisition or learning of skills and use
    of skills on the job.

47
Outcomes Used in Evaluating Training Programs
(continued)
  • Affective Outcomes
  • Include attitudes and motivation.
  • Trainees perceptions of the program including
    the facilities, trainers, and content.
  • Results
  • Determine the training programs payoff for the
    company.

48
Outcomes Used in Evaluating Training Programs
(continued)
  • Return on Investment (ROI)
  • Comparing the trainings monetary benefits with
    the cost of the training.
  • Direct costs
  • Indirect costs
  • Benefits

49
How do you know if your outcomes are good?
  • Good training outcomes need to be
  • Relevant
  • Reliable
  • Discriminate
  • Practical

50
Importance of Training Cost Information
  • To understand total expenditures for training,
    including direct and indirect costs.
  • To compare costs of alternative training
    programs.
  • To evaluate the proportion of money spent on
    training development, administration, and
    evaluation as well as to compare money spent on
    training for different groups of employees.
  • To control costs.
Write a Comment
User Comments (0)
About PowerShow.com