Evaluation of

Training
by
Jayadeva de Silva

Raman K. Attri

Formative Evaluation
Formative evaluation (also known as
internal) is a method of judging the
worth of a program while the program
activities are forming (in progress). This
part of the evaluation focuses on the
process.

Jayadeva de Silva

Jayadeva de Silva

Summative
The summative evaluation (also know as
external) is a method of judging the worth
of a program at the end of the program
activities (summation). The focus is on the
outcome.

Jayadeva de Silva

Jayadeva de Silva

Jayadeva de Silva

The Ten Factors of Developing

a Training Program
1.
2.
3.
4.
5.

Determine needs
Set objectives
Determine subject content
Select qualified applicants
Determine the best schedule
Jayadeva de Silva

The Ten Factors of Developing

a Training Program
6. Select appropriate facilities
7. Select qualified instructors
8. Select and prepare audiovisual
aids
9. Co-ordinate the program
10. Evaluate the program
Jayadeva de Silva

Donald Kirkpatrick

Jayadeva de Silva

10

Kirkpatrick

Jayadeva de Silva

Evaluating
The reason for evaluating is to

determine the effectiveness of a

training program. (Kirkpatrick,
1994, )

Jayadeva de Silva

Reasons for Evaluating

Kirkpatrick gives three reasons
why there is a need to evaluate
training:

Jayadeva de Silva

Reasons for Evaluating

1.To justify the existence of the
training department by showing
how it contributes to the
organizations objectives and
goals.

Jayadeva de Silva

14

Reasons for Evaluating

2. To decide whether to continue
or discontinue training
programs.

Jayadeva de Silva

Reasons for Evaluating

3. To gain information on how to
improve future training programs.
(Kirkpatrick, 1994, )

Jayadeva de Silva

16

Kirkpatrick: Evaluating
Training Programs
What is quality training?
How do you measure it?
How do you improve it?
Jayadeva de Silva

'Training Evaluation
Senior management
The trainers
Line management
The training manager
The trainee

Jayadeva de Silva

18

Jayadeva de Silva

19

4-Tier Practical Training

Value Measurement Model

Training Expectation Measurement

Trainee feedback scores on initial impression on to
what extent training met their expectations of
learning, skills and knowledge.

Tier-1

Expectations

Tool: Survey Form

Stu
dent
feed
bac
k

Jayadeva de Silva

20

4-Tier Practical Training

Value Measurement Model

Training Expectation Measurement

Trainee feedback scores on initial impression on to
what extent training met their expectations of
learning, skills and knowledge.

Tier-1

Expectations

Tier-2
Improvement

Tool: Survey Form

Stu
dent
feed
bac
k

Training Improvement Measurement

Assessment of trainees training exposure and
expertise gained on same set of tasks before and
after training. Includes comparison of in-training &
Post-training test performance, If applicable.

Pre-Training
Vs PostTraining
Assessment

Tool: Pre-training and Post-training Survey Forms.

Post-training performance tests.

Jayadeva de Silva

21

4-Tier Practical Training

Value Measurement Model

Training Expectation Measurement

Trainee feedback scores on initial impression on to
what extent training met their expectations of
learning, skills and knowledge.

Tier-1

Expectations

Tier-2
Improvement

Tool: Survey Form

Stu
dent
feed
bac
k

Training Improvement Measurement

Assessment of trainees training exposure and
expertise gained on same set of tasks before and
after training. Includes comparison of in-training &
Post-training test performance, If applicable.

Pre-Training
Vs PostTraining
Assessment

Tool: Pre-training and Post-training Survey Forms.

Post-training performance tests.

Training effectiveness Measurement

Tier-3

Effectiveness

Post-training normalized feedback scores and its

quarterly trends. Feedback from trainee Manager
on visible incremental changes in trainee skills,
service parameters and on-job behavior.

Post-Training OnJob Behavior

Survey

Tool: Quarterly Follow-on Survey from Managers

Jayadeva de Silva

22

4-Tier Practical Training

Value Measurement Model

Training Expectation Measurement

Trainee feedback scores on initial impression on to
what extent training met their expectations of
learning, skills and knowledge.

Tier-1

Expectations

Tier-2
Improvement

Tool: Survey Form

Stu
dent
feed
bac
k

Training Improvement Measurement

Assessment of trainees training exposure and
expertise gained on same set of tasks before and
after training. Includes comparison of in-training &
Post-training test performance, If applicable.

Pre-Training
Vs PostTraining
Assessment

Tool: Pre-training and Post-training Survey Forms.

Post-training performance tests.

Training effectiveness Measurement

Tier-3

Effectiveness

Post-training normalized feedback scores and its

quarterly trends. Feedback from trainee Manager
on visible incremental changes in trainee skills,
service parameters and on-job behavior.

Post-Training OnJob Behavior

Survey

Tool: Quarterly Follow-on Survey from Managers

Tier-4

Impact

Improvement in business
indicators, revenue figures
or success parameters
Jayadeva de Silva

Training Impact Measurement

Impact of training for improving revenues,
enhancing business or other success factors
driving the training needs.
Tool: Comparison of Baseline data and Quarterly
23
business data, Impact Factor calculation sheets

Tier-1: how closely does

Training meet the
expectations of trainees?
Feedback score collected
from each trainee at end of
the class
Record general impression
about the Training on a
numerical scale of 1-5 or
similar
Average score reflects how
closely
training
met
expectations of field and
how close it is aligned with
business needs identified
earlier which drives the
training.
Jayadeva de Silva

24

Reactions.
Reaction may best be defined as how well
the trainees liked a particular training
program."
Reactions are typically measured at the
end of training.

Jayadeva de Silva

25

Jayadeva de Silva

26

FEEDBACK FORM
Please take a few minutes to fill out this feedback form. Your feedback will help us strengthen the course delivery.
Course Title Name of the facilitator You may provide feedback of the program on the following criteria by putting a () mark in the space provided.
4 Exceeds Expectation; 3 Meets Expectation; 2 Needs Improvement; 1 Unsatisfactory

Subject Knowledge
Presentation Style
Communication
Examples, Cases, Simulation & Exercises
Relevance
Supporting Materials
Overall Evaluation

1. What did you like most in the workshop?

2. What did you dislike in the workshop?
3. What other changes would you suggest in case the same course is conducted in future?
Learning & Development
Jayadeva de Silva
Human Resources

27

Verification of Reactions
Typically 'happy sheets'
Feedback forms based on subjective personal
reaction to the training experience
Verbal reaction which can be noted and
analyzed
Post-training surveys or questionnaires
Subsequent verbal or written reports given by
delegates to managers back at their jobs

Jayadeva de Silva

28

Jayadeva de Silva

29

Jayadeva de Silva

30

Jayadeva de Silva

31

Jayadeva de Silva

32

Jayadeva de Silva

33

Jayadeva de Silva

34

Jayadeva de Silva

35

Jayadeva de Silva

36

Jayadeva de Silva

37

Jayadeva de Silva

38

Jayadeva de Silva

39

Tier-2: how well Training

improved skills of trainees?
Collect Pre-training and post-training
data on trainees skills before and
after the training. A well drafted skill
based survey can be used.
Compare improvements in exposure
or expertise gained by the trainee as
seen on post-training survey by
comparing it with pre-training survey.
Conduct in-training skill tests as part
of the qualification criteria to measure
the improvement in the learned skills.

Jayadeva de Silva

40

Learning.
What principles, facts, and techniques
were understood and absorbed by the
participants?"
What the trainees know or can do can
be measured during and at the end of
training

Jayadeva de Silva

41

Jayadeva de Silva

42

Jayadeva de Silva

43

Verification of Learning
WHAT?
What knowledge was acquired?
What skills were developed or enhanced?
What attitudes were changed?
HOW?
Tests before and after the training
Interview or observation can be used before and after
training.
Measurement and analysis is possible and easy on a
group scale
Reliable, clear scoring and measurements need to be
established

Jayadeva de Silva

44

Tier-3: How effective does the training prove

in changing On-job-behavior and skills of
trainee?
Collect quarterly feedback from
trainees manager on his
observations on improvement of
trainees on-job performance/
behavior or skills after attending
training.
Analyze the trends in key onjob-performance parameters or
indicators as seen on survey
with
respect
to
previous
quarters.
Convert delta into normalized
scores to indicate the value
created by training in the work
efficiency of the trainee
Ensure continual measurement
of effectiveness after regular
intervals to assess long term
Jayadeva de Silvavalue of the training.
45

Behavior.
Changes in on-the-job behavior,Behavior
changes are acquired in training and they
then transfer (or don't transfer) to the work
place.
What skills did the learner develop, that is,
what new information is the learner using
on the job?

Jayadeva de Silva

46

When evaluating change in

behavior, decide:
When to evaluate
How often to evaluate

How to evaluate

Jayadeva de Silva

Jayadeva de Silva

48

Jayadeva de Silva

49

Jayadeva de Silva

50

Jayadeva de Silva

51

Jayadeva de Silva

52

Jayadeva de Silva

53

Verification of Behaviour
What?
Whether the trainee is able to transfer the learning
to the work environment
New learning is demonstrated
Whether the trainee is motivated

How?
self-assessment can be useful, using
carefully designed criteria and measurements
cooperation and skill of observers, typically linemanagers, are important factors, and difficult to
control
Use of focus groups
Jayadeva de Silva

54

Tier-4: To what extent training does impact in

improving business / revenue or service
parameters specific to trainees or group of
trainees?
Measure
quarterly
the
business
indicators of the trainees job or service
parameters based on nature of job.
Business indicators could be collected
based on individual job or group
responsible for the said function.
Ideally business indicator data before
the training should be used as baseline.
Record the business parameters or
governing service parameters on
quarterly basis
Calculate qualitative or
quantitative
impact factor based on normalized delta.

Jayadeva de Silva

55

Results.
Reduction of costs;
Reduction of turnover and absenteeism;
Reduction of grievances;
increase in quality and quantity or production;
or Improved morale which, it is hoped, will lead to
some of the previously stated results.
These factors are also measurable in the
workplace

Jayadeva de Silva

56

Jayadeva de Silva

57

Jayadeva de Silva

58

Jayadeva de Silva

59

Jayadeva de Silva

60

The Four Levels

Reaction
Learning
Behavior
Results

Jayadeva de Silva

Jayadeva de Silva

62

The Four Levels represent a

sequence of ways to evaluate
(training) programs.As you
move from one level to the
next, the process becomes
more difficult and timeconsuming, but it also provides
more valuable information.
(Kirkpatrick, 1994,)
Jayadeva de Silva

When all conditions are met,

the employee must:
Realize an opportunity to use the
behavioral changes.
Make the decision to use the
behavioral changes.
Decide whether or not to continue
using the behavioral changes.
Jayadeva de Silva

Return on Impact (RoI) is a new approach

which measures the difference training has
created on those governing business/ service
or revenue parameters which drives training
needs.

Jayadeva de Silva

65

The four conditions Kirkpatrick

identifies for changes to occur:
Desire to change
Knowledge of what to do and
how to do it
Work in the right climate
Reward for (positive) change
Jayadeva de Silva

Jayadeva de Silva

67

An individual and group

exercise
Group 1- As an HR Manager,how will you
evaluate:The output effectiveness of a
supervisory training programme?
Group 2- The Outputs to be Evaluated
after a Sales Training programme
Group 3- The Outputs to be Evaluated
after a Customer Relations Training
programme
Jayadeva de Silva

68

Outputs to be Evaluated after

Supervisory and Management Training
increased output
reduced absenteeism and
tardiness
reduced cost of new hires
reduced turnover
increased number of employee
suggestions
climate survey data (morale and
attitudes)
Jayadeva de Silva

69

Outputs to be Evaluated after

Sales Training
sales volume
average sale size
add-on sales
close-to-call ratio
ratio of new accounts to
old accounts
number of items per order

Jayadeva de Silva

70

Outputs to be Evaluated After

Customer Relations Training
accuracy of orders
size of orders
number of transactions per day
adherence to credit procedures
number of lost customers
amount of repeat business
number of referrals
number of complaints

Jayadeva de Silva

71

Questions & Concerns

Jayadeva de Silva

72