Measuring ROI of Training

Measuring
ROI of
Training
www.exploreHR.org 1
You can download this brilliant presentation at:
www.exploreHR.org
Visit www.exploreHR.org for more
presentations on HR management and
management skills
www.exploreHR.org 2
Agenda
1. Measuring the Effectiveness of Training

Program
2. Measuring ‘Return on Investment’ of Training
3. Enhancing the Effectiveness and ROI of

Training
www.exploreHR.org 3
Measuring the Effectiveness of
Training Program
www.exploreHR.org 4
Training Process
Training Training Training

Training
Need Objectives Evaluation
Delivery
Analysis
What are Objective Techniques Measure

the training should be include on- reaction,
needs for measurable the-job- learning,
this person and training, behavior,
and/or job? observable action and results
learning, etc.
www.exploreHR.org 5
The Four Levels of Evaluation
Level 1 - Reaction
Level 2 - Learning
Four Levels
of Training
Level 3 – Behavior Effectiveness
Application
Level 4 – Business
Impact
www.exploreHR.org 6
Evaluate trainees’ reactions to Level 1 -

the program. Did they like the Reaction
program? Did they think it
worthwhile?
Test the trainees to determine if Level 2 -

they learned the principles, Learning
skills, and facts they were to
learn.
www.exploreHR.org 7
Ask whether the trainees’ behavior on the job Level 3 –

changed because of the training program. For Behavior
example, are employees in the store’s complaint Application
department more courteous toward disgruntled
customers than previously?
What final results were achieved in terms of the Level 4 –

training objectives previously set? Did the number Business
of customer complaints about employee drop? Did Impact
the reject rate improve? Was turnover reduced,
and so forth.
www.exploreHR.org 8
Value of Frequency of Difficulty of

Information Use Assessment
Level
Least Frequent Easy
valuable
I. Reaction
II. Learning
III. Behavior
Most Infrequent Difficult
IV. Results valuable
www.exploreHR.org 9
Level 1 - Reaction
Evaluate trainees’
reactions to the program:
Did they like the program?
Level 1 - Did they like the
Reaction
facilitators?
Did they like the training
accommodation and
facilities?
www.exploreHR.org 10
Guidelines for Evaluating Reaction
1. Determine what you want to find out

2. Design a form that will quantify reactions
3. Encourage written comments and suggestions
4. Get 100 percent immediate response
5. Get honest response
6. Develop acceptable standards
7. Measure reactions against standards, and take
appropriate action
8. Communicate reaction as appropriate
Sample of
Reaction
Please give us
Form
Program :
Facilitator :
Level 2 - Learning
Measuring learning means

determining one or more of
the following :
Level 2 - • What knowledge was
Learning learned?
• What skills were
developed or improved?
• What attitudes were
changed?
Guidelines for Evaluating Learning
1. Use a control group if practical

2. Evaluate knowledge, skills and/or attitudes both
before and after the program
3. Use a paper-and-pencil test to measure knowledge
4. Use a performance test to measure skills
5. Get 100 percent immediate response
6. Use the results of evaluation to take appropriate
action
Pretest and Posttest Scores on Change Management Training

Example :
www.exploreHR.org
Pre Te 15
Level 3 – Behavior Application
• The frequency of
application of new
skills/knowledge/
Level 3 – attitudes (on the job)
Behavior
Application • The effectiveness of the
skills/knowledge/
attitudes (as applied on
the job)

2. Allow time for behavior change and application to take
place
3. Evaluate both before and after the program if practical
4. Survey and/or interview one or more of the following :
trainees, their immediate supervisor, their subordinates,
and others who often observe their behavior
5. Get 100 percent response or a sampling
6. Repeat the evaluation at appropriate times
7. Consider cost versus benefits
Example of Survey to Measure Behavior Application
Instruction:
The objective of this questionnaire is to determine the extent to which those who
attended the recent program on Leadership have applied the principles and techniques
that they learned there to the job.
Circle the answer that you consider appropriate for each question.
5 = Much more 4 = More 3 = Same 2 = Less 1 = Much less
Level 4 – Business Results
• Indicate the extent to which you

think this program has
influenced each of these
measures in your work unit,
Level 4 – department, or business unit:
Business • Productivity
Results • Quality
• Customer Response Time
• Cost Control
• Employee Satisfaction
• Customer Satisfaction
• Other

2. Allow time for results to be achieved
3. Measure both before and after the program if practical
4. Repeat the evaluation at appropriate times
5. Consider cost versus benefits
6. Be satisfied with evidence if proof is not possible
Performance Indicators
HARD DATA INDICATORS
• Downtime duration
• Number of defect products
• Sales volume
• Production unit
• Customer satisfaction index
• Response time to orders
• Number of accidents at work
• Others
Performance Indicators
SOFT DATA INDICATORS (intangible impacts)
• Job satisfaction
• Conducive working relationship
• Effective communication
• Stress rate
• Quality in decision-making
Example : Measuring Training Results
200.00
Program : TQM
Training
150.00
Results after 3
120 units
months of
100.00
training, number
80 units
of defects
dropped to 80 50.00
units/day
0.00
Before training After training
Example : Measuring Training Results
Program : 50.00
Sales Training
40.00
Results after 3
months training,
training 30.00
30 units
number of sales
per salesman 20.00 20 units
increase to 30
units/month. 10.00
0.00
Before training After training
Measuring
Return on Investment of
Training
Level 5 : Return on Investment of Training
Level 1 - Reaction
Level 2 - Learning
Level 3 – Behavior
Application
Level 4 –
Business Impact
Level 5 – Return
on Investment of
Training
Criteria for Selecting Programs for Levels
4 and 5 Evaluation
• Importance of the program in meeting the

organization’s goals
• Cost of the program
• Visibility of the program
• Size of the target audience
• Extent of management interest
Benefits of ROI of Training
• Measure contribution
• Set priorities
• Focus on results
• Alter management perceptions of training
ROI of Training Model
Isolate the Convert Data

Collect
Data Effects of to Monetary
Training Values
Identify
Intangible Calculate ROI
Benefits of Training
Tabulate
Program
Costs
Return on Investment Formula
Net Program Benefits
ROI = X 100
Program Costs
Example :
• Costs per program (25 participants) $ 88,500
• Benefits per program (1st year) $230,625
$ 230,625 – 88,500
ROI = X 100
$ 88,500
ROI = 161 %
Collecting Data
• Identify appropriate
Collect performance indicators
Data
• Develop a collection plan
Example of Performance Indicators
Output Time
• Units produced • Equipment downtime

• Items sold • Overtime
• Work backlog • Time to project
• New accounts opened completion
• Productivity • Processing time
• Inventory turnover • Repair time
• Etc. • Lost time days
• Etc
Example of Hard Indicators
Cost Quality
• Unit costs • Scrap

• Variable costs • Waste
• Overhead costs • Rejects
• Operating costs • Error rates
• Number of cost • Rework
reduction • Product defects
• Etc. • Product failure
Example of Performance Results
• Some performance results after training

program:
• Scrap was reduced from 11 % to 7.4 %
• Absenteeism was reduce from 7 % to 3.25 %
• The annual turnover rate was reduced from 30
% to 16 %
• Lost time accidents were reduced 95 %
Isolating the Effects of Training
Using
Control Group
Trend
Methods to Lines
Isolate the
Effects of Participants
Training Estimate
Supervisors of
Participants
Estimate
Using • A control group arrangement can be

Control used to isolate training impact.
Group • With this strategy, one group
receives training, while another,
similar group does not receive
training.
• The difference in the performance of
the two groups is attributed to the
training program.
Trend • Trend lines are used to project the

Lines values of specific output variables if
training had not been undertaken.
• The projection is compared to the
actual data after training, and the
difference represents the estimate
impact of training.
Trend Lines Analysis
Actual sales performance

At the beginning of
The
May, a Sales training difference
Volume of Sales
Program session was represents

held the estimate
impact of
training.
Trend Projection
Jan Feb Mar Apr May Jun Jul Aug
• This method rests on the assumption that

Participants and
participants (and their supervisors) are
Supervisors of
capable of estimating how much a
Participants
Estimate of performance improvement is related to the
Training’s Impact training program.
• Because their actions have produced the
improvement, participants (and their
supervisors) may have very accurate input
on the issue.
• They should know how much of the
change was caused by applying what they
have learned in the program.
• Typical Questions to Estimate :

Participants and
Supervisors of • What percent this improvement
Participants can be attributed to the application
Estimate of
of skills/techniques/knowledge
Training’s Impact
gained in the training program?
• What confidence do you have in
this estimate, expresses as a
percent?
• What other factors contributed to
this improvement in performance?
Example of a Participant’s Estimation
The confidence percentage is multiplied by the

F
estimate (50 % x 70 %) to produce a usable training
factor value of 35 %
Example of a Participant’s Estimation
•
F
The confidence percentage is multiplied by the estimate (50 % x 70 %) to
produce a usable training factor value of 35 %
•
•
No.
This adjusted percentage is then multiplied by the actual amount of
improvement (post-program minus pre-program value) to isolate the portion
attributed to training
The adjusted improvement is now ready for conversion to monetary values,
and used in the return on investment
Converting Data to Monetary Values
• Steps to Convert Data to Monetary

Converting
Values
Data to
Monetary 1. Focus on a unit of improvement
Values 2. Determine a value of each unit
3. Calculate the change in performance

data
4. Determine an annual amount of change
5. Calculate the annual value of

improvement
Steps to Convert Data to Monetary Values
An example to illustrate the steps to convert data to monetary values
Steps Illustration
1. Focus on unit • One grievance reaching step two in the

improvement four-step grievance resolution process
2. Determine a • Using internal experts, the cost of an

value of each average grievance was estimated to be $
unit 6,500 when considering time and direct
costs (V = $ 6,500)
Steps Illustration
3. Calculate the • Six months after the program was completed,

change in total grievances per month reaching step two
performance declined by ten.
data • Seven of the then grievance reductions were
related to the program as determined by
supervisors (isolating the effects of training)
4. Determine an • Using the six month value, seven per month

annual amount (grievance reductions), yields an annual
for the change improvement of 84 (7 x 12 months)
Steps Illustration
5. Calculate the • Annual value = 84 x $ 6,500 = $ 546,000

annual value of
improvement
Tabulating Cost of the Program
Tabulating • Tabulating the costs involves

cost of the monitoring or developing all of the
program related costs of the program
targeted for the ROI calculation.
• Cost components that should be

Tabulating included are :
cost of the
program • The cost to design and develop
the program
• The cost of all program materials
provided to each participant
• The cost for facilitator
• The cost of the facilities of the
training program
• Cost components that should be

Tabulating
included are :
cost of the
program • Travel, lodging, and meal costs
for the participants
• Salaries, plus employee benefits
of the participants who attend
the training
• Administrative and overhead
costs of the training function,
allocated in some convenient
way
An Example to Illustrate ROI analysis
Average Weekly Sales

Post Training Data
Weeks after
training Trained Group Control Group
1 US$ 9,723 9,698
2 9,978 9,720
3 10,424 9,812
13 13,690 11,572
14 11,491 9,683
15 11,044 10,092
Average for weeks 12,075 10,449
13, 14, 15
Annualized Program Benefits

Data US$
Average weekly sales
Trained Group 12,075
Control Group 10,449
Increase 1,626
Profit Contribution from Training 2 % 32.50
(Training Impacts)
Total weekly improvement 1,495
(32.5 x 46 participants)
Total annual benefit 71,760
($ 1,495 x 48 weeks)
Note : 46 participants were still in job after 3 months

Cost Summary : An Illustration

ROI (%):
$ 71,760 - $ 29,090
= x 100
$ 29,090
= 146 %
Facilitation fees
Enhancing Training
Effectiveness
How Effective is Your Training Program?
Broad and Newstrom (1992) report studies have

shown less than 30% of what is actually taught
transfers to the job in a way that enhances
performance.
performance
Source : Broad, M., & Newstrom, J. W. (1992). Transfer of training: Action packed
strategies to ensure high payoff from training investments. Reading, MA: Addison-
Wesley.
Source of Barriers to Training Transfer
• Lack of reinforcement on the job
• Interference from immediate (work) environment
• Nonsupportive organizational culture
• Trainees’ perception of impractical training programs
• Separation from the inspiration or support of the trainer
The Transfer Partnership
Trainee Trainee recognizes need for

new skills
Trainer designs and/or

Trainee Trainer
delivers learning experiences
Manager supports learning

Trainee Trainer
and application on the job
Manager
The Transfer Matrix
Time Periods
Before During After
Manager
Role Players
Trainer
Trainee
Before Training
• Build transfer of training into supervisory

Manager performance standards
• Collect baseline performance data
• Involve supervisors and trainees in needs
analysis process
• Involve trainees in program planning
• Brief trainees on the importance of the training
(course objective, content, process, and
application on the job)
• Review instructional content and materials
• Plan to participate in training sessions
• Encourage trainees attendance at all sessions
Before Training
• Align the training plan with the organization's

Trainer strategic plan
• Systematically design instruction
• Provide proactive opportunities
• Design a peer coaching component for the
program and its follow-up activities
Before Training
• Provide input into program planning

Trainee • Actively explore training options
• Participate in advance activities
During Training
• Prevent interruptions
Manager • Transfer work assignment to others
• Monitor attendance and attention to training
• Recognize trainee participation
• Participate in transfer action planning
• Review information on employee in training
• Plan assessment of transfer of new skills to the
job
During Training
• Develop application-oriented objectives

Trainer • Answer the “WIIFM” question
• Manage the unlearning process
• Provide realistic work-related tasks
• Give individualized feedback
• Provide job performance aid
During Training
• Maintain an ideas and application notebook

Trainee • Participate actively
• Form support groups
• Plan for applications
• Create behavioral contracts
After Training
• Plan trainees’ reentry

Manager • Provide opportunities to practice new skills
• Have trainees participate in transfer-related
decisions
• Reduce job pressures initially
• Give positive reinforcement
• Schedule trainee briefings for co-workers
• Set mutual expectations for improvement
• Arrange proactive (refresher) sessions
• Provide and support the use of job aids
After Training
• Provide follow-up support

Trainer • Conduct evaluation surveys and provide
feedback
• Develop and administer recognition system
• Provide refresher/problem-solving sessions
After Training
• Practice self-management
Trainee • Review training content and learned skills
• Develop a mentoring relationship
• Maintain contact with training ‘buddies’
References/Recommended Further Readings
1. Donald Kirkpatrick, Evaluating Training Programs : The Four

Levels, Berrett-Koehler Publishers. You can obtain this excellent book at this link :
http://www.amazon.com/Evaluating-Training-Programs-Four-Levels/dp/1576753484/ref=sr_1_1?ie=UTF8&s=books
2. Jack J. Phillips and Patricia Phillips, In Action : Measuring

Return On Investment, American Society for Training &
Development. You can obtain this excellent book at this link :
http://www.amazon.com/Action-Measuring-Return-Investment/dp/1562860089/ref=sr_1_3?ie=UTF8&s=books&qid=
3. Mary Broad and John Newstrom, Transfer of training: Action

Packed Strategies to Ensure High Payoff from Training
Investments, Addison-Wesley. You can obtain this excellent book at this link :
http://www.amazon.com/Transfer-Training-Action-Packed-Strategies-Investments/dp/0738205672/ref=sr_1_1?ie=U
End of Material
If you find this presentation useful, please consider telling others

about our site (www.exploreHR.org)

Measuring ROI of Training

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Measuring ROI of Training

Caricato da

Copyright:

Formati disponibili

Measuring

1. Measuring the Effectiveness of Training

2. Measuring ‘Return on Investment’ of Training

3. Enhancing the Effectiveness and ROI of

Training Training Training

What are Objective Techniques Measure

Evaluate trainees’ reactions to Level 1 -

Test the trainees to determine if Level 2 -

Ask whether the trainees’ behavior on the job Level 3 –

What final results were achieved in terms of the Level 4 –

Value of Frequency of Difficulty of

1. Determine what you want to find out

Measuring learning means

1. Use a control group if practical

Pretest and Posttest Scores on Change Management Training

1. Use a control group if practical

• Indicate the extent to which you

1. Use a control group if practical

HARD DATA INDICATORS

SOFT DATA INDICATORS (intangible impacts)

Before training After training

Before training After training

• Importance of the program in meeting the

• Cost of the program

• Visibility of the program

• Size of the target audience

• Extent of management interest

• Alter management perceptions of training

Isolate the Convert Data

• Costs per program (25 participants) $ 88,500

• Benefits per program (1st year) $230,625

• Units produced • Equipment downtime

• Unit costs • Scrap

• Some performance results after training

Using • A control group arrangement can be

Trend • Trend lines are used to project the

Actual sales performance

Program session was represents

Jan Feb Mar Apr May Jun Jul Aug

• This method rests on the assumption that

• Typical Questions to Estimate :

Example of a Participant’s Estimation

The confidence percentage is multiplied by the

• Steps to Convert Data to Monetary

3. Calculate the change in performance

4. Determine an annual amount of change

5. Calculate the annual value of

An example to illustrate the steps to convert data to monetary values

1. Focus on unit • One grievance reaching step two in the

2. Determine a • Using internal experts, the cost of an

3. Calculate the • Six months after the program was completed,

4. Determine an • Using the six month value, seven per month

5. Calculate the • Annual value = 84 x $ 6,500 = $ 546,000

Tabulating • Tabulating the costs involves

• Cost components that should be

• Cost components that should be

Average Weekly Sales

Annualized Program Benefits

Note : 46 participants were still in job after 3 months

Cost Summary : An Illustration

Broad and Newstrom (1992) report studies have

• Lack of reinforcement on the job

• Interference from immediate (work) environment

• Nonsupportive organizational culture