RESEARCH

includes research articles that focus on the analysis and resolution of managerial and academic issues based on analytical and empirical or case research

A Study on Return on Investment of Training Programme in a Government Enterprise in India

K S Subramanian, Vinita Sinha and Priya D Gupta

Executive Summary

Return on investment (ROI) has become one of the most challenging and intriguing issues facing the human resources development (HRD) and performance improvement fields. With ROI, decision makers evaluate investments by comparing the magnitude and timing of expected gains to the magnitude and timing of investment costs. A good ROI means that investment returns compare favourably to investment costs. This study is aimed at testing the feasibility of a Return on Investment (ROI) model in the context of a training programme in the Indian Oil Corporation (IOC), a government organization, to see whether the success of a training programme can be measured more accurately in monetary terms. It is meant to enable the management to understand the direct impact on the business bottom-line as a causal effect of the training imparted and to decide whether to continue or discontinue the programmes. IOC conducted in-house training programmes in 2004 which included a blend of behavioural and functional training. The programmes conducted at this organization were evaluated using the most comprehensive evaluation models and discussed in terms of return on investment. The two programmes chosen for the study were: The Threshold Programme and the Excellence in Project Management, each having 30 participants. The ROI analysis plan document captures information on several key issues necessary to develop the actual ROI calculation: Significant data items The method for isolating the effects of the training and education programme The method for converting data into monetary values The cost categories, noting how certain costs should be prorated The anticipated intangible benefits The communication targets to receive the information Other issues or events that might influence programme implementation.

KEY WORDS ROI Evaluation Training Programme Government Enterprise Indian Oil Corporation India

With a proper planning around a proven framework, realistic evaluation targets, and shared responsibilities for major steps, the ROI process can be implemented in a costeffective, systemic manner and can assist the resource-constrained training function to reap financial benefits that leaders understand and have come to expect. It is now strategically imperative that training be conducted with the clear understanding that if people are truly the organizations greatest asset, then training is beyond doubt, the greatest investment and must hence be utilized wisely.

VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

31

eturn on investment (ROI) has become one of the most challenging and intriguing issues facing the human resources development (HRD) and performance improvement field. The interest in ROI during the 1990s was phenomenal, more so in the new millennium. This topic appears on almost every HRD conference and convention agenda. Articles on ROI appear regularly in HRD practitioner and research journals. Several books have been developed on the topic and consulting firms have sprung up almost overnight to tackle this critical and important issue. ROI is one of the several approaches to evaluating and comparing investments. With ROI, decision makers evaluate investments by comparing the magnitude and timing of expected gains to the magnitude and timing of investment costs. A good ROI means that investment returns compare favourably to investment costs. In the last few decades, this approach has been applied to asset purchase decisions (computer systems, factory machines, or service vehicles, for example), go-no-go decisions for projects and programmes of all kinds (including marketing, recruiting, and training programmes), and to more traditional investment decisions (such as the management of stock portfolios or the use of venture capital).

lect number of programmes. It can determine if the benefits of the programme, expressed in monetary terms, have outweighed the costs and thus whether it has made a contribution and is actually a good investment or not. Sets Priorities By calculating ROIs in different areas, one can determine which programmes contribute the most to the organization, allowing priorities to be established for high-impact training. Focuses on Results Measurement of ROI is a resultbased process which brings a focus on results with all programmes. The process requires instructional designers, facilitators, participants, and support groups to concentrate on measurable objectives what the programme is attempting to accomplish. Thus, the process has the added benefit of improving the effectiveness of all the training programmes. Alters Management Perceptions of Training The ROI process, when applied consistently and comprehensively, can convince the management group that training is an investment and not an expense. Managers will see training as making a viable contribution to their objectives, thus increasing the respect for the function. This is an important step in building partnership with management.

WHY MEASURE ROI OF A TRAINING PROGRAMME?

Several issues are driving the increased interest in, and application of, the ROI process, the most common being: the pressure from clients and senior managers which show that the return on their training investment is probably the most influential drive the competitive economic pressures that are causing intense scrutiny of all expenditures the general trend towards accountability with all staff support groups that is causing some HRD departments to measure their contribution to justify the existence of the training department by showing how it contributes to the organizations objectives and goals to decide whether to continue or discontinue the training programmes.

BASIC ROI ISSUES AND TRENDS

Many professionals argue that most models of the ROI process ignore or provide very little insight into two key elements essential to developing the ROI: Isolating the effects of training Converting data into monetary values While most executives can logically conclude that training can pay off in important bottom-line measures such as productivity improvements, quality enhancements, cost reduction, and time savings, frustration comes from the lack of evidence to show that the process is really working. Organizations have moved from training for activity to training with a focus on bottom-line results and this shift is evident from the beginning to the end of the process.

The ROI Analysis Plan

The ROI analysis plan is a continuation of the data collection plan. This plan document captures information on several key issues necessary to develop the actual

BENEFITS OF ROI
Measures Contribution ROI makes it possible for the HRD staff to know the specific contribution from a se-

32

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

ROI calculation. The key issues include: Significant data items, usually Level 4 (Business Impact), but in some cases could include Level 3 (Application and Implementation) data. However Five levels of measurement are said to be currently in practice, (Phillips, 1997a) (Annexure 1) The method for isolating the effects of the training and education programme The method for converting data into monetary values The cost categories, noting how certain costs should be prorated The anticipated intangible benefits The communication targets to receive the information Other issues or events that might influence programme implementation. These two planning documents are necessary to successfully implement and manage the ROI process.

itself. IOC used training as a major tool to change the attitude sets and skill levels of its employees. Conducting programmes all over the country right through the year, it has succeeded in not only holding its own in todays dynamic business environment, but has also ventured into businesses outside the country successfully.

Training Vision
Be a learning organization, and inculcate skills and competencies required for becoming a transnational, integrated energy company and for value creation for the stakeholders.

Training Mission
Develop employees by enhancing their knowledge, skills, and competencies in functional areas general, strategic management, and advanced functional management, in line with the business of the company. Provide post-experience management education for renewal of executives to global standards. Achieve and sustain customer respect through worldclass products and services. Sustain learning environment through creativity, competence, and recognition of contribution.

Criteria for an Effective ROI Process

The following criteria were developed for an effective ROI process with inputs from hundreds of education and training managers and specialists (Phillips, 1997a). It must be simple, void of any complex formula. It must be economical with the capability to become a routine part of training and development without requiring any additional resources. The assumptions, methodology, and techniques used must be credible. Ideally, the process must strike a balance between maintaining a practical and sensible approach and a sound theoretical base for the process. The ROI process must account for other factors that influence the output variable. The ROI process must be applicable to both hard and soft data. Post-1990, when the economy was liberalized, the country was given firm indications that it would no longer be functioning in a closed, regulated, and secure environment and that it would eventually have to face the open market forces. If a company had to survive in such an environment, it would require a drastic change right from the roots. A lax, laid-back organization would have to re-invent itself into a more competitive player in the industry and a force to reckon with in order to re-invent
VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

SURVEY OF RELATED LITERATURE

Many organizations around the globe are using costsaving approaches so that they can begin conducting ROI evaluation within their current budget while others are using such approaches in order to increase the number of ROI studies they conduct. General cost saving approaching for measuring programmes at the ROI level introduced by Phillips (1997a) have been proven to significantly decrease resource requirements while still providing sound, credible data. Despite these factors, establishing an evaluation culture is no easy task. In many ways, implementing a system-wide ROI effort is similar to implementing a large-scale change initiative. The concept of ROI has been used for centuries. The 75th anniversary issue of Harvard Business Review (HBR) traced the tools used to measure the results in organizations. In the early issues of HBR, during the 1920s, ROI was the emerging tool to place a value on the payoff of investments. In recent years, the application of the con-

33

cept has been expanded to all types of investments including training and education, change initiatives, and technology (Phillips, 2000a). With increased adoption and use, it appears that ROI is here to stay. Today, hundreds of organizations, representing manufacturing, service, non-profit, and government, are routinely using ROI calculations for education and training programmes. A professional society, The ROI Network, with over 500 members, allows practitioners an opportunity to share information and tools around ROI. The networks have been formed within the organizations to focus on the ROI and accountability issue. Almost 1,000 individuals have been certified to implement the process in their organizations. Three casebooks have been developed to show specific applications of ROI (Phillips, 1994; 1997; 2000c). A fourth casebook describes successful implementation of the ROI process (Phillips, 1998). This level of interest and activity is evidence that the ROI process is here to stay. There are good reasons why return on investment is so significant. Although the viewpoints and explanations may vary, some things are very clear. First, in most organizations, education and training budgets have continued to grow year after year. As expenditures grow, accountability becomes a more critical issue. A growing budget creates a larger target for internal critics, often prompting the development of an ROI process. Second, Total Quality Management and Continuous Process Improvement have drawn increased attention to measurement issues. Today, organizations measure processes and outputs that were not previously measured, monitored, and reported. This measurement focus has placed increased pressure on the education and training function to develop measures of programme success. A paper (Buckberry, 2004) has been developed to provide to members of Computer Education and Management Association (CEdMA) some basic introductory information and ideas to assist in the preparation of their own customized ROI processes. CEdMAs clients are typically purchasers of IT software and hardware, and CEdMA members are responsible for the provision of training to these clients. The question this paper addresses is, How do we help customers understand and justify for themselves the need to invest properly and comprehensively in training, and how do we present the comparative benefits of different approaches to training? Implementing some form of measurement proc-

ess is as important for managing training programmes and investment, as it is for any other project requiring significant financial investment by a business. Training programmes consume resources (i.e., they take peoples time and money), but they are also critical to maximizing the return on investment in other programmes or products (e.g., the effective introduction of software systems requires users to be able to use the systems effectively if the potential benefit of the software systems is to be realized in practice), as well as generally improving the productivity of the workforce. If ROI is to be successfully managed and measured, it is important that the process be included early in the planning cycle for the training programmes. (Buckberry, 2004). A study was conducted by Reed (1986) to determine the net impact of Job Training Partnership Act (JTPA) training. (A measure of net impact expresses only those gains due to training and not those due to other reasons.) Job service applicants were chosen as a comparison group whose recent labour market experiences would parallel those of JTPA participants. All results were positive. For men who participated in JTPA in 1983-84, the estimated effect of training was an additional $1,400 earned during 1985. It appeared likely that the benefit persisted in 1986. For white women, the estimated effect of training was an additional $1,000 earned in 1985. A separate estimate of the impact on earnings was made for recipients of Aid to Families with Dependent Children (AFDC). For them, the effect was an additional $1,200 in 1985 earnings. Because their initial earnings were so low, this amounted to approximately 100 per cent of the 1979 income. The reduction in AFDC grant amounts attributable to training was another measure of impact used. Twelve months after enrollment, AFDC recipients who participated in training were 86 per cent more likely not to be receiving assistance than were their counterparts who did not participate. Collins, Collins and Jensen (2009) concluded from their study that wisely allocating financial resources is essential to the success of every health care organization. Therefore, health care managers must be able to determine if dedicating the necessary funds for employee training results in an adequate return on investment. This case study examines how training programmes can be evaluated in terms of business results and describes one method, simple regression analysis that health care managers may use, to help determine if the training was financially beneficial to the organization.

34

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

Philips and Philips (2009) describe the ROI methodology, a measurement process that was developed almost 30 years ago and refined over the years to the point that it is now becoming a staple for many HR functions. During difficult times in the economy, nothing is more important to top executives than knowing the true value of a particular project or programme. Show me the money has become a battle cry for many executives demanding that any new HR project or programme shows its value even before it is implemented and, certainly, the impact and return on investment (ROI) after it has been implemented. Around the globe, HR executives are taking a look at the ROI process as a way to show credible values, including financial ROI. This article describes why and how ROI is used to show the contribution of HR programmes and improve them further so that they can add more value, build support for HR, enhance commitments, and concretize important business relationships. This method can be used to show the value of major programmes and projects and establish HR as a business partner. With the ROI process, the HR staff and the client would know the specific contribution of an HR programme.

Dependent Variable: Change in the attitude sets and skill levels of the employees and direct impact on the business bottom-line have been considered as dependent variable in the present study. It includes change in knowledge, skills, and competencies enhancement in functional areas as well as change in attitude and the behavioural aspects of the employees.

Programmes/Sample
The two programmes chosen for the study were: 1) Threshold Programme (30 Participants) 2) Excellence in Project Management Programme (30 Participants) Threshold Programme This particular programme was chosen for the purpose of ROI calculation since this is one of Indian Oils most elite programmes. The programme has been designed to enhance the general management skills of senior managers of the company and to help them acquire a comprehensive strategic perspective so that they are prepared to lead the organization with confidence to meet the emerging challenges. This is the programme where the best of the best are trained to enable them to cross over from middle management into the rungs of top management. Excellence in Project Management Programme Due to the increased investment envisaged by the corporation to meet the current challenges in technological, environmental, and customer expectations in terms of quality in delivery, the Excellence in Project Management Programme becomes one of the most vital ingredients in the recipe for success.

METHODOLOGY Objectives
This study is aimed at testing the feasibility of a Return on Investment (ROI) model within the organization so that the success of a training programme or a set of training programmes can be measured more accurately in monetary terms, thus enabling the management to understand the direct impact on the business bottom-line as a causal effect of the training imparted.

Variables
Independent Variable: The effects of behavioural and functional training imparted to the employees have been coined as independent variable. IOC used training as a major tool to change the attitude sets and skill levels of its employees. It conducted 2,434 in-house training programmes which included a blend of behavioural and functional training. The training was intended to develop employees by enhancing their knowledge, skills, and competencies in functional areas/general/strategic management and advanced functional management, in line with the business of the company.

Collecting Post-Programme Data

The questionnaires for Threshold Programme Evaluation and Excellence in Project Management Programme (Refer to Annexure 3) have been implemented to ascertain the degree of success in meeting the objectives of the training programme at IOC.

Converting Data into Monetary Benefits

Several methods such as calculating the cost of quality, using historical costs, and internal and external experts inputs and converting employee time could have been used but due to lack of time and data, the method chosen for this study was to use the estimates from participants.

VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

35

In some situations, programme participants are capable of estimating the value of a soft data improvement. This method seems appropriate when participants are capable of providing estimates of the cost (or value) of the unit of measure improved by applying the skills learned in the programme. The advantage of this approach is that the individuals closest to the improvements are also often the ones most capable of providing the most reliable estimates of its values.

Notes on Cost Evaluation

1) The cost of conducting needs assessment has been excluded from the costs. This might include time of staff members conducting the needs assessment, direct fees and expenses for external consultants who conduct the needs assessment and internal services and supplies used in the analysis. 2) A final charge is the cost of overhead, the additional costs in the training function not directly related to any particular programme. Typical items might include the cost of clerical support, the departmental Table 2: Costs Considered in the Excellence in Project Management Programme at IOC
No. Cost Element Consolidated Amount (Rupee Value) 19,200 1,68,000 14,875 27,500 34,755 3,850 2,40,000 5,08,180

Estimating Programme Costs

Most of the data relating to programme costs was available from the Project Completion Reports of the programmes. The only additional cost which has been included is the cost of participants time which had not been previously taken into account. This element has been considered while trying to estimate programme costs because this represents the time that the participants have spent attending the programme which they would have otherwise spent at the workplace. The CTC (cost to company) of the participants represent an expense that should be included. For situations where the programme has been conducted, these costs can be estimated using average or midpoint values for salaries in typical job classifications (Tables 1 and 2).

1 2 3 4 5 6 7

Faculty honorarium Stay @ Rs. 1,500 per day/participant Catering @ Rs. 125 per day/participant Photocopying, stationary and brochure Air and local travel Photography and books Cost of participants time @ Rs. 8,000 per participant (assumed average CTC) Total

Source: HR (Dept.), IOC.

Table 1: Costs Considered in the Threshold Programme at IOC

No. 1 Cost Element Faculty Honorarium: Vedanta Thomas Intl. APTECH Comp. Update IIM-A Stay @ Rs. 1,500/- per day/participant for 22 SMs/CMs and @ Rs. 2,400/- for 8 DGMs for 28 days Catering @ Rs. 125/- per day/participant for 30 persons for 28 days Photocopying Stationary incl. Ties and Bags Air Travel and Local Travel (If borne by IIPM) Others: (Foreign Module) a. Air Fare @ Rs. 35,726 for 31 people b. Visa + Ins. @ Rs. 2,700 for 31 people c. Foreign Allowance (27 people + 1 coordinator) Cost of participants time @ Rs. 90,000 per participant (assumed average CTC) Total Amount (Rupee Value) 7,500 29,160 12,900 20,000 11,98,800 Consolidated Amount (Rupee Value)

2 3 4 5 6 7

12,68,360 8,01,600 1,05,000 43,000 18,220 5,000

11,07,506 8,37,000 24,14,250

43,58,756 27,00,000 92,999,36

Source: HR (Dept.), IOC.

36

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

office expenses, salaries of training managers, and other fixed costs. Organizations might obtain an estimate for allocation by dividing the total overhead by the number of programme participant days for the year. For the sake of simplicity, such allocation of overheads has been excluded from programme cost tabulation.

Benefits/Costs Ratio One of the earliest methods for evaluating training investments is the benefits/costs ratio (BCR). This method compares the benefits of the programme to the costs in a ratio. BCR = Programme Benefits Programme Costs

Calculating the Return

The return has been calculated in three different ways:

In simple terms, BCR compares the annual economic benefits of the programme to the costs of the programme.

Calculation of Return*
The Threshold Programme 1) Benefits/Costs Ratio This is calculated as follows: BCR = Programme Benefits Programme Costs 32,50,000 15,49,989 2.097 = = Excellence in Project Management Programme 1) Benefits/Costs Ratio This is calculated as follows: BCR = Programme Benefits Programme Costs 9,87,750 1,86,333 5.3

= =

2) ROI Formula This is calculated as follows: (Programme Benefits Programme Costs) Programme Costs (32,50,000 15,49,989) 15,49,989 109.68%

2) ROI Formula This is calculated as follows: (Programme Benefits Programme Costs) Programme Costs (9,87,750 1,86,333) 1,86,333 430% (approx)

ROI(%)

x 100

ROI (%)

x 100

= = 3) Payback Period Payback Period

x 100

= = 3) Payback Period

x 100

= = =

Total Investment Annual Savings 15,49,989 17,00,011 0.91 years or 11 months (approx) = = Net Benefits - Costs Breakup Payback Period =

Total Investment Annual Savings 1,86,333 8,01,417 0.23 years or 3 months (approx)

Net Benefits - Costs Breakup

Costs Net Benefits Costs

Net Benefits

* The costs of both programmes have been adjusted on the basis of usable responses received. For example, there were thirty participants in the Threshold Programme but only five usable responses were received. Therefore, the cost taken into consideration for the programme was onesixth of the total cost incurred.

VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

37

A BCR of one means that the benefits equal the costs. A BCR of two, usually written as 2:1, indicates that for each rupee spent on the programme, two rupees were returned as benefits. ROI Formula Perhaps the most appropriate formula for evaluating training investments is net programme benefits divided by cost. The ratio is usually expressed as percentage when the fractional values are multiplied by 100. ROI can thus be expressed as: ROI (%) = Net Programme Benefits Programme Costs x 100

5) The values are adjusted for the amount of the improvement related directly to the programme. These five adjustments create a very credible value that is usually considered to be an understatement of the benefits accrued due to the training imparted. Tables 3 and 4 give the estimates of training impact from participants in the different programmes.

LIMITATIONS OF THE STUDY

The method chosen to isolate the impact of training is meant to obtain information directly from the programme participants. The effectiveness of this approach rests on the assumption that participants are capable of determining or estimating how much of a performance improvement is related to the training programme. This is one of the major methodological limitations of such studies. Erroneous, incomplete, and extreme information could have distorted the analysis. While calculating ROI through this method, the participants should know how much of the change was caused by applying their learning from the programme. Apart from this, the following limitations have been figured out in the present study: 1) The exact figure for cost of participants time was not available. Hence there is a possibility that the estimations made are not accurate. 2) Suitable responses from all the participants were not available. This means that the effectiveness of both training programmes taken into consideration has been an extrapolation of the responses, which were made available. The assumption here is that similar benefits would have accrued from the participants who did not make their response available. 3) There was no possibility of even experimenting with the control groups since the project commenced posttraining. 4) The participants of both the programmes were posted all over the country, making distance a barrier for conducting in-depth interviews. The presence of participants across the nation also made it difficult to obtain post-training data from more than one source, i.e., the participants themselves. This was a unique problem that was uncovered particularly with the Threshold Programme participants. Since they primarily occupy high leadership positions, they do not

The ROI value is related to the BCR by a factor of one. This means, for example, that a BCR of 2.56 is the same as an ROI value of 156 per cent. An ROI on a training investment of 60 per cent means that an additional 60 per cent of the costs are reported as earnings. An ROI on training investment of 150 per cent indicates that the costs have been recovered and an additional 1.5 multiplied by the costs is captured as earnings. Payback Period The payback period is a common method for evaluating capital expenditures. With this approach, the annual cash proceeds (savings) produced by the investment are equated to the original cash outlay required by the investment to arrive at some multiple of cash proceeds equal to the original investment. Measurement is usually in term of years or months. Payback Period = Total Investment Annual Savings

DATA ANALYSIS AND RESULTS

The calculations are based on a series on impact questions. Five adjustments are made to ensure that the data is credible and accurate: 1) The participants who do not complete the questionnaire or provide usable data on the impact questions are assumed to have made no improvement. 2) Extreme and unrealistic data have been omitted. 3) Only annualized values are used as requested in the responses. 4) The values are adjusted to reflect the confidence level of participants.

38

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

Table 3: Estimates of Training Impact from Participants in the Threshold Programme

Participant Improvement (Rupee Value) 30,00,000 25,00,000 20,00,000 40,00,000 30,00,000 Basis Confidence Level(%) 30 60 40 30 60 Isolated Effect Percentage Estimate(%) 50 40 80 70 40 Conservative Integration (Rupee Value) 4,50,000 6,00,000 6,40,000 8,40,000 7,20,000 32,50,000

1 2 3 4 5

Better decision making Efficient management Quick and accurate decisions Efficient handling of situations and increased teamwork within department Good decisions Total Benefits

Table 4: Estimates of Training Impact from Participants in Excellence in Project Management Programme
Participant Improvement (Rupee Value) 1,60,000 3,00,000 3,00,000 2,00,000 2,75,000 3,00,000 4,00,000 2,00,000 3,50,000 1,40,000 1,50,000 Basis Confidence Level (%) 90 60 70 50 60 70 80 70 85 50 100 Isolated Effect Percentage Estimate(%) 50 50 60 50 40 50 40 50 50 60 60 Conservative Integration (Rupee Value) 72,000 90,000 1,26,000 50,000 66,000 1,05,000 1,28,000 70,000 1,48,750 42,000 90,000 9,87,750

1 2 3 4 5 6 7 8 9 10 11

Additional Sales Decreased time overruns Reduced cost of rework Minimizing startup hiccups Reduced maintenance problems Increased site effectiveness Reduced maintenance problems Reduced project time Reduced on-site problems Efficient time and labour management Under budget Total Improvement

have any activity within their purview which directly impacts the bottom line of the organization. It is generally their ability to handle the employees, who directly boost the bottom line well, which is their real contribution. This indirect effect makes quantifying the benefits derived due to training rather difficult.

RECOMMENDATIONS AND IMPLICATIONS FOR FUTURE RESEARCH

For organizations implementing the ROI concept for the first time, it is recommended that only one or two courses be selected for an initial calculation, as a learning process (Phillips, 1997). If appropriate and feasible, participants should receive prior communication about the requirement for a follow-up questionnaire. This minimizes some of the resistance to the process, provides an opportunity to explain in some more detail the circumstances surrounding the evaluation, and positions the followup evaluation as an integral part of the programme not as an add-on activity that someone initiated three
VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

months after the programme. Management involvement at the local level might prove critical to the response rate success. Managers can distribute the questionnaires themselves, make reference to the questionnaire at staff meetings, follow up to see if the questionnaire has been completed and generally show the support for completing the questionnaire. This direct supervisor support might cause some participants to respond with usable data. Even if it is an abbreviated form, participants should see the results of their study. More importantly, participants must understand that they will receive a copy of the study when they are asked to provide the data. This promise might increase the response rate, as some individuals want to see the results of the entire group along with their particular input. It is difficult to evaluate an entire HRD function such as management development, career development, executive education or technical training within the ROI umbrella. ROI is more effective when applied to one programme that can be linked to a direct payoff.

39

For this reason, ROI evaluation must be a micro-level activity that will usually focus on a single programme or a few tightly integrated programmes. This decision to evaluate several programmes or just one programme should include consideration of the objectives and timing of the programme. Attempting to evaluate a group of programmes conducted over a long period becomes quite difficult. The cause and effect relationship becomes more confusing and complex. Incentives might be used to obtain a greater response results from the participants. For example, one might pin a ten rupee note to the questionnaire and add a heading on the questionnaire which states Please fill out this form over a cup of coffee. Another incentive might be to send a pen along with the questionnaire along with a note which states Kindly use this pen to fill out the form. These methods have been known to boost response rates.

Training can no more be a mundane task which exists because it always has. It is now strategically imperative that training be conducted with the clear understanding that if people are truly the organizations greatest asset, then training is beyond doubt, the greatest investment and must hence be utilized wisely. By evaluating training programmes with the ROI in mind, training functions can be perceived in a more credible light. Programmes aligned with organization strategy are offered, while others that add little value are redesigned and sometimes eliminated. With proper planning around a proven framework, realistic evaluation targets, and shared responsibilities for major steps, the ROI process can be implemented in a cost-effective, systemic manner and can assist the resource-constrained training function to present their work in terms of financial benefits that leaders understand and have come to expect. Specifically, part one of this ROI series emphasized the following cost-savings approaches: 1. 2. 3. 4. 5. Plan for evaluation early in the process Build evaluation into the training process Share the responsibilities for evaluation Require participants to conduct major steps Use short-cut methods for major steps.

CONCLUSION
Any organization wanting to improve systems, procedures, or even attitudes must plan accordingly. This planning must be integrated and aligned with the business. The effort to use training as a tool to help balance an organization in a dynamic environment must be continuous. There must be constant matching of individual and organizational needs in a system which is biased towards neither. Once this matching is completed and training programmes carried out, it is then not only important to evaluate the training but to realize how much it has boosted the bottom line. Here is where the ROI comes to the fore possibly on helping management realize that training is truly an investment and not an expense.

Part two will continue to describe practical application of five additional cost-saving approaches to ROI implementation: 6. Use sampling to select the most appropriate programmes for ROI analysis 7. Use estimates in the collection and analysis of data 8. Develop internal capability 9. Streamline reporting 10. Utilize technology.

Annexure 1: Five Levels of ROI Evaluation

The evaluation levels categorize data, reporting a chain of impact as reaction leads to learning, to application, to impact, and to return on investment.
ROI Evaluation Level 1. Reaction & Planned Action 2. Learning 3. Application and Implementation 4. Business Impact 5. Return on Investment
Source: Phillips, 1997.

Measurement Focus Measures participant satisfaction with the programme and captures planned actions Measures changes in knowledge, skills, and attitudes Measures changes in on-the-job behaviour and progress with application Captures changes in business impact measures Compares programme monetary benefits to the programme costs.

40

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

Annexure 2: Training at IOC

IOC conducts on an average approximately 2,000 training programmes per year which are conducted in-house, leveraging expert faculty from renowned institutes/organizations and well-trained business leaders from within the company. The training plan contains a variety of programmes which meet intrinsic needs of different functions. Some of the programmes are made mandatory at the corporate level such as Vigilance Awareness Programme, Communication & Presentation Skills, Gender Sensitivity Programme, and Security Sensitization. However, majority of the Divisions carry out additional programmes based on Here and Now Needs of the customer divisions such as Safety and Hazardous Operations, Safety Integrated Learning, Hydrotreatment Processes for Middle Distillates, and Treasury Management. regular basis and Enterprise Resource Planning Solutions and related training. Focus on quality, ISO Certification/accreditation, adoption of Business Excellence Model by strategic business units and benchmarking with national/international training providers. Train critical position holders extensively in India and abroad to handle business ambiguities and complexities and gear up for the new business outside India. Develop in-house Petroleum based international MBA programme with in-built foreign module to create a global mindset and entrepreneurial qualities among executives. Depute sizable number of executives for MBA courses in other Institutes of India for creating a mix of executives with general MBA and petroleum-based MBA. Conduct Management Development Training Programmes in niche business schools in India and abroad for understanding global business trends. Make a paradigm shift from Supply and Distribution to Marketing Strategies and CRM Train Executives for business integration, diversification, and entering into joint ventures and handling newly acquired companies.

Training Strategy
Align and integrate learning with the companys business. Identify skill and competency gaps at all levels and create groups for delivering in-house trainings. Identify courses outside IOC/India and depute key employees handling new technology, practices or new business to attend these courses. Set up a world-class institute for creating critical mass of highly trained middle and senior level executives for occupying top level positions to the organization and the industry/nation. Set up a learning centre for training of marketing executives on customer-orientation, marketing of products and other marketing specific programmes. Organize cutting-edge training on quality, cost, attitude, customer service and communication network. Conduct brain-storming sessions with the key executives, senior and middle management groups, unions and working levels, for deciding strategies for the changed scenario and involvement of employees. Organize regular interaction with top management for deliberations on specific issues in an organized way. Impart training to senior top managers in advance technologies and latest managerial tools and techniques for maintaining cutting edge of the organization and developing leaders with all-round qualities. Have multi-pronged communication about the imminent competition across the organization on a
VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

Focus Areas of Training

Corporate Focal Area Strategic and General Management Operations & Technology Management Finance Management Human Resource Management & Organizational Behaviour related courses Information Technology & Enterprise Resource Planning related trainings (SAP) Marketing Management Internal Audit, Security and Vigilance Gender Sensitization Women Development

Refineries Division Projects Management Operations Management Programmes on Quality Initiatives Maintenance and Inspection Fire and Safety

41

Product Customization Petrochemicals Materials Management Human Resource Management Finance and Accounting Management Tendering Procedure and Contracts Management Information Systems Management Corporate Communications and Media Management General Management courses such as: Executive Stress Management, YOGA for Good Health and Rejuvenation

Pipelines Transportation Management Operations of Pipelines Fire and Safety Management Materials Management Maintenance and Upkeep of Pipelines/Pumps Finance Management Human Resource Management Contracting and Tendering Total Quality Management

R&D Marketing Engineering Services Customer Orientation Brand Management and CRM Productivity Courses Marketing Strategy and Market Research: Tools and Techniques Quality Programmes Human Resource Management Finance Management Supply and Distribution courses Simulation Games Fire and Safety Management Information Systems Management R&D Management Purchase and Work Procedures Best Practices and Benchmarking Creativity and Innovation Product Knowledge Analytical Techniques in Petroleum Analysis Basics of Tribology Oily Sludge Management New Environmental Regulations Fundamentals of Lubrications Petroleum Refining Technology Hydroprocessing Workshop

Annexure 3: Questionnaires for Programme Evaluation Threshold Programme Evaluation

1) Listed below are the objectives of the Threshold programme. After reflecting on the programme, please indicate the degree of success in meeting the objectives:
Failed Limited Success Generally Successful Completely Successful

As a result of this programme, participants will be able to: a. Develop a general management orientation through highlighting the inter-linkages across decisions in functional area b. Strengthen the understanding of key issues and challenges in strategy formulation and implementation c. Enhance the awareness about the threats, constraints and opportunities that have arisen from operating in increasingly deregulated product, services and resource markets d. Sensitize the participants to the need to create value through appropriate corporate actions for long-term survival and growth of the organization

42

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

2) Please rate on a scale of 1-5, the relevance of each of the programme elements to your job by indicating (1) not relevant and (5) very relevant 1 Interactive Activities Group Discussions Networking Opportunities Reading Materials / Video Programme Content 2 3 4 5

3) Please indicate the degree to which your use of the following actions was enhanced as a result of your participation in the training programme:
No Little Some Change Change Change Significant Change Momentous Change No Opportunity to Use Skill

Taking decisions with a sound analysis of the same Making the unit/department more market-oriented Focusing on customer analysis and customer value Effectively using the information technology within the organization Understanding and managing co-workers and superiors Effectively measuring and managing risk Engaging in vertical integration, diversification and/or mergers and acquisitions Ability to envision for your organization Enhancing your leadership traits Better handling the competition faced by your company 4) Please identify any specific accomplishments that you can link to this training programme (on time schedules, project completion, response times, etc)

VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

43

5) What specific value in Indian Rupees can be attributed to the above accomplishments/ improvements (use first year values only)? While this is a difficult question, try to think of specific ways in which the above improvements can be converted into monetary units. Please indicate the basis for your calculation. INR Basis:

6) What level of confidence do you place on the above estimations? (0% = No Confidence, 100% = Certainty) 7) Other factors generally tend to affect performance as well. Please indicate the per cent of the above improvements that is related directly to this programme.

% %

8) Indicate the extent to which you think this programme has influenced each of these measures in your work unit, department or business unit:
No Influence Some Influence Moderate Influence Significant Influence Momentous Influence

Productivity Customer Response time Cost Control Employee Satisfaction Customer Satisfaction Quality Other 9) What barriers, if any, have you encountered that have prevented you from using skills or knowledge gained in this programme?

10) What specific suggestions do you have for improving the programme?

44

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

Excellence in Project Management Programme Evaluation

1) Listed below are the objectives of the Excellence in Project Management programme. After reflecting on the programme, please indicate the degree of success in meeting the objectives:
Failed Limited Success Generally Successful Completely Successful

As a result of this programme, participants will be able to: Grasp the concepts of Project Management in the current scenario Grasp the latest trends in Project Management for optimum utilization of resources available for the projects Reduce the execution time by using modern construction equipment Identify key factors for improvement in time, cost and quality of projects 2) Please rate on a scale of 1-5, the relevance of each of the programme elements to your job by indicating (1) not relevant and (5) very relevant 1 Interactive Activities Group Discussions Networking Opportunities Reading Materials / Video Program Content 2 3 4 5

3) Please indicate the degree to which your use of the following actions was enhanced as a result of your participation in the training programme:
No Little Some Change Change Change Significant Change Momentous Change No Opportunity to Use Skill

Minimizing the non-contributing actions per project Arranging the site for maximum effectiveness Assigning appropriate work for involved employees Dealing with problems on site Keeping the project members focused Accomplishing project objectives Evaluating the project Implementing action plans Planning a follow-up activity

VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

45

No Little Some Change Change Change

Significant Change

Momentous Change

No Opportunity to Use Skill

Organizing projects efficiently Understanding of the project life cycle Linking realistic objectives to stakeholder needs Establishing dependable monitoring techniques Estimating project costs Preparing realistic time schedules Using modern construction equipment efficiently in mega projects Controlling costs efficiently Increasing the level of safety at construction sites Monitoring of the project Minimizing start-up hiccups post-project completion Envisaging maintenance problems at project completion stage and minimizing the same 4) Please identify any specific accomplishments that you can link to this training programme (on-time schedules, project completion, response times, etc.)

5) What specific value in Indian Rupee can be attributed to the above accomplishments/improvements (use first year values only)? While this is a difficult question, try to think of specific ways in which the above improvements can be converted into monetary units. Please indicate the basis for your calculation. INR Basis:

6) What level of confidence do you place on the above estimations? (0% = No Confidence, 100% = Certainty)

46

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA

7) Other factors generally tend to affect performance as well. Please indicate the per cent of the above improvements that is related directly to this programme.

8) Indicate the extent to which you think this programme has influenced each of these measures in your work unit, department or business unit:
No Influence Some Influence Moderate Influence Significant Influence Momentous Influence

Productivity Customer Response time Cost Control Employee Satisfaction Customer Satisfaction Quality Other 9) What barriers, if any, have you encountered that have prevented you from using skills or knowledge gained in this programme?

10) What specific suggestions do you have for improving the programme?

REFERENCES
Buck Berry, Norman (2004). Summary Process for Measuring ROI of Training, Version Draft 5, May 2, Prepared for CEdMA Europe. Collins, Sandra K; Collins, Kevin S and Jensen, Steven C (2009). Determining Return on Investment for Training Using Simple Regression: A Hypothetical Case Study for the Health Care Industry, The Health Care Manager, 28(1), 30-37. Hasset, J (1992). Simplifying ROI, Training, September, 54. Kirkpatrick, D L (1975). Techniques for Evaluating Training Programs, Evaluating Training Programs, Alexandria, VA: American Society for Training and Development. Phillips, J J (1994). In Action: Measuring Return On Investment, Vol. 1. Alexandria, VA: American Society for Training
VIKALPA VOLUME 37 NO 1 JANUARY - MARCH 2012

and Development. Phillips, J J (1996). How Much is the Training Worth? Training and Development, 50(4), pp. 20-24. Phillips J J (1996). Accountability in Human Resource Management, Gulf Publishing Co. Phillips, J J (1997). Handbook of Training Evaluation and Measurement Methods, 3rd Edition, Houston, TX: Gulf Publishing. Phillips, J J (1997). Return on Investment in Training and Performance Improvement Programs, Butterworth Heineman. Phillips, J J (1997a). Return On Investment in Training and Performance Improvement Programs. Houston, TX: Gulf Publishing.

47

Phillips, J J (1998). Implementing Evaluation Systems and Processes, Alexandria, VA: American Society for Training and Development. Phillips, J J (2000a). The Consultants Scorecard, NY: McGrawHill. Phillips, Jack and Phillips, Patti (2009). Measuring Return on Investment in HR, Strategic HR Review, 8(6), 12-19.

Pine, J and Tingly, JC (1993). ROI of Soft Skills Training, Training, February, 55-66. Reed, David (1986). The Return on Investment from Indianas Training Programs Funded through the Job Training Partnership Act, Report, Indiana State Office of Occupational Development, Indianapolis, October.

Acknowledgement. We acknowledge the contribution of B Vasanth Gopalan, an ex-student of SCMHRD, who collected the data from IOC and conducted the study. We are also thank-

ful to IOC for granting us permission to publish the results of our analysis.

K S Subramanian is the Director of Symbiosis Centre of Management & Human Resource Development, Pune. A Postgraduate in Commerce from Wadia College, he had won a Gold medal in his Masters with Marketing and Human resources from the Symbiosis Institute of Business Management. He served as a commissioned officer in the Indian Army in Infantry, Anand Business Group, Aditya Birla Group, Grasim and Indo Gulf Division, as Vice President of the Site for Dabhol Power plant, and BPL technology group for their cellular venture in Maharashtra and Goa. e-mail: director@scmhrd.edu Vinita Sinha is a faculty (OB&HR) at the Symbiosis Centre of Management & Human Resource Development, Pune. She has completed her doctorate in Psychology, has a Masters in Psychology with specialization in OB, and a Post Graduate Diploma in Human Resource Management from IMT Ghaziabad, Post Graduate Diploma in Health Psychology and Behaviour Modification from Amity University. She was the 1st rank holder in the college in both the years of her Masters. Her past work experience lies with the Great Lakes Institute of Management, Chennai; BIMTECH, Greater Noida and EMPI Business School, New Delhi. She has published numerous research papers, articles, and book reviews in the refereed international

and national journals and has authored two books entitled, Implications of Hardiness and Social Support: Psychological Wellbeing-A Recent Perspective and The Burnout Process with Lambert Academic Publications, Germany. e-mail: vinita_sinha@scmhrd.edu Priya D Gupta is an Associate Professor in the Symbiosis Centre for Management and Human Resource Development (SCMHRD), Pune. She is pursuing her doctoral degree in the area of Training and Development from the Tata Institute of Social Sciences, Mumbai. She has done her Masters in Human Resource Management and also holds a Diploma in Training and Development from the Indian Society of Training and Development, New Delhi. She has about 12 years of experience in the area of Human Resource Management, her core areas of expertise being Training and Development, Performance Management, and Employee Relations. She has conducted workshops in the areas of self-development, motivation, leadership, communication; team building, human resource management, etc., for different organizations. She was the editor of SCMRDs first peer reviewed Journal, OPUS (Organization People and Us), for three years. e-mail: priya_gupta@scmhrd.edu

48

A STUDY ON RETURN ON INVESTMENT OF TRAINING PROGRAMME IN A GOVERNMENT ENTERPRISE IN INDIA