6th INTERNATIONAL CONFERENCE ON EDUCATION May 21-23, 2004 Athens, Greece

A cost-effectiveness analysis of vocational training in Spain

Gonzlez Veiga, C., cveiga@uniovi.es Cueto Iglesias, B., bcueto@uniovi.es Mato Daz, J. jmato@uniovi.es University of Oviedo, Spain.

A COST-EFFECTIVENESS ANALYSIS OF VOCATIONAL TRAINING IN SPAIN

Education for Employment and Vocational Training in Spain is organized in three subsystems. First, Technical Vocational Education, included in the education system. Second, Vocational Training, which is mainly directed at updating the skills of the unemployed and to help them back into employment. Third, Continuing Education taught by firms and by public and private organisms, directed the employed population and aimed at retraining and at the maintenance of professional skills. The three subsystems of Vocational Education have been developed in parallel until the present time, with little coordination between them. This is partly caused by differences in target groups, although the subsystems have in common the employment and the professional integration of the participants as basic goals. The o bjectives of this paper are to present each one of the three subsystems and to compare them in terms of cost-effectiveness. To this aim, the basic indicator is the average cost per job held by the participants in vocational education and training in the Spanish region of Asturias. Several variants of this indicator are used that take into account different time horizons and different estimations of employment rates.

1.

Introduction

In order to fight unemployment the European Union institutions promote different training measures that are co-financed via the Structural Funds (the European Social Fund). Employment training aims to produce the skills that are required in the labour market, which would allow the trainees to fulfil the needs of firms and t o increase economic growth as well as reducing unemployment. As happens in other countries, in Spain there exist different measures regarding employment training. However, they are not directed at the same target population nor are they equally effective. The main objective of this paper is to analyse the results obtained by the different types of employment training (Technical Vocational Education, Vocational Training and Continuous Training). Cost-effectiveness analysis was chosen as the method to analyse and compare these training measures. This is a useful means to take into account not only the results of employment training but also the different costs of policies that have as a common objective getting people into jobs. The search for efficiency in p ublic expenditure, always a reasonable objective, seems an appropriate goal given the sharing of training costs by the EU institutions. Although data used in this paper correspond to a specific Spanish region, the design of the research and the techniques used are such that potential comparisons would be of interest and go beyond the framework of this study.
The paper is organised as follows. In the second part the different subsystems of Spanish professional training are briefly explained and their relationship with co-financing European Social Fund measures. The third part describes the main features of participants in the training programmes. After that, the fourth part compares the employment results obtained in each of the training subsystems and the fifth introduces cost-effectiveness analysis and applies it to a new comparison of the measures. The main conclusions and bibliographic references end the paper.

2.

The employment training system in Spain

Up to the present time, employment training in Spain does not form an integrated system but it is carried out through three independent subsystems or channels. Firstly, Technical Vocational Education depends on the education authorities. It covers a Basic Professional Education (Formacin Profesional de Base) received by secondary education students and a Specific Professional Education (Formacin Profesional Especfica). This is taught via intermediate level modules directed at students who finish compulsory secondary education (enseanza secundaria obligatoria, ESO) at sixteen and through upper level modules directed at students who finish non-compulsory secondary education al eighteen. Also, the Programas de Garantia Social take care of early school leavers who do not finish the compulsory ESO. Secondly, Vocational Training is oriented towards the needs of the unemployed and depends on the Labour authorities. As an active labour market policy, Vocational Training tries mainly to improve the professional careers of the unemployed by qualifying and updating their professional skills. Finally, Continuous Vocational Training covers a group of training measures carried out by firms, firms organisations and trade unions, which try to improve the qualifications and skills of the employed population as well as to retrain workers. In the past, the three subsystems of employment training have been developed in parallel with little coordination. Each one is directed at a different target population

but, however, the common goal is the improvement of the working perspectives of the participants. This basic common feature allows us to suggest a comparative evaluation of training according to effectiveness in the access of participants to employment. An interesting trait of employment training is that all three subsystems receive important funding from the European Union. Axis number four of the Community Support Framework (CSF) is dedicated to human resource policies. Therefore, training measures have been included in the Integrated Operating Programme of the CSF 2000-2006 in the region where this study takes place, Asturias, so as to obtain funding from the European Social Fund.

3.

Characteristics of participants in the training programmes

This paper presents part of a greater research project dealing with the evaluation of ESF activities in the objective n1 region of Asturias (Spain)1 and with the evaluation of the working patterns of technical education graduates2 . Within this evaluation project surveys have been carried out on representative samples of participants in the measures that form the Integrated Operating Programme of the CSF 2000-2006. The evaluation required data on the implemented measures and on the participants, emphasizing the employment status of the latter. Therefore, two types of data have been used. On the one hand, administrative registers of the Asturian Government have produced information on the costs of each measure and the number of participants. On the other hand, surveys of samples of participants have produced data on individual socio-economic and employment characteristics both at the time of participation and after. In 2001, the reference year for the evaluation, expenditure in training measures amounted to more than 16.8 millions of euros and the number of participants exceeded 14,000 people (Table 1). The most important measure in terms of expenditure is Professional Education Modules (Ciclos Formativos) and the greatest measure in terms of participants is Continuous Training. At this stage showing some differences between measures is in order. First, participants in Professional Education Modules (Ciclos Formativos) are mostly full time students and therefore are considered as out of the labour market. The duration of these courses is longer than the academic year. Second, vocational training is directed at the unemployed although it may happen at times that participants are not in the labour market or are studying. These courses tend to last about 250 hours. Finally, Continuous training of the working population is carried out taking into account the greater opportunity cost of workers time and it is for this reason that the average duration of the courses is shorter, about 70 hours. In order to compare the results of the different training measures some basic features of the participants should be considered. Technical education and Youth training cover the same population: young people with very similar ages and with large shares of work experience. Participants are also comparable between technical education and training of the unemployed, although some differences appear: age averages of vocational training and technical education add up to 32 and 22 years old respectively and work experience differs from 78,9% in vocational training to 37,0% in technical education. Participants in the programme for early school leavers (Garanta Social) and training of the long term unemployed differ to a greater extent but share one thing in
1 2

See Paeda et al. (2003). See Gonzlez Veiga et al. (2003).

common: both populations have special difficulties regarding their entering the labour market and, therefore, training of these people tends to be more costly. Continuous training is slightly different, for the participants already have a job and the objective of the programme is to adapt their skills to the new situations or to new technology.

4.

Results and impact of technical vocational education, vocational training and Continuous training

In order to evaluate the results of training measures two indicators have been used: the rate of access to employment and the employment rate. The first one, the rate of access to employment is defined as the proportion of participants in the measure who finished the programme and have got at least one job since the programme ended. The second indicator, the employment rate, is estimated as the proportion of participants who finished the programme and are employed when the evaluation takes place. Therefore, the employment rate does not include those who are currently unemployed but did work some time after training (these people are added to the numerator when calculating the rate of access to employment). It must be noted that about 20 months have passed since the end of training and the evaluation3 . Table 2 presents result indicators for the training measures subject to evaluation. About 90% of the participants finish the courses with one exception given by the programme for early school leavers (Garanta Social), which shows a completing rate close to 80%. Regarding employment and access to employment rates, the highest values are those of Continuous training. This is reasonable given the fact that these courses are received by working people (and thus the rate of access to employment is 100%). The best results are found in technical education modules and the values tend to decrease as the participants belong to groups with g reater difficulties to enter the labour market: the long term unemployed, youth or early school leavers. Overall, these can be considered as good results since all employment rates are above the range of 40%. Probably the most important question regarding the evaluation of training programmes is to estimate the extent to which the results are due to the programme and the influence of other factors. The previous analysis could be improved if the gross employment rates (both current and access to employment) are substituted by indicators considering just the net effects of the programmes. This means determining causality between completing training and the further acquisition of a job. In order to estimate net effects the deadweight effect must be controlled for, ie. the amount of jobs that would have been obtained without taking part in the training measures. Estimation of net effects has been carried out by asking the participants in training to what extent the courses determined their finding of jobs 4 . Table 3 shows
3

This number is an average and therefore it is an approximation. Technical professional education is taught during the school year and all courses finish at about the same time (they have different durations). However, vocational training consists of shorter courses that are taught at different times. 4 In order to estimate this effect, the questionnaire to participants in the diifferent measures included the following questions: Participants in subsidies to self-employment were asked if, in the absence of the subsidy, they would have carried out the business project. Participants in all technical education and vocational training were asked about the usefulness of the course in order to get a job.

the results, indicating that the greatest effect is found in technical education, which is followed by training targeting the groups with greater difficulties in entering employment (the long term unemployed and youth). Table 4 presents the main r esult indicators as well as those that estimate training impact. These data form the basis for the cost-effectiveness analysis that follows. Multiplying A times B the total number of participants who completed training is obtained. Using the C proportion g ives us the number of participants who have had at least one job after training, this being a result indicator. However, as it has been stated, not every job was related to training participation. Jobs related to training participation appear under the net effect (D) and the product of C times D gives us the net rates of access to employment. Multiplying D times A finally produces an estimation of participants who got a job related to their training.

5.

Comparative evaluation of training programmes by means of the costeffectiveness technique

The cost-effectiveness technique was used in this analysis as it allows us to rank different ways of reaching the same goal regarding the efficiency of each alternative. This is a popular method in health economics but it is not to well-known in other fields such as policy evaluation. However, cost-effectiveness is one of the techniques that was indicated by the European Commission in order to improve the evaluation of Structural Funds (European Communities, 1999). As a result of its little presence in the field of policy evaluation, just a few papers can be cited that have used cost-effectiveness. Taylor and Wren (1997) evaluate the efficiency of different regional employment incentives by comparing the cost per job among other indicators. These authors estimate that each job costs between 500 and 700 pounds sterling. Also, Taylor and Wren (1997) present other papers that used cost-effectiveness to evaluate different measures of company public support (Wren and Taylor, 1997; Hart et al, 1993); amongst their conclusions the low cost per job created can be cited.

5.1.

The cost-effectiveness technique

From the point of view of target oriented evaluation, efficiency analysis requires the identification of the most adequate measures to reach a certain objective. To this end cost benefit analysis and cost effectiveness analysis represent the most interesting evaluation techniques because they override the simple analysis of management efficiency to focus on the efficiency of policy impact. Cost-effectiveness analysis is a complete technique of economic evaluation that studies both the costs and the consequences of the assessed programmes (Drummond et al., 1999). The technique compares the effects of two or more treatments on a certain objective by looking at effectiveness units and relating these with costs. Cost-effectiveness differs from cost-benefit analysis in the measurement of programme effects: whereas the former considers physical units, the latter takes into account monetary units. However, both techniques value costs in monetary units. A basic element of cost-effectiveness analysis is the choice of the effectiveness measure. This choice must be based upon the objectives of the evaluated

Participants in subsidies to salaried employment were asked if, in the absence of the subsidy to the firm, they would have been hired.

programmes and their likely results. Regarding employment policies the main result is access to jobs and employment maintenance. Certainly there may be employment quality differences related to, for example, wages, working days and the adequacy of employment to the training received. But considering each one of these would be an extremely difficult task and, therefore, the effectiveness measurements chosen are employment and access to employment rates after 20 months. Valuing the costs of these programmes is no easy task since, for instance, some costs may be shared by different programmes5 . We have adopted criteria used to accept co-financing by the ESF within the Integrated Operating Programme of the CSF. Thus, rather than total costs it is co-financing by the ESF that we use. This changes to some extent the interpretation of cost-effectiveness results because the final rates will help determine the best uses of ESF funds among the different policies that may be co-financed. The analysis is based upon comparing essentially three indicators: cost per person taking part in the programme (cost per participant); cost per job accessed to by the participants after leaving the programme (cost per job) and cost per job that is maintained at the time when evaluation took place (cost per current job). The last two indicators will consider just the participants who finished training (Table 5).

5.2.

Using cost effectiveness analysis to evaluate different training programmes

Previously a preliminary comparison of programme results or programme effectiveness was presented (part 4). Now an estimation of average costs of the programmes follows. Firstly the cost per participant is calculated by dividing total costs by the total number of people who benefited from each programme. Interpreting cost per participant must take into account that courses mean duration is quite different among programmes: vocational education courses are longest, followed by vocational training. Continuous training shows a reduced cost per participant due to the short duration of training courses. The next step is cost per graduate. The total number of participants who finished the programme comes from using the survey graduating rate on the total participants. Cost per graduate requires dividing total costs by the number of graduated participants. As it has been anticipated, the most expensive measures are those targetting the most disadvantaged groups: early school leavers and the long term unemployed. Comparing the results of technical education with those of Youth vocational training the latter is slightly higher since hours taught are much greater in the former. As already stated, Continuous training cannot be directly compared to other measures: courses are shorter and therefore the cost is lower (Table 6). In order to analyse the cost-effectiveness of these measures in terms of their employment results we estimate the average cost per job. The rate of access to employment is used to this end and the highest costs correspond to the programme for early school leavers followed by vocational training to the long term unemployed. Differences with technical education and training are increased now because this measure has a greater rate of access to employment and, therefore, cost per job is shorter in this measure than in Youth vocational training. Leaving aside Continuous training, the most economical measure is the training of the unemployed given its higher rate of access to employment.
5

Cost accounting criteria in the different programmes does not always correspond to the needs of the researcher.

The following step is to consider employment maintenance. The number of jobs maintained has been t aken from the surveys carried out on participants in the various measures, more specifically from the labour market status during the week before the survey. This has allowed us to calculate the cost per job that is maintained or cost per current job (Table 7). As stated, not just employment maintenance but also employment quality matters. This has been considered too by distinguishing permanent and temporary work contracts and thus calculating the number of equivalent permanent jobs. This has been done by weighting temporary work contracts by 0,5, thus assuming that two temporary jobs values as much as one indefinite job. 6 In order to further improve the analysis the net effect of these measures was taken into account. The deadweight effect or inertia effect, consisting in the jobs that would have been found even if the programmes did not take place (had the trainee not received the course). Thus, the objective is estimating cost per job but only for jobs that are due to the participation in the evaluated measures. Results obtained slightly modify the impression produced by evaluating just employment or current employment of each measure. Introducing questions that try to assess causality between policies and jobs is a way of comparing results that, at least, allows us to obtain the magnitude order of costs per net job. The smallest net effects belong to the groups with less difficulties in accessing work, such as workers in Continuous training. These have greater deadweight effects and thus they have higher costs per net job. For instance, Continuous training presents the lowest costs per job and costs per current job but, at the same time, its costs per net job is very high given the little incidence of the policy over employment maintenance. Something similar occurs in the training of the unemployed, a measure that has a high cost per net job due to an important deadweight effect in access to employment of the participants in this measure who are not young or long term unemployed.

6.

Concluding remarks

Confronting current labour market problems, authorities at the local, regional, national and even the European level promote the so-called active labour market policies. These have a great range of action and it is convenient to evaluate them in order to help evidence-based decision making regarding which policies deserve priority and which ones could be adjusted. However, evaluations are not frequent and little research is directed at this objective. Evaluation is required, first, because the results of policies do not always correspond to their original objectives. Also, evaluating is convenient in order to determine which one of the measures at hand is more effective and which one is more adequate to solve the employment problems of the different groups.
All evaluations have some limits and simplifying assumptions that lead us to read them carefully, mainly given the heterogeneity of groups and policies. However, using objective criteria to know the degree of objective fulfilment and to clarify all difficulties and limitations is helpful for the decision making process.

This paper compared cost-effectiveness of education and training measures carried out in a Spanish region. Specific results are linked to the economic reality of Asturias. However, it is perhaps the main value of the paper is to help define indicators and to use them in cost-effectiveness empirical analysis, something that could be applied in other regions or in similar circumstances.
6

Choice of weights is subject to discussion and it could be fixed at 0,6 or 0,4. We have chosen 0,5 to show the differences between both types of contracts.

7.

References
Drummond, M. F., Stoddart, G. L. and Torrance, G. W. (1991): Mtodos para la evaluacin econmica de los programas de atencin de la salud. Daz de Santos Editores, Madrid. European Communities (1999): Evaluating socio-economic programmes. Principal evaluation techniques and tools. MEANS collection, volume 3. Gonzalez Veiga, C.; Mato Daz, F. J. and Cueto, B. (2003): Evaluacin de la insercin laboral de los titulados de ciclos formativos en Asturias. Consejo de Asturias de la Formacin Profesional. http://www.educastur.princast.es/consejofp/scripts/descargar _titulados.asp Paeda, C., Hernndez, M. and Mato, F. J. (2003): Informe de evaluacin intermedia del Programa Operativo Integrado del Principado de Asturias 2000-2006. Periodo 2000-2002 . Mimeo. Taylor, J. and Wren, C. (1997): UK regional policy: an evaluation. Regional Studies, vol. 31, 9, pp. 835-848.

Table 1 Training co-financed by ESF

ESF measure Professional Education Modules (Ciclos Formativos) Programme for early school leavers (Garanta Social) Training of the remaining unemployed Training of the long-term unemployed Youth training Cost (thousands of euros) 6,959.6 1,270.3 2,552.8 2,149.1 Participants 4,335 516 2,064 959 1,420 4,528

TECHNICAL VOCATIONAL EDUCATION

VOCATIONAL TRAINING

2,415.6 CONTINUOUS Continuous Training 1,471.8 TRAINING Source: Intermediate Evaluation of EFS measures in Asturias (Spain).

Table 2 Basic result indicators of the measures

ESF measure Degree rate Continuous Training 93.8 Professional Ed. 95.1 Modules Training of the 91.8 75.6 unemployed Training of the long94.1 70.3 term unemployed Youth training 91.1 65.2 Prog. for early school 81.8 56.1 leavers Source: Intermediate Evaluation of EFS measures in Asturias (Spain). Rate of access to employment 100 81.8 Employment rate 82.2 60.4 51.1 46.9 43.5 38.9

Table 3 Gross and net rates of access to employment

ESF measure

Rate of access to Rate of access to Net effect employment employment

(net) 51.1 35.5 12.5 30.5 20.0 7.8 41.8 19.9 9.5 21.4 13.0 7.8

(gross) Professional Education TECHNICAL 81.8 Modules (Ciclos Formativos) VOCATIONAL Programme for early school EDUCATION 56.1 leavers (Garanta Social) Training of the remaining 75.6 unemployed VOCATIONAL Training of the long-term 70.3 TRAINING unemployed Youth training 65.2 CONTINUOUS Continuous Training 100.0 TRAINING Source: Intermediate Evaluation of EFS measures in Asturias (Spain).

10

Table 4 Main results of training measures

ESF measure Beneficiaries (A) % degree (B) Gross Rate of Access to employment (C) 81.8 Net Effect (D) 51.1

Professional Education Modules (Ciclos 4,335 95.1 TECHNICAL Formativos) VOCATIONAL Programme for early EDUCATION 516 81.8 school leavers (Garanta Social) Training of the 2,064 91.8 remaining unemployed VOCATIONAL Training of the long959 94.1 TRAINING term unemployed Youth training 1,420 91.1 CONTINUOUS Continuous Training 4,528 93.8 TRAINING Source: Intermediate Evaluation of EFS measures in Asturias (Spain).

56.1 75.6 70.3 65.2 100.0

35.5 12.5 30.5 20.0 7.8

11

Table 5 Indicators used in the cost-effectiveness analysis

Indicator Cost per participant = Definition Total programme costs Number of participants in the programme Total programme costs Number of participants who graduated Total programme costs Number of graduates who got at least one job Total programme costs Graduates who currently work

Cost per graduate Cost per job Cost per current job

= = =

12

Table 6 Cost per participant and cost per graduate

Measure Professional Education Modules (Ciclos Formativos) Programme for early school leavers (Garanta Social) Training of the remaining unemployed Training of the long-term unemployed Youth training Continuous Training Total Cost (thousands of euros) 6,959.6 1,270.3 2,552.8 2,149.1 2,415.6 1,471.8 Cost per participant (euros) 1,605.4 2,461.8 1,236.8 2,241.0 1,701.1 325.0 Cost per graduate (euros) 1,688.2 3,009.6 1,347.3 2,381.5 1,867.3 346.5

13

Table 7. Cost per job and cost per current job (euros)
Measure Professional Education Modules (Ciclos Formativos) Programme for early school leavers (Garanta Social) Training of the remaining unemployed Training of the long-term unemployed Youth training Continuous Training Cost per job held 2,063.8 5,364.6 1,782.1 3,387.6 2,864.0 346.5 Cost per current job 2,795.0 7,736.7 2,636.6 5,077.8 4,292.7 421.6

14

Table 8 Cost per job (euros)

Measure Professional Education Modules (Ciclos Formativos) Programme for early school leavers (Garanta Social) Training of the remaining unemployed Training of the long-term unemployed Youth training Continuous Training

Cost per equivalent permanent job

4,464.8 12,948.4 4,336.5 8,924.1 6,997.0 535.7

Cost per net current job

Cost per current job 2,628 6,233 2,303 4,918 3,739 379 Cost per net current job 5,142.0 17,556.5 10,014.1 14,902.6 18,697.3 22,314.6

15