Paper 3:

The Module Level Indicator - an EQF-based instrument for equivalence checks of vocational
and higher education modules
Wolfgang Müskens, Carl von Ossietzky University of Oldenburg (Germany)

1. Accreditation of prior vocational learning

Germany has a highly differentiated vocational education system. After completion of an

initial apprenticeship, lasting up to 3.5 years, many working people attend long-term
vocational training programmes that lead to regulated vocational qualifications. A business
specialist (Betriebswirt) for example has usually over ten years of work experience and has
attended executive vocational training for approx. seven years.

His transition from vocational education to higher education should be facilitated by the
accreditation of his vocational knowledge and skills. A business specialist who decides to
read for a Bachelor programme at university therefore should be exempted from study
modules, which were already part of his vocational education.

Figure 1: Vocational vs. higher education in Germany

In 2005 the German Federal Ministry of Education and Research launched the programme
ANKOM (Recognition of Vocational Competences in Higher Education)1 to develop
instruments and methods to assess and evaluate prior vocational learning for accreditation.

The ANKOM pilot project at Oldenburg University2 primarily explored accreditation for a
Business Administration Bachelor programme. The methods and tools developed for the

1
ANKOM (“Anrechnung beruflicher Kompetenzen auf Hochschulstudiengänge”) http://ankom.his.de
2
ANKOM project “Qualifikationsverbund Nord-West”: http://www.web.uni-oldenburg.de/anrechnung/index.html

1
accreditation process, however, can also be transferred to other degree programmes. In the
CREDIVOC3 project, which is funded by the Lifelong Learning Programme of the European
Commission, we intend to investigate to what extent the instruments are also suitable for the
accreditation of vocational technician qualifications (“Staatlich geprüfte/r Techniker/in”) to
higher education.

The ANKOM projects had the following aims:

o A more flexible transition from extra-university education (in particular vocational
education) to higher education institutions should be facilitated.
o Redundancies at the interface between vocational and higher education should be
avoided.
o Incentives to encourage life-long learning should be established.
o Education should become more flexible.
o Prior learning should be considered more fairly in future with regards to embarking on
a degree.
o The route to gaining a degree should be shortened. (Müskens, 2006, p. 23)

2. Blanket recognition

As a result of the ANKOM project students at Oldenburg University can apply various kinds
of accreditation to shorten the duration of their studies.

If a student has already completed vocational training he or she is exempt from taking certain
modules, upon presentation of a final certificate, without having to take an examination for
these modules. Students can be exempted from various modules, depending on their
vocational qualification. We call this kind of accreditation “blanket recognition”.

The basic idea behind blanket recognition is that students who have already completed
certain vocational training programmes are exempt from taking modules containing learning
outcomes they have already acquired during vocational training. Any holder of a certain
qualification (e.g. Master craftsman, Certified senior clerk) gets exemption from equivalent
university modules.

3
CREDIVOC project “Transparency and Mobility through Accreditation of Vaocational Learning Outcomes”:
http://www.credivoc.eu

2
The prerequisite for blanket recognition is that the university decides which modules students
with certain vocational qualifications can be exempt from taking. We call this one-off testing
of the correspondence between vocational qualifications and higher education the
“equivalence check”.

If blanket recognition is to be given to a student, all he or she has to do is to present his or

her vocational qualification certificate to the examination office. The university guarantees
that certain compliant courses can indeed be recognised in the equivalence check. With
blanket recognition, no individual assessment of a student’s knowledge and skills is made.
The grade of an accredited module results from the respective grade gained in the vocational
qualification being recognised.

Blanket recognition of prior learning was principally rendered possible at German higher
education institutions due to a resolution of the Kultusministerkonferenz (KMK)4 of 28 June
2002, in which it was stipulated that: “Knowledge and skills obtained outside higher
education can be accredited towards a degree in the context of a – possibly also blanket –
classification if they [...] are equivalent in content and level to the part of the degree
programme that is to be replaced […].” (KMK, 2002). As the maximum possible accreditation
of extra-university skills, the KMK further recommended in this resolution that: “Knowledge
and skills obtained outside higher education can replace a maximum of 50% of a degree.”
(KMK, 2002)

In the following year, this KMK resolution was substantiated and supplemented by a joint
recommendation by the BMBF, the Standing Conference of the Ministers of Education and
Cultural Affairs of the Länder in the Federal Republic of Germany (KMK) and the HRK
(German Rectors’ Conference). This recommendation of 8 July 2003 states: “Demanding
qualifications in the area of further training are particularly suited to test the desired stronger
link between higher education institutions and different qualification routes as well as learning
locations outside higher education institutions. […] For this, ECTS credit points should be
awarded for qualifications proven by examination in the context of vocational training, which
should be able to be accredited when embarking on a degree programme at the respective
higher education institution.” (HRK-Plenum, 2003, p. 2).

3. Equivalence check

4
Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of
Germany

3
A core element of the recognition process is a so called “equivalence check”. The objective of
this check is to find out to what extent different modules from vocational and higher education
have similar contents and the same level.

Methods and tools to compare the equivalence of vocational qualifications and degree
programmes were developed within the ANKOM projects. In order for a module to be
recognised, the learning outcomes to be accredited must correspond to the learning
outcomes of the module with regard to both content and level.

The equivalence check is carried out by experts (“evaluators”) who appraise the degree
programme and the vocational qualification. The experts assess
o the extent to which the contents of the module are covered by learning outcomes of
the vocational qualification
o whether the level of the learning outcomes and skills to be accredited corresponds to
the level of the module

Figure 2: Equivalence check

4. The Module Level Indicator

In order to compare subjects or learning units from continuing vocational training with higher
education study modules we had to develop a level comparison procedure or instrument.
When doing this, we had to consider the different concepts of “level” in vocational and higher
education. Because of these different concepts we decided to apply the European
Qualifications Framework for Lifelong Learning (EQF) as a common benchmark (European
parliament, 2007).

4
A simple procedure of application of the EQF is the direct classification: We asked experts to
assign directly an EQF level to a certain module or learning unit. However, the evaluators
reported major difficulties when applying this direct classification procedure:
o The EQF is tailored to comprehensive qualifications, not to learning units or modules.
o The third column of the EQF seems to be inapplicable to higher education modules.
o The experts perceived their own ratings as less reliable.

As a consequence, we decided to develop a more reliable instrument for level comparisons

of modules or learning units. We called this instrument the “Module Level Indicator”. Although
this instrument avoids direct EQF ratings, the EQF descriptors were utilized as a basis for
item formulation.

Figure 3: A tool to compare the level of learning units or modules

In the current version (v 2.1) the MLI consists of 51 items. These items were constructed
based on
o the EQF descriptors,
o descriptors of the German Higher Education Qualifications Framework5,
o the European Higher Education Area Qualifications Framework6,
o Lecturers’ suggestions (gathered by means of semi-structured interviews).

5
KMK (2005)
6
Bologna Working Group on Qualifications Frameworks (2005)

5
Figure 4: Construction of the Module Level Indicator

The 51 items of the MLI are aggregated to 9 scales. Each of these scales consists of 5 to 10
items. Most of the items explicitly refer to learning outcomes. The MLI usually is completed
by independent experts. Alternatively or in addition to these expert ratings also lecturers or
students may fill in this questionnaire.

The MLI was designed as an instrument that:

o allows differentiated descriptions of learning units by means of multiple dimensions,
o offers reliable scales (according to classical test theory),
o is applicable to different modules, different programmes of study, different
instructional designs and different kinds of examination, and
o predicts direct EQF ratings of modules validly.

5. Method

Experts rated N=175 modules or learning units by means of the MLI. Principal component
analyses with orthogonal Varimax rotation revealed a 9 factor solution. Based on these 9
dimensions the scales of the MLI version 2.1 were constructed and named.

Further 84 modules were rated by experts by means of the MLI version 2.1. The reliabilities
of the 9 MLI scales were obtained based on these 84 ratings. The internal consistencies
(Cronbach’s alpha) of the scales vary from .62 to .95, but only one of the scales shows a
consistency below .75. The scales were standardized based on these 84 module ratings.

6
Figure 5: Scales of the MLI and their reliability

Figure 6: Examples of MLI items (from version 2.1)

6. MLI results

The results of the MLI ratings of a single module or learning unit may be presented as a bar
chart, which visualizes the features of the rated module. The 9 specific scales are also
aggregated to a comprehensive total score. Based on the MLI total score it is possible to
compare the general level of different modules or learning units. Therefore this total score

7
provides the basis for equivalence checks. Learning units from vocational education with a
MLI total score equally high as higher education modules may be accredited.

Figure 7: MLI results – Profile of a learning unit from vocational education

Figure 8: MLI results – Comparison of Senior Industrial clerk qualification and Bachelor Business Administration

7. Construct validity

Does the MLI validly measure the EQF level of a module or learning unit? To answer this
question we compared N=91 MLI module ratings (by experts) with direct EQF classifications
of these modules (performed by the same experts).

The relation between the MLI total scores and the direct EQF classifications may be
interpreted as construct validity of the instrument. Simple bivariate Pearson correlation

8
between the two measures was r=.80. However, the scatterplot indicates that there might be
an even stronger nonlinear relation. The coefficient of determination of a nonlinear square
model regression analysis results in r2=.70. That means that 70% of the EQF direct rating
variance is explained by the MLI total scores.

Figure 9: Construct Validity of the Module Level Indicator

This nonlinear relation will be considered in a future re-standardisation of the MLI scores.

Bibliography

Bologna Working Group on Qualifications Frameworks (2005): A Framework for Qualifications of 

the European Higher Education Area. Link http://www.vtu.dk

European Parliament (2007): European Parliament legislative resolution of 24 October 2007 on the 

proposal for a recommendation of the European Parliament and of the Council on the establishment of 

the European Qualifications Framework for lifelong learning, Link 

http://www.europarl.europa.eu/sides/getDoc.do?pubRef=­//EP//TEXT+TA+P6­TA­2007­

0463+0+DOC+XML+V0//EN

HRK­Plenum (2003): Empfehlung des Bundesministeriums für Bildung und Forschung,der Konferenz 

der Kultusminister der Länder und derHochschulrektorenkonferenz an die Hochschulen zur Vergabe 

von Leistungspunkten in der beruflichen Fortbildung und Anrechnung auf ein Hochschulstudium vom 

8.7.2003.

9
KMK (2002): Anrechnung von außerhalb des Hochschulwesens erworbenen Kenntnissen und 

Fähigkeiten auf ein Hochschulstudium. ­ Beschluss der Kultusministerkonferenz vom 28.06.2002. 

Online in the  Internet: http://www.kmk.org/doc/beschl/anrechnung.pdf [last accessed: 29.07.2008].

KMK (2005). Qualifikationsrahmen für Deutsche Hochschulabschlüsse ­ Im Zusammenwirken von 

Hochschulrektorenkonferenz, Kultusministerkonferenz und Bundesministerium für Bildung und 

Forschung erarbeitet und von der Kultusministerkonferenz am 21.04.2005 beschlossen. Link 

http://www.kmk.org/doc/beschl/BS_050421_Qualifikationsrahmen_AS_Ka.pdf

Müskens, Wolfgang (2006): Pauschale und individuelle Anrechnung beruflicher Kompetenzen auf 

Hochschulstudiengänge – das Oldenburger Modell.­ In: Hochschule & Weiterbildung, 1, pp. 23­30.

10