LCCTV Manual

A Methodology for Designing Employment-Speci c Language Tests

Language Competence Certication Tools for the Vocations

Published by Westflische Wilhelms-Universitt Mnster Arbeitsbereich Linguistik H erstrae , Mnster http://santana.uni-muenster.de Copyright All rights reserved. Project Consortium

is work has been funded with support from the European Commission within the Leonardo da Vinci Innovation Transfer Project Language Competence Certi cation Tools for the Vocations ( ). See http://lcctv.uni-muenster.de for more information. is publication re ects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Typeset by Johannes Schwall and Prinzipalsatz Typographie Mnster Printed in Germany by diedruckerei.de, Neustadt a. d. Aisch

Leonardo da Vinci Project Language Competence Certi cation Tools for the Vocations

LCCTV Manual
A Methodology for Designing Employment-Speci c Language Tests

Contents
Background Methodology Identifying the skills and competences needed to work at the various levels in an industry . . . . . . . . Converting the occupational skills into the linguistic skills and competences needed to perform at the di erent levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compiling vocabulary which is characteristic of the industry . . . . . . . . . . . . . . . . . . . . . . . . . Identifying grammatical structures that feature at each level . . . . . . . . . . . . . . . . . . . . . . . . . . Establishing test speci cations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Piloting and testing the tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Language Competence Framework of Reference for the Vocations (LCFRV) National registry for the Construction industry (Bulgaria) . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tables of linguistic skills and competences, assessment criteria and types of exercises appropriate for levels within the construction industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Work levels of a ship crew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Table of linguistic skills and competences for the main levels (A, B and C) within the marine industry . Table of linguistic skills, assessment criteria and types of exercises for level A within the marine industry . . . . . . . . . . . . . . . . . 12 3 3

Background
One of the founding principles of the European Union is that there should be free movement both of capital and people across European borders. Indeed, statistics now show that signi cant numbers of European workers have moved and are moving to other member countries of the for the purposes of employment. It is common, however for immigrant employees (legal and illegal) to have very little knowledge of the language of their host country and this presents enormous problems both for themselves and for their prospective employers. ere are some work situations in foreign countries in which knowledge of the language is irrelevant or of little importance at least for the purposes of carrying out work. But in most cases a high level of competence is a must and it is o en observed that immigrant employees, though they may be well quali ed and skilled, frequently lack the degree of foreign language competence to work e ectively. is o en results in both wastage of highlyquali ed people and poor performance. Sometimes employment agencies that send workers abroad do conduct exams to check the linguistic competences of the applicants, but in most cases what they evaluate is the applicants knowledge of General English, German, French, etc. e language they are tested in does not re ect the register, jargon and vocabulary of the work they will be doing. To address these problems the European Union is attempting to introduce instruments which will facilitate the transfer of quali cations and competences for academic or professional purposes across borders. is project partnership, Language Competence Certi cation Tools for the Vocations ( ) is part of this work. One of the contributions of the project to the attempt to make quali cations and competences transparent is the following manual. e manual is a step-by-step approach to developing foreign language competence tests (also called performance tests) for the vocations. e manual is intended to assist language teachers in initial and tertiary vocational training, examination bodies and panels, to prepare foreign language tests which can identify the levels of language competence of the test takers for work purposes.

Methodology
Since each industry has its own jargon, each industry needs its own set of language competence tests. When putting together tests, test writers nd that it is necessary to put the context of the language at centre stage. But, language test writers are rarely experts in an industry as well as being language test writers, so without being an expert in that particular industry, how is a writer to know and understand the context and write tests that really do test a candidates ability to understand and express enough of a foreign language to ourish in a job in a foreign country? ere is no magic formula for this. For the test writer, an understanding of other peoples work contexts involves a lot of reading, asking questions and observation. Regular contact with language students working within that industry is an enormous help to test writers and some work contexts are simply more familiar to us than others and therefore easier to work on. For example, though Medicine is highly jargonized we have all seen medical soap operas on , visited the doctor, swallowed tablets and visited hospitals. Medical language is obscure and very specialized but at least the smells and sounds of a clinic are part of our daily life the same with Construction and Tourism. It is not so easy with industries like Carpet Manufacturing or Ship Building. A familiarity with the context is gained by repeated exposure to the documentation of an industry and frequent contact with its practitioners. In this way a test writer builds up, quite randomly, a corpus of industry-speci c terms. As familiarity with the industry grows it becomes necessary to employ the terms of the industry to express the concepts and practices that are speci c to the industry and gradually the test writer becomes something of an expert in the industry and its activities at least in a theoretical and abstract way. ough this expertise is acquired only through interaction with the industrys practitioners and not through any actual eld experience, it is, nevertheless, required. What is also required is knowledge of language learning syllabuses and the necessary creativity to design test items that do what they are supposed to do. e following is a summary of the steps to constructing a language competence test: ) Identify the occupational skills and competences needed to work at the various levels in an industry ) Convert the occupational skills into the linguistic skills and competences needed to perform at the different levels ) Compile vocabulary characteristic of the industry ) Identify grammatical structures that feature in the communicative situations of each level in the industry ) Establish the type of test that is to be used and develop its speci cations ) Construct test items and pilot the test ese steps are explained below.

Step 1 Identifying the skills and competences needed to work at the various levels in an industry
Within an industry it is usually possible to establish ve or six broad ranks. e names of these ranks vary from industry to industry (compare nursing with seafarers, for example) but basically they can be reduced to something like: a) Apprentice/trainee b) Skilled and quali ed employee (non-managerial) c) First line manager. is is a supervisor who oversees the performance of line tasks. Titles associated with such positions are: foreman, shi boss, sergeant, section head, and ward manager. d) Middle manager. is is someone who manages supervisors overseeing day-to-day operations. e) Senior manager. is is someone in authority who has responsibility for planning and directing work usually with the power to hire or re employees or to promote them. Job descriptions are widely available that will help in compiling information about the duties of each of these ranks and test writers use them as well as gathering large amounts of information from other sources about the work of each rank. Seeing the complexities of hierarchies and roles prompts ideas for test items as well as giving further insight into the work context. (For illustration and clari cation see appendices (page ) and (page ).) For the purposes of this manual we are assuming that the level of skill and responsibility in a job is re ected in the level of competence in a foreign language needed to perform that job. So, senior managers, because they may need to communicate to a varied range of people and deal with complex texts, require a higher level of foreign language competence than, say, trainees. is is not to say that an apprentice level employee will not have foreign language skills as developed as his Managing Director. In fact this is frequently the case and is particularly so in countries like Bulgaria, where the majority of former vocational schools have foreign language teaching programmes which lead students to / level of the whereas their bosses from older generations may well have not studied foreign languages at school.

Step 2 Converting the occupational skills into the linguistic skills and competences needed to perform at the different levels
Language tests are generally of two types which are determined by their purpose: achievement tests and competence/pro ciency tests. Achievement tests are associated with teaching and they are designed to test a students retention and comprehension of things that have been taught and the students ability to apply them. Competence/pro ciency tests however are the ones that are the concern of this manual. Pro ciency tests look to the future use of language in our case in employment and test a candidates ability to cope. Test writers have to consider the contexts in which employees operate and how and when the employees need to communicate. Test writers have to construct miniscenarios and narratives in their heads either for the purposes of composing a script for a listening test, writing a text for a reading comprehension item or simply generating individual sentences for gap ll exercises. Test writers need to imagine people in an industry communicating by phone, fax and over lunch tables, imagine them in various roles in the industry and capture what they say, write and read, then convert this into test items. ere is very little di erence between writing exercises for a language course book and writing test items. If there is a di erence it is in the level of scrutiny that test items get and the consequent importance of getting them right. e following table illustrates the correlation between employment levels in an industry, in this case Construction, and the foreign language competence required at each level of employment. Drawing on the information in the rst four columns a test writer needs to be at such a point of familiarity with an industry to be able to either know or imagine the communicative functions that employees at the various levels of seniority have to perform. ese communicative functions, along with knowledge of grammar syllabuses in common use in language teaching will form the basis for the tests.

Position/level Apprentice/unskilled labourer (Level of education: primary and usually secondary schooling) Skilled worker (Level of education: primary and usually secondary schooling)

Areas of employment Construction and maintenance of buildings, roads, railways, dams and canals

Job description Work that involves routine manual tasks, using tools and operating equipment

Technician/supervisor (Level of education: at least secondary schooling and usually tertiary level training)

Company manager (Level of education: at least secondary schooling but very often nowadays tertiary level training)

Building work such as: brick laying, masonry, stone cutting, working with concrete etc. Jobs include: carpenters, joiners, specialists in repair of roofs, oorings and facings, plasterers, glazers, insulation workers, plumbers, pipeline workers, electricians and painters. Construction and related specialist work such as hydro electricity, building underground, water supply and sewage, bridges, roads, dams and airports. Tasks may include: surveying and cartography, land regulation, land measuring etc. Construction, consultancies, project management, planning and design

Skilled work often supported by a combination of several years experience as an unskilled labourer and/or qualications and training usually in a specic sub-eld.

Occupational competences Digging, lime slaking, drilling, surveying and measuring clearing, loading, unloading, laying of gravel, carrying bricks and other materials, assembly of basic every day equipment Specic knowledge and skills for tools-based work and knowledge of all stages of the production process, tools and materials needed, as well as of the nature and use of the nal product. Often supervisory and training roles are required. The ability to apply concepts, learned methods, procedures and processes to specialist areas of construction. It is usually necessary to perform technical tasks and monitor and control technical facilities and systems. To perform business activities such as negotiating with customers, suppliers and clients. When accountable to report and discuss at meetings. Often technical knowledge is required but not always many companies are run by experts in business rather than experts in construction. To formulate policy and be accountable for the overall performance of a companys products and services. Represents the company before other organizations including state authorities. May have to prepare and write technical or scientic papers.

Communicative functions Understanding instructions, making requests, completing simple forms

Understanding but also giving instructions, providing simple explanations and descriptions, completing simple work-related forms

Technical tasks in construction involving construction, maintenance and repair, as well as sometimes taking a design role.

Making recommendations, summarizing, working with mathematics and measurements, writing technical reports

Civil engineers and architects (Level of education: tertiary)

Heading construction and engineering projects employment includes: architects, urban planners, hydrologists, engineers of all kinds and specialisms. Research and consultancy

Management of companies (often their own companies). These people plan, manage and coordinate the activities of the company, conduct negotiations with suppliers and customers and other organizations, plan and monitor the utilization of company resources and employ workers. Planning, management and coordination of construction and design projects. Evaluating and analyzing research and surveys as well as devising architectural documentation, urban plans and landscapes.

Exchanging ideas and discussing at an abstract or theoretical level, making and justifying decisions, negotiating with other parties, reporting orally and in written form

Formulating and presenting ideas and suggestions based on in-depth analyses, giving opinions and recommendations, discussing policies, preparing high calibre work documentation

Step 3 Compiling vocabulary which is characteristic of the industry

ere are numerous sources from which to extract industry-speci c vocabulary. ey include the following: ) and any other language-for-speci c-purposes course books and teaching materials (if available on the market) Glossaries and dictionaries of industry-related terms Industry-speci c documentation (rules, codes of practice, health and safety standards, technical specications, any other authentic documentation) Specialist journals and other editions such as newsletters, bulletins, circular letters, etc. Special publications of the respective Chambers of Industry such as, for example, Construction Chambers, professional bodies, organisations and associations Chapter headings, section headings and sub headings of key books that summarize or introduce the industry Specialist websites and forums ) Conference proceedings on topics related to the industry in question ) programmes and video materials discussing various aspects of that industry All are essential sources of reference for test writers. By immersion in them a test writer can get insight into the industrial context and the material will generate ideas and sentences that can usefully be integrated into test items. e skill to grade vocabulary, however, is probably more a matter of a combination of experience and intuition rather than any systematic procedure. ere are vocabulary lists provided by big publishers which grade vocabulary according to English language pro ciency levels and they are extremely useful (see the Oxford . word list: Oxford University Press). It must be noted though that such things are lists of General English vocabulary. For lists of vocational speci c words and terms the test writer is pretty much on his own.

) )

) )

Step 4 Identifying grammatical structures that feature at each level

With a few adaptations and cross-overs between levels, a grammar syllabus for a well established language course which has been matched up to the Common European Framework of Reference for Languages can provide a framework for planning the shapes and forms of the sentences that will both feature in a test and will be tested within the test. e following list of grammar items and structures is taken from the syllabus of Penguin Books (Longman Corpus Network). ough the details of the syllabus relate to the English language, the principles behind the divisions in the list can be transferred to other languages. ) ) level Simple one clause sentences or two clauses joined Singular and plural forms Possessive forms Pronouns, determiners, interrogatives, articles numbers, possessives, quanti ers Common constructions of adjectives Positive imperative verbs Question forms Adverbs of frequency, place and time Conjunctions ) Phrasal verbs Irregular verbs Inde nite pronouns Direct and reported speech

Level As above and in addition: Sentences with more than two main clauses Conditional sentences Inde nite pronouns Re exives and relative pronouns Relative clauses Intensifying adjectives More complex verb forms (in English: Present Perfect Simple) Passive verb forms Obligations and suggestions Question tags Level As above and in addition: Complex sentences including if clauses Reciprocal pronouns (in English: each other) More complex intensi ers (in English: nearly as, just as) Simple passive verb forms Other more complex verb forms (in English: past perfect/have something done/past continuous passive) Reported speech with past tense reporting verbs level As above and in addition: More complex verbs (in English: past perfect continuous/future continuous etc.) Conditional sentences and sentences with embedded relative clauses

Level As above and in addition: Quanti ers Negative imperative forms of verbs Past verbs for prediction and intention Two clauses joined with because As above and in addition: Inde nite pronouns Quanti ers and distributives Comparative and superlative adjectives Negative questions Expressing future meaning

Step 5 Establishing test specications

It is not surprising that tests are analyzed and criticized more intensely than almost any other educational device. A er all, careers, hopes and ambitions, fortunes and pride are usually at stake and everybody wants to be sure that a test does actually test what it says it will, be no harder or easier than it needs to be, nor too long or too short and that it is an accurate and fair way of measurement. tics of a test and supply detailed information about length of papers, item types etc. to the test writers. It is usual for test publishers to issue very clear speci cations to test writers who are commissioned to write test items. ey use these speci cations over and over in order to maintain continuity so that this years exam does exactly the same thing, albeit with di erent content, as last years did and next years will do etc. e test speci cations underpin the development of 1) Specications content and purpose the test items each time the test is written. ey are the It is extremely important to draw up speci cations for main tool for guiding a writer in inventing and formtests. Speci cations are descriptions of all the characteris- ing test items which are consistent in content and which

have similar capacity to measure language competence. Speci cations for tests are documents which evolve over time, with small additions and alterations taking place constantly as they are ne tuned. Alderson et al. in Language Test Construction and Evaluation (Cambridge University Press) say that test speci cations should include: A clear statement of purpose of the test A description of the test takers A statement of the test level Reference to any existing and suitable language course book e number of sections/papers e time for each section/paper e weighting for each section/ paper Details about text types Information about text lengths A statement of the language skills and elements to be tested Detailed information about test tasks and test items Rubrics criteria for marking 2) Industry-specic proles For the purposes of creating a clear pro le of potential candidates who will be taking the test, it is necessary to have descriptions of typical performances at each level and descriptions of what candidates at each level should be able to do in the real world. ese linguistic skills and work performance can be organized in a framework of work-based references using the as a model. By way of a model, appendix (page ) presents a table of the linguistic skills and competences appropriate for levels within the construction industry. e table

includes the linguistic skills that the tests aim to assess, the minimum level linguistic requirements, the test specications per skill (accompanied by assessment criteria) and examples of test tasks that can be used to write the test items. Appendix (page ) presents a table of the linguistic skills and competences for the main levels within the marine industry while Appendix (page ) presents a table of the linguistic skills, assessment criteria and types of exercises for level within the marine industry. 3) Test items Apart from the level of the test, its length and the language skills it is testing, test writers consider the best and most appropriate types of test items to use. e design of test items depends on the language skill that is being tested (compare testing vocabulary knowledge with testing speaking skills). In non-oral tests, test items are o en in xed response format, especially at the lower levels; multiple-choice being the most common. e usual approach with constructing multiple-choice test items is to present a range of responses that you anticipate based on typical confusions or misunderstandings. In the language of test writers these are known as distractors. e correct option is known as the key. Test speci cations give clear instructions on the number of distractors to be used in any test item; their length and style. ere are common variations in multiple choice test items as well as a number of well used alternatives. A list of common task types for testing (and practicing) reading and listening, compiled by N. Canham ( ), is presented below:

4) Common types of language testing items Examples for Reading:

Task type Broad Multiple Choice Questions Multiple Choice Questions with unnished sentence stems True or false (t/f ) Short answer questions Sentence completion Matching headings Multiple matching Information transfer Personal response Reapplying ideas Noticing grammar Finding vocabulary Example Which of these options best describes the writers purpose? The department closed because (a, b, c, d) Decide whether the following statements are true or false according to the text. Which department is most likely to close? . . . The managers decided to close the . . . department. Match each heading (af ) with the correct paragraph/section. There is one heading you do not need. For each question, choose which option is referred to. Which department (1) is understaffed? (2) will close next week? Complete the table/owchart/form/diagram based on the information in the passage. To what extent do you agree with the writers view? The writer suggests some ways of dealing with staff disputes. Which solutions would be most useful in these situations? Underline examples of relative pronouns used in the passage. Find words/phrases in the text with similar meanings to the expressions below.

Examples for Listening:

Task type Multiple Choice Questions True or false (t/f ) Matching Information transfer Sentence completion Example The department closed because a, b, c Decide whether the following statements are true or false according to the speaker. Listen to ve speakers talking about the place where they work. Match each speaker (15) with the opinion they express (af ). There is one opinion that you do not need. Complete the table/owchart/form/diagram based on the information in the recording. The managers decided to close the . . . department.

Oral skills are much harder to objectively judge than receptive skills such as reading and listening. Test items which are designed to do it range from the simple: examiner asks question, candidate responds; to the complex such as prepared presentations. However, in all cases it

is necessary to set as many parameters as possible for the test writers and to provide model answers for the test markers and a clear list of descriptors by which to judge a candidates performance.

Step 6 Piloting and testing the tests

When the rst dra of a test has been written it needs to go through a rigorous and critical editing. Test writers usually pilot a test on colleagues and then on students at di erent levels and from di erent backgrounds. ey get someone from the industry to read a later dra looking for faux pas and wrong vocabulary usage and all the time test items which cause contention or generate too many variants are changed and re ned. When test writers have got somewhere close to a nal dra , data is collected through a piloting done with numbers of students. Each test item is analysed to check its rubric for ease of understanding, its level of di culty and its placing and position in the test. e piloting of tests should also take into consideration some of the following issues identi ed by in the Guidelines for Good Practice in Language Testing and Assessment. ese include consideration of the following: ) Size of the pilot sample, and how it compares with the test population ) e nature of the information that is to be collected during the piloting stage. (For example: teachers views on the test, students opinions on the test and test items or suitability of levels etc.) ) How the pilot data will be analysed. ) How to agree on changes to the test and how to integrate these changes into the next version of the test. e piloting of tests should be based on systematic procedures for review, revision and editing of items and tasks to ensure that they match the test speci cations and comply with item writer guidelines. An important element of the test speci cations is the rubrics criteria for marking for each skill that is being tested. e table on the opposite page details the rubrics criteria for marking as per level and skill.

A1 Test takers are assessed on their ability to: comprehend extended speech or work-related presentation and grasp explicit as well as implicit details identify key details in long and fairly complex speech identify opinions and/or attitudes and support them with a list of reasons transfer and organise information in the form of a table/owchart/form/ diagram comprehend extended speech or work-related presentation and grasp nuances and even minute details draw conclusions based on implications identify opinions and/or attitudes and reasons for these

A2 Test takers are assessed on their ability to:

B1 Test takers are assessed on their ability to:

B2 Test takers are assessed on their ability to:

C1 Test takers are assessed on their ability to:

Listening

identify work-related vocabulary and simple phrases when spoken slowly and clearly demonstrate understanding of spoken instructions respond appropriately to spoken warnings (within the scope of the listening section)

identify high-frequency work-related vocabulary follow and identify key details in longer but and phrases when spoken at a normal speed straightforward speech on work matters such as understand and respond apropriately to jobdescriptions, explanations and suggestions or recspecic instructions, directions and warnings ommendations on how work-related tasks should be performed. identify the sequence of procedures/tasks guided by the sequence markers used (e.g. rst, then, next, after you do this, before you do this, as soon as you do this, not before, nally, etc.)

Reading

identify factual work-related information understand key points in short and simple written messages in work situations follow not only the main elements but also identify sift factual information and identify the main topic and key the details in longer and more complex technical ideas in a comparatively long text documentation and other work-related regulations identify important elements such as ndings, comments and recommendations

identify factual work-related information in texts (e.g. product/material descriptions and instructions for use) understand key points in short and simple written messages on subjects directly related to their work provide description, clarication or instructions as requested without any serious omissions participate in discussions on working practices by asking for or giving proper explanation about key details and using the appropriate markers to signal the importance of the various items discussed provide detailed clarication on things/ideas in response to questions justify an opinion in a discussion or conversation, using the appropriate linguistic markers justify a decision providing comments negotiate on a work-related issue and reach an agreement

comprehend and identify information in long and complex specialist texts deduce the views of the author and identify key details/ideas

Spoken Interaction

respond appropriately in work situationd in a provide and ask for work-related information way which does not hinder communication using appropriate grammatical structures in a use the precise terminology (vocabulary preciway which does not hinder communication sion) or paraphase words in a conversation so as explain key details by using precise terminology to make himself/herself understood (vocabulary precision) or paraphasing words so provide and ask for personal or work-related as to make himself/herself understood information using appropriate grammatical structures

formulate and present ideas in work-related conversations or discussions express and justify an opinion while elicitating an opinion from his/her interlocutor invite comments and/or suggestions and explain his/her position without any serious errors

Spoken Production

describe the operation of a machine or the way a tool is used even if with very simple sentences explain the sequence of procedures using very simple sequence markers (e.g. rst, second, then, after that, next etc.) provide personal information about himself or work colleagues

provide structured/well organised process-related describe work materials or a sequence of work procedures using a series of phrases and simple information by signalling sequence or any other sentences communication-related element (e.g. importance, give simple directions to colleagues or lower urgency, necessity, etc.) level staff provide clear, logical and properly sequenced in express attitudes such as agreement or disagree- structions using appropriate vocabulary and grammar ment state his/her opinion and give reasons for it without any serious errors

provide detailed description or make a fairly detailed presentation without any serious omissions. justify a viewpoint on a job-related problem compare solutions and make recommendations using the appropriate linguistic markers to signal his intentions.

provide a detailed description and/or opinion giving points in a structured way and adding conclusions without any serious errors justify an opinion on a complex issue at work

Writing

provide work-related factual information even if with very simple and short sentences provide personal information for ofcial purposes such as lling in a job application or an immigration form

provide work-related factual information and make suggestions with simple and short sentences provide personal information for ofcial purposes such as lling in a job application or an immigration form

provide job-specic factual information in written form (e.g. as in time-sheets, supplies list or material recording form) accurately and without any serious errors report in written form on various aspects of work without any serious grammatical and/or communication errors

provide a properly-justied recommendation or objection express a professional opinion in the form of a using the appropriate layout, structure and register to exwritten recommendation comparing advantages press his/her views and disadvantages and/or suggesting solutions report on ndings, make comments and give recommenda- state in a highly formal way his/her entions without any serious errors. quiry/problem, request approval and thank for summarise work-related information and present it in the the support using appropriate vocabulary and with appropriate job-specic form without any serious grammati- gramatical precision cal and/or communication errors.

1) Marking Scheme A marking scheme should re ect the level of pro ciency of the test taker as per each skill tested, e.g. Listening ; Reading ; Speaking and Writing . ese should be used to produce a summative assessment, presented as an average grade, based on the results of the components/skills. Since the majority of member countries have their own marking systems which vary considerably, we suggest the percentage system as universal and easily understable by any -citizen, but even more so by employers. e benchmark is xed at a minimum of per each skill following the assumption that it is the minimum required level of linguistic competence to perform e ciently on the job. Whatever the results from the test, each test taker has to receive a written document which details two things. First, the results of the test for each skill in percentage plus the average grade, and a test report section which gives a descriptive summary of the test takers performance (identifying his/her areas of insufcient knowledge/skills) and makes recommendations for improvement. e assumption is that each attempt should be registered and yet another assumption is that if the test taker does not meet the benchmark but is close to it, the decision be made by the employer. We suggest the following format for the test report page. 2) Test Report Page e Test Report Page should comprise the following information:

e Personal details section will include headings providing identi cation details but also any other meaningful information such as native language, second language, foreign language to be used on the job, industry sector, position applied for (optional) etc. e Test Results section will include the following information:
Skill tested Listening Reading Speaking Writing Average grade Result achieved 70 % 90 % 75 % 80 % 79 % (rounded) Skill requirements met Yes Yes Yes Yes

e above table gives a clear picture of the weaknesses in the language competences of the test taker but yet gives a fair account of his strengths which may help the employer decide in favour of appointing the job applicant despite his test results, depending on the speci city of the particular work context. e Descriptive assessment section will provide the following information:
Description of the linguistic competences demonstrated at the exam or weaknesses identied The test taker can . . . , but . . . The test taker has fully operational competence for this level. The test taker can . . . , but . . . The test taker has fully operational competence for this level. Skill tested Areas of linguistic competence in need of attention (recommendations) The test taker needs to improve his ability to . . .

Listening 70 % Reading 90 %

Personal details Test results (for each skill) plus average grade Writing 80 % Descriptive assessment (for each skill) plus recommendations for improvement

Speaking 75 %

Result achieved

The test taker needs to improve his ability to . . .

Sample tests and specications:

Within the Project, sample tests and speci cations have been developed according to this methodology. ere are tests and their respective speci cations for all main levels in the Construction and Maritime industries. ey are included on the - or can be downloaded (along with other materials) from the project website (see http://lcctv.uni-muenster.de.)

10

Language Competence Framework of Reference for the Vocations (LCFRV)

11

Language Competence Framework of Reference for the Vocations (LCFRV)

e Language Competence Framework of Reference for the Vocations ( ) aims to meet the needs of the labour market by providing linguistic criteria for workrelated language competence certi cation that can be used to identify the level of the linguistic skills of mobile workers/employees in the respective industry sector. It can be used by employers, testing bodies or national policy makers to assist them in making linguistic workoriented competences more transparent and easily understandable. e describes levels of work-related linguistic competences based on work levels which can easily guide employers and recruiters of mobile workforce in deciding on the employability of the migrant workers. e levels are based on what the person can do in the target language and include all the four skills as they are used in work situations.

Global scale of language competence levels for the vocations:

C2 Voc Rarely applicable in work situations. Can only be found in academic contexts. Can understand longer specialist texts, and recognise all nuances in meaning. Can express himself/herself uently and spontaneously using the whole range of professional terminology specic to his/her eld or respective subeld. Can also use language exibly and effectively for social, learning and other purposes. Can produce clear, well-structured, detailed text on complex work-related and sector-specic subjects, showing appropriate use of organisational patterns, connectors and cohesive devices. Can understand the main ideas of complex job-specic texts and participate in technical discussions in his/her eld of specialisation. Can interact with native speakers justifying decisions in work-related context by specifying the advantages and disadvantages of various work-related options. Can produce clear, detailed text on sector-specic subjects and express views on a problematic issue using the appropriate markers. Can understand the key points of clear standard interaction on occupational matters connected with everyday work life. Can deal with most situations likely to arise while working by giving simple instructions as well as reasons for opinions and plans. Can produce simple connected text on workrelated topics such as brief reports which belong to his/her immediate work obligations. Can describe work procedures and voice concerns connected with specic work situations. Can make suggestions and comments on various aspects of work both in spoken and written form. Can understand high frequency work-related vocabulary and phrases. Can follow simple instructions and directions. Can describe work materials/tools/devices/equipment, and sequence of work procedures; can express agreement or disagreement on matters related to his/her work environment. Can recognise and use the names of materials/tools/devices/equipment that s/he works with as well as basic phrases related to his/her work environment. Can interact in a simple way by asking and answering questions connected with personal identication and job-related tasks. Can ll in job application or immigration forms giving personal details and details of work qualications, too.

Procient User C1 Voc

B2 Voc Independent User

B1 Voc

A2 Voc Basic User A1 Voc

e table on the following page shows a detailed description of the levels. Note that level is not included because it is rarely applicable in work situations. It might, but only in academic and research contexts which do not involve the target groups the project aims to reach. e levels that have been speci ed show the minimum language levels required for the respective industry positions. ey do not exclude cases when employees holding lower positions have a higher level of linguistic competence for various reasons that have been detailed in the methodology. More o en than not, understanding through reading is much easier than speaking which explains the larger

number of tasks that can be achieved in Reading, whereas Spoken production is o en the last skill to be successfully acquired by learners and a major concern for providers. ere have been witnessed paradoxical situations when a specialist in a technical eld has demonstrated evidence that s/he is able to understand the contents of a handbook/manual without being able to produce a single sentence which could be classi ed as normal speech. Employees may demonstrate di erent levels of linguistic performance for the four skills but whatever their language competence is, it has to be certi ed properly so as to enable employers to decide whether the worker can cope on the job or not.

12

level required for an aplevel required for a A1 Minimum prentice/unskilled worker/employee A2 Minimum skilled worker/employee level required for a B1 Minimum rst-line manager Can understand extended speech and presentations and follow even complex lines of argument on work topics. Can understand TV news and documentaries directly related to his/her job. level required for a C1 Minimum senior professional/manager

level required for a B2 Minimum company manager

Listening

UNDERSTANDING

Can recognise the names of materials/tools/ devices/ equipment that s/he works with, as well as basic phrases concerning his/her work, work colleagues and work surroundings/ environment when people speak slowly and clearly. Can understand safety instructions if spoken slowly and respond to safety warnings in case of danger. Can read articles and reports concerned with work matters which include attitudes or viewpoints.

Can understand the highest frequency workrelated vocabulary and phrases. Can follow short, work-specic simple instructions and directions. Can understand safety instructions if spoken slowly and respond to safety warnings in case of danger.

Can understand the key points of clear, standard speech on work matters and follow descriptions, explanations and suggestions or recommendations on how work-related tasks should be performed.

Can understand extended speech with its nuances and follow a lecture or presentation on specialist topics without too much effort. Can grasp details and draw conclusions which are based on implications only and are not signalled explicitly.

Reading

Can understand familiar brand names, abbreviations and words comprising professional jargon (terms) and very simple and short sentences (e.g. precautions) on labels, packagings, boxes and other types of containers, as well as notices on noticeboards which are connected with his/her work. Can deal with most situations likely to arise while working and can enter unprepared into conversations on work-related topics that cover his/her job responsibilities. Can interact relatively uently with native speakers and participate actively in discussions in work contexts. Can justify decisions and sustain his/her views on work-related matters.

Can read simple product/material/tool/device or equipment descriptions and instructions for use. Can also understand short personal notes/messages from colleagues or supervisors.

Can understand texts that consist of high frequency job-related vocabulary. Can understand the description of work processes, read user manuals, technical specications of systems and equipment, standards and safety regulations.

Can understand long and complex specialist texts such as specialised articles and long and detailed technical instructions related to his/her occupation.

Spoken Interaction

SPEAKING

Can interact in a simple way provided the other person speaks slowly and what s/he says is directly related to the work situation or work surroundings. Can ask and answer very simple questions connected with personal identication and job-related tasks. Can produce simple sentences meant to describe work procedures, give instructions and voice jobrelated concerns. Can briey give reasons and explanations for opinions and plans.

Can participate in simple task-related conversations and job-specic exchange of information especially when s/he is on the answering side.

Can express himself/herself uently without preparation. Can use language effectively for work purposes. Can formulate ideas and opinions with precision and participate in work-related discussions.

Spoken Production

Can use simple phrases and sentences to describe work-related activities and/or work colleagues.

Can use a series of phrases and simple sentences to describe work materials, tools, devices and equipment as well as the sequence of work procedures (e.g. to a colleague or apprentice); can give simple directions and express agreement or disagreement.

Can present clear, detailed descriptions on subjects related to his/her occupation. Can explain a viewpoint on work issues giving reasons for his/her opinion. Can compare options and make recommendations supported by appropriate arguments.

Can present clear, detailed descriptions of complex subjects related to his/her work, integrating subthemes, developing particular points in a structured way and producing appropriate conclusions.

Writing

WRITING

Can write a short, simple message for colleagues or supervisors informing them of work issues or problems that have arisen in the course of work. Can ll in job application or immigration forms giving personal details such as name, nationality, marital status, etc, as well as educational/training background and details of work qualications.

Can write short, simple messages for colleagues or supervisors informing them of work issues or problems that have arisen in the course of work. Can ll in a job application or immigration form giving personal details such as name, nationality, marital status, etc, as well as educational/training background and details of work qualications.

Can write simple connected text on work-related topics like short reports, suggestions and comments on various aspects of work. Can ll in questionnaire- type technical documentation which does not require writing accuracy and precision.

Can write clear, detailed text on subjects related to his/her occupation. Can write a report, passing on information or giving reasons in favour of or against a particular proposal or discussion point connected with his/her work.

Can express himself/herself in clear, wellstructured text, pointing at advanges or disadvantages, giving reasons for his/her opinion or presenting solutions to work-related problems. Can write a formal business letter or report underlining issues of importance using appropriate style.

13

Appendix 1 National registry for the Construction industry (Bulgaria)

Position/level (for the respective industry sector) Apprentice in construction First level educational entry prerequisites: primary or secondary schooling

Positions by codes (national registry of construction jobs) 93130001 general labourer, construction of buildings 93130002 general labourer, maintenance of buildings; 93120001 lime slakers; 9312 0002 digger; 93120003 worker, construction; 93120004 worker, water supply wells; 93120005 worker, drills; 93120006 worker, road maintenance; 93120007 worker, dam maintenance; 93120008 worker, maintenance; 93120009 worker, maintenance of railways and railway facilities; 93120010 worker, land-surveying and measuring operations; 93120012 worker digger, canals and barrow pits; 93120013 worker shoveller Builder 712 Construction builders; 7121 Builders second level educational using conventional materials; 7122 Masons entry prerequisites: primary and stone-cutters; 7123 Concrete workor secondary schooling ers and related jobs; 7124 Carpenters and joiners; 7129 Builders and other related unskilled workers; 713 Workers engaged in nishing works on buildings and other related jobs; 7131 Workers engaged in construction, maintenance and repairs of roofs; 7132 Workers engaged in oorings and facings; 7133 Plaster workers; 7134 Insulation workers; 7135 Workers engaged in window glazing; 7136 -Plumbers, pipeline workers and other related; 7137 Construction electricians and other related jobs; 714 Painters, chimney sweepers, faade cleaners and other related jobs; 7141 House painters and other related jobs; 7142 Lacquer painters and other related jobs; 7143 Workers engaged in the cleaning of building constructions

Job description These separate groups encompass workers who perform simple and routine tasks in construction and maintenance of roads, dams and other construction facilities, in the building of sites, connected with the use of manual tools and more often than not, involving physical efforts.

Occupational competences Manual work: wheeling out, clearing, loading, unloading materials and others; laying of gravel; carrying bricks and other materials; similar tasks in the construction of roads, dams and buildings; removing debris; simple manual assembly of elements that do not require the strict adherence to a specied technology. Competences may include supervision of other labourers.

712 These groups encompass workers who build, maintain and repair the interior and exterior of the foundations, walls and other parts of buildings and other constructions.; 713 This level also includes workers who deal with nishing work in the construction of buildings and other constructions such as mounting, maintaining and repairing roofs, oors, walls, isolation systems, window glazing or other types of framing; installing waterpipes, sewage, electricity mains. The second level refers to workers who prepare surfaces, paint or apply other similar materials to buildings and other constructions, cover walls and ceilings with wallpaper or other materials; sweep roofs and clean the exterior of buildings and other constructions.

Construction tech3112 Construction technician; 31123001 Technician, hydro construction works; nician/supervisor Third level educational en- 31123002 Technician, water supply and try prerequisites: secondary sewage; 31123003- Technician, geodesy, phoschooling togrammetry and cartography; 31123004 Technician, civil constrution (constructor); 31123005 Technician, civil construction (hydraulic works); 31123006 Technician, landmeasuring and land regulation; 31123007 Technician, investors supervision; 31123008 Technician, in charge of a railway maintenance team; 31123009 Technnician, civil engineering and architecture; 31123010 Technician, underground construction; 31123011 Technician, transport construction

This group of positions encompasses employees who fulll technical tasks in construction, as well as design, building, operation, maintnenance and repair of buildings and other facilities such as water supply and sewage, bridges, roads, dams and airports.

Specic knowledge and skills for manual or tools-based work, so as to reduce the physical efforts and time needed for the implementation of specic tasks. Knowledge of all stages of the production process, tools and materials needed, as well as of the nature and use of the nal product. Ability to use construction materials and methods, to build and repair, foundations, walls and other parts of buldings made from bricks, stones and other similar materials, to mount scaffoldings, to mount and dismount various wooden constructions, to cover buldings with various materials, to lay various oorings, to plaster walls and ceilings, to isolate walls, oors and ceilings, to cut and glaze windows, to install pipes, sewage, electricity mains and other similar equipment. To prepare surfaces and coat them with paint and other materials, to be able to cover interior walls and ceilings with wall paper or other materials; to sweep chimneys and clean facades of buildings and other constructions. The ability to perform supervising functions. The ability to apply concepts and practical methods in the eld of construction engineering, to perform technical tasks connected with geology, engineering drawing and the specic production process. To monitor and control technical facilities and systems, to inspect the application of standard measures and safety procedures when using machinery and equipment; to inspect the production processes and products, to fulll the orders of managers and experts, to monitor the work of other workers.

15

Position/level (for the respective industry sector) Company manager fourth level educational entry prerequisites: secondary schooling

Positions by codes (national registry of construction jobs) 1313 managers in construction and design; 13135001 manager, design; 13135002 manager, construction

Job description This group encompasses individuals who manage their own cmpanies or, on behal of other owners, small construction and design companies; who plan, manage and coordinate the activities of the company; who conduct negotiations with suppliers and customers and other organizations; who plan and monitor the utilisation of company resources and employ workers.

Occupational competences To be able to manage smes or perform business activities for themselves or on behalf of other owners, aided by not more than one manager and several assistants; to be able to plan the everyday activities of the company; to perform the day-to-day management of the business and evaluate the results achieved, to plan and monitor the use of resources and deal with the selection of staff; to hold negotiations with suppliers, clients and other developers, to report the results to the company owners, if applicable; to supervise the work of other staff. To be able to dene and formulate the company or organisation policy; to be able to manage, coordinate and be in charge of the overall performance or the activities of their interior or regional structures; to be able to represent enterprises or organisations before other enterprises and organisations including state authorities and bodies, to be able to monitor the performance of the enterprises or organisations and to report them to the board of directors. Architects with engineering and other related specialists must be able to research and consult, control the construction of buildings, the layout of urban areas, consult and manage their maintenance and repair, to research and consult on technological aspects of specic materials, products, processes and the efciency of production and work organisation. They must be able to survey, study and devise architectural documentation and urban plans, landscape and transport systems. They must also prepare scientic papers and argumentation as well as monitor the work of other employees.

Civil engineers and architects fth level educational entry prerequisites: higher education

1223 Managers/Supervisors of construction and design units 12236001 Head of construction; 12237002 Head of construction site; 12237003 Head of construction engineering department; 12236004 Head of construction department; 12237005 Head of construction team; 12233006 Technical supervisor; 214 Architects, engineers, and other related jobs; 21417001 Architect, design; 21417002 Architect, housing, industrial and road construction; 21417003 Architect, engineering equipment; 21417004 Architect, interior; 21417005 Architect, horizontal planning; 21417006 Chief architect; 21417007 Landscape architect; 21417008 Urban architect; 21427001 Engineer, hydrologist; 21427002 Engineer, water supply and sewage; 21427003 Engineer, railway construction; 21427004 Engineer, investors supervision; 21427005 Engineer, irrigation works; 21427006 Engineer, constructor; 21427007 Engineer, bridge construction; 21427008 Engineer, harbour construction; 21427009 Engineer, industrial and civil construction; 21427010 Engineer, Road construction; 21427011 Engineer, sanitary construction; 21427012 Civil engineer; 21427013 Engineer, building construction; 21427014 Engineer, hydro-construction; 21427015 Engineer, construction of chimneys; 21427016 Engineer, construction of domes and towers; 21427017 Engineer, construction technologist; 21427018 Engineer, tunnel construction; 21427019 Engineer, hydro-power construction 21427020 Engineer, hydro-meliorative construction

This group encompasses individuals who plan, manage and coordinate activities in construction and design. As well as people who design housing, commercial and industrial buildings; who research, survey and devise architectural documentation, urban plans, who do landscape and road engineering, who plan and monitor their construction, maintenance and development. This group includes positions for staff that carries out surveys and provides consultancy; that design and manage the construction of housing and road sites; who do research and provide consultancy on the technological aspects of certain construction materials.

16

Appendix 2 Tables of linguistic skills and competences, assessment criteria and types of exercises appropriate for 5 levels within the construction industry

e level requirements are based on the Language Competence Framework of Reference for the Vocations ( that was devised as part of this project (see page ).

A1
Listening

Minimum level required for an apprentice/unskilled worker Can recognize the names of materials and tools that s/he works with, as well as basic phrases concerning his/her work, work colleagues and work surroundings/environment when people speak slowly and clearly. Can understand safety instructions if spoken slowly and respond to safety warnings in case of danger.

Reading

Can understand familiar brand names, words comprising professional jargon (terms) and very simple and short sentences (e.g. precautions) on labels, packaging, boxes and other types of containers, as well as notices on noticeboards which are connected with his/her work.

Test specications per skill (Assessment criteria) Given a set of pictures showing various materials and tools and listening to their names pronounced by a speaker, the test taker will match the picture with the name with at least 90 % accuracy (e.g. if given 10 pictures the test taker will match correctly at least 9 of them). Given a choice of 10 situations with 3 or 4 options each and listening to work-related basic phrases the test taker will respond appropriately in 9 out of 10 situations. Given a set of pictorial warning symbols and listening to the speaker the test taker will match the warning with the symbol with at least 90 % accuracy (e.g. if given 10 symbols the test taker will match correctly at least 9 of them). Given a list of general occupational terms (names of materials /tools/labels/precautions, etc.) commonly used on the job, or a matching list of such terms and pictures, the employee will dene the words or match the words and pictures with at least 90 % accuracy (e.g. if given 10 pictures the test taker will match correctly at least 9 of them). Given a set of sentences the test taker will identify the names of materials, key information on labels, packaging, boxes and other types of containers. Given a set of brief notices the test taker will identify their meaning from a choice of three options two of which function as distractors.

Spoken Can interact in a simple way proInteraction vided the other person speaks slowly and what s/he says is directly related to the work situation or work surroundings. Can ask and answer very simple questions connected with personal identication and job-related tasks. Spoken Can use simple phrases and senProduction tences to describe work-related activities and/or work colleagues.

Given simple questions directly related to a specic work situation, the test taker will respond appropriately in a way which does not hinder communication even if with some mistakes. Given simple questions related to personal identication, the test taker will provide accurate answers giving the information required. Given a picture of specic tools or a machine the test taker will list ve common operations performed on the machine or with the tools, or list a sequence of task-related activities needed to perform the respective task. Given key words and certain work and every day situations, the test taker will write a simple note containing the appropriate message. Given blank samples of a job application or immigration form, the test taker will ll it in with 90 % accuracy.

Types of test tasks that can be used (examples only) Look at the pictures of materials/tools and number them in the order that you hear them. Listen to what the speaker says and write down only a few important pieces of information. a) write down the ve tools/devices that you need to work with. b) write down the ve things that you need to do. c) write down the ve times that the speaker mentions. Look at the warning symbols and listen to their interpretations. Match a symbol with its interpretation. Any other as appropriate. Match the terms on the left with their denitions on the right. Match the pictures on the left with the terms on the right. Label the following materials/tools. Label the various parts of the machine/device in the picture. Underline the names of materials in the following sentences. Write the names of the following verbs beside the pictures that illustrate them. Write the names of the following objects beside the pictures that illustrate them. Read the notices carefully and choose a reply which matches their meaning. Read the job advertisements and answer the following questions. Answer the questions with very simple words but giving the required information. Discuss your daily work schedule with your immediate superior/boss. Ask questions connected with your daily tasks. Ask your work colleagues about their name, nationality, age, job, family, and hobbies and interests. Tell your colleague what your tasks for the day are. Ask your colleague to give you some tools. Explain to your new colleague how the machine works (use 5 sentences to do this). Add some safety warnings (3), too. Write a short note informing your work colleagues and/or your boss about a problem at work or warning them about a risky situation such as a machine out of order, no materials, dangerous ground, etc. Fill in the blank with the necessary information. Use key words only.

Writing

Can write a short, simple message for colleagues or supervisors informing them of work issues or problems that have arisen in the course of work. Can ll in job application or immigration forms giving personal details such as name, nationality, marital status, etc, as well as educational background and details of work qualications.

17

A2
Listening

Minimum level required for a skilled worker Can understand the highest frequency work-related vocabulary and phrases. Can follow short, workspecic simple instructions and directions. Can understand safety instructions if spoken slowly and respond to safety warnings in case of danger.

Reading

Can read simple product/material descriptions and instructions for use. Can also understand short personal notes/messages from colleagues or supervisors.

Test specications per skill (Assessment criteria) Given a set of pictures showing various materials and tools and listening to their names pronounced by a speaker, the test taker will match the picture with the name with at least 90 % accuracy (e.g. if given 10 pictures the test taker will match correctly at least 9 of them). Given a choice of 10 situations with 3 or 4 options each and listening to work-related basic phrases the test taker will respond appropriately in 9 out of 10 situations. Given a set of instructions and listening to the speaker the test taker will number the instructions in the order that s/he hears them with at least 90 % accuracy (e.g. if given 10 instructions the test taker will number correctly at least 9 of them). Given a description of a material or tool, the employee will dene the main properties of the material or demonstrate understanding of the instructions for use both for materials and tools with at least 90 % accuracy (the 90 % will depend on the number of properties/instructions for use mentioned in the description). Given a short personal note/message from a colleague or superior the test taker will identify key information concerning time, tasks, problems mentioned, suggestions, and alike with at least90 % accuracy (the 90 % will depend on the number of information items used in the note/message).

Types of test tasks that can be used (examples only) Look at the pictures of materials/tools and number them in the order that you hear them. Listen to what the speaker says and tick the correct response in the following situations. Listen to the speaker and number the instructions in the order that you hear them. Look at the warning symbols and listen to their interpretations. Match a symbol with its interpretation. Any other as appropriate.

Spoken Interaction

Can participate in simple taskrelated conversations and jobspecic exchange of information especially when s/he is on the answering side.

Given simple questions directly related to a specic work situation, the test taker will respond appropriately in a way which does not hinder communication even if with some mistakes. Given a situation where information on working practices/or any other work issues needs to be exchanged, s/he will be able to ask questions requesting explanation about key details without any serious mistakes. Given a series of jumbled phrases/sentences the test taker will describe work materials or the sequence of work procedures following the logic of the respective industry sector with at least 90 % accuracy. Given an exemplary work situation, the test taker will give the appropriate directions to a work colleague and express agreement or disagreement connected with work-related operations performed by a co-worker without any serious mistakes.

Spoken Production

Can use a series of phrases and simple sentences to describe work materials, sequence of work procedures (e.g. to a colleague or apprentice), give simple directions and express agreement or disagreement.

Writing

Can write short, simple messages for colleagues or supervisors informing them of work issues or problems that have arisen in the course of work. Can ll in a job application or immigration form giving personal details such as name, nationality, marital status, etc, as well as educational background and details of work qualications.

Given a work-related problem and some key words and phrases, the test taker will write a short note explaining the problem and suggesting solutions. Given blank samples of a job application or immigration form, the test taker will ll it in with 90 % accuracy.

List the 5 (any number as appropriate) properties mentioned in the description. Match the instructions for use with their explanations on the right. Number the instructions for use in their logical sequence. Underline the names of materials in the following descriptions. Match the instructions for use with the tools. Write the names of the following objects beside the pictures that illustrate them. Read the notices carefully and choose a reply which matches their meaning. List the key piece of information in the personal note/message. List the 3 tasks (any number as appropriate) which are mentioned in the message. Answer the questions with very simple words but giving the required information. Discuss your daily work schedule with your immediate superior/boss. Ask questions connected with your daily tasks. Ask questions requesting explanation about new materials/tools to be used. Ask your work colleagues about their name, nationality, age, job, family, and hobbies and interests. Produce a brief description of a work material and its properties related to the work task to be performed. Tell your colleague what the sequence of work procedures for a certain operation is. Give directions to your new colleague as to where certain work facilities are. Explain to your new colleague how the machine/tool works (use 7 sentences to do this). Add some safety warnings (3), too. Any other as appropriate Write a short note informing your work colleagues and/or your boss about a problem at work or warning them about a risky situation such as a machine out of order, no materials, dangerous ground, etc. Fill in the blank with the necessary information. Give details as required.

18

B1
Listening

Minimum level required for a rst-line manager (e.g. foreman, construction/building technician Can understand the key points of clear, standard speech on work matters and follow descriptions, explanations and suggestions/recommendations on how work-related tasks should be performed.

Test specications per skill (Assessment criteria)

Types of test tasks that can be used (examples only)

Reading

Can understand texts that consist of high frequency jobrelated vocabulary. Can understand the description of work processes, read blueprints, technical specications of systems and equipment, standards and safety regulations.

Spoken Interaction

Can deal with most situations likely to arise while working and can enter unprepared into conversations on work-related topics that cover his/her job responsibilities.

Given a summary paragraph with missing information and listening to a description of a work process pronounced by a speaker, the test taker will supply the missing information with at least 90 % accuracy (e.g. if given 10 gaps/blank spaces the test taker will ll in correctly at least 9 of them). Given a work situation requiring work-related decision-making and listening to a set of 15 suggestions/recommendations on how work-related tasks should be performed the test taker will identify the 10 appropriate suggestions with at least 90 % accuracy. Given a short speech on 6 (or as appropriate) daily tasks to be performed by the work team the test taker will list correctly at least 5 of them. Given a description of a work process, the test taker will dene the main stages of the process with at least 90 % accuracy (the 90 % will depend on the number of stages mentioned in the description). Given technical specications of machinery/equipment the test taker will identify key information as required in the answer sheet with at least 90 % accuracy (the 90 % will depend on the number of information items used in the answer sheet). Given a book of standards or safety regulations, the test taker will identify the standards or regulations according to tasks, problems or work situations mentioned, with at least 90 % accuracy. Given questions directly related to a specic work situation and requiring description, clarication or giving instructions, the test taker will respond appropriately providing all necessary details as requested without any serious omissions. Given a situation where information on working practices/or any other work issues needs to be discussed s/he will be able to engage in it providing proper explanation about key details or requesting such without any serious errors.

Listen to the speaker and ll in the gaps with the missing information. You have the following problem at work (specify a problem) and you need to make recommendations connected with it. Listen to the speakers suggestions and tick the appropriate ones. Listen to the speaker and number the instructions in the order that you hear them. Listen to a speaker explaining the tasks to be performed for the day. Take notes and list the 5 tasks that the speaker mentions. Any other as appropriate. List the 5 (any number as appropriate) stages mentioned in the description of a work process. Read the technical specications and provide answers to the following qustions: Match the following emergency situations with the appropriate set of safety rules. Match the quantities with the technical indicators as appropriate. Match the instructions for use with the tools. Read the descriptions and match the equipment with its functions. Any other as appropriate. Answer the questions and provide a description or clarication giving all necessary details. Discuss your responsibilities with your immediate superior/boss. Ask questions connected with the daily tasks of your team. Anwer questions providing explanation as to how new materials/tools are to be used. Ask questions and get information about a new employee with regard to their name, nationality, age, job, family, and qualications. Produce a brief description of a work process and its stages related to the work task to be performed. Give instructions to a subordinate listing the sequence of work procedures for a certain task. State your opinion on the following work-related problem and give reasons for it. Any other as appropriate.

Spoken Production

Can produce simple sentences meant to describe work procedures, give instructions and voice job-related concerns. Can briey give reasons and explanations for opinions and plans.

Writing

Can write simple connected text on work-related topics like short reports, suggestions and comments on various aspects of work. Can ll in questionnairetype technical documentation which does not require writing accuracy and precision.

Given a work-related process or procedure the test taker will describe the different stages or the sequence of operations following the logic of the respective industry sector without any serious omissions. Given a work situation, the test taker will give the appropriate instructions to lower level employees. Given a problematic work situation which requires decisionmaking the test taker will provide solutions briey stating his opinion and giving reasons without any serious errors. Given a work-related document such as a time-sheet, supplies list or material recording form, the test taker will complete the forms accurately without any serious errors. Given a daily-report form, the test taker can produce a short report including comments and suggestions on various aspects of work without any serious grammatical and/or communication errors.

Fill in the following document using information from the box. Write a brief report listing work done for the day and giving details as required. Write comments and/or suggestions connected with the following situation at work: Any other as appropriate.

19

B2
Listening

Minimum level required for a company manager (e.g. engineer) Can understand extended speech and presentations and follow even complex lines of argument on work topics. Can understand TV news and documentaries directly related to his/her job.

Test specications per skill (Assessment criteria) Given a summary paragraph with missing information and listening to an extended speech or work-related presentation delivered by a speaker, the test taker will supply the missing information with at least 90 % accuracy (e.g. if given 10 gaps/blank spaces the test taker will ll in correctly at least 9 of them). Given a set of sub-headings and listening to a presenter the test taker will take notes and list key details related to each sub-heading (number to be specied by the test giver) with at least 90 % accuracy. Given a short speech on work matters the test taker will identify opinions and/or attitudes and support them with a list of reasons (numbers to be specied according to the length of speech) with at least 90 % accuracy. Given an explanation on a work process or practice, the test taker will transfer the information in the form of a table/owchart/form/diagram with 90 % accuracy and no omissions. Given an article related to the respective industry, the test taker will dene the main topic of the the article and identify the key ideas in it with at least 90 % accuracy (the 90 % will depend on the number of ideas/suggestions mentioned in the article). Given a work-based report the test taker will identify its main elements such as ndings, comments and recommendations with at least 90 % accuracy (the 90 % will depend on the number of linguistic items specied by the test giver).

Types of test tasks that can be used (examples only) Listen to the speaker and ll in the gaps with the missing information. Listen to the speaker and list the key details that go with each sub-heading. Use a maximum of three words per item. Listen to the presenter and decide whether the following statements are true or false. Listen to a colleague expressing his/her opinion on a certain work issue. List the three reasons s/he gives to support his/her opinion. Listen to ve speakers talking about the place where they work. Match each speaker 15 with the opinion they express AE. Complete the table/owchart/form/diagram based on the information in the recording. Any other as appropriate. Read the article and a/ decide what it is about; b/ list the 5 (any number as appropriate) ideas/suggestions mentioned in the article. Read the following statements and decide if they are true or false according to the opinion of the author. Match each heading AF with the correct paragraph/section. Complete the following sentences using information from the text. Complete the table/owchart/form/diagram based on the information in the passage. The writer makes recommendations about dealing with . . . (Specify as appropriate according to text). Which recommendations would be most useful in these situations? Any other as appropriate. Answer the questions and provide a detailed description or clarication on the following topic. Discuss the following work problem with your colleague. Ask and answer as many questions as needed to reach a decision. Ask questions to nd out your colleagues opinion on the following problem and request justication. Discuss the following news with your colleague. Give and ask for comments and suggestions providing/adding as many details as possible. Listen to the speaker and give your opinion supporting it with the appropriate reasons. Read the following description and comment on the main features and technical specications of the machine/equipment. Express your opinion on the following problem and give reasons to support it. Compare the two alternatives presented in the text describing the advantages and disadvantages of both methods and make recommendations about which one should be used and why. State your opinion on the following work-related problem and give reasons for it. Any other as appropriate. Read the following text and write a recommendation or objection to what is proposed in the text providing reasons for your opinion. Read the following materials and write a short report on the use of . . . (specify topic) giving your ndings and making comments and recommendations. Read the following information and write a memo to all those concerned. Any other as appropriate.

Reading

Can read articles and reports concerned with work matters which include attitudes or viewpoints.

Spoken Can interact relatively uInteraction ently with native speakers and participate actively in discussions in work contexts. Can justify decisions and sustain his/her views on work-related matters.

Given a specic work situation which requires discussion that includes clarifying things/ideas or reaching an agreement on work related matters, the test taker will respond appropriately providing all necessary details and/or expressing opinion supported by reasons using the appropriate linguistic markers without any serious errors. Given a decision-making situation the test taker will state his/her decision and provide justication for it following the logic of the problem in question. S/he will invite comments and/or suggestions from his peers and reach an agreement with his interlocutor without any serious errors.

Spoken Can present clear, detailed Production descriptions on subjects related to his/her occupation. Can explain a viewpoint on work issues giving reasons for his/her opinion. Can compare options and make recommendations supported by appropriate arguments.

Given a detailed description of a piece of equipment, the test taker will make a straightforward presentation introducing the main features and technical specications of the product targeted at lower level employees without any serious omissions. Given a work situation, the test taker will express his views giving reasons for his/her opinion. Given a problematic work situation with two alternatives solutions the test taker will compare the solutions dwelling on the advantages and disadvantages of either options and making recommendations on which one to use without any serious errors and using the appropriate linguistic markers to signal his intentions. Given a work-related subject the test taker will produce a properlyjustied recommendation or objection using the appropriate layout, structure and register to express his/her views with 90 % accuracy. Given the task to survey work-related information the test taker will produce a brief report stating his/her ndings, making comments and giving recommendations without any serious errors. Given an issue at work, the test taker will produce a memo adhering to the specicity of this piece of writing without any serious grammatical and/or communication errors.

Writing

Can write clear, detailed text on subjects related to his/her occupation. Can write a report, passing on information or giving reasons in favour of or against a particular proposal or discussion point connected with his/her work.

20

C1
Listening

Minimum level required for a senior professional/manager (e.g. civil engineer, architect) Can understand extended speech with its nuances and follow a lecture or presentation on specialist topics without too much effort. Can grasp details and draw conclusions which are based on implications only and are not signalled explicitly.

Test specications per skill (Assessment criteria)

Types of test tasks that can be used (examples only)

Reading

Can understand long and complex specialist texts such as specialised articles and long technical instructions related to his/her occupation.

Given a summary paragraph with missing information and listening to an extended speech or workrelated presentation delivered by a speaker, the test taker will supply the missing information with at least 90 % accuracy (e.g. if given 10 gaps/blank spaces the test taker will ll in correctly at least 9 of them). Given a set of statements and listening to a presenter the test taker will identify the opinion of the author with at least 90 % accuracy. Given a short speech on work matters the test taker will draw conclusions based on information which does not convey things explicitly with at least 90 % accuracy. Given an article related to the respective industry, the test taker will deduce the views of the author and/or identify key details/ideas in it with at least 90 % accuracy (the 90 % will depend on the number of ideas/suggestions mentioned in the article). Given a specialist article and a set of subheadings the test taker will match the headings with the respective sections of the article with at least 90 % accuracy (the 90 % will depend on the number of sections in the article).

Spoken Interaction

Can express himself/herself uently without preparation. Can use language effectively for work purposes. Can formulate ideas and opinions with precision and participate in work-related discussions.

Spoken Production

Can present clear, detailed descriptions of complex subjects related to his/her work, integrating subthemes, developing particular points in a structured way and producing appropriate conclusions. Can express himself/herself in clear, well-structured text, pointing at advantages or disadvantages, giving reasons for his/her opinion or presenting solutions to work-related problems. Can write a formal business letter or report underlining issues of importance using appropriate style.

Given a work related topic the test taker can ask and answer questions aimed at giving and receiving detailed infrmation without any serious errors. Given a situation where the test taker is supposed to express an opinion s/he will do so formulating his/her ideas with precision and checking them against the opinion of his/her interlocutor. Given a situation which requires discussion, the test taker will respond appropriately providing all necessary details and/or expressing opinion supported by reasons using the appropriate linguistic markers without any serious errors. Given a decision-making situation the test taker will invite comments and/or suggestions from his peers, give recommendations himself/herself and explain his position without any serious errors. Given a work-related subject, the test taker will provide a detailed description/opinion giving points in a structured way and adding conclusions without any serious errors. Given a work situation, the test taker will express his views giving reasons for his/her opinion.

Listen to the speaker and ll in the gaps with the missing information. Listen to the following presentation and tick the statements which reect the views of the speaker. Listen to the following short speech and provide 3/5/7 (as appropriate) conclusions based on what you hear. Decide whether the following statements are true or false according to the speaker. Listen to ve speakers talking about a specialist topic. Match each speaker 15 with the opinion they express AF. There is one opinion that you do not need. Any other as appropriate. Read the article and tick the statements which reect the views of the author. Read the article and list the 3/5/7 (any number as appropriate) suggestions mentioned in it.Use a maximum of 3 words. Read the article and tick the suggestions mentioned in it. Match the subheadings with the sections from the article. There is one subheading that you do not need. Read the following statements and decide whether the author of the article agrees with them or not. Any other as appropriate. Read the following text which has some missing information. Ask your partner questions to supply the missing information. Then answer the questions of your interlocutor providing a detailed description/comment or clarication as appropriate. (integrated activity) Comment on the following topic asking and answering questions aimed at nding out the other persons opinion as well as expressing yours. Provide reasons for your opinion and follow the logic of the topic in question. Find out as much information as possible about . . . (any work-related topic) by asking the necessary questions and agree on a set of recommendations with your partner. Discuss the following topic explaining your position and negotiating a win-win solution. Read the following materials (3 different resources) and present a detailed description of the subject matter in a structured and logical way. Express your opinion on the following work situation and give reasons for it. Present a work-related proposal and justify it as best as you can. Use the following ideas/topics to help you. Any other as appropriate. Present a written argument/case/proposal to an informed reader/work colleague/peer using your own ideas, knowledge and experience. Write a letter/e-mail/report to the investors explaining why one work method failed, making an alternative suggestion and explaining its advantages by providing as many details as needed in order to get approval. Write a proposal presenting solutions to a work-based problem providing proper justication for your ideas. Any other as appropriate.

Writing

Given a work-related subject the test taker will produce a written recommendation pointing at advantages and disadvantages of one work method or another and/or suggesting solutions (in the form of a proposal) using the appropriate layout, structure and register to express his/her views with 90 % accuracy. Given the task to write a business letter the test taker will produce a formal letter stating his/her enquiry/problem, requesting approval and thanking for the support without any serious errors.

21

Appendix 3 Work levels of a ship crew

e following diagram shows the work levels of a ship crew and provides brief job descriptions. master chief o cer second o cer third o cer boatswain able seaman ordinary seaman deckboy chief engineer second engineer third engineer oiler wiper

Operational Level Second O cer: Traditionally he is the ships navigator and watch keeper. Maintenance of the electronic equipments of the ship is under his responsibility so; he regularly controls bridge equipments and updates charts. Also, he is the medical o cer of the ship; he applies medical rst aid on board. ird O cer: Besides watch keeping and navigation, the third mate is in charge of safety equipments. Preventing and ghting re on board, operating life saving appliances are under his responsibilities.

Support Level Boatswain: He is the uppermost one of the unlicensed men on board. He plans daily works and charges the other deck hands with the routine works such as cleaning and painting the hull, superstructure or deck equipments.

Able Seaman: His role as a helmsman is to keep a steady course and keep Master: sharp look-out during the voyage. His role as a deckhand e master is responsible for organizing and managing is to ful ll the daily works given by the boatswain. He the crew, running the ship and the safety of the crew, performs for keeping the ship in good conditions and passengers and the cargo. He is the representative of the maintenance. owner, agent in any operation and also he is last man to abandon the ship in case of emergency. He is the super- Ordinary Seaman: visor of the other o cers and the crew so; he is in a way Like able seaman, ordinary seaman performs both as the manager of the ship. a helmsman and a deckhand. His duties are versatile, launching and recovering lifeboats, rigging and securing Chief O cer: cargo, cleaning the deck and equipments, removing dust A er the master, the chief o cer is second in command. and rust, scraping and chipping. He is the head of deck department and he behaves as an assistance of the master. As navigation is the primary Deckboy: responsibility of all the o cers, he is in charge of navi- He is the inexperienced and unquali ed one in the hierargation and he is on duty for watch. He supervises chical structure of the crew. He assists with all the works cargo handling, safe delivery of the cargo, watch keeping on board and his responsibilities depend on the type of arrangements and stability of the vessel. ship and the number of crew.

Management Level

22

Appendix 4 Table of linguistic skills and competences for the 3 main levels (A, B and C) within the marine industry

A
Listening UNDERSTANDING

SUPPORT LEVEL Minumum level required for seaman and boatswain

OPERATIONAL LEVEL Required for second ofcer and third ofcer

MANAGEMENT LEVEL Required for master and chief ofcer

Reading

Can recognise the name of deck equipment and tools that he works with. Can understand safety and emergency instructions, wheel orders, manoeuvring instructions and signals if spoken slowly. Can respond to safety warnings in emergency situations. Can understand any warning sentences on hatch cover, manhole, main deck, cargo box, container, training manuals, safety and re plans. Can also understand short personal notices. Can ask and answer simple questions about job related tasks. Can inform other people about his job and give instructions. Can use simple phrases and sentences to describe work related activities.

Can understand vhf and emergency communication, navigational warnings, meteorological warnings, port authority instructions, pilot instructions, master orders.

Can understand extended sentences, any marine crises management communication, any authority messages (psc, classiport authority, etc.), including long and complex sentences.

Can read any navigational reports, meteorological reports,sailing direction books,map abbreviations, routing charts, tide tables, admiralty radio signal books, training manuals, port guides, pilot books, etc. Can express himself effectively on vhf communication and emergency communication, can answer and ask questions on manoeuvring ship on bridge or deck. Can produce effective sentences meant to describe his work. Can explain opinions and plans for any situations. Can write clear sentences, can ll port log, navigation journal, any documents dealing with ships navigation.

Can understand cargo documents, important legal records, stability calculation books, ships certicates, navigation journal records, any medical and navigational books.

Spoken Production SPEAKING

Spoken Interaction

WRITING

Writing

Can write short,simple messages to the ship management staff informing them of workrelated issues. Can ll isps documents, can give crew details such as name, nationality, position, rank, etc.

Can express himself uently. Can use language effectively for work purposes. Can explain his orders and instructions effectively to subordinate staff. Can provide detailed descriptions of ship situations at any time. Can present clearly ships cargo name, nature of cargo, air draft, draft, ships particulars and other special information. Can express himself in clear well structured texts, can ll cargo documents, immigration forms, notice of readiness, bills of loading, legal forms, sea protest and statement of fact.

23

Appendix 5 Table of linguistic skills, assessment criteria and types of exercises for level A1 within the marine industry

A1
Listening

UNDERSTANDING

SUPPORT LEVEL/Minimum level required for seaman and boatswain Can recognise the name of deck equipment and tools that he works with. Can understand safety and emergency instructions, wheel orders, manoeuvring instructions and signals if spoken slowly. Can respond to safety warnings in emergency situations.

Reading

Can understand any warning sentences on hatch cover, manhole, main deck, cargo box, container, training manual, safety and re plan. Can also understand short personal notices.

Test specications per skill (Assessment criteria) Given a set of pictures showing different deck equipment, safety imo signs, tools and general alarm signals and listening to their names pronounced by the speaker, the test taker will match the pictures with the name with at least 80 % accuracy. Given a set of pilot rudder orders, manoeuvre orders and safety warning instructions and listening to the speaker the test taker will group the orders or instructions. Given a list of general technical terms (ship manoeuvre orders, cargo operations terms, names of materials) used on the ship, the test taker can match the words and pictures with at least 80 % accuracy. Given a set of sentences about deck equipment, the test taker will identify the name of the equipment.

Types of test tasks that can be used (examples only) Listen to what the speaker says and write down a few important pieces of information. Look at the warning symbols and listen to their interpretations.Match a symbol with its interpretation.

Match the terms on the left with their denitions on the right. Match the pictures on the left with the terms on the right. Label the following materials and tools. Write the names of the following objects beside the pictures that illustrate them. Read the notices carefully and choose a reply which matches their meaning. Answer the questions with very simple words but giving the required information. Discuss your daily work with your boss. Ask your work colleagues about their identication. Talk about your daily work on board the ship. Give some information about maritime rules. Ask your colleagues about the safety rules on board the ship, etc. Write a short note giving information about what should be done in an emergency situation at sea such as (grounding, ooding, re, abandoning, listing etc.) or any problem at ship related to your work.

Spoken Production SPEAKING

Can ask and answer simple questions about job related tasks. Can inform other people about his job and give instructions. Can use simple phrases and sentences to describe work related activities.

Spoken Interaction

24

WRITING

Writing

Can write short, simple messages to the ship management staff informing them of work related issues. Can ll isps documents, can give crew details such as name, nationality, position, etc.

Given some questions about seaman work, ship internal communication and crew identication, the task taker will produce accurate answers. Given pictures of a ship part, deck equipment, safety imo sign and isps warning sign, the test taker will provide their names and explain what they mean. Given blank samples of seaman works, cargo operation or deck works, the test taker will ll it in with 80 % accuracy. Given key words the test taker will write a simple explanatory notice about the word/term.

This project has been funded with support from the European Commission. This publication reects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

Arbeitsbereich Linguistik Herstrae Mnster Germany http://lcctv.uni-muenster.de

Language Competence Certication Tools for the Vocations