SCORING AND REPORTING OF RESULTS EXPLANATION OF SCORING The speaking and writing sections are graded cording to scales established by Cambridge Michigan Assessments (see appendices I and 2 of this The speaking section is conducted and assessed report by trained local oral examiners, ‘The writing seetion is assessed in the United States by raters trained and certified wcording (© our standards, All writing section responses re scored by at least wo raters, The listening and the grammar, vocabulary, and (GYR) sections of the ECCE are scored by reading computer at the University of Michigan. A sealed score is caleulaced using an advanced mathematical model based on Item Response Theory (IRT). IRT scores are not the same as number-right scores, but there is a very high correlation between number of correct answers and the IRT scores, Item Response Theory ensures that the ability required to pass the multiple-choice sections (che listening and GYR) remains the same from year to Scores on the different sections of the ECCE are categorized into five performance levels High Pass (HP) Pass (P} Low Pass (LP) Borderline Fail (BF) Fail (P) For the speaking and writing, ratings are ¢ asA=HPB=P.C=LPD = BE, and E and GVR, scaled scores established by IRT scoring tegorized F For listening used to categorize an examinee’s score into a performance level. The same pre-established standardized scaled scores are used from year to year to categorize scores from the ind GVR sections. nine’ scores (the rating for speaking and and GYR) meet the writing: the scaled score for listenin; cutoff level fora LP, P, HP, ch a pass for that section of the three sections with a Low Pass (or higher) and receive no examinee is considered as am. Examines who pass less than a Borderline Fail in one section are awarded an certificate, PROCEDURES FOR REPORTING SCORES All examinees receive an individual Report of Examination Results that shows their overall examination result and their performance results on each section of the exam. The overall examination result is shown as ci Pass or Fail The four EC bands, These levels of performance, tion scores are reported in five Jom highese to ECCE Five Levels of Performance Writing & Speaking High Pass (HIP) 840-1000 A Pass (P) 750-835 B Low Pass (LP) 650-745 c Borderline Fail (BF) 610-645 D Pail 0-605 E ‘The speaking section level and writing section reported with a brief description of whac co's resules mean, Numeric lev che exami are reported for the listening section and the GVR. (grammar/vocabulary/reading) section. Examinees whose overall examination result is Pass are awarded an ECCE certifica INTERPRETING ECCE RESUL’ The ECCE is aimed at the B2 level of the Common, European Framework of Reference for Languages Learning, Teaching, Assessment (Council of Europe, 2001) ( Independent Users who: users at this level are considered PR), Lang Can understand che main ideas of complex cexc on both concrete and abstract copics, including technical discussions in his/hy of specialization. Can interact with a de of fluency and spontaneity that makes x imteraction with native speakers quite possible without strain for either party. C: detailed rexe on a wide range of subjects and produce clear, explain a viewpoint on a topical issue giving the advantages and disadvantages of various options (Council of Europe, 2001: 24) ECCE certificate holders can be expected to understand spoken English on a variety of topics, both h. They can conversational and more planned spe be expected to sustain a face-to-face oral interaction quite intelligibly, They can be expected wo be able to ask information and clarification questions and grasp information delivered co them at a normal pace, They can be expected to be able to communicate their ideas and show involvement in a topic or situation and diseuss ideas or opinions without communication breakdowns. They have sufficient concrol of grammar and vocabulary to express themselves quite comprchensibly both orally and in writi be expected to organize and explicitly connect their ideas. They can give their opinion and develop their argument with supporting details, Vasiability in linguisticcontrol in their speaking or writing does nor significantly interfere with communication. The can be expected to read and understand prose and nonprose texts from a variety of popular and public sources using a range of effective reading strategies and skills though they can be expected to need resources for understanding texts with low frequency vocabulary, highly specialized terms, or low frequeney idioms, ‘When interpreting an ECCE score report, itis important to remember that the ECCE estimates the examinees’ true competency in English based on their ability to perform on the test tasks. Also, temporary factors untelaced to an examinee’ English competeney, such as fatigue, anxiety, or illness, may affect exam results. ‘When using test scores for decision-making, check the date the ECCE was taken, While the certificate is valid for the holder’ lifetime, language abiliey changes over time, This ability can improve with active use and further study of the language, or it may diminish if the holder does noc continue co seudy or to use English on a regular basis. 1 is also important to remember that test performance is only one aspect to be considered. Communicative language ability consists of both knowledge of language and knowledge of the world, Therefore, one would need to consider how factors other than language affect how well someone can communicate For example, in che general context of using English in a work setting, the ability to function effectively involves not only knowledge of English, but also other knowledge and skills such as intellectual knowledge and other job skills TEST-TAKING POPULATION This section presents an overview of the examinees, who took the May 2011 ECCE, providing information about the countries in which the exam was administered, as well as the distributions for examinee age and gender. A total of 40,134 examinees in 20 councties took the May 2011 ECCE the examination was administered Table 2 shows the countries in which Vable 2: ¢ the ECCE was in May 2011 Countries Whe Administered Argentina Costa Rica Paraguay Belgium Ecuador Peru Bolivia Germany Poreugal Brazil Greece Spaie Bulgaria laly Uruguay Chile Lebanon Vietnam Colombia Mexico Every examinee completes a registration form, which asks for gender and date of birth, Tables 3 and 4 show the distribution of examinees by age and gender. The examinees for data shows that the majority of ECCI this administration were under oxency years old, with a sizeable proportion in che 13-16 age group. This also shows that many examinees take the EC CE when they are in secondary education and before higher or further education, The majority of the examinees who took the May 2011 ECCE were female, but the test population is somewhat equally balanced between females and males. This distribution is similar to previous administrations of the exam, Table 3: Distribution of ECCE Examinees by Age Number of % of MII Tes Age Examinees Population <2 5 0.19 13-16 33044 82.33 I-19 2735 681 20-22 650 All 23-25 140 2.84 26-29 672 Lor 30-39 521 Lo 240 296 0.74 Missing data 1 0 Table 4: Distribution of ECC Number of Exami Examinees by Gender % of MII Test Go Male 18394 Female 21739 Missing data 1 0