Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Information Pack
Prepared June 2015
Contacts
Name
Position
Phone number
Email address
Charles Hamilton
Director Client Relations International
+44 (0)20 7010 2641
Charles.Hamilton@Pearson.com
Annabelle Llanes
Clients Relations Manager
+44 (0) 779 542 6179
annabelle.llanes@pearson.com
Pearson Education Limited is a registered company in England and Wales whose registered office is at
Edinburgh Gate, Harlow, Essex CM20 2JE, United Kingdom, company Registration number 872828
PTE Academic is a registered trademark of Pearson Education Limited
Contents
Introduction ............................................................................................................................................................................ 3
Purpose of the document........................................................................................................................................................ 3
Alignment of PTE Academic to Industry Standards ................................................................................................................. 3
Background to PTE Academic ................................................................................................................................................. 4
PTE Academic measurement of Reading, Writing, Speaking and Listening ............................................................................. 4
Automated Scoring, Test Objectivity and Reliability ............................................................................................................... 5
Scoring Written English Skills .................................................................................................................................................. 8
Scoring Spoken English Skills ................................................................................................................................................... 9
Test Error and Accuracy ......................................................................................................................................................... 9
Mapping to the Common European Framework (CEFR)...................................................................................................... 11
The PTE Academic Score Scale and the CEFR ...................................................................................................................... 12
What PTE Academic Scores Mean........................................................................................................................................ 12
Test Relevance ...................................................................................................................................................................... 13
Language Proficiency Requirements ...................................................................................................................................... 15
Booking and Payment Process .............................................................................................................................................. 15
Test Preparation ................................................................................................................................................................... 16
Test and Test Centre Security .............................................................................................................................................. 17
Threats to Secure English Language Testing .......................................................................................................................... 18
Fraud Risk Mitigation ............................................................................................................................................................. 18
Identifying Fraud .................................................................................................................................................................... 19
Score Reports and Sending Scores ........................................................................................................................................ 20
2015 Recognition .................................................................................................................................................................. 21
Summary ............................................................................................................................................................................... 21
Introduction
As a licensed United Kingdom Tier 4 visa-sponsoring Higher Education Institution you have the ability to choose how to
assess English language competence for students studying at degree level and above. (See page 28 section 5.11 Tier 4 of
the Points Based System: Guidance for Sponsors) The aim of this document is to provide our HEI partners with a
comprehensive source of information about the Pearson Test of English Academic (PTE Academic) to address any
questions you may have as you continue to use PTE Academic for vouching purposes.
Purpose of the document
PTE Academic provides a best-in-class high stakes English proficiency test that meets and frequently exceeds Home Office
SELT standards. PTE Academic delivers high security, industry-leading accuracy and ease of user access. Our technology
allows us to mitigate error, control fraud, eradicate bias and deliver results faster than any other equivalent assessment.
Above all we offer a fair and consistent computer-based test in every location where we operate.
Alignment of PTE Academic to Industry Standards
PTE Academic was specifically constructed as a secure English language test. As a result its security, methodology, delivery,
accuracy and governance meets or exceeds industry best practice.
The Information Pack covers the attributes of PTE Academic in detail but, for quick reference, the table below identifies a
list of broad industry best practices for High Stakes tests and an indication of whether PTE Academic meets them.
Best Practice for High Stakes English language testing
PTE Academic
Ofqual
Test Taker Voice (audio) file captured and available on Score Verification Website
Candidate photograph captured on test day and made available on score report
Up to 24 hours
5 Days
Biometric security process with Palm-vein scanning comparing 1 to 1 and 1 to many images
(Please note: The minimum and maximum timings indicated for the sections of each part of the test do not add up to the
total timings stated. This is because different versions of the test are balanced for total length. No test taker will get the
maximum or minimum times indicated.)
Automated Scoring, Test Objectivity and Reliability
The best tests are objective. It should not matter to the test taker when or where they take the test or who scores it. Test
takers and institutions should be confident of receiving a score which reflects the test takers ability.
Automated Scoring and Objectivity
One aspect of test objectivity is how a test is scored. PTE Academic, TOEFL iBT and IELTS use different approaches for
assessing test taker performance automated scoring only, human raters combined with automated scoring, and human
raters only. PTE Academic only uses automated scoring.
PTE Academic
TOEFL iBT
IELTS
Automated Scoring
Human rating
Scoring Method
Automated Scoring
Human rating
Human rating
Speaking
IELTS is a registered trademark of the University of Cambridge ESOL Examinations Syndicate.TOEFL iBT is a registered trademark
of Educational Testing Service (ETS).
(See How IELTS is marked and TOEFL iBT Test Scores)
Research indicates that, in many ways, automated scoring provides more objective results than humans do. Unlike human
judgment, which is prone to be influenced by a variety of factors, an automated scoring system is impartial. This means that
the system is not distracted by language-irrelevant factors such as a test takers appearance, personality or body language
(as can happen in spoken interview tests).
As the worldwide leader in publishing and assessment for education, Pearson uses several of our proprietary, patented
technologies to automatically score test takers performance on PTE Academic.
Academic institutions, corporations and government agencies around the world have selected Pearsons automated
scoring technologies to measure the abilities of students, staff or applicants.
An extensive field test program was conducted to evaluate PTE Academic test items as well as to obtain the data
necessary to train the automated scoring engines to assess PTE Academic items.
Test data was collected from more than 10,000 test takers from 38 cities in 21 countries who participated in PTE
Academics field test. These test takers came from 158 different countries and spoke 126 different native languages.
The data from the field test were used to train the automated scoring engines for both the written and spoken PTE
Academic items.
By combining the power of a comprehensive field test, in-depth research and Pearsons proven, proprietary automated
scoring technologies, PTE Academic fills a critical gap by providing a state-of-the-art test that accurately measures the
English language speaking, listening, reading and writing abilities of non-native speakers
Because PTE Academic scoring is automated, the responses are rated objectively and consistently, no matter where in the
world the test is taken. There are no regional variances in standards meaning you can have complete confidence in the
communication skills of your students.
PTE Academic treats regional accents equally. Human assessors tend to value accents they are used to higher than accents
they are unfamiliar with. You can therefore be confident in the PTE Academic scores of international students from all over
the world. (See Objective_Factsheet.pdf)
The table below captures the reported reliability estimates for PTE Academic, TOEFL iBT and IELTS. PTE Academic has
the highest reliability estimates for both the overall score and the communicative skills scores.
PTE Academic
0.972
TOEFL iBT
0.943
IELTS
0.96
Overall Reliability
Communicative skills reliability
Reading
0.92
0.85
0.90
Listening
0.91
0.85
0.90
Writing
0.91
0.74
0.81-0.89
Speaking
0.91
0.88
0.83-0.86
IELTS is a registered trademark of the University of Cambridge ESOL Examinations Syndicate.TOEFL iBT is a registered trademark
of Educational Testing Service (ETS).
(See IELTS analysis of test data/test performance 2013 and TOEFL iBT Research Reliability and Comparability of
TOEFL iBT Scores)
This demonstrates that PTE Academic generally produces more consistent and reliable results than competitive tests.
It is widely recognised that the scores human raters give can be influenced by irrelevant factors, particularly when only one
person rates the test takers performance. Automated scoring has the benefit of removing this effect as it is indifferent to a
test takers appearance and personality, and is not effected by issues such as examiner tiredness, mood and leniency.
Automated scoring also allows individual features of a language sample (e.g. vocabulary or pronunciation) to be analysed
independently, so that weakness in one area of language does not affect scoring in others.
Although PTE Academic is computer-based and scoring is automated, responses to all new tasks that require constructed
responses (e.g. essays) are initially human scored so that the intelligent scoring systems can be appropriately trained and
calibrated. (See pages 53 to 64 Score Guide)
PTE Academic only uses expert human raters who are retrained and certified before each rating session. PTE Academics
automated scoring is based on the collective wisdom of a large pool of skilled human raters (over 200) and not, as in
human scoring, on the views of one or a handful of individuals. As a result every single response receives a replicable,
objective and completely impartial score.
The speech recognizer was trained to treat all their accents equally using the same criteria. Most human raters are likely to
be more accustomed to some accents than others which will make it difficult for them to treat accents they are not familiar
with equally. Some raters may be accustomed to perhaps half a dozen accents but it is extremely unlikely that they will be
able to treat over a hundred different foreign accents as objectively as PTE Academic.
The impartiality of automated scoring means that test takers can be confident that they are being judged solely on their
language performance, and stakeholders can be confident that a test takers scores are generalizable that they would
have earned the same score if the test had been administered in Beijing, Brussels or Bermuda.
7
score contributes to the communicative skills score for writing, as well as to the overall score reported for performance on
the entire test.
Scoring Spoken English Skills
The spoken portion of PTE Academic is automatically scored using Pearsons Ordinate technology. Ordinate is the
product of years of research in speech recognition, statistical modelling, linguistics and testing theory. This technology is
used by organizations such as the U.S. Department of Homeland Security, schools of aviation around the world, the
Immigration and Naturalization Service in the Netherlands, and the U.S. Department of Education
The technology uses a proprietary speech processing system that is specifically designed to analyse and automatically score
speech from native and non-native speakers of English. In addition to recognizing words, the system locates and evaluates
relevant segments, syllables and phrases in speech and then uses statistical modelling techniques to assess spoken
performance.
To understand the way that the Ordinate technology is taught to score spoken language, think about a person being
trained by an expert rater to score speech samples during interviews.
First, the expert rater gives the trainee rater a list of things to listen for in the test takers speech during the interview. Then
the trainee observes the expert testing numerous test takers, and, after each interview, the expert shares with the trainee
the score he or she gave the test taker and the characteristics of the performance that led to that score.
Over several dozen interviews, the trainees scores begin to look very similar to the expert raters scores. Ultimately, one
could predict the score the trainee would give a particular test taker based on the score that the expert gave.
This, in effect, is how our system is trained to score, only instead of one expert teaching the trainee, there are many expert
scorers feeding scores into the system for each response, and instead of a few dozen test takers, the system is trained on
thousands of responses from hundreds of test takers.
Furthermore, the system does not need to be told what features of speech are important; the relevant features and their
relative contributions are statistically extracted from the massive dataset when the system is optimized to predict human
scores.
Independent studies have demonstrated that Ordinates automated scoring system can be more objective and more
reliable than many of todays best human-rated tests, including one-on-one oral proficiency interviews.
Where a candidate produces a response that is, for example, off topic, or is insufficiently similar to the training responses
used to calibrate the system, the response is referred for human marking.
All tests contain an element of error. The size of the error component is a function of the reliability of a test and is known
as the Standard Error of Measurement (SEM).
SEM is used to track proximity between observed and true scores. The range of scores in which the true score is expected
to be found represents the confidence interval associated with an observed score. Generally speaking, an interval of one or
two SEMs around the observed score is used to discuss accuracy within the academic-testing world. The smaller the SEM,
the more accurate the test.
PTE Academic, TOEFL iBT and IELTS each report the SEM of their tests. To compare these SEMs, we need to transfer all
scores onto the same scale to make sure we are comparing like with like. Using the concordance tables in the PTE
Academic Score Guide (See pages 49 and 50 Score Guide) TOEFL iBT and IELTS scores can be placed on the same scale
as PTE Academic. The scale used is the Global Scale of English, which runs from 10 to 90.
In the tables below, we use a 95% confidence level and an example score of 59. This shows that for PTE Academic, you
can be 95% confident that the test takers true score falls within the 10-point range shown in the table. For TOEFL iBT the
range is 18 points, and for IELTS it is 22 points
Test
SEM
PTE
Academic
TOEFL iBT
Min
Max
59
58.5-59.4
54
64
5.64
87
86.5-87.4
75
99
59
0.23
IELTS on GSE
2 x SEM
Rounded
From
2.32
TOEFL on
GSE
IELTS
Example
Score
6.5
6.25-6.74
59
68
5.8
7.2
50
Range
GSE
50
72
10
points
23
points
95%
Interval
10
points
18
points
1.4
band
22
points
Score of 59
47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70
PTE Academic
10 points 54-64
PTE Academic
TOEFL iBT
18 points 50-68
TOEFL iBT
IELTS
22 points 48-70
IELTS
IELTS is a registered trademark of the University of Cambridge ESOL Examinations Syndicate.TOEFL iBT is a registered trademark
of Educational Testing Service (ETS).
10
The purpose of assessment is to precisely capture a test taker's true level of ability. PTE Academic is the most accurate
test of academic English in the world. We demonstrate the accuracy of our tests through a lower standard error of
measurement and the precision of our scores. (See Accurate Factsheet.pdf)
Mapping to the Common European Framework (CEFR)
To ensure comparability and interpretability of test scores, PTE Academic was constructed to align to the Common
European Framework of Reference (CEFR), which is recognized as a standard across Europe and in many countries across
the world. (See Aligning PTE Academic Test Scores to the Common European Framework of Reference for Languages)
PTE Academic differs from other tests and exams which claim alignment to the CEFR in that the test was designed to
measure language competence according to the principles of the CEFR. PTE Academic assesses candidates ability in all
four language areas of reading, writing, speaking and listening, from A1 to C2 within the CEFR and specifically language
competencies in the range from upper B1 to lower C2.
Pearson involves external experts in the fields of language education when developing assessments. This includes
assessment methodologists, statistical analysts and psychometricians as well as educational practitioners such as language
school representatives, teachers and teacher trainers. Technical Advisory Groups for our PTE Academic SELT and the
Global Scale of English are chaired by Professor John de Jong, a Pearson employee, who has been extensively involved in
the development of the CEFR.
The CEFR includes a set of consecutive language levels defined by descriptors of language competencies. The six-level
framework was developed by the Council of Europe (2001) to enable language learners, teachers, universities or potential
employers to compare and relate language qualifications by level.
Alignment of PTE Academic to the CEFR levels provides a means to interpret PTE Academic scores in terms of the level
descriptors of the CEFR. As these descriptors focus on what an English language learner can do, scores that are properly
aligned to the CEFR give educators and institutions more relevant information about a test takers ability.
The relationship between the PTE Academic score scale with the descriptive scale of the CEFR is based on item-centred
and test taker-centered methods. For the item-centered method, the CEFR level for items was estimated by item writers,
reviewed and, if necessary, adapted in the item-reviewing process.
For the test taker-centered method, three extended responses (one written and two spoken) per test taker were each
rated by two independent, trained raters. If there was a disagreement between the two independent raters, a third rating
was gathered and the two closest ratings were retained.
A dataset of over 26,000 ratings (by self-reporting test takers, by items and by raters) on up to 100 different items was
analysed using the computer program FACETS.
Estimates of the lower boundaries of the CEFR levels, based on the item-centered method, correlated at .996 with those
based on the test taker-centered method, which effectively means that the two methods yielded broadly the same results.
11
12
The table below shows for the CEFR levels A2 to C2 which PTE Academic scores predict the likelihood of a test taker
performing successfully on the easiest, average and most difficult tasks within each of the CEFR levels.
Assuming a score of 51 we would expect the following level of English language competency;
PTE
Academic
Score
51-58
Common
European
Framework Level
Scores in this
range predict
success on the
easiest tasks at B2
Level Proficiency
Mapping of PTE Academic to the CEFR is reviewed each time the item bank is updated. The procedure is carried out by a
statistical analyst and psychometrician and then checked by a second psychometrician who is not directly involved in the
testing programme. The last update of the mapping was in April 2014.
Test Relevance
The degree that a test reflects the real life demands of study is an important factor in determining English language
proficiency. Authenticity is an integral part of PTE Academic ensuring that students are able to use their English effectively
in academic settings. We do this through the use of genuine academic test content, setting academically relevant tasks, and
by measuring skills in an integrated way, for example - assessing the ability to listen to a lecture and then provide an oral
summary. Test takers also hear a range of accents in the test, from British and American to non-native speakers. (See
pages 8 to 35 Score Guide)
13
Not all English language tests are the same when it comes to the type and format of the questions. 11 of the 20 PTE
Academic question types are what we refer to as integrated i.e. questions that contain tasks that address more than one
language skill.
Integrated tasks are those that test more than one language skill, and therefore reflect the combinations of skills students
need at university. For example, PTE Academic test takers are asked to listen to a lecture, take notes and then provide an
oral or written summary, or they have to read and summarize written information, or understand and repeat what they
have heard.
This table below demonstrates that PTE Academic has by far the largest number of integrated tasks of all major tests of
academic English. Indeed, over half the test is made up of integrated tasks.
Number of Integrated Task
Types
PTE Academic
11
47-55
TOEFL iBT
IELTS
n/a
n/a
IELTS is a registered trademark of the University of Cambridge ESOL Examinations Syndicate.TOEFL iBT is a registered trademark
of Educational Testing Service (ETS).
PTE Academic includes these integrated tasks to reflect the real life language skills that students will need to apply in an
academic environment. Research highlights the importance and advantages of using integrated tasks to improve test validity
and increase authenticity.
Another way to increase test authenticity is to ensure the use of authentic content. Content for the questions used in PTE
Academic is taken from real-life situations which test takers will encounter in an academic environment. Question writers
are required to use actual texts as stimulus material and no question without a source reference is accepted (See Relevant
Factsheet.pdf.)
Reading texts appropriate for PTE Academic include study texts of academic interest and texts related to all aspects of
student life and the lectures are genuine academic lectures, not actors reading scripts.
Tests that use only one specific English variety put test takers not familiar with that variety at a clear disadvantage. We
create an international flavour for PTE Academic by selecting texts and settings drawn from Australia, Canada, New
Zealand, the United Kingdom and the United States of America. Using the main varieties of English ensures that all test
takers are on an equal footing. Furthermore PTE Academic is the only academic English test to include non-native English
accents. This reflects the diversity of English that students are likely to experience at any university where English is the
medium of instruction, and where students will be taught by foreign professors and teaching assistants.
14
This use of international varieties of English and non-native accents makes PTE Academic highly relevant to todays modern
international academic institutions.
Language Proficiency Requirements
The requirement is for a student to be able to demonstrate a minimum proficiency of CEFR B2 level for degree level study
or above. A B2 equates to a PTE Academic score of at least 51. It is the university that decides on the score required for
admission provided that this is at B2 or above.
Our experience suggests that most universities require:
For some university programs, the communicative skills scores for Listening, Speaking, Reading and Writing may provide
useful, additional information for making admissions decisions.
For example, institutions may:
set the admission requirement based on the minimum overall score alone, without taking into account
communicative skills scores in admission decisions;
set the admission requirement based on the minimum overall score in combination with a higher minimum on one
of the communicative skills scores, because it is considered particularly important for the program the test taker
wants to enter;
set the admission requirement based on the minimum overall score in combination with a lower minimum on one
of the communicative skills scores, because it is considered less important for the program the test taker wants to
enter.
Other combinations of the overall score and one or more of the communicative skills scores may be considered.
When the booking is complete a booking confirmation is displayed onscreen and a confirmation email is automatically and
instantly sent to the candidate.
Information included in the confirmation email includes:
Candidates can login anytime to reschedule or cancel their booking within the parameters of the PTE Academic business
rules. The easy-to-follow process is similar to that followed when first booking the exam.
(See Test session live search and Test taker registration and booking page)
Candidates cannot book multiple tests and may only book another test once they have received the scores for the
previous test. Through use of a proprietary matching algorithm we can ascertain if a candidate has an account. If a
candidate is determined to be new login details are granted. If the candidate has an existing record, they are notified and
their existing username is emailed to them.
Test Preparation
For the best preparation, we recommend the PTE Academic scored practice test
Our scored practice test is a full three-hour, computer-based mock test. At the end, the candidate will receive an example
score report, which will help show their strengths and weaknesses and highlight areas to improve. The candidate can
choose from two different version of the scored test, or try both for the ultimate preparation.
16
Test takers can choose from a range of free preparation materials (See Free PTE Academic Preparation Material) to get
ready for PTE Academic including downloading our offline practice test, watching hours of skills videos on YouTube or
reading our test tips and test tutorial.
A candidate can prepare for PTE Academic using a range of official course books. They can choose from The Official
Guide to PTE Academic or Practice Tests Plus to understand the test format and prepare for specific item types. Or, if
they are studying over a longer period, choose PTE Academic at level B1 or B2 as your official course book.
The Official Guide to PTE Academic provides comprehensive information about the test, over 200 practice tasks
on CD-ROM, analysis of sample answers, test-taking tips and more. This is the first and only official guide for PTE
Academic written by the developers of the test itself and includes everything that a candidate needs to know about the
test.
Test and Test Centre Security
PTE Academic is taken at a secure test centre, where the candidate presents their ID, checks in with the test
administrator, and sits the test in a secure computer room. (See Secure Factsheet.pdf)
Pearson uses progressive identification verification to make sure that only authorised and verified individuals are able to sit
exams. The identity management solution includes:
ID Documents - Any candidates testing outside their country of origin use their passport or travel document as
identification. Candidates ID documents are checked to confirm that they contain a photo which matches that of
the candidate and that these docs have not expired. The ID document is checked to ensure the biographic details
(name, date of birth, nationality, gender and passport number as presented) match those in which the test booking
was made. The candidates photo ID is checked to ensure it matches the candidate.
Photos - A digital photo is taken of the candidate at registration on the test day(s) and compared with the
candidates photo ID.
Palm Vein Scan - All around the world, test centers that deliver PTE Academic are equipped with advanced palm
vein recognition technology. These devices capture and recognize the unique patterns in a test takers palm veins
using non-intrusive scanning technology. During the check-in process a comparison is made between the pattern
recorded and Pearsons No Test List. This ensures that individuals who have previously been prohibited from
testing are not able to do so under an assumed identity or as a proxy for another test taker. Furthermore all test
taker identities are automatically verified when taking and returning from breaks. The system also provides an
enhanced service, known as one-to-many matching. It is configured to compare each test takers biometrics to an
entire test taker database prior to the release of the test results. The system detects duplicate records and where
duplicates are identified the results are automatically withheld whilst an investigation takes place.
Signature - Candidates are asked at registration on the Test day to provide a sample of their signature and this is
compared to that on the ID documents provided.
17
Voice Sample - A voice sample (30 second unscored personal introduction) of all candidates is recorded and may
be made available to institutions via the score report website.
A typical test centre configuration would look like the image right
with ceiling-mounted cameras, separated and standard sized
testing booths and video feeds from each workstation.
Threats to Secure English Language Testing
Even with the highest levels of security applied to the test centre and test process there will always be attempts at fraud.
The main threats are as follows;
Proxy Testing - an imposter takes the test on behalf of the registered person. Attempts may be systematic and
organised, or carried out in a limited way by individuals.
In-room cheating: collusion between test takers, use of notes/cheat sheets, swapping of seats or test papers, use
of communication devices.
Test administrator/examiner collusion: active participation in test fraud by personnel involved in test
administration and scoring/marking.
Counterfeiting of reports/certificates
Content harvesting and leaks: coordinated memorisation of test content, use of surveillance equipment to record
or photograph content. Attempts may be systematic and organised, or carried out in a limited way by individuals.
Content theft: in paper-based testing theft or copying of hard-copy test materials in transit or on-site, in
computer-based testing hacking of or unauthorised access to systems used to store or transmit test content.
18
19
Where the system establishes an anomaly associated with unusual test scores or characteristics of the administration of the
test a hold is automatically placed on the result for investigation by the security team.
In this way every test is thoroughly examined prior to the release of results to the test taker.
The enabling skills scores are also provided in the score report. They
provide information about particular strengths and weaknesses of a test takers ability to communicate in speaking or
writing. This information may be useful to determine remedial or further English study and coursework required to
improve a test takers score. The enabling skills scores should not be used when making admissions decisions because the
measurement error is too large.
The graphic display of the scores in the report allows the candidate and the institution to quickly view the candidates
strengths and weaknesses, and how these relate to the overall performance
In the context of some university programs, the communicative skills scores may provide useful, additional information for
making admissions decisions.
For example, institutions may set the admission requirement;
based on the minimum overall score alone, without taking into account communicative skills scores in admission
decisions;
based on the minimum overall score in combination with a higher minimum on one of the communicative skills
scores, because it is considered particularly important for the program the test taker wants to enter;
based on the minimum overall score in combination with a lower minimum on one of the communicative skills
scores, because it is considered less important for the program the test taker wants to enter.
20
Other combinations of the overall score and one or more of the communicative skills scores may be considered.
Once test takers have received their scores electronically via a secure online portal, they are free to assign their score to
any recognising institution or program. To ensure you maximise the security of PTE Academic scores, you should always
verify scores through the Score Report Website (SRW) see SRW Login Page); you should not accept paper or PDF
copies of the score report.
The score report website lets you:
21