update: institution exams
An overview of the Institution’s
examination process
This article describes the procedures used by the Institution
in setting the Associate-Member (AM) and the Chartered
Membership (Part 3) examinations and assessing the
performance of candidates
T
he examination procedures have
evolved over a period of many
years as a result of the ongoing
review carried out by the Institution’s
Examinations Panel. The Panel consists
of corporate members of the Institution
who have served as Chief Examiners or
who have otherwise gained extensive
experience of assessment procedures
and the Institution’s examinations.
The ongoing review process and the
setting and marking of the Institution’s
examinations is led by the
Examinations Panel and administered
by the Institution’s Examinations
Officer who also serves as Secretary to
the Panel. The Examinations Panel is
responsible for developing examination
policy and for providing guidance to
candidates on all aspects of the exami-
nation process.
The review process
The Examinations Panel meets at least
four times each year to review all
aspects of the Institution’s examina-
tions. The Panel reports formally to the
Institution’s Membership Committee
which, in turn, reports to the Council of
the Institution.
In addition, the Panel meets with the
chief examiners and the marking exam-
iners on a regular basis to identify
examples of best practice, to discuss
improvements to any aspect of the
examination process, to seek feedback
on any draft proposals for change and to
assist with the training of newly
appointed examiners. The committee
structure not only considers and
approves proposals for change made by
the Panel but also provides guidance on
how matters of Institution policy will
affect the examination process.
Setting the questions
Each question of each examination is
set by a chief examiner. A chief exam-
iner is always either a member or fellow
of the Institution who has previously
gained several years experience as a
marking examiner (see below).
The first draft of each question is
considered in detail by the
Examinations Panel. The Panel reviews
each question to ensure:
• a level of technical challenge that is
appropriate to each examination;
• accuracy and clarity;
• consistency of style.
The development of each examination
paper is co-ordinated by a member of
the Examinations Panel. It is the
responsibility of each examination co-
ordinator to develop the final draft
version of the paper, taking into account
the views of the other members of the
Examinations Panel and each chief
examiner, following the revisions to the
first draft. The final draft of each paper
is approved by the chairman of the
Examinations Panel prior to printing. A
further check of the final printed
version of each examination paper is
carried out by the examinations co-ordi-
nator prior to the examination date.
Marking
Each examination script is marked by
two independent marking examiners.
Marking examiners are corporate
members of the Institution with exten-
sive design experience in one or more of
the specialist subject areas covered by
the examinations such as concrete,
masonry, steel and timber building
structures; bridges and offshore struc-
tures. The Institution uses an anony-
mous marking system; candidates are
only identified by a reference number
until the marking process is completed.
Approximately 1 month after the
examination date, a trial marking
meeting is held. This is attended by
members of the Examinations Panel,
the chief examiners and the marking
examiners. Following an initial briefing
by the chairman of the Examinations
Panel, trial marking of each question is
carried out by the marking examiners.
This is led by the chief examiner for
each question who produces marking
guidance for each question.
Even though the marking examiners
for each question are following the same
marking scheme identified by the chief
examiner, it is inevitable that differ-
ences in marking will occur. To take
account of this the marks awarded by
each marking examiner for each candi-
date are adjusted statistically to
produce a set of normalised marks
(normalisation of marks is a standard
technique that has been used for many
19 March 2002 – The Structural Engineer|13
years within the UK examination
system). The normalisation process
adjusts the mean and standard devia-
tion of the marks awarded by each
marking examiner to be the same as the
mean and standard deviation of the
marks awarded by the entire group of
marking examiners for that question.
Hence each candidate is awarded 4
marks – a raw mark and a normalised
mark from each of the two marking
examiners.
Approximately 3 months after the
examination date, the four marks for
each candidate are considered in detail
by the Examinations Panel. The answer
of any candidate who is judged to be a
borderline pass or fail is identified by
the Panel for adjudication by the appro-
priate chief examiner.
The chief examiner reviews, in detail,
the answer paper of each candidate
identified for adjudication and decides if
the candidate should pass or fail. Advice
is sought from the Examinations Panel
when considered necessary. Throughout
this process the anonymity of all the
candidates is maintained. Hence, the
performance of each candidate is consid-
ered by two independent marking
examiners and any borderline candi-
dates are considered further by the
chief examiner. Candidates with extenu-
ating circumstances are also considered
during the adjudication process.
On completion of the adjudication
process, the Examinations Panel and
chief examiners make recommendations
for the award of prizes.
Quality assurance
The Examinations Panel has developed
a thorough quality assurance procedure
for all stages of the examination
process. The procedure is divided into
two sections covering the setting of the
examinations and the assessment. All
aspects of the examination process are
subject to independent external audit.
Feedback
Feedback is given to unsuccessful candi-
dates (taken from a standard list and in
the form of sentences) which best
summarises candidate performance.
Additionally, marking examiners
receive feedback with regard to their
marking performance. The feedback
given to candidates and marking exam-
iners is constantly under review by the
Examinations Panel.
Appeals
The Institution has developed an
Examination Appeals procedure. se