This article was downloaded by: [RMIT University]

On: 16 August 2015, At: 18:53

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: 5 Howick Place, London, SW1P 1WG

British Journal of Audiology

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/ijap20

Categories of Auditory
Performance: Inter-User Reliability
a b
Sue Archbold Coordinator , Mark E. Lutman & Thomas
c
Nikolopoulos
a
Nottingham Paediatric Cochlear Implant Programme,
Ropewalk House, Nottingham
b
Institute of Sound and Vibration Research, University of
Southampton
c
Department of Otorhinolaryngology, Queen's Medical
Centre, University Hospital, Nottingham
Published online: 16 Apr 2015.

To cite this article: Sue Archbold Coordinator, Mark E. Lutman & Thomas Nikolopoulos (1998)
Categories of Auditory Performance: Inter-User Reliability, British Journal of Audiology, 32:1,
7-12

To link to this article: http://dx.doi.org/10.3109/03005364000000045

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information
(the “Content”) contained in the publications on our platform. However, Taylor
& Francis, our agents, and our licensors make no representations or warranties
whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and
views of the authors, and are not the views of or endorsed by Taylor & Francis. The
accuracy of the Content should not be relied upon and should be independently
verified with primary sources of information. Taylor and Francis shall not be liable
for any losses, actions, claims, proceedings, demands, costs, expenses, damages,
and other liabilities whatsoever or howsoever caused arising directly or indirectly in
connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
 Terms & Conditions of access and use can be found at http://www.tandfonline.com/
page/terms-and-conditions
Downloaded by [RMIT University] at 18:53 16 August 2015
 British Journal ofAudiology, 1998,32,7-12

Short paper

Categories of auditory performance: inter-user

reliability
Sue Archbold', Mark E. Lutman2 and Thomas Nikolopoulos3
'Nottingham Paediatric Cochlear Implant Programme, Ropewalk House, Nottingham, 'Institute of
Sound and Vibration Research, University of Southampton and 'Department of
Downloaded by [RMIT University] at 18:53 16 August 2015

Otorhinolaryngology, Queen's Medical Centre, University Hospital, Nottingham

(Received 14 February 1997, accepted 16 July 1997)

Abstract
Categories of Auditory Performance (CAP) describes a scale used to rate outcomes from paediatric
cochlear implantation in everyday life. It differs from more technical measures by being readily applied and
easily understood by non-specialist professionals and by parents. Being based on subjective assessments,
there is a need to establish whether ratings by different persons are comparable. Therefore, an analysis of
inter-user reliability was undertaken using ratings from 23 children followed up at various intervals after
implantation. Analysis relating scores by local teachers of the deaf and the teachers of the deaf at the
implant centre revealed very high inter-user reliability (correlationcoefficient 0.97). This result establishes
the reliability of CAP as an outcome measure for use in cochlear implant programmes.

Key words: paediatric cochlear implant, outcome, parents, health-care purchasers

Introduction prolonged period of time. Such measures need to

Paediatric cochlear implantation in the UK has
grown rapidly in recent years, with 600 children
implanted by the end of 1996 (Summerfield, per-
sonal communication). There remains a need for
standardized assessment of outcomes on a
national basis (Summerfield and Marshall, 1995).
Few measures of benefit from cochlear implanta-
tion are readily understood by parents involved in
making the decision for their child to proceed, or
by non-specialist professionals who may be in the
position of advisors, or by health-care purchasers
who have to prioritize public expenditure. There is
wide variation among children in progress follow-
ing implantation, and there is a shortage of mea-
sures that can be applied successfully for use with
all children, both prior to implantation and for
a number of years after implantation. This
is required to demonstrate progress over a
Address for correspondence: Sue Archbold, Coordinator,
Nottingham Paediatric Cochlear lmplant Programme,
Ropewalk House, 113 The Ropewalk, Nottingham NGI
6HA
have a wide range of outcome levels in order to
avoid floor and ceiling effects, and to be scaled in a
way that spreads milestones as evenly as possible
across the range. Thus, a child progressing accord-
ing to the generally expected timescale will
achieve milestones at somewhat regular intervals.
While formal objective measures are impor-
tant, they may not be generally understood by
non-specialists, and may not reflect the function-
ing of the child in everyday situations at home
and at school. Vidas, Hassan and Parnes (1992)
found that clinical tests of speech and hearing
may not reflect performance at home and school
following implantation and, like Selmi (1985)
and Cunningham (1990), that the assessments by
parents, teachers and clinic-based therapists dif-
fer widely. Cunningham (1990) found that par-
ents considered that their children performed
significantly better in structured (training) situa-
tions than in unstructured (real-life) situations.
Parents, health-care purchasers and consumers
may find it more valuable to know how the child
 8 Sue Archbold, Mark E. Lutman and Thomas Nikolopoulos
is functioning at home and at school rather than
the results obtained in a clinical setting.
The difficulties of assessing young deaf chil-
dren are well known; tests that are available have
problems of low testhe-test reliability and in con-
sistency of presentation (Boothroyd, 1991;
Osberger et al., 1991; Staller et al., 1991). Cate-
gories of Auditory Performance (CAP) was
designed to provide an outcome measure that
would be accessible to parents, professionals and
health-care purchasers alike (Archbold et al.,
1995). It is now used in many cochlear implant
centres in the UK and Europe. Some teachers of
the deaf are also finding it useful for hearing aid
wearers. It provides a scale on which children’s
Downloaded by [RMIT University] at 18:53 16 August 2015
developing auditory abilities can be rated in eight
categories in order of increasing difficulty. The
categories are:
Displays no awareness of environmental
sounds
Awareness of environmental sounds
Responds to speech sounds
Recognizes environmental sounds
Discriminates at least two speech sounds
Understands common phrases without
lipreading
Understands conversation without lipreading
with a familiar talker
Can use the telephone with a familiar talker
The guidelines (see Appendix) for completion
have been revised following use by implant cen-
tres and teachers of the deaf, to ensure that ambi-
guity is minimized. Although the guidelines aim
to standardize the measure, there is a danger that
different users may rate the same child differ-
ently, which would limit its usefulness. There-
fore, the aim of this study was to test the
reliability of this measure by determining its
inter-user repeatability.
Methods
Children in the Nottingham programme are cate-
gorized prior to implantation, at 0,3,6,12 months
after implantation, and thereafter annually. In
this context, 0 months means within a few days of
initial stimulation. In this study 23 children,
selected at random, were categorized by their
implant centre teacher of the deaf, and indepen-
dently by their own teachers of the deaf.
Although the implant centre and local teachers
would have discussed each child in many ways
concerning the abilities mentioned in the CAP,
they did not discuss the completion of CAP itself
in any respect. Four implant centre teachers and
23 local teachers were involved in the study;
opportunity was given for clarification of the cat-
egories, but this was not found to be necessary. A
range of children were included in the study,
including those with congenital and acquired
losses, those with short and long durations of
deafness, and those in a variety of educational
settings. Details of the children are provided in
Table 1.
The children had a range of duration of implant
use, and assessments were compared at a variety
of intervals from pre-implant to five years post-
implant to ensure that the conclusions would be
applicable over a range of intervals. Thus, ratings
for 23 children at different intervals were cross-
checked, and the results for the implant teachers
compared with the local teachers.
Results
Table 2 compares the scores of the local teachers
and the implant centre teachers of the deaf, and
clearly demonstrates a high level of agreement.
There were only three discrepancies out of the 23
cases; the choices of categories only varied by one
in each case. Regression analysis carried out on
the scores revealed that the correlation coeffi-
cient between the two assessments was 0.97. The
three discrepancies in categorization were at the
immediate post implant interval, six months after
implantation, and four years after implantation.
In two instances the local teacher of the deaf
rated the child in a higher category, and in one
instance, the implant centre teacher of the deaf
rated the child in a higher category.
Discussion
The agreement between the local teachers, who
often see the children daily, and the implant cen-
tre teachers, who see the children less often, indi-
cates that CAP gives a repeatable measure, and
hence is reliable. This is demonstrated for a
group of children who illustrate the range of chil-
dren presenting for implantation, in terms of
both biographic data and communication man-
agement. The results demonstrate that the
assessments of the children’s local teacher and
that of their implant centre teacher are very simi-
lar; there are few discrepancies in score, and
these are small. It is useful to note that while the
implant centre teachers of the deaf had access to
 Categories of auditory perfownance 9

Table 1. Biographical details of the children used in the repeatability study (n = 23)

Sex Age of Age of Duration Educational Communication

onset (mths) implant (mths) of deafness setting management

M 17 43 27 P TC
F 0 52 52 U TC
M 0 53 53 S OIA
F 0 88 88 S OIA
M 0 51 51 S TC
F 0 41 41 M TC
M 2 37 34 P OIA
M 0 81 81 S TC
M 0 62 62 U TC
F 0 62 62 S TC
Downloaded by [RMIT University] at 18:53 16 August 2015

M 0 74 74 S TC
F 0 186 186 U TC
F 12 38 26 P OIA
F 0 52 52 M TC
M 0 44 44 P TC
M 15 25 10 P TC
M 4 21 17 P TC
F 0 48 48 P TC
M 0 64 64 U TC
M 33 78 45 U OIA
F 4 21 18 P TC
F 17 84 67 S TC
M 0 69 69 U OIA

Mean: 4.5 59.7 55.3

Educational placement: P = Pre-school, M = Mainstream, U = UnitResource base in mainstream school,
S = School for deaf or hearing impaired
Communication management: O/A = OraYAural; TC = Total Communication
(We recognize this term is contentious, but it is used here to describe the use of sign with spoken
language.)

the formal measures of benefit such as the IOWA
Closed-Set Sentence Test, the local teachers did
not, and completed the rating from their own
observations. In summary, the ratings were
repeatable, whether completed from clinic-based
tests or from observations in everyday life.
The repeatability of CAP has also been
demonstrated across a range of intervals, from
before implantation to five years after implanta-
tion. This is important, given the need for out-
come measures that are easily understood, and
can also be used for large groups of children and
for children over a long period of time. While
CAP does not replace formal clinic-based mea-
sures, it provides a global measure that can be
readily understood by non-professionals,
thereby demonstrating the benefits seen from
implantation and the time-scale over which they
are achieved. This information is important to
parents, professionals and health-care pur-
chasers to help them make informed decisions. It
also gives insight into the long-term commitment
that will be necessary.
Conclusion
With increasing demands being made on health-
care and educational resources, those of us work-
ing with deaf children must be able to present
evidence of their achievements through readily
understood measures. Categories of Auditory
Performance provides one means of rating and
presenting information that is quick and easy to
 10 Sue Archbold, Mark E. Lutman and Thomas Nikolopoulos

Table 2. The ratings of the local teachers of the deaf (LTOD) and the implant centre teachers of
the deaf (ICTOD) for the same children at the same intervals using Categories of Auditory
Performance; differences are given in bold type

Assessment Pre Pre Post Post 3mth 3mth 3mth 3mth 6mth
interval
LTOD 0 0 1 1 2 4 4 4 4
ICTOD 0 0 1 2 2 4 4 4

Assessment 6mth 6mth 6mth 6mth 6mth 12mth 12mth 12mth

interval
LTOD 4 4 4 4 3 4 4 4
ICTOD 4 4 4 4 4 4 4 4

Assessment 12mth 12mth 24mth 36mth 48mth 60mth

Downloaded by [RMIT University] at 18:53 16 August 2015

interval
LTOD 5 6 5 5 5 6
ICTOD 5 5 5 5 5 6

use, and illustrates progress in a group of children
with a wide range of achievement over a long
period of time. The present study has demon-
strated its repeatability, with an extremely high
degree of agreement between users, attesting to
the robustness of the measure. CAP is already
used for many children with cochlear implants,
and this study has illustrated its ability to be used
by professionals who are working with the chil-
dren in home and local schools, as well as those
who are more clinic-based. It would be interest-
ing in future to provide parents with the scale, and
notes for completion, and compare their results
with those of the professionals working with their
children.
Work is currently in progress to compare the
results on C A P with those of formal perfor-
mance-based measures, such as the IOWA
Closed-Set Sentence Test, and also to look at
those factors known to influence progress follow-
ing implantation, such as length of deafness.
Meanwhile, it continues to be a useful measure of
outcome from cochlear implantation, illustrating
benefit to parents and purchasers in a readily
understood form.
References
Archbold S, Lutman M, Marshall D. Categories
of Auditory Performance. In: Clark M, Cowan
RSC eds. International Cochlear Implant,
Speech and Hearing Symposium, Melbourne.
Ann of Otol, Rhino1 Laryngoll995; 104 Suppl
166: 312-4.
Boothroyd A. The assessment of speech percep-
tion capacity in profoundly deaf children. Am J
Otol1991; SUPPI12:67-72.
Cunningham JK. Parents’ evaluation of the
effects of 3M/House cochlear implants in chil-
dren. Ear Hear 1990; l l: 375-81.
Osberger MJ, Miyamoto RT, Zimmerman-
Phillips S, Kemink JL, Stroer BS, Firzst JB,
Novak MA. Independent evaluation of the
speech perception abilities of children with the
Nucleus 22 channel cochlear implant system.
Ear Hear 1991; 12:151-64.
Selmi A. Monitoring and evaluating the educa-
tional effects of the cochlear implant. Ear Hear
1985; 6(3): 52s-59s.
Staller SJ, Beiter AL, Brimacombe JA, Mecklen-
burg DJ, Arndt P. Paediatric performance with
the Nucleus 22 channel cochlear implant sys-
tem. Am J Otoll991; 12: 126-36.
Summerfield AQ, Marshall DH. Cochlear
implantation in the UK 1990-1994. London:
HMSO, 1995.
Vidas S, Hassan R, Parnes LS. Real-tife perfor-
mance considerations of four paediatric multi-
channel cochlear implant recipients. J Otol
1992;21: 387-93.
Appendix: Categoriesof auditory performance
Working definitions
0. Displays no awareness of environmental
sounds. Wearing appropriate aids with good
earmoulds, the child does not alert sponta-
 Categories of auditory performance 11

neously to any environmental sounds. Nor has
the child been reported to alert to environ-
mental sounds.
1. Awareness of environmental sounds. The
child has been observed to make a sponta-
neous reaction to about half a dozen different
environmental sounds (at home, at school, in
the clinic or outdoors). The reaction need not
indicate that the child recognizes the sound,
only that he or she has detected it.
2. Responds to speech sounds. The child will
obey a simple command, such as the instruc-
tion ‘Go’ to perform an action such as rolling a
ball at a skittle, when delivered in a normal 6. Understands conversation without lipreading
conversational sound level at a distance of 1-2
Downloaded by [RMIT University] at 18:53 16 August 2015
leel, lool, laal) presented with live voice at a
conversational level without lipreading.
5. Understands common phrases without lipread-
ing. The child is able to identify common
phrases in a familiar constraining context. For
example, the child can perform the IOWA
Closed-Set Sentence Test at Level A; or the
child can identify simple, familiar questions in
a known context (e.g. ‘What’s your name?’,
‘Where’s mummy?’, ‘How old are you?’); or
the child can identify a picture correctly from a
limited set when the picture is described ver-
bally.
with a familiar talker. The child can carry out a

feet. simple unscripted conversation with a familiar

3. Recognizes environmental sounds. The child
has been observed to identify a range of about
half a dozen environmental sounds consis-
tently in everyday life (e.g. doorbell, tele-
phone, parent’s voice, traffic etc). Observers 7. Can use the telephone with a familiar talker.
are confident that the child is monitoring his
or her environment via audition.
4. Discriminates at least two speech sounds. The
child can discriminate consistently any combi-
nation of two of Ling’s five sounds (lssl, Ishl,
talker (e.g. a parent or teacher) without
lipreading in a quiet setting. The child must be
able to respond correctly to simple questions
without interaction breaking down.
The child can sustain a simple unscripted con-
versation on the telephone with a familiar
talker. The child must be able to respond cor-
rectly to simple questions without interaction
breaking down.