This article was downloaded by: [Monash University Library]

On: 05 February 2015, At: 02:21

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Aphasiology
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/paph20

Preliminary psychometric
evaluation of an acute aphasia
screening protocol
a b c
Michael A. Crary , Nancy J. Haak & Anne E. Malinsky
a
Department of Communicative Disorders , University of
Florida Health Science Center , Gainesville, FL
b
Department of Speech-Language Pathology , Upreach
Rehabilitation Hospital , Gainesville, FL
c
Blake Memorial Rehabilitation Center , Bradenton, FL
Published online: 29 May 2007.

To cite this article: Michael A. Crary , Nancy J. Haak & Anne E. Malinsky (1989) Preliminary
psychometric evaluation of an acute aphasia screening protocol, Aphasiology, 3:7, 611-618,
DOI: 10.1080/02687038908249027

To link to this article: http://dx.doi.org/10.1080/02687038908249027

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information
(the “Content”) contained in the publications on our platform. However, Taylor
& Francis, our agents, and our licensors make no representations or warranties
whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and
views of the authors, and are not the views of or endorsed by Taylor & Francis. The
accuracy of the Content should not be relied upon and should be independently
verified with primary sources of information. Taylor and Francis shall not be liable
for any losses, actions, claims, proceedings, demands, costs, expenses, damages,
and other liabilities whatsoever or howsoever caused arising directly or indirectly in
connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
Terms & Conditions of access and use can be found at http://www.tandfonline.com/
page/terms-and-conditions
 1989, VOL. 3,
APHASIOLOGY, NO. 7, 611-618

Preliminary psychometric evaluation of an

acute aphasia screening protocol

M I C H A E L A. C R A R Y ' , N A N C Y J . HAAK2 and

ANNE E. MALINSKY3
'Department of Communicative Disorders, University of Florida Health
Science Center, Gainesville, FL
'Department of Speech-Language Pathology, Upreach Rehabilitation Hospital,
Gainesville, FL
'Blake Memorial Rehabilitation Center, Bradenton, FL
Downloaded by [Monash University Library] at 02:21 05 February 2015

(Received 9 March 1988; revision received 15 October 1988; accepted 16 December

1988)

Abstract
Results of preliminary psychometric evaluations of the Acute Aphasia Screening
Protocol are presented. This protocol demonstrates good concurrent validity with
the Western Aphasia Battery and good content and construct validity when
compared to existing aphasia batteries. Test-retest reliability is high, indicating
temporal stability of the procedure. Preliminary interjudge reliability is high
within and across patients. These results indicate that the AASP may be a useful
clinical tool for aphasia assessment when used for its specified purposes. The need
for additional psychometric evaluation is described.

Introduction
Aphasia screening tests may be useful for various clinical situations. Frequently, acute
patients are unable to endure a detailed aphasia examination; yet information
concerning the extent of acute aphasia may be helpful in delineating a patient's abilities
at that point in time. Furthermore, a screening procedure that estimated the severity
of aphasia and provided a brief profile of communicative strengths and deficits might
be useful in monitoring a patient's progress until such time as a more detailed
evaluation was appropriate. T h e Acute Aphasia Screening Protocol (AASP) was
developed for two primary purposes: (1) to provide an objective, systematic
assessment of acute aphasic patients who were not able to tolerate a more detailed,
lengthier assessment, and (2) as a short objective assessment of patients' early
progress. In developing this procedure attention was given to several features: (1) test
content similar to longer aphasia batteries representing a range of receptive and
expressive spoken language tasks; (2) a simple scoring system; (3) a total score
representing the severity of aphasic impairment; (4)the ability to profile a patient's
performance across general aspects of communicative abilities; and (5) a portable
procedure that could be administered in a variety ofenvironments without reliance on
excessive materials. The AASP is a numerically indexed scale that permits objective

Address for correspondence: M. A. Crary, Ph.D., Box J-174,University of Florida Health

Science Center, Gainesville, FL 32610, USA.
0268-7038/89 $3.00 @ 1989 Taylor 81 Francis Ltd.
 612 M . A . Crary, N.J . Haak and A . E. Malinsky

scoring and reporting of patient performance. The procedure requires approximately

ten minutes for administration and scoring and utilizes only those objects that are
readily available in many environments as test stimuli.
The AASP evaluates four general areas of communicative performance: (1)
AttentiodOrientation to Communication; (2) Auditory Comprehension; (3) Ex-
pressive Abilities; and (4) Conversational Style. Each area requires multiple
judgments by the examiner or responses by the patient to a range of tasks. Tasks were
chosen to represent a variety of communication functions traditionally examined in
aphasia. Table 1 depicts the AASP scoring system.

Table 1. Acute aphasia screening protocol scoring system.

Test area Score
Downloaded by [Monash University Library] at 02:21 05 February 2015

Attentionlorientation to communication -15

Auditory comprehension -115
object recognitionlselection (5)
yeslno questions (5)
commands (5)
Expressive abilities -120
Object naming (5)
repetition (5)
word fluency (4)
automatic (serial) speech (6)
Conversational style -110
interacts verbally (1)
effort (3)
output (3)
information (3)
Total -150

Cumulative score (Total X2) -%

The first area of evaluation (ATT/ORNT) is comprised of a checklist of a range of

communicative interactions from ‘awake and alert’ to ‘speaks, gestures, or otherwise
conveys appropriate communicative content’. A binary scoring system (1 or 0) is used
for a total of five points.
Assessment of Auditory Comprehension includes three subtests: (1) object
recognition/selection; (2) yes/no questions; and (3) following commands. Each of
these contains five test items for a total of 15 points.
The items used in the object recognition/selection task must be those that will be
found repeatedly in the patient’s environment. We have used items such as a pillow,
window, sink, ceiling and TV. We have found it best to use general items that would
be present in multiple environments in the event that the patient is tested in different
rooms at subsequent meetings. It is imperative that the same five items be used for
each patient at each test administration.
The five yes/no questions include factual and abstract information requiring
confirmation/denial responses. The patient is asked to confirm hidher name and the
fact that they are in a hospital. Subsequently, they are asked if they ‘live here’. If the
patient responds appropriately or with a question regarding the nature of the question
 Psychometric evaluation of screening protocol 613

(e.g. ‘Do you mean in the hospital?’), a correct response is indicated. Finally, the
patient is asked to confirm/deny their own gender and that of the examiner.
Three single stage commands and one two-stage command are used to test the
patient’s comprehension and nonverbal response to spoken requests. The three single
stage commands involve manipulation of the patient’s body (close eyes, raise hand,
make fist). The two-stage command ‘point to the window, then to the door’) permits
partial credit for correct response to either half of the stimulus.
The Expressive Abilities area incorporates four subtests evaluating the patient’s
ability to name objects, repeat words and phrases, generate animal names, and
produce serial speech in the form of counting and days of the week. The total possible
score for this subtest is 20 points.
The object naming task uses five different items (key, watch, eye, pen and hand).
Other items may be substituted but must be used each time the protocol is
Downloaded by [Monash University Library] at 02:21 05 February 2015

administered to a given patient. The repetition task incorporates four stimuli: one
single-syllable word, one tri-syllable word, one three-word phrase and one five-word
phrase.
The Conversational Style score is based upon the examiner’s judgement of four
aspects of verbal interaction including effort ofproduction, amount of spoken output,
and amount of conveyed information. Judgements are made from a three item list of
possibilities in each area reflecting minimal, normal or excessive attributes for each
aspect and derived from a brief conversation with the patient.
Scores from each subtest are added and a percentage of the 50 possible points is
computed. This percentage is referred to as the Cumulative Score.
Clinical assessment protocols must be subjected to basic psychometric evaluations
in order to provide indices of their measurement potential. Spreen and Risser (1981)
describe several psychometric characteristics that must be detailed in aphasia test
development. These same issues are reviewed extensively in the American
Psychological Association’s Standards for Educational and Psychological Tests
(1974). This manual details multiple aspects of validity and reliability estimates and
their relative importance for various testing purposes. Following the psychometric
guidelines offered by these two sources, the purpose of this paper is to detail the results
of preliminary psychometric evaluations of validity and reliability estimates of the
Acute Aphasia Screening Protocol.

Validity Estimates
Face validity is generally considered to be the weakest form ofvalidity. It concerns the
relationship between test content and purpose. Shewan and Kertesz (1980) discuss
face validity for the Western Aphasia Battery (WAB) by comparing its similarity with
other aphasia tests and the relevance of the various subtests to the examination of
aphasia and the classification of aphasic subtypes. Similarly, two relevant purposes of
the AASP are: (1) estimating the severity of acute aphasic impairment, and ( 2 )
profiling the patient’s abilities across general aspects of communicative abilities. The
cumulative score of the AASP is intended as the index of aphasia severity. This score is
derived from patient performances on multiple language tasks from both the
comprehension and expression realms. Many of the tasks in the AASP are similar to
those found in lengthier aphasia batteries. These include three separate auditory
comprehension tasks, two naming tasks, a repetition task, and a serial speech task. In
 614 M . A. Cvary, N.J . Haak and A . E. Malinsky

addition, the AASP contains scales requiring clinical judgement concerning basic
functional communication abilities of the patient. For example, is the patient
attempting to communicate, is this communication meaningful to the situation, etc.
In this respect, the content of the AASP is related directly to its stated purposes, thus
giving the face appearance of validity.
Content validity refers to the relationship between the test items and the test
area-in this case aphasia, or more specifically, communication abilities of aphasic
patients. Stated simply ‘content validity refers to the adequacy of sampling from the
domain of behaviors to be measured’ (Spreen and Risser 1981; p. 75). The AASP
would not seem as rigorous as other aphasia batteries in this respect owing to its
brevity. For example, reading and writing abilities are not evaluated in the present
form of the AASP. Still, the range of spoken language behaviors sampled by the
AASP is similar to the WAB (Kertesz 1982), the BDAE (Goodglass and Kaplan 1983)
Downloaded by [Monash University Library] at 02:21 05 February 2015

and other aphasia batteries. Given the obvious differences between the AASP and
lengthier aphasia batteries, and the stated purposes ofthe AASP, it demonstrates good
content validity when used for its specified purposes.
Two additional types of validity have been estimated statistically for the AASP.
Concurrent validity refers to the ability to predict a patients performance on a
criterion test from performance on the AASP. Construct validity reflects how well a
test measures the theoretical construct that it is intended to measure. Both forms may
be approached through comparison with a ‘standard’. In this case, a standard is an
established aphasia assessment procedure that has received significant psychometric
evaluation.
Concurrent and construct validity characteristics of the AASP were evaluated by
comparing the results of the screener to those from the WAB (Kertesz 1982)
administered to the same patients. The WAB was chosen as a criterion test as it has
received extensive psychometric evaluation (Shewan and Kertesz 1980), has been used
in studies of acute and recovering aphasia (Kertesz and McCabe 1977, Lomas and
Kertesz 1978) and it has a simple numerical scoring procedure lending itself to
statistical evaluation.
For purposes of comparison 48 acute aphasic patients were evaluated with both
assessment procedures. Table 2 depicts the patient characteristics of age, months post
onset (mpo), and the aphasia quotient (AQ) and aphasia type results from the WAB.
All 48 patients demonstrated aphasia secondary to single episode, left-hemisphere
thrombo-embolic strokes. These patients were selected from a larger population of

Table 2. AASP-WAB comparison: patient characteristics (N=48).

Variable Mean Range
Age (years) 69.42 21-87
Months post-onset 0.56 0.10-1 .o
Aphasia quotient (AQ) 46.72 0.50-92.6
Aphasia type Number of patients
global 12
Wernicke 10
Broca’s 4
Conduction 1
Isolation syndrome 1
Transcortical motor 2
Transcortical sensory 3
Anomic 15
 Psychometric evaluation of screening protocol 615

patients who were tested within the hospital setting. Presence of aphasia was
determined by an aphasia quotient on the WAB less than 93-8. The mean aphasia
quotient for the group was 46.72 with a range from 0.50 to 92.6. The patients ranged in
age from 21 to 87 years with a mean of69.42 years. Patients were tested within one of
two hospital settings, an acute care facility or a rehabilitation hospital. O n the average
patients completed both protocols at approximately two weeks post-onset (0.56 mpo)
with a range of 3-30 days post-onset (0.10 to 1.0 mpo). For the purposes of this study
‘acute aphasia’ was defined as aphasia present within the first month post-onset. Type
of aphasia was determined by the WAB classification system. All types of aphasia
identified by the WAB were included in our sample. The majority of patients
presented anomic (31%), global (25%) or Wernicke (21%) types of aphasia.
Both procedures were administered to each patient within one week of the other.
The screening protocol was always administered first. Table 3 presents the results of
Downloaded by [Monash University Library] at 02:21 05 February 2015

Table 3. AASP-WAB comparison: correlation analyses.

AASP subtests WAB subtest r*

Cumulative score Aphasia quotient 0.93

Auditory comprehension Auditory comprehension 0.85
Expressive abilities Spontaneous speech 0.84
Information Content 0.89
Fluency 0.69
Conversational style Spontaneous speech 0.72
Information Content 0.61
Fluency 0.75
Repetition Repetition 0.92
Object naming/word fluency Naming ~
0.95
*All correlations significant at the 0.0001 level

correlation analyses between comparable aspects of both protocols. WAB scores used
in the A Q calculation were compared to the four ‘area’ scores from the AASP. The
WAB A Q was compared to the AASP cumulative score. Note that the WAB has no
subtest which is directly comparable to the AttentiodOrientation area of the AASP.
In those instances where no direct comparison was possible, a single subtest or
combination of subtests from the screening protocol was compared to performance
on the WAB. For example, the repetition subtest from the AASP was compared to the
repetition score from the WAB. A combination of object naming and word fluency
from the AASP was compared to the naming score from the WAB. Both the
Expressive Abilities score and the Conversational Style score from the AASP were
compared to the Spontaneous Speech score from the WAB. All correlations were
positive and highly significant. These data indicate a strong relationship between
corresponding areas of the AASP and the WAB. Collectively, the similarity between
test items ofthe AASP and other aphasia tests and the strong, significant relationships
between the AASP and the WAB suggest good validity characteristics for this
screening procedure.

Reliability Estimates
T o date, two types of reliability estimates have been completed for the AASP.
Preliminary interjudge reliability, or the extent to which different raters agree when
scoring the same patients, was evaluated by having ten judges rate three video-taped
 616 M . A . Crary, N.J . Haak and A . E . Malinsky

administrations of the screening tool. The patients’ aphasia spanned a large range of
severity as indicated by the aphasia quotient (13.6, 44.0, 67.3). The judges were
graduate students in a seminar on the topic of aphasia. No instructions nor training
were provided to the judges prior to scoring the video tapes. This ‘blind’ scoring was
employed in an attempt to identify potential scoring confusions toward the goal of
developing improved administration instructions.
Table 4 presents the mean and range of interjudge correlations collapsed across all
scores obtained from the AASP for each of the three patients. The judges were highly
similar in their scoring of individual patients. These data indicate good interjudge
reliability for individual patient performance on the AASP.

Table 4. Interjudge reliability within patients.

Patient Mean r Range
Downloaded by [Monash University Library] at 02:21 05 February 2015

A 0.97 0.91-1.0
B 0.94 0.75-1.0
C 0.94 0.8 1-1 .O

Table 5 presents the mean and range of interjudge correlations for the cumulative
score and the four test areas of the screener collapsed across the three patients. O n the
AttentiodOrientation to Communication scale eight of the ten judges scored all three
patients exactly the same with the maximum possible score for this scale. Two of the
judges differed from the other eight by one item. However, these two judges were in
perfect agreement with each other. Given the limited variability across patients and
judges for this scale the mean correlation of 1.00 should not be considered precise.
Rather it is intended as a reflection of the high agreement among the judges. The
lowest correlations were found in the two expressive areas, Expressive Abilities and
Conversational Style. However, even in these areas, the mean correlations would be
considered moderately high. Nine of the ten judges produced very high correlations
for the Expressive Abilities task. The remaining judge demonstrated lower
correlations with all other judges. In checking this judge’s score sheets, we noticed
that the discrepancies primarily were the result of uncertainty in scoring misarticu-
lated responses. There was considerably more disagreement among judges in the
Conversational Style scores. We believe this to be a reflection ofthe inherent difficulty
in making judgements regarding conversational characteristics of aphasic patients.
The lower correlations on both of these expressive test areas demonstrate the need for
detailed instructions for scoring the respective tasks or scales. The high correlations in
the Auditory Comprehension area suggest that scoring of these tasks is relatively
straightforward. These interjudge reliability data are to be used to strengthen the
scoring procedures for the AASP.
Test-retest reliability was evaluated to estimate the temporal stability of the
AASP. Ten patients who were within one month post onset were given two

Table 5. Interjudge reliability within AASP test area.

Test area Mean r Range
Cumulative score 0.95 0.75-1.0
Attentionlorientation 1.00 -
Auditory comprehension 1.oo 0.99-1.0
Expressive abilities 0.87 0.45-1.0
Conversational stvle 0.70 0.0-1.0
 Psychometric evaluation of screening protocol 617

administrations of the screening protocol. Both AASP administrations were

completed within one week with a mean interval of five days between tests. Table 6
presents the results of test-retest correlations for the Cumulative Score and the four

Table 6. Test-retest reliability.

Test area r*

Cumulative score 0.98

Attentiodorientation 1.00
Auditory comprehension 0.92
Expressive abilities 0.95
Conversational style 0.94
*All correlations significant at the 0.0001 level.
Downloaded by [Monash University Library] at 02:21 05 February 2015

evaluation areas of the screening protocol. All correlations were positive and highly
significant. Though scores from the second administration were slightly higher
suggesting recovery of communicative ability, t-test comparisons demonstrated no
significant differences between the two administrations on any of the five measures.
Table 7 presents the mean scores for the four evaluation areas and the cumulative score
from the AASP taken from the two administrations.

Table 7. Test-retest reliability: mean AASP scores.

Test area Test 1 Test 2
Cumulative score 55.6 60.2
Attentiodorientation 90.0 90.0
Comprehension 74.5 75.2
Expressive abilities 35.0 44.0
Conversational style 50.0 54.0

Summary
The preliminary psychometric data on the Acute Aphasia Screening Protocol are
encouraging. This short and simple clinical assessment appears to demonstrate good
content and construct validity, and good temporal reliability. When administered
according to specific scoring instructions, we expect that future evaluation will
demonstrate very high interjudge and intrajudge reliability. Given these characteris-
tics, the AASP may be a useful clinical tool for the condensed evaluation of acute
aphasia when used as intended. In addition, it seems to have potential for estimating
the severity of aphasia in that the cumulative score correlates highly with the WAB
AQ.
Though encouraged by these preliminary findings, we realize the information
gaps that remain with reference to this protocol. In this respect, psychometric and
clinical evaluation studies are continuing. Future research will compare the AASP to
aphasia tests originating from different philosophic perspectives from the WAB
(including other screening protocols), examine construct validity via factor analysis
techniques, reevaluate interjudge reliability in judges trained on specific scoring
conventions with patients presenting a wider of abilities, and estimate intrajudge
 618 M . A . Crary, N.].Haak and A . E. Malinsky

reliability. Through the processes of psychometric evaluation and modification we

feel that the AASP will become a valuable tool for the aphasiologist.

References
GOODGLASS, H., and KAPLAN,E. (1983) Boston Diagnostic Aphasia Examination. Philadelphia: Lea and
Febiger.
LOMAS,J., and KERTESZ, A. (1978) Patterns of spontaneous recovery in aphasic groups: a study of adult
stroke patients. Brain and Language, 5 , 388-401.
KERTESZ, A. (1982) Western Aphasia Battery. New York: Grune and Stratton.
KERTESZ, A. and MCCABE,P. (1977) Recovery patterns and prognosis in aphasia. Brain, 100, 1-18.
SHEWAN, C. and KERTESZ, A. (1980) Reliability and validity characteristics of the Western Aphasia Battery
( W A B ) . Journal of Speech and Hearing Disorders, 45, 308-324.
SPREEN, 0. and RISSER,A. (1981) Assessment of aphasia. In M. T. Sarno (ed.), Acquired Aphasia, New
Downloaded by [Monash University Library] at 02:21 05 February 2015

York: Academic Press.

Standardsfor Educational and Psychological Tests-Revised. (1974) Washington, DC: American Psychological
Association, Inc.