This article was downloaded by:[University of Florida]

On: 8 May 2008

Access Details: [subscription number 768222542]
Publisher: Psychology Press
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Clinical Neuropsychologist

Publication details, including instructions for authors and subscription information:
http://www.informaworld.com/smpp/title~content=t713721659

Utilization Rates of Ecologically Oriented Instruments

Among Clinical Neuropsychologists

Laura A. Rabin a; Leslie A. Burton b; William B. Barr c

a
Department of Psychology, Brooklyn College/City University of New York,
Brooklyn, NY, USA
b
Department of Psychology, Fordham University, Bronx, NY, USA
c
NYU Comprehensive Epilepsy Center, New York University School of Medicine,
New York, NY, USA
First Published on: 08 December 2006
To cite this Article: Rabin, Laura A., Burton, Leslie A. and Barr, William B. (2006)
'Utilization Rates of Ecologically Oriented Instruments Among Clinical
Neuropsychologists', The Clinical Neuropsychologist, 21:5, 727 743
To link to this article: DOI: 10.1080/13854040600888776
URL: http://dx.doi.org/10.1080/13854040600888776

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,
re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly
forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be
complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be
independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or
arising out of the use of this material.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

The Clinical Neuropsychologist, 21: 727743, 2007

http://www.psypress.com/tcn
ISSN: 1385-4046 print=1744-4144 online
DOI: 10.1080/13854040600888776

CE UTILIZATION RATES OF ECOLOGICALLY

ORIENTED INSTRUMENTS AMONG CLINICAL
NEUROPSYCHOLOGISTS
Laura A. Rabin1, Leslie A. Burton2, and William B. Barr3
1

Brooklyn College=City University of New York, Department of Psychology,

Brooklyn, 2Fordham University, Department of Psychology, Bronx, and
3
NYU Comprehensive Epilepsy Center, New York University School of
Medicine, New York, NY, USA
The ecological validity of neuropsychological instruments has become an important topic in
recent decades, as neuropsychologists are asked to address real-world outcomes with
increasing frequency. Although novel instruments that tap skills required for everyday functioning have been developed, it is unclear whether these instruments are migrating from
research laboratories into the applied settings of clinical neuropsychologists. The current
study surveyed assessment practices of neuropsychologists with regard to their utilization
of instruments designed with ecological concerns in mind. Respondents included 747 North
American, doctorate-level psychologists (40% usable response rate) affiliated with Division
40 of the American Psychological Association, National Academy of Neuropsychology, or
the International Neuropsychological Society. Results indicated that approximately onethird of respondents reported use of ecologically oriented instruments (EOIs), and these
instruments were generally utilized with much less frequency than traditional measures.
Additionally, certain practice demographics affected usage rates of EOIs. Study findings
are interpreted in the context of a growing body of literature that calls attention to the
importance of developing and utilizing instruments that are able to handle the complex,
real-world issues increasingly addressed during the neuropsychological assessment process.
Keywords: Clinical neuropsychology; Ecological validity; Neuropsychological assessment; Survey

INTRODUCTION
Concern about the ecological validity of neuropsychological instruments has
grown in recent years, as the field of clinical neuropsychology has moved away from
purely descriptive, diagnostic endeavors toward prescriptive, treatment-oriented
assessments that bridge the gap between diagnosis and rehabilitation (Boyd, 1988;
Chaytor & Schmitter-Edgecombe, 2003). Clinicians increasingly are asked assess
patients real-world capacities including rehabilitation potential, optimal living
Address correspondence to: L. A. Rabin, Ph.D., Department of Psychology, Brooklyn College,
2900 Bedford Avenue, Brooklyn, NY 11201, USA. E-mail: lrabin@brooklyn.cuny.edu
Accepted for publication: June 21, 2006. First published online: December 8, 2006.
# 2006 Psychology Press, an imprint of the Taylor & Francis group, an Informa business

Downloaded By: [University of Florida] At: 15:21 8 May 2008

728

LAURA A. RABIN ET AL.

arrangements, functional independence, and adaptation to everyday life (Cubic &

Gouvier, 1996; Heinrichs, 1990; Leonberger, 1989). In addition, neuropsychologists
help predict the impact of behavioral deficits on patients functioning in specific
environments (Johnstone et al., 1995; Long, 1996). For example, recent surveys have
indicated that prediction of work behavior is among the top reasons for conducting a
neuropsychological evaluation (Guilmette, Faust, Hart, & Arkes, 1990; Rabin, Barr,
& Burton, 2005). A consequence of this new undertaking is that instruments initially
designed to detect neuropathology are now used to address various social and
vocational outcomes. It is unclear whether this is an appropriate and valid use of
neuropsychological tools (Cripe, 1996; Guilmette & Kastner, 1996).
Egon Brunswik (1955, 1956) introduced the concept of ecological validity into
the psychological literature, and he used the term to describe conditions under which
one could generalize from results of controlled, systematic experiments to naturally
occurring events in the real world. Brewer (2000) described ecological validity as one
of the three essential components of external validity, alongside robustness and relevance. Neuropsychologists typically consider issues of ecological validity when evaluating strengths and weaknesses of their assessment techniques. An important aspect
of ecological validity is verisimilitude, or degree of similarity between data collection
methods and skills required in the open environment.1 For example, does an instrument created to measure memory actually contain items that resemble everyday
tasks requiring memory processes? Verisimilitude is essential during the design phase
of an instrument, when an investigator should explore the relationship between a
tests task demands and a predicted behaviors situational demands. For example,
with a test of visuospatial skills, such as the Line Orientation Test (Benton, Hannay,
& Varney, 1975), one must consider the extent to which line orientation discriminations are required for a specific task such as driving an automobile. Verisimilitude
investigations also explore the match between the way skills are tapped in an assessment task and the requirements of relevant real-world situations. For example, a
memory test that does not permit rehearsal may not accurately predict memory
behaviors in the natural environment where rehearsal is common (Franzen &
Wilhelm, 1996).
Verisimilitude concerns have been addressed through the development of new
ecologically oriented instruments (EOIs), meant to simulate everyday behaviors
and assess the types of difficulties patients encounter in their daily lives. This process,
however, is complicated. Many clinical neuropsychologists are comfortable with
their current tools and unlikely to develop or seek out innovative instruments
(Piotrowski & Keller, 1989; Sweeney, Clarkin, & Fitzgibbon, 1987; Williams, 1996).
1

The construct verisimilitude is related (but not identical) to other types of validity including face
and content. Face validity is concerned with how a measure or procedure appears on the surface. Does a
test look valid or seem like a reasonable way to gain the desired information? Content validity is based
on the extent to which a measurement reflects the specific intended domain of content. Content validity is
related to face validity, but involves more rigorous statistical tests and established theories for support
than face validity, which only requires an intuitive judgment. While both face and content validity overlap
with ecological validity, the latter is used specifically to refer to the topographical similarity or degree of
relation between the method of data collection and skills and behaviors that occur in natural settings.
Instruments designed with ecological goals in mind tend to be face valid but may or may not possess content validity (Anastasi & Urbina, 1997; Kline, 1999).

Downloaded By: [University of Florida] At: 15:21 8 May 2008

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

729

Second, it is both costly and difficult to construct an instrument that assesses thefull complexity of real-world behaviors (Williams, 1996). Third, even if an instrument appears similar to everyday tasks, there still may be aspects of the testing
situation that limit its applicability to the real world (Chaytor & Schmitter-Edgecombe,
2003). Lastly, everyday task demands vary considerably from one patient to the next.
For example, financial management poses very different problems for patients on social
security than for those with complex investment portfolios. Thus, when designing and
utilizing a neuropsychological instrument, one must consider the relevance of, and
generalizability to, patients everyday lives (Heaton & Pendleton, 1981).
Despite the inherent difficulty in trying to capture the richness and diversity of
real-world experiences in laboratory tests, researchers have attempted to create ecologically relevant checklists, questionnaires, rating scales, and standardized tests and
test batteriese.g., Everyday Memory Inventory (EMI; West, 1985); Rivermead
Behavioral Memory Test (RBMT; Wilson, Cockburn, & Baddeley, 1985); Executive
Function Route-Finding Task (EFRT; Boyd & Sautter, 1993); Behavioral Assessment of the Dysexecutive Syndrome (BADS; Wilson, Alderman, Burgess, Emslie,
& Evans, 1996); Everyday Attention Questionnaire (EAQ; Martin, 1983); Test of
Everyday Attention (TEA; Robertson, Ward, Ridgeway, & Nimmo-Smith, 1994).
The Appendix provides a list of measures designed with ecological concerns in mind.
Available studies suggest that some EOIs can provide useful information about cognitive functioning relevant to patients, caregivers, and rehabilitation workers.2 These
instruments also can be used to monitor change following remediation of cognitive
dysfunction. It is important to note that while EOIs tend to be more face valid than
traditional tests, only a subset have been validated empirically in terms of their ability to capture the essence of everyday cognitive skills. Therefore a tests designation
as an EOI is not meant to imply that it possesses ecological validity or is necessarily superior to traditional measures.
In recent decades, investigators have used survey research to characterize the
rapid growth occurring within the field of neuropsychology. This research has
focused on a broad range of topics, including practitioner characteristics and activities, education and training, views on professional matters, and basic test usage (for
review see Rabin et al., 2005). An important question, not yet addressed in the literature, is whether instruments designed with ecological goals are migrating from the
laboratory to the applied settings of clinical neuropsychologists. The current study
surveyed utilization rates of both traditional and ecologically oriented neuropsychological instruments to determine the degree to which EOIs are actually being
incorporated into assessment batteries. From a theoretical perspective, we hypothesized that certain neuropsychologists would be more likely than others to utilize
EOIs. These include neuropsychologists who devote substantial clinical time to
rehabilitation and=or remediation efforts, work in settings where rehabilitation concerns are frequently addressed, or conduct neuropsychological evaluations to answer
rehabilitation-related referral questions. The value of survey research lies in its
2

See Alderman, Burgess, Knight, and Henman (2003); Boyd and Sautter (1993); Higginson, Arnett,
and Voss (2000); Martzke, Swan, and Varney (1991); Norris and Tate (2000); Robertson, Ward,
Ridgeway, and Nimmo-Smith (1996); Shiel (1990); West and Crook (1992); Wilson (1991, 1993); Wilson,
Cockburn, Baddeley, and Hiorns (1989); Youngjohn, Larrabee, and Crook (1992).

Downloaded By: [University of Florida] At: 15:21 8 May 2008

730

LAURA A. RABIN ET AL.

ability to answer important practice-related questions, reveal trends, and identify

issues critical to the continued success of the field of clinical neuropsychology.
Through its exploration of ecological issues, the current study aimed to promote
more effective applications of neuropsychological evaluations.
METHOD
Potential Participants
Potential participants were 2,004 randomly selected members of American
Psychological Association (APA) Division 40, National Academy of Neuropsychology (NAN), and=or the International Neuropsychological Society (INS). To select
the sample, the year 2000 membership directories of NAN, INS, and APA Division
40 were combined and membership overlap eliminated. Only members in possession
of doctoral degrees (i.e., Ph.D., Psy.D., or Ed.D.) and residing in the United States
(U.S.) or Canada were selected for inclusion. The resulting population of 5,840 members likely encompassed most practitioners of neuropsychology in the U.S. and
Canada. Random numbers were then assigned to each of the 5,840 cases, and
2,004 were drawn to form the sample, representing approximately one-third of the
population. A questionnaire packet consisting of an explanatory letter, questionnaire, stamped self-addressed envelope, and incentive was mailed to all potential
participants in January 2001. Respondents received informed consent through a
cover letter that accompanied the questionnaire. In order to enhance the response
rate, potential participants received a follow-up letter and second incentive 4 weeks
after the initial mailing; questionnaires returned within 4 months of the initial mailing were included in the analyses. A total of 51 surveys were returned undeliverable
due to relocation, lack of proper forwarding address, or both reasons. After
responses were recorded, respondents names were deleted from the mailing list
and not linked to their responses. In addition, any identifying information (e.g.,
comments written on letterhead) was separated from the questionnaire in order to
maintain anonymity of response.
Questionnaire and Procedure
In the first part of the questionnaire, respondents provided basic demographic
and practice-related information including: gender, age, degree type and field, board
certification status, percentage of time devoted to various professional activities,
primary work settings, referral sources and questions, patient characteristics, and
battery approach. They also reported their most frequently used assessment instruments. In the second part of the questionnaire, respondents read a vignette about a
traumatic brain injury (TBI) patient experiencing cognitive difficulties arising from a
sports-related accident. Respondents received one of three versions of the case, varying according to the patients occupation and level of education: (a) a doctor with
20 years of education, (b) a police officer with 16 years of education, and (c) a
school bus driver with 12 years of education.3 These occupations were selected for
3

Refer to Rabin, Burton, and Barr (in press) for findings related to the effect of patient occupation=
education variables on the choice of neuropsychological assessment instruments.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

731

two main reasons: (a) they encompass and require varying degrees of professional
skill, and (b) they demand high levels of competence and precision on the part of
those employed in each job. Therefore, patients who attempt to return to any of
these occupations before adequate recovery from their head injuries may endanger
the lives of many others. Based on the information presented in the case study,
respondents were asked to list the instruments they would use to evaluate the
patients memory, attention, and executive functioning. Respondents also listed
the instruments they would use to assess specific symptomatology in each of those
cognitive domains. Subsequently, they reported any additional instruments they
would use in predicting the patients ability to return to work.
RESULTS
Response Rate and Organizational Affiliations
A total of 879 surveys were returned, representing a 44% response rate. However, 132 surveys were deemed unusable due to such factors as retirement, death,
inactivity in the practice of clinical neuropsychology, or similarly being identified
with APA Division 40, NAN, or INS on the basis of intellectual interest alone. Thus,
the questionnaire was inappropriate for 15% of the sampled respondents who did
not offer neuropsychological services. Subtracting these 132 from the original
2,004 questionnaires yielded an adjusted response rate of 40%, reflecting the opinions and practices of 747 clinical neuropsychologists. Study participants belonged
to one of seven membership affiliation categories: members of APA Division 40
exclusively, members of NAN exclusively, members of INS exclusively, members
of Division 40 and NAN, members of Division 40 and INS, members of NAN
and INS, and members of Division 40, NAN, and INS. Overall, the percentage of
returned questionnaires per participant category was consistent with each categorys
initial percentage breakdown. No pattern of bias in the characteristics of respondents versus nonrespondents was discerned (v2 1.20, df 6, ns). Notably, potential
respondents who were members of all three organizations returned the greatest percentage of usable surveys (20.7%), followed by members of Division 40 exclusively
(18.5%) and Division 40 and NAN (17.5%).
Relevant Demographic and Practice-Related Information
The average age of the respondents was 48 (SD 8.7, range 2985 years), with
40% women and 60% men. With respect to type of doctoral degree, the percentage
holding Ph.D.s was highest (87%), followed by Psy.D.s (9%) and Ed.D.s (3%). A
total of 22% of respondents were board certified in neuropsychology by ABPP
(16%) and ABPN (6%), respectively. The majority of respondents received their
degrees in clinical psychology (62%), followed by counseling psychology (11%),
clinical neuropsychology (11%), school psychology (5%), and neuroscience (2%).
Respondents reported professionally practicing neuropsychology for an average of
13.2 years (SD 7.7, range 150 years). Most respondents (74%) performed 115
neuropsychological assessments per month, with an additional 16% performing
1630 assessments per month. A total of 96% of respondents indicated that they

Downloaded By: [University of Florida] At: 15:21 8 May 2008

732

LAURA A. RABIN ET AL.

were currently performing neuropsychological assessments, and the average respondent spent 42% of his or her professional time conducting neuropsychological
assessments followed by psychotherapy, research=teaching, and rehabilitation=
cognitive remediation.
Respondents reported serving a diversity of diagnostic groups and spending
the greatest percentage of professional time with adults, followed by young adults
and older adults. The top five referral sources were neurologists, psychiatrists, psychologists, attorneys, and general medical practitioners. The majority of respondents
(62%) identified private or group practice as their neuropsychological work setting,
with medical hospitals (34%) and rehabilitation facilities (17%) proving fairly common as well; 36% worked in more than one setting. The most frequently endorsed
assessment referral questions were determination of diagnosis, rehabilitation and=or
or treatment planning, and forensic determination. The majority of respondents
(68%) favored a flexible battery approach to test selection (i.e., variable but routine
groupings of tests for different types of patients), with an additional 20% favoring a
flexible approach (i.e., based on the needs of an individual case), and 11% favoring a
standardized battery. Overall, there was great consensus regarding the types of abilities evaluated during a neuropsychological assessment. For example, attention, construction, executive functions, intelligence, language, motor skills, verbal and
nonverbal memory, and visuospatial skills were endorsed as frequently assessed,
while auditory and tactile perception reportedly were assessed occasionally.
Respondents also reported the frequency with which they utilized certain types of
information in their assessments. The top five sources of information were medical=psychiatric history, neuropsychological test data, the referral source, psychosocial history, and measures of mood and affect. Information gained from work
records and projective personality testing were the least used.
General Test Usage and Utilization of Ecologically
Oriented Instruments
Table 1 presents a rank-ordered list of the top five instruments reported in the
areas of memory, attention, executive functioning, and return to work. For complete
test usage data derived from this study, see Rabin et al. (2005). Table 2 presents a
rank-ordered list of the top five ecologically oriented instruments (EOIs) reported
within each functional domain. One study goal was to determine usage rates of
EOIs, in response to the case study, as compared to more traditional instruments.
Respondents were designated as falling into one of two categories: Category A
respondents reported one or more instruments from the list of EOIs; Category B
respondents did not report using any instruments from the list of EOIs. Results
revealed that 35% of respondents (n 258) reported use of at least one EOI, while
65% respondents (n 479) did not report using any EOIs. Notably, of respondents
who reported use of at least one EOI, 64% used exactly one and 95% used three or
less. Table 3 presents a complete breakdown of the number of EOIs per respondent
in those reporting utilization of such instruments.
A related goal was to determine whether neuropsychologists actively engaged
in ecologically relevant professional activities were more likely than others to utilize
EOIs. These included neuropsychologists who (a) devoted a substantial portion of

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

733

Downloaded By: [University of Florida] At: 15:21 8 May 2008

Table 1 Top five assessment instruments by functional category

Rank
MEMORY
1
2
3
4
5
ATTENTION
1
2
3
4

% of
% of
responses respondents

Instrument

Wechsler Memory Scale

(WMS, Revised & III)
California Verbal Learning
Test (I & II)
Rey-Osterrieth Complex Figure Task
Boston Naming Test
Wechsler Adult Intelligence Scale
(WAIS, Revised & III)

488

12.1

70.8

374

9.3

54.3

312
231
173

7.7
5.7
4.3

45.3
33.5
25.1

Trail Making Test

WAIS=WMS Digit Span Subtest
Paced Auditory Serial Addition Task
Stroop Neuropsychological Screening
Test=Stroop Test
Continuous Performance Test (I & II)

421
318
288
208

11.2
8.4
7.6
5.5

63.7
48.1
43.6
31.5

204

5.4

30.9

494
268
262
260
154

15.1
8.2
8.0
7.9
4.7

75.5
41.0
40.1
39.8
23.5

213

13.2

39.9

85

5.3

15.9

72
67
64

4.5
4.1
4.0

13.5
12.5
12.0

5
EXECUTIVE FUNCTIONING
1
Wisconsin Card Sorting Test
2
Rey-Osterrieth Complex Figure Task
3
Halstead Category Test
4
Trail Making Test
5
Controlled Oral Word Association
Test=FAS
RETURN TO WORK
1
Minnesota Multiphasic Personality
Inventory (I & II)
2
Wechsler Adult Intelligence Scale
(WAIS, Revised & III)
3
Driving evaluationa
4
Beck Depression Inventory (I & II)
5
Clinical interviewa
a

Denotes nonstandardized or nonreferenced tests from the Appendix. A complete list of responses is
available on request.

their professional time to rehabilitation and=or cognitive remediation efforts, (b)

worked in rehabilitation settings, and (c) conducted neuropsychological assessments
for rehabilitative purposes. Respondents were assigned to one of two categories
based on the percentage of professional time spent in various professional activities.
Category A respondents devoted 15% or more of their professional time to rehabilitation and=or cognitive remediation, and Category B respondents devoted less than
15% of their professional time to those activities.4 As shown in Table 4, a chi-square
4

The figure 15% (or
6 hours per week) was selected in advance as the cut point for what constitutes a substantial portion of professional time in this analysis. The number was arrived at on the basis
of the authors professional judgment and after review of previous neuropsychological survey research,
which specified the distribution of clinical time in various professional activities, work status, and mean
time required to complete a neuropsychological evaluation (e.g., Putnam, DeLuca, & Anderson, 1994;
Sweet, Moberg, & Suchy, 2000).

734

LAURA A. RABIN ET AL.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

Table 2 Top five EOIs by functional category

Rank

% of
% of
n responses respondents

Instrument

MEMORY
1
2
3
4
5
ATTENTION
1
2
3
4
5

Rivermead Behavioral Memory Test

Autobiographical Memory Interview
Contextual Memory Testa
Brief Test of Attention
Prospective Memory Test

44
12
11
5
3

1.1
0.3
0.3
0.1
0.1

6.4
1.7
1.6
0.7
0.4

Test of Everyday Attention

Behavioral Inattention Test
Observe patients compensatory strategiesa
Clinical interview with functional questionsa
Review work record and past
job responsibilitiesa

14
2
2
1
1

0.4
0.1
0.1
< 0.1
< 0.1

2.1
0.3
0.3
0.2
0.2

28
13

0.9
0.4

4.3
2.0

0.2

0.8

4
3

0.1
0.1

0.6
0.5

72
43
37
15

4.5
2.7
2.3
0.9

13.5
8.1
6.9
2.8

10

0.6

1.9

EXECUTIVE FUNCTIONING
1
Tinkertoy Test
2
Behavioral Assessment of the
Dysexecutive Syndrome
3
Behavior Rating Inventory of
Executive Function
4
Six Elements Test
5
Dysexecutive Questionnaire
RETURN TO WORK
1
Driving evaluationa
2
Functional assessmenta
3
Structured work triala
4
Clinical assessment of current job
demands, expectations, requirementsa
5
Interview with coworkers=supervisorsa

a
Denotes nonstandardized or nonreferenced tests from the Appendix. A complete list of responses is
available on request.

analysis revealed a significant relationship, v2(1, n 732) 7.31, p .005, between

professional activity and use of EOIs. Of respondents who devoted more than
15% of their professional time to rehabilitation and=or cognitive remediation,
46% used at least one EOI. By contrast, only 33% of respondents who devoted less
Table 3 Breakdown of respondents utilization of EOIs
# of EOI(s) per respondent
1
2
3
4
5
6
7
Total

% of respondents using
EOIs

165
53
27
6
3
2
2
258

64
21
10
2
1
1
1
100

Downloaded By: [University of Florida] At: 15:21 8 May 2008

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

735

Table 4 Relationship between professional activity, work setting, or assessment referral question and use
of EOIs

Respondent category
Professional activity
Category A: Devote
15% of professional
time to rehab and=or cognitive remediation
Category B: Devote < 15% of professional
time to rehab and=or cognitive remediation
Work setting
Category A: Work in rehab settings
Category B: Do not work in rehab settings
Assessment referral question
Category A: Endorse
1 rehab-related referral questions
Category B: Endorse 0 rehab-related referral questions


p < . 01;



Use 0 EOIs
% (n)

Use > 1 EOIs

% (n)

54.3 (69)

45.7 (58)

67.3 (407)

32.7 (198)

7.73

50.9 (81)
68.8 (397)

49.1 (78)
31.2 (180)

17.47

63.3 (324)
68.2 (150)

36.7 (188)
31.8 (70)

1.62

v2

p < .001.

than 15% of their professional time to rehabilitation and=or cognitive remediation

reported using such instruments.
Respondents next were assigned to one of two categories based on where they
performed their neuropsychological work. Category A respondents performed a portion of their neuropsychological work in a rehabilitation facility, physical medicine
and rehabilitation department within a hospital, vocational rehabilitation agency,
community re-entry program, and=or brain injury day-treatment center. Category
B respondents did not perform their neuropsychological work in any of those
settings. A chi square analysis revealed a significant relationship, v2(1,
n 736) 17.47, p < .0001, between work site and use of EOIs. As shown in Table
4, 49% of respondents who worked in a rehabilitation setting used at least one EOI,
while only 31% of those who did not work in a rehabilitation setting used these
instruments. We also explored whether respondents who conducted neuropsychological assessments for rehabilitative purposes were more likely to use EOIs.
Respondents were assigned to Category A if they endorsed one or more of the following rehabilitation-related assessment referral questions: (1) assess capacity to
work, (2) assess capacity for independent living, and (3) rehabilitation=treatment
planning. Category B respondents did not endorse any of those referral questions.
As shown in Table 4, a chi-square analysis results failed to reveal a statistically significant relationship, v2(1, n 732) 1.62, p .203, between assessment referral
question and use of EOIs.

DISCUSSION
The present study employed a survey questionnaire to determine utilization
rates of instruments designed with ecological concerns in mind. Results indicated
that approximately one third of respondents (35%) used at least one ecologically
oriented instrument (EOI). Of respondents who reported EOI usage, the vast
majority used at most one or two. Additionally, although several standardized EOIs

Downloaded By: [University of Florida] At: 15:21 8 May 2008

736

LAURA A. RABIN ET AL.

received mention by a reasonable amount of study respondents (e.g., 6% and 4%

reported use of the RBMT and Tinkertoy Test, respectively) this represents but a
fraction of the total percent of responses (approximately 2%). Overall, even the most
popular EOIs were vastly less common than the top traditional instruments in the
areas of memory, attention, and executive functioning (24 to 71% usage as compared to .2 to 6%, see Tables 1 & 2). Since this was the first study to explore this
issue, there is no baseline with which to compare the observed percentages. One
can speculate, however, that usage rates of EOIs may rise as the practical consequences of test scores become increasingly important to neuropsychologists. Thus,
as neuropsychologists increase their activity in applied, rehabilitative settings that
emphasize the prediction of behavior, so too will the desire and demand for instruments designed with ecological objectives.
We also investigated whether various practice demographics affected usage
rates of EOIs. Results indicated that respondents who devoted a substantial portion
of their professional time to rehabilitation and=or cognitive remediation efforts or
who worked in rehabilitation settings (e.g., physical medicine and rehabilitation
department, vocational rehabilitation agency, brain injury day-treatment center)
were more likely to use EOIs than those who did not. Results did not support the
hypothesized relationship between respondents endorsement of rehabilitationrelated referral questions (i.e., rehabilitation=treatment planning, assessment of
work capacity or independent living) and use of EOIs. It therefore appears that neuropsychologists are no more likely to use EOIs for assessments that focus on outcome predictions than for those that focus on diagnostic, forensic, or other
concerns. Notably, a surprisingly high percentage of respondents (70%) endorsed
at least one rehabilitation-related assessment referral question. Thus, while the overwhelming majority of respondents receive rehabilitation-related referrals, only a
third of all respondents use EOIs. This finding highlights the disparity between
the proportion of neuropsychologists who conduct assessments that focus on
ecological issues and the proportion who use instruments designed for ecological
purposes.
Future research might expand on the present findings by investigating other
variables potentially related to the utilization of EOIs (e.g., respondents training
background, patient populations served, knowledge of the existence of such instruments). Qualitative research could explore neuropsychologists reasons for including
specific instruments in their assessment batteries. Additionally, neuropsychologists
could be asked to complete rating scales that assess the degree to which various
instruments are viewed as useful in addressing ecological issues. Research also
should systematically evaluate the ability of EOIs to add valuable information
beyond that obtained by other procedures such as traditional tests, clinical interview,
behavioral observations, or collateral report. Because neuropsychologists statements about everyday functioning can significantly impact patients lives, determination and improvement of the ecological validity of neuropsychological
instruments represent important directions for future research.
A limitation of the current study relates to the construction of the list of EOIs
presented in the Appendix. Since the vast majority of neuropsychological methods
have yet to be validated ecologically, list inclusion was determined through recourse
to research literature, where instruments were described as having been designed or

Downloaded By: [University of Florida] At: 15:21 8 May 2008

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

737

utilized with ecological concerns in mind. Since this was the first study to attempt to
compile existing EOIs into a single list, some eligible instruments undoubtedly were
overlooked. It is also likely that some instruments do not actually address ecological
concerns, and grouping all tests together may obscure this fact. Future research
should attempt to distinguish tests with proven ecological validity from those that
merely purport to possess this quality. Additionally, some neuropsychologists may
object to our inclusion of subjective report instruments in light of research suggesting
low correlations between such measures and objective test results (e.g., Herrmann,
1982; Jorm et al., 2004; Rabbitt & Abson, 1990). Future research will attempt a more
systematic and valid approach to list construction. Finally, study respondents were
repeatedly asked to list the instruments they would use to assess or predict a
patients functioning and behavior. Several respondents found the word instrument limiting because they felt that it excluded such techniques as clinical interviews, direct observations, etc. Certainly, it was not the investigators intention to
limit or alter respondents test usage reporting in any way. In future research, the
word instrument will be replaced with, or supplemented by, broader more inclusive terms like technique or method.

Conclusions and Implications for the Practice of Clinical

Neuropsychology
The continued success and growth of field of clinical neuropsychology depends
on its ability to meet the changing needs of its consumers. Todays neuropsychological assessments are most useful when they not only aid in the detection
and description of deficit, but also provide information about ecological issues, such
as rehabilitation potential and future functional independence. Overall, survey
results shed light on usage rates of ecologically oriented instruments by clinical neuropsychologists. These findings can be used to inform clinicians about the current
practices of their colleagues and to help establish acceptable standards of assessment
competence in the field. Study results also may encourage closer examination of
commonly used EOIs, leading to their incorporation into test batteries and investigation of their validity and utility with various clinical populations. A related goal is
to continue to clarify the relationship between neuropsychological assessment techniques and everyday cognitive skills and functions. This research will enable neuropsychologists to evaluate their own assessment batteries and modify existing
practices to incorporate the latest, most ecologically meaningful techniques. It is
hoped that these efforts will culminate in improved neuropsychological evaluations,
which offer diagnostic and predictive information essential to the provision of optimal patient care.

ACKNOWLEDGMENTS
These data were derived from the dissertation of Laura A. Rabin. The authors
gratefully acknowledge the many INS, NAN, and APA Division 40 members who
took time for their busy schedules to complete and return this questionnaire.

738

LAURA A. RABIN ET AL.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

REFERENCES
Alderman, N., Burgess, P. W., Knight, C., & Henman, C. (2003). Ecological validity of a
simplified version of the multiple errands shopping test. Journal of the International
Neuropsychological Society, 9, 3144.
Anastasi, A. & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ:
Prentice Hall.
Bennett-Levy, J. & Powell, G. E. (1980). The subjective memory questionnaire (SMQ). An
investigation into self-reporting of real-life memory skills. British Journal of Clinical Psychology, 19, 177188.
Benton, A. L., Hannay, H. J., & Varney, N. R. (1975). Visual perception of line direction in
patients with unilateral brain disease. Neurology, 25, 907910.
Berry, J. M., West, R. L., & Dennehey, D. M. (1989). Reliability and validity of the Memory
Self-Efficacy Questionnaire. Developmental Psychology, 25, 701713.
Boyd, T. A. (1988). Clinical assessment of memory in children: A developmental framework
for practice. In M. G. Tramontana & S. R. Hooper (Eds.), Assessment issues in child
neuropsychology (pp. 177204). New York: Plenum Press.
Boyd, T. M. & Sautter, S. W. (1993). Route-finding: A measure of everyday executive functioning in the head-injured adult. Applied Cognitive Psychology, 7, 171181.
Brewer, M. B. (2000). Research design and issues of validity. In H. T. Reis & C. M. Judd
(Eds.), Handbook of research methods in social and personality psychology (pp. 316).
Cambridge, UK: Cambridge University Press.
Broadbent, D. E., Cooper, P. F., FitzGerald, P., & Parkes, K. R. (1982). The cognitive failures
questionnaire (CFQ) and its correlates. British Journal of Clinical Psychology, 21, 116.
Brunswik, E. (1955). Symposium of the probability approach in psychology: Representative design
and probabilistic theory in a functional psychology. Psychological Review, 62, 193217.
Brunswik, E. (1956). Perception and the representative design of psychological experiments
(2nd ed.). Berkeley, CA: University of California Press.
Butler, R. W., Anderson, L., Furst, C. J., Namerow, N. S., & Satz, P. (1989). Behavioral
assessment in neuropsychological rehabilitation: A method for measuring vocationalrelated skills. The Clinical Neuropsychologist, 3, 235243.
Chaytor, N. & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13, 181197.
Crewe, N. M. & Athelstan, G. T. (1981). Functional assessment in vocational rehabilitation. A
systematic approach to diagnosis and goal setting. Archives of Physical Medicine and
Rehabilitation, 62, 299305.
Cripe, L. I. (1996). The ecological validity of executive function testing. In R. J. Sbordone &
C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 171202). Boca
Raton, FL: St. Lucie Press.
Crockett, D., Tuokko, H., Koch, W., & Parks, R. (1989). The assessment of everyday
functioning using the present functioning questionnaire and the functional rating scale
in elderly samples. The Clinical Gerontologist, 8, 325.
Crook, T. H. & Youngjohn, J. R. (1993). Development and treatments for memory disorders:
The necessary meeting of basic and everyday memory research. Applied Cognitive
Psychology, 7, 619630.
Cubic, B. A. & Gouvier, W. D. (1996). The ecological validity of perceptual tests. In
R. J. Sbordone & C. J. Long (Eds.), Ecological validity of neuropsychological testing
(pp. 203224). Boca Raton, FL: St. Lucie Press.
Dobbs, A. R. & Rule, B. G. (1987). Prospective memory and self-reports of memory abilities
in older adults. Canadian Journal of Psychology, 41, 209222.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

739

Einstein, G. O., Smith, R. E., McDaniel, M. A., & Shaw, P. (1997). Aging and prospective
memory: The influence of increased task demands at encoding and retrieval. Psychology
and Aging, 12, 479488.
Franzen, M. D. & Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in
neuropsychological assessment. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity
of neuropsychological testing (pp. 91112). Boca Raton, FL: St. Lucie Press.
Gilewski, M. J., Zelinski, E. M., & Schaine, K. W. (1990). The Memory Functioning Questionnaire for assessment of memory complaints in adulthood and old age. Psychology
and Aging, 5, 482490.
Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2000). Behavior rating inventory of
executive function. Odessa, FL: Psychological Assessment Resources.
Green, P., Iverson, G. L., & Allen, L. (1999). Detecting malingering head injury litigation with
the Word Memory Test. Brain Injury, 13, 813819.
Guilmette, T. J., Faust, D., Hart, K., & Arkes, H. R. (1990). A national survey of psychologists
who offer neuropsychological services. Archives of Clinical Neuropsychology, 5, 373392.
Guilmette, T. J. & Kastner, M. P. (1996). The prediction of vocational functioning from
neuropsychological data. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity of
neuropsychological testing (pp. 387411). Boca Raton, FL: St. Lucie Press.
Heaton, R. K. & Pendleton, M. G. (1981). Use of neuropsychological tests to predict adult
patients everyday functioning. Journal of Consulting and Clinical Psychology, 49, 807821.
Heinrichs, R. W. (1990). Current and emergent applications of neuropsychological assessment: Problems of validity and utility. Professional Psychology: Research and Practice,
21, 171176.
Herrmann, D. J. (1982). Know thy memory: The use of questionnaires to assess and study
memory. Psychological Bulletin, 92, 434452.
Higginson, C. I., Arnett, P. A., & Voss, W. D. (2000). The ecological validity of clinical tests of
memory and attention in multiple sclerosis. Archives of Clinical Neuropsychology, 15,
185204.
Johnstone, B., Frank, R. G., Belar, C., Berk, S., Bieliauskas, L. A., Bigler, E. D. et al. (1995).
Psychology in health care: Future directions. Professional Psychology: Research and Practice, 26, 341365.
Jorm, A. F., Butterworth, P., Anstey, K. J., Christensen, H., Easteal, S., Maller, J. et al.
(2004). Memory complaints in a community sample aged 6064 years: Associations with
cognitive functioning, psychiatric symptoms, medical conditions, APOE genotype, hippocampus and amygdale volumes, and white-matter hyperintensities. Psychological Medicine, 34, 14951506.
Kline, P. (1999). Handbook of psychological testing (2nd ed.). New York: Routledge.
Kopelman, M., Wilson, B. A., & Baddeley, A. D. (1990). The autobiographical memory interview. Bury St. Edmunds, UK: Thames Valley Test Company.
Kuriansky, J. & Gurland, B. (1976). The performance test of activities of daily living. The
International Journal of Aging and Human Development, 7, 343352.
Leonberger, F. T. (1989). The question of organicity: Is it still functional? Professional Psychology: Research and Practice, 20, 411414.
Lezak, M. D. (1982). The problem of assessing executive functions. International Journal of
Psychology, 17, 281297.
Loewenstein, D. A., Amigo, E., Duara, R., Guterman, A., Hurwitz, D., Berkowitz, N. et al.
(1989). A new scale for the assessment of functional status in Alzheimers disease and
related disorders. Journal of Gerontology: Psychological Sciences, 44, 114121.
Long, C. J. (1996). Neuropsychological tests: A look at our past and the impact that ecological
issues may have on our future. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity
of neuropsychological testing (pp. 114). Boca Raton, FL: St. Lucie Press.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

740

LAURA A. RABIN ET AL.

Marsh, R. L., Hicks, J. L., & Landau, J. D. (1998). An investigation of everyday prospective
memory. Memory and Cognition, 26, 633643.
Martin, M. (1983). Cognitive failure: Everyday and laboratory performance. Bulletin of the
Psychonomic Society, 21, 97100.
Martzke, J. S., Swan, C. S., & Varney, N. R. (1991). Posttraumatic anosmia and orbital frontal
damage: Neuropsychological and neuropsychiatric correlates. Neuropsychology, 5, 213225.
McGlone, J., Gupta, S., Humphrey, D., Oppenheimer, S., Mirsen, T., & Evans, D. (1990).
Screening for early dementia using memory complaints from patients and relatives.
Archives of Neurology, 47, 11891193.
Norris, G. & Tate, R. L. (2000). The behavioral assessment of the dysexecutive syndrome
(BADS): Ecological, concurrent, and construct validity. Neuropsychological Rehabilitation, 10, 3345.
OShea, M. F., Saling, M. M., Bladin, P. F., & Berkovic, S. F. (1996). Does naming contribute
to memory self-report in temporal lobe epilepsy? Journal of Clinical and Experimental
Neuropsychology, 18, 98109.
Piotrowski, C. & Keller, J. W. (1989). Psychological testing in outpatient mental health facilities: A national study. Professional Psychology: Research and Practice, 20, 423425.
Ponsford, J. & Kinsella, G. (1991). The use of a rating scale of attentional behavior. Neuropsychological Rehabilitation, 1, 241257.
Putnam, S. H., DeLuca, J. W., & Anderson, C. (1994). The second TCN salary survey: A survey of neuropsychologists part II. The Clinical Neuropsychologist, 8, 245282.
Rabbitt, P. & Abson, V. (1990). Lost and found: Some logical and methodological limitations of self-report questionnaires as tools to study cognitive aging. British Journal of Psychology, 77, 6973.
Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical neuropsychologists in the United States and Canada: A Survey of INS, NAN, and Division 40
members. Archives of Clinical Neuropsychology, 20, 3365.
Rabin, L. A., Barr, W. B., & Burton, L. A. (in press). Effect of patient occupation=education variables on the choice of neuropsychological assessment instruments. Applied Neuropsychology.
Riege, W. H. (1982). Self-report and test of memory aging. Clinic Gerontologist, 1, 2336.
Robertson, I. H., Ward, T., Ridgeway, V., & Nimmo-Smith, J. (1994). The Test of Everyday
Attention. Bury St. Edmunds, UK: Thames Valley Test Company.
Robertson, I. H., Ward, T., Ridgeway, V., & Nimmo-Smith, J. (1996). The structure of normal human attention: The Test of Everyday Attention. Journal of the International Neuropsychological Society, 2, 525534.
Rogers, J. C. (1987). Performance Assessment of Self-Care Skills (PASS). Unpublished Performance Test.
Seidenberg, M., Haltiner, A., Taylor, M. A., Hermann, B. B., & Wyler, A. (1994). Development and validation of a Multiple Ability Self-Report Questionnaire. Journal of Clinical
and Experimental Neuropsychology, 16, 93104.
Shallice, T. & Burgess, P. W. (1991). Deficits in strategy application following frontal lobe
damage in man. Brain, 114, 727741.
Shiel, A. (1990). An investigation of the relationship between unilateral neglect and ADL dependency in right hemisphere stroke patients. Unpublished masters thesis, Southampton University Medical School, Southampton, UK.
Spikman, J. M., Deelman, B. G., & van Zomeren, A. H. (2000). Executive functioning, attention and frontal lesions in patients with chronic CHI. Journal of Clinical and Experimental
Neuropsychology, 22, 325338.
Sunderland, A., Harris, J. E., & Baddeley, A. D. (1983). Do laboratory tests predict everyday
memory? A neuropsychological study. Journal of Verbal Learning and Verbal Behavior,
22, 341357.

Downloaded By: [University of Florida] At: 15:21 8 May 2008

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

741

Sunderland, A., Harris, J. E., & Gleave, J. (1984). Memory failures in everyday life following
severe head injury. Journal of Clinical Psychology, 6, 127142.
Sweeney, J. A., Clarkin, J. F., & Fitzgibbon, M. L. (1987). Current practice of psychological
assessment. Professional Psychology: Research and Practice, 18, 377380.
Sweet, J. J., Moberg, P. J., & Suchy, Y. (2000). Ten-year follow-up survey of clinical neuropsychologists: Part I. Practices and beliefs. The Clinical Neuropsychologist, 14, 1837.
Vilkki, J., Ahola, K., Holst, P., Ohman, J., Servo, A., & Heiskanen, O. (1994). Prediction of
psychosocial recovery after head injury with cognitive tests and neurobehavioral ratings.
Journal of Clinical and Experimental Neuropsychology, 16, 325338.
West, R. L. (1985). Memory fitness over 40. Gainesville, FL: Triad Publishing Company.
West, R. L. & Crook, T. H. (1992). Video training of imagery for mature adults. Applied Cognitive Psychology, 6, 307320.
Williams, J. M. (1996). A practical model of everyday assessment. In R. J. Sbordone &
C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 129145). Boca
Raton, FL: St. Lucie Press.
Wilson, B. A. (1991). Long term prognosis of patients with severe memory disorders.
Neuropsychological Rehabilitation, 1, 117134.
Wilson, B. A. (1993). Ecological validity of neuropsychological assessment: Do neuropsychological indexes predict performance in everyday activities? Applied and Preventive
Psychology, 2, 209215.
Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., & Evans, J. J. (1996). Behavioral
Assessment of the Dysexecutive Syndrome (BADS). Bury St. Edmunds, UK: Thames
Valley Test Company.
Wilson, B. A., Cockburn, J., & Baddeley, A. (1985). The Rivermead Behavioral Memory Test
Manual. Bury St. Edmunds, UK: Thames Valley Test Company.
Wilson, B. A., Cockburn, J., Baddeley, A. D., & Hiorns, R. (1989). The development and validation of a test battery for detecting and monitoring everyday memory problems. Journal
of Clinical and Experimental Neuropsychology, 11, 855870.
Wilson, B. A., Cockburn, J., & Halligan, P. (1987). The Behavioral Inattention Test. Bury St.
Edmunds, UK: Thames Valley Test Company.
Wilson, B. A., Ivani-Chalian, R., & Aldrich, F. (1991). The Rivermead Behavioral Memory
Test for Children Aged 510 Years: Manual. Bury St. Edmunds, UK: Thames Valley Test
Company.
Winterling, D., Crook, T., Salama, M., & Gobert, J. (1986). A self-rating scale for assessing
memory loss. In A. Bes, J. Cahn, S. Hoyer, J. P. Marc-Vergnes, & H. M. Wisniewski
(Eds.), Senile dementias: Early detection (pp. 482486). London: John Libbey Eurotext.
Youngjohn, J. R., Larrabee, G. J., & Crook, T. H. (1992). Discriminating age-associated
memory impairment from Alzheimers disease. Psychological Assessment, 4, 5459.

APPENDIX
Memory
Tests=test batteries:
Autobiographical Memory Interview (AMI; Kopelman, Wilson, & Baddeley, 1990).
Everyday Memory Inventory (EMI; West, 1985).
Memory Assessment Clinics Computerized Everyday Memory Battery (CEMB;
Crook & Youngjohn, 1993).
Rivermead Behavioral Memory Test (RBMT; Wilson, Cockburn, & Baddeley, 1985).

Downloaded By: [University of Florida] At: 15:21 8 May 2008

742

LAURA A. RABIN ET AL.

Rivermead Behavioral Memory Test for Children (RBMT-C; Wilson, Ivani-Chalian,

& Aldrich, 1991).
Questionnaires:
Everyday Memory Questionnaire (EMQ; Sunderland, Harris, & Baddeley, 1983).
Memory Assessment Clinics Self-Rating Scale Questionnaire (MAC-S; Winterling,
Crook, Salama, & Gobert, 1986).
Memory Complaints Inventory (MCI; Green, Iverson, & Allen, 1999).
Memory Complaint Questionnaire (MCQ; OShea, Saling, Bladin, & Berkovic, 1996).
Memory Failure Questionnaire (MFQ; Sunderland, Harris, & Gleave, 1984).
Memory Observation Questionnaire 2 (MOQ2; McGlone, Gupta, Humphrey,
Oppenheimer, Mirsen, & Evans, 1990).
Memory Self-Efficacy Questionnaire (MSEQ; Berry, West, & Dennehey, 1989).
Memory Self-Report Questionnaire (MSRQ; Riege, 1982).
Memory Questionnaire (MQ; Gilewski, Zelinski, & Schaine, 1990).
Multiple Ability Self-Report Questionnaire (MASQ; Seidenberg, Haltiner, Taylor,
Hermann, & Wyler, 1994).
Subjective Memory Questionnaire (SMQ; Bennett-Levy & Powell, 1980).
Attention
Tests=test batteries:
Behavioral Inattention Test (BIT; Wilson, Cockburn, & Halligan, 1987).
Test of Everyday Attention (TEA; Robertson, Ward, Ridgeway, & Nimmo-Smith,
1994).
Questionnaires:
Everyday Attention Questionnaire (EAQ; Martin, 1983).
Rating Scale of Attention (RSA; Ponsford & Kinsella, 1991).
Executive Functioning
Tests=test batteries:
Behavioral Assessment of the Dysexecutive Syndrome (BADS; Wilson, Alderman,
Burgess, Emslie, & Evans, 1996).
Ecological Planning Task (Spikman, Deelman, & van Zomeren, 2000).
Executive Function Route-Finding Task (EFRT; Spikman, Deelman, & van
Zomeren, 2000).
Multiple-Errands Test (Shallice & Burgess, 1991).
Six Elements Test (Shallice & Burgess, 1991).
Spatial Learning Task (SLT; Vilkki, Ahola, Holst, Ohman, Servo, & Heiskanen, 1994).
Tinkertoy Test (Lezak, 1982).

ECOLOGICAL UTILITY OF ASSESSMENT TECHNIQUES

743

Downloaded By: [University of Florida] At: 15:21 8 May 2008

Questionnaires:
Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, &
Kenworthy, 2000).
Dysexecutive Questionnaire (Wilson, Alderman, Burgess, Emslie, & Evans, 1996).
Other
Standardized:
Behavioral Assessment of Vocational Skills (BAVS; Butler, Anderson, Furst,
Namerow, & Satz, 1989).
Cognitive Failures Questionnaire (CFQ; Broadbent, Cooper, FitzGerald, & Parkes,
1982).
Functional Assessment Inventory (FAI; Crewe & Athelstan, 1981).
Functional Rating Scale (FRS; Crockett, Tuokko, Koch, & Parks, 1989).
Loewenstein Direct Assessment of Functional Status (LDAFS; Loewenstein et al.,
1989).
Performance Activities of Daily Living (PADL; Kuriansky & Gurland, 1976).
Performance Assessment of Self-Care Skills (PASS; Rogers, 1987).
Nonstandardized or nonreferenced:
Clinical Assessment of Job Demands=Job Expectations=Current Job Requirements.
Clinical Interview with Functional Questions (e.g., job-related scenarios, judgment
questions).
Contextual Memory Test=Recall of Recent Life Events.
Formal Driving Evaluation.
Functional Assessment=Direct Observation of Patient in Environment.
Interview with Patients Coworkers and Supervisors.
Patient Competency Rating Scale.
Patients impression about whether instruments capture real-world deficits.
Prospective Memory Tests (e.g., Dobbs & Rule, 1987; Einstein, Smith, McDaniel, &
Shaw, 1997; Marsh, Hicks, & Landau, 1998).
Formal Review of Supports Available on the Job (e.g., gradual return, interim supervision).
Structured Work Trial=Peer Review.
Executive Function Behavioral Rating Scale.
Frontal Behavioral Checklist.