This article was downloaded by: [University of Southern Queensland]

On: 08 October 2014, At: 09:57

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Journal of Personality Assessment

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/hjpa20

SCORS–G Stimulus Characteristics of Select Thematic

Apperception Test Cards
a b c a
Michelle B. Stein , Jenelle Slavin-Mulford , Caleb J. Siefert , Samuel Justin Sinclair ,
a a d a
Megan Renna , Johanna Malone , Iruma Bello & Mark A. Blais
a
Psychological Evaluation and Research Laboratory (PEaRL) , Massachusetts General Hospital
and Harvard Medical School
b
Department of Psychology , Augusta State University
c
Department of Psychology , University of Michigan–Dearborn
d
Department of Psychiatry, New York University , Langone Medical Center and School of
Medicine
Published online: 27 Aug 2013.

To cite this article: Michelle B. Stein , Jenelle Slavin-Mulford , Caleb J. Siefert , Samuel Justin Sinclair , Megan Renna ,
Johanna Malone , Iruma Bello & Mark A. Blais (2014) SCORS–G Stimulus Characteristics of Select Thematic Apperception Test
Cards, Journal of Personality Assessment, 96:3, 339-349, DOI: 10.1080/00223891.2013.823440

To link to this article: http://dx.doi.org/10.1080/00223891.2013.823440

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 Journal of Personality Assessment, 96(3), 339–349, 2014
Copyright C Taylor & Francis Group, LLC
ISSN: 0022-3891 print / 1532-7752 online
DOI: 10.1080/00223891.2013.823440

SCORS–G Stimulus Characteristics of Select Thematic

Apperception Test Cards
MICHELLE B. STEIN,1 JENELLE SLAVIN-MULFORD,2 CALEB J. SIEFERT,3 SAMUEL JUSTIN SINCLAIR,1 MEGAN RENNA,1
JOHANNA MALONE,1 IRUMA BELLO,4 AND MARK A. BLAIS1
1
Psychological Evaluation and Research Laboratory (PEaRL), Massachusetts General Hospital and Harvard Medical School
2
Department of Psychology, Augusta State University
3
Department of Psychology, University of Michigan–Dearborn
4
Department of Psychiatry, New York University, Langone Medical Center and School of Medicine
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014

There has been surprisingly little research into the stimulus properties of the Thematic Apperception Test Cards (TAT; Murray, 1943). This study
used the Social Cognition and Object Relations Scale–Global Rating Method (SCORS–G; Stein, Hilsenroth, Slavin-Mulford, & Pinsker, 2011;
Westen, 1995) to explore the stimulus properties of select TAT cards in a clinical sample. The SCORS–G is a theoretically based and empirically
validated measure of object relations that has widely been used in TAT research. A sample of 80 patients referred for psychological assessment at a
large Northeastern hospital were administered the TAT (Cards 1, 2, 3BM, 4, 13MF, 12M, and 14) as part of their assessment battery. Trained raters
scored the narratives using the SCORS–G. The SCORS–G ratings were analyzed to determine the nature and degree of object representation “pull”
both across and within the TAT cards. The results showed that Cards 3BM and 13MF exhibited the greatest card pull for negative pathological
object representations, and Card 2 displayed the highest card pull for adaptive and mature object representations. Both clinical and research related
implications are discussed.

The Thematic Apperception Test (TAT; Murray, 1943) is a free-
response and implicit measure of personality. Murray described
it as “a method of revealing some of the dominant drives, emo-
tions, sentiments, complexes, and conflicts of personality” (p. 3).
In considering what accounts for individual differences Aronow,
Weiss, and Reznikoff (2001) posited that there are three contrib-
utors to the TAT response: card stimulus, testing environment,
and the patient’s inner world. It is typically left up to the exam-
iner to differentiate what aspects of an individual’s responses
can be attributed to each of these sources. To date, the major-
ity of the TAT literature has focused on understanding how the
patient’s inner world and experience becomes reflected in TAT
responses. There is a relative dearth of research examining the
role of the testing environment and the stimulus pull of indi-
vidual TAT cards. This study begins to address this gap in the
literature by analyzing the stimulus properties of a select group
of commonly employed TAT cards.
Stimulus pull can be defined as the “tendency of the stimulus
to evoke or predispose certain perceptual and/or affective re-
sponses in the subject (Peterson & Schilling, 1983, p. 273). “It
denotes a reality which does not come from within the subject
. . . but corresponds more closely to the actual picture perception,
a more or less objective report of what’s out there” (Peterson
& Schilling, p. 266). To date, the literature on stimulus pull
is limited and empirical studies are “embarrassingly lacking”
(Peterson & Schilling, 1983; Zubin, Eron, & Schumer, 1965,
p. 60). Further, a significant portion of the existent research on
stimulus pull is contained in doctoral dissertations and book
Received July 28, 2012; Revised February 25, 2013.
Address correspondence to Michelle B. Stein, Psychological Evaluation and
Research Laboratory (PEaRL), Massachusetts General Hospital and Harvard
Medical School, One Bowdoin Square, 7th Floor, Boston, MA 02114; Email:
mstein3@partners.org
339
chapters. As a result, clinicians and researchers might have dif-
ficulty accessing this valuable information. There are a number
of potential reasons for the present state of TAT research. For
example, such research is burdensome in terms of obtaining
and rating a suitable number of protocols. Further, TAT rating
scales are not widely known nor routinely employed in clinical
practice. And there is no single widely accepted scoring system
for the TAT like the Comprehensive System for the Rorschach
(Exner, 1995). Jenkins (2008) reported that the most common
measures used to rate TAT narratives include the Social Cogni-
tion and Object Relations Scale (SCORS; Westen, 1995) and the
Defense Mechanism Manual (DMM; Cramer, 1991). However
to our knowledge, neither of these measures has been employed
to examine stimulus characteristics of individual cards.
Although generally underexamined, there have been some
studies examining stimulus pull within and across individual
cards. The TAT cards used in this study (1, 2, 3BM, 4, 13MF,
12M, and 14) are the focus of the literature review, even though
additional cards might have been used in previous research.
Table 1 provides a description of each card and Table 2 provides
a summary of prior research findings. Early empirical research
on TAT stimulus pull focused on rating the frequency of themes
and emotions evoked by the cards in both patient and nonpa-
tient samples (Eron, 1948, 1950, 1953; Rosenzweig & Fleming,
1949). There was also research into the perceptual clarity as-
sociated with the cards (e.g., the extent to which people were
viewing the characters, objects, and social perception of charac-
ters similarly; Murstein, 1972; Rosenzweig & Fleming, 1949).
Eron (1950) developed normative data as well as created TAT
rating scales for emotional tone and checklist for themes and
interpretations levels within and across cards. Eron’s work on
developing normative data was groundbreaking. It allowed re-
searchers and clinicians to determine the extent to which patient
narratives deviated from those of a particular diagnostic group
 340
TABLE 1.—Thematic Apperception Test card descriptions.
TAT Card Description (Murray, 1943)
1 A young boy is contemplating a violin which rests on a table in
front of him.
2 Country Scene; in the foreground is a young woman with books
in her hand, in the background a man is working in the fields
and an older woman is looking on.
3BM On the floor against a couch is the huddled form of a boy with his
head bowed on his right arm. Beside him on the floor is a
revolver.
4 A woman is clutching the shoulders of a man whose face and
body are averted as if he were trying to pull away from her.
13MF A young man is standing with downcast head buried in his arm.
Behind him is the figure of a woman lying in bed.
12M A young man is lying on a couch with his eyes closed. Leaning
over him is the gaunt form of an elderly man, his hand
stretched out above the face of the reclining figure.
14 The silhouette of a man (or woman) against a bright window. The
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
rest of the picture is totally black.
and from stimulus properties of a card such as themes evoked,
emotional tone, perceptual accuracy, and verbal productivity
and word count.
Other researchers followed suit and began using specific rat-
ing scales to examine unique aspects of stimulus pull. For ex-
ample, Alvarado (1994) asked students to make comparisons of
the relative similarity of cards over multiple trials and employed
a more experimental or social psychological method for exam-
ining stimulus pull. Using a visual triad task (e.g., subjects are
asked to make comparisons of the relative similarity of items in
each domain), 21 TAT cards (1, 2, 3BM, 4, 5, 6BM, 6GF, 7GF,
8BM, 8GF, 9BM, 9GF, 10, 12BG, 12F, 12M, 13MF, 17BM,
18BM, 18GF, and 20) were presented to subjects over multiple
trials. Participants were asked to rate similarity between cards
as well as intensity of emotional response evoked by each card.
Consensus analysis was used to determine whether responses
of subjects were idiosyncratic or whether a common response
existed within the group. Using this method, Alvarado found
that Cards 1 and 2 were rated as being similar to each other;
likewise, 13MF and 4 were rated as similar and 3BM and 12M
were ranked as similar. Interestingly, the results also indicated
that the emotional tone of the card partially mediated the sim-
ilarity effect. Alvarado interpreted these findings as suggesting
that participants rated cards as being more or less similar based
on the degree to which they “pulled” for common aspects of ob-
ject representations such as affect tone, emotional investment,
and self-esteem and identity.
Ehrenreich (1990) examined stimulus pull for Cards 1, 2,
3BM, 12M, and 13 MF using narrative content gathered from a
nonpatient sample. He assessed drive expression (Pine, 1960),
defensive patterns (as measured across four tiers ranging from
mature to extremely pathological), dependency (e.g., rated as
dependent, interdependent, independent/autonomous, or nonin-
teractive, p. 16), and locus of control. He found that narratives
for Cards 3BM and 13MF contained the lowest level of defenses,
whereas narratives for Card 2 tended to contain the highest, most
adaptive level defenses. With the exception of locus of control,
narratives for all cards differed significantly from one another,
suggesting that individual cards “elicited specific themes and
were differentially arousing” (p. 15) across measures of drive
expression, defense, and dependency.
STEIN ET AL.
More recently, Seedwaldt (2006) conducted a study focus-
ing on stimulus characteristics across and within TAT cards (1,
2, 3BM, 4, 5, and 13MF) using a well-validated method for
conceptualizing interpersonal themes, the Core Conflictual Re-
lationship Theme (CCRT; Luborsky & Crits-Christoph, 1998).
The CCRT was originally created to be used with Luborsky’s
Relationship Anecdotes Paradigm interview or from transcripts
in supportive-expressive therapy. In this study, the CCRT was
adapted for the TAT. Notably, aspects of Murray’s scoring sys-
tem (needs and press) informed the CCRT standard categories.
The CCRT was developed to identify a “central relationship
pattern, script, or schema that each person follows in conduct-
ing relationships” (Luborksy & Crits-Christoph, 1998, p. 3).
The CCRT uses three components—the Wish (W), Response
from Other (RO), and the Response of Self (RS)—to identify
repetitive relationship patterns expressed in narrative data (a
more detailed description of the CCRT method can be found
in Luborsky & Crits-Christoph, 1998). In Seedwaldt’s study,
Cards 1, 2, 4, and 13MF exhibited strong and moderate stim-
ulus pull for W, RO, and RS. In contrast, Card 3BM had high
stimulus pull for content pertaining to the RO and RS, but weak
card pull for W. Overall, this study highlights that these selected
TAT cards vary in their degree of stimulus pull and elicit dif-
ferent interpersonal themes. An important strength of this study
is the use of an empirically based scale for coding TAT nar-
ratives, the CCRT method, that has both research and clinical
relevance.
Aronow et al. (2001), Bellak and Abrams (1997), and Holt
(1978) provided comprehensive reviews of the research on stim-
ulus pull. They also provided insights from their clinical expe-
rience on the common interpersonal and emotional themes seen
across cards, some of which were described earlier. These are
predominantly summarized in Table 2.
To expand this line of research, we used the Social Cognition
and Object Relations Scale–Global Rating Method (SCORS–G;
Stein, Hilsenroth, Slavin-Mulford, & Pinsker, 2011; Westen,
1995) to examine the stimulus pull of the TAT cards listed in
Table 1 as they relate to the emotional and cognitive aspects
of interpersonal perception and functioning. These are the TAT
cards, which we administer in our clinic and have data for. The
SCORS–G is a widely used empirically based measure for cod-
ing TAT narrative data in a reliable and valid manner (see Stein
et al., 2011, for details). One advantage of the SCORS–G is that
it rates multiple domains of psychological functioning. Westen
et al. (1991) stated that the TAT is a “particularly good test for
assessing object relations because the stimulus is unambigu-
ously social and the subjects are likely to provide enough detail
in describing characters and relationships as to provide con-
siderable access to cognitive and affective motivation patterns
related to interpersonal functioning” (p. 56). For these reasons,
the SCORS–G appears to be an ideal measure in examining the
stimulus properties of selected TAT cards.
In summation, there is limited research on stimulus pull, with
the majority of studies on this topic being conducted several
years ago before the most common coding systems for TAT
narratives were developed. Previous studies have also tended
to limit examination of pull to the global level and most have
utilized nonclinical and college samples. This study builds on
previous research and pursues four goals. First, using a clinical
sample, we examined seven TAT cards for pull using an expert-
rater coding system (i.e., the SCORS–G). This system allows
 STIMULUS CHARACTERISTICS AND SCORS–G 341

TABLE 2.—Summary of research on stimulus pull.

Emotional Tone Frequently Occurring Themes CCRT Wish7 CCRT Response of Other7 CCRT Response of Self 7

Card 1 1. Neutral to 1. Aspiration3, 13 1. To achieve and help others 1. Controlling 1. Disappointed and
moderately sad1, 2 2. Parental pressure/conflict3, 4, 5, 6 2. To feel good and comfortable 2. Rejecting and opposing depressed
2. Most frequent 3. Belongingness1 2. Self-controlled and
outcome is 4. Achievement5, 6, 13 self-confident
happy1 5. Intermediate levels of defense14
6. External locus of control14
Card 2 1. Moderately 1. Ambivalence between personal ambition 1. To achieve and help others 1. Rejecting and opposing 1. Self-controlled and
happy1, 8 and families ties/role conflict3, 5, 9 2. Strong self-confident
2. Most frequent 2. Aspiration/occupational 2. Disappointed and
outcome is concern/achievement1, 4, 5, 6, 10, 11 depressed
happy1, 8 3. Economic pressure1, 4
4. Parental pressure1, 4, 6
5. Conception of family relationships and
attitudes as giving, supportive versus
barren and depriving13
6. High level defenses14
7. Internal locus of control14
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014

Card 3BM 1. Moderately to 1. Depressive themes3, 5, 9, 13 1. To be distant and avoid 1. Rejecting and opposing 1. Disappointed and
very sad1, 2, 8, 9 2. Suicide1, 3, 5, 9 conflicts 2. Helpful depressed
3. Loss of relationship/grief 3, 5, 9 2. Helpless
4. Failure1, 5
5. Punishment/parental pressure1, 5
6. Behavior disorder1
7. Coping resources5
8. Guilt13
9. Low-level defenses14
10. External locus of control14
Card 4 1. Neutral to mildly 1. Male–female roles/sexual attitudes and Female protagonist Female protagonist Female protagonist
negative1 conflicts such as a love triangle or 1. To be loved and 1. Rejecting and 1. Unreceptive
2. Outcome is infidelity3, 5, 9, 13 understood opposing Male protagonist
moderately 2. Pressure/departure/nurturance to and from 2. To be close and accepting 2. Upset 2. Self-controlled and
happy1 partner1, 4 Male protagonist Male protagonist self-confident
3. Impulsivity, aggression, and 3. To be distant and avoid 3. Rejecting and 3. Disappointed and
dangerousness2, 4, 5 conflicts opposing depressed
4. To achieve and help
others
Card 13MF 1. Sad1, 8 1. Sex1, 3, 5, 6, 9, 13 1. To be loved and understood 1. Rejecting and opposing 1. Disappointed and
2. Guilt and 2. Aggression1, 3, 5, 6, 9, 10 2. To be distant and avoid 2. Upset depressed
remorse1, 3, 4, 5 3. Hostility10, 11 conflicts 2. Anxious and ashamed
4. Suicide1, 5 3. To be close and accepting
5. Death/illness1, 4, 5, 6, 12
6. Low-level defenses14
7. External locus of control14
Card 12M 1. Moderately sad1 1. Hypnotism1 N/A N/A N/A
2. Religion1
3. Illness/death1
4. Emotional attitudes toward father5
5. Ideas regarding therapy/therapist5, 9, 13
6. Intermediate levels of defense14
7. External locus of control14
Card 14 1. Positive affect1, 5 1. Happy reminiscence1, 4, 5 N/A N/A N/A
2. Aspiration/hope/ determination1, 4, 5, 13
3. Dreaming, looking, seeking inspiration,
and relaxing1, 5
4. Contemplation9
5. Occupational concerns4
6. Curiorsity1
7. Frustrations/worries13

Note. CCRT = Core Conflictual Relationship Theme (Luborsky & Crits-Christoph, 1998).
1
Eron (1950). 2Goldfried and Zax (1965). 3Bellak & Abrams (1997). 4Eron (1953). 5Holt (1978). 6Rosenzweig & Fleming (1949). 7Seedwaldt (2006). 8Alvarado (1994). 9Aronow,
Weiss, & Reznikoff (2001). 10Campus (1976). 11Murstein (1965). 12Eron (1948). 13Rapaport, Gill, & Schafer (1968). 14Ehrenreich (1990).

us to examine evidence for pull at the global level (i.e., several
dimensions of object relations influenced) and the specific level
(i.e., one or two dimensions influenced). Second, if evidence
for pull was found, we wanted to determine which cards were
more neutral, which had higher levels of global pull, and which
had higher levels of specific pull. Third, we compared and con-
trasted our results to previous findings for all cards containing
some level of pull. Finally, we examined how word count was
related to SCORS–G variables across specific cards. The over-
arching purpose of this study was to determine and highlight
any evidence for card pull on the SCORS–G. In so doing, and
by contextualizing our findings within the preexisting literature,
we hoped to outline various needs for future research that would
have implications for SCORS–G scoring and interpretation.
 342 STEIN ET AL.

METHOD (e.g., 5, 6 or 7) indicate healthy responses. The first variable is

Participants
Participants were 80 outpatients (48 male) referred for psy-
chological assessments in the outpatient psychiatry department
of a large Northeastern academic medical center. All partici-
pants were assessed between September 1, 2008 and July 1,
2010. On average, the sample was 38 years old (SD = 15.0) with
14.5 years of education (SD = 3.0). The most common refer-
ral diagnoses were depressive disorder (47%), anxiety disorder
(18%), and bipolar disorder (17%). However, these are outpa-
tients who have a complex clinical picture, often have multiple
diagnoses, and are being referred for a diagnostic clarification.
Approximately 41% of participants had history of psychiatric
hospitalization. Our sample was primarily White (91%).
Procedure
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
complexity of representations of people (COM), which evalu-
ates internal states and how well the patient is able to see internal
states in the self and other when reporting narratives. It also as-
sesses the patient’s relational boundaries and ability to integrate
both positive and negative aspects of self and others. Affective
quality of representations (AFF) examines a patient’s expec-
tations of others within a relationship and the description of
significant relationships in the past. It assesses emotional tone
of the narrative. Emotional investment in relationships (EIR)
assesses a patient’s ability for intimacy and emotional sharing.
Emotional investment and values in moral standards (EIM) as-
sesses the extent to which the patient uses abstract thought in
relation to morality and compassion for others. Understanding
of social causality (SC) assesses the extent to which the patient
understands human behavior. Experience and management of

aggressive impulses (AGG) assesses the patient’s ability to tol-

Outpatients received a battery of standard clinical instruments erate and manage aggression appropriately. Self-esteem (SE)
including the TAT. After the TAT cards were administered, they assesses the patient’s self-concept, and identity and coherence
were transcribed (written verbatim as patient was creating nar- of self (ICS) assesses a patient’s level of fragmentation and
rative), deidentified, and scored by two independent raters using integration. More thorough descriptions of the eight SCORS
the SCORS–G. The SCORS–G was used to rate all TAT narra- variables, global rating method, and various training examples
tives from each patient. Raters (M. Stein and J. Slavin-Mulford) are provided in the manuals developed by Stein et al. (2011) as
are expert raters who have used this scale for at least 5 years and well as Westen (1995).
previously completed manualized training on the SCORS–G
(Hilsenroth, Stein, & Pinsker, 2007; Westen, 1995). Good
(> .60) to excellent (> .74) reliability was attained by both raters RESULTS
in previous research (Eudell-Simmons, Stein, DeFife, & Hilsen- SCORS–G Reliability
roth, 2005; Pinsker, Stein, & Hilsenroth, 2007; Slavin, Stein,
Intraclass correlation coefficients (ICCs) were used to calcu-
Pinsker-Aspen, & Hilsenroth, 2007; Stein, Hilsenroth, Pinsker-
late SCORS–G ratings of TAT narratives. We calculated ICC (2)
Aspen, & Primavera, 2009; Stein, Pinsker-Aspen, & Hilsenroth,
and Spearman–Brown corrected two-way random effects model
2007; Stein, Slavin-Mulford, Sinclair, Siefert, & Blais, 2012).
(2, 1). Shrout and Fleiss (1979) reported the magnitude for in-
Each rater independently rated all 544 TAT narratives employed
terpreting ICC values in which poor is < .40, fair = .40–.59,
in this study. It is important to note that although the “raw” TAT
good = .60–.74, and excellent >.74. As Table 3 shows, ICC
narratives were reviewed as part of the clinical assessment, as-
(2) fell in the good to excellent range and ICC (2, 1) fell in
sessing clinicians did not have access to the SCORS–G ratings.
the excellent ranges for all SCORS–G variables. The coefficient
Also, the SCORS–G raters were blind to the clinical assessment
alpha for each SCORS–G variable was also calculated and these
results.
fell in the excellent range. Alpha differs from ICC (2, 1) simply
by virtue of the fact that it is based on the Pearson correlation
Measures
Thematic Apperception Test. The standard TAT protocol
used at the clinic consists of Cards 1, 2, 3BM, 4, 13MF, 12M TABLE 3.—Interrater reliability of the Social Cognition and Object Relations
and 14, routinely given in this order. However the test battery is Scale–Global Rating Method (SCORS–G) variables for Thematic Apperception
applied flexibly and some subjects were given fewer than seven Test (TAT) narratives.
cards (due to symptom acuity, fatigue, or both). All participants
SCORS Coefficient
were administered Cards 1, 2, 3BM, and 4. Ninety-five percent Variables M SD ICC(2) ICC(2,1) Alpha
of participants were administered Cards 13MF and 14. Card
12M was administered to 92.5% of participants. Clinicians in COM 3.35 .55 .65 .79 .79
the clinic are trained to employ standard TAT instructions: AFF 3.30 .49 .79 .88 .88
EIR 2.99 .51 .75 .86 .86
Now I’m going to show you some cards with pictures on them. The EIM 3.62 .38 .73 .85 85
pictures are of people in various situations and what I want you to do is SC 3.17 .64 .67 .80 .80
AGG 3.53 .43 .83 .90 .90
make up a story around the picture. Like all stories, yours should have SE 3.74 .30 .67 .80 .80
a beginning, middle, and ending. Tell me what led up to the picture, ICS 4.43 .41 .68 .81 .81
how it turns out, and what the people feel and think. Here’s the first M .72 .84 .84
one. Make up a story around this picture.
Note. N = 80; 544 TAT narratives. ICC = intraclass correlation coefficient; (2) = Model
2, two-way random effect; (2,1) = Model 2, two raters, Spearman–Brown correction for
Social Cognition and Object Relations Scale–Global two-way random effect; COM = complexity of representations; AFF = affective quality of
Rating Method. This scale consists of eight variables that are representations; EIR = emotional investment in relationships; EIM = emotional investment
in values and moral standards; SC = understanding of social causality; AGG = experience
scored on a 7-point anchored scale where lower scores (e.g., 1, and management of aggressive impulses; SE = self-esteem; ICS = identity and coherence
2 or 3) indicate more pathological responses and higher scores of self.
 STIMULUS CHARACTERISTICS AND SCORS–G 343

TABLE 4.—Estimated variance components and percentage variance explained from two-facet random crossed models ( p × r × c).

COM AFF EIR EIM

df MS σ2 % df MS σ2 % df MS σ2 % df MS σ2 %

Participant ( p) 72 3.576 .154 12% 71 4.085 .241 36% 71 3.916 .149 11% 71 2.020 .069 10%
Rater (r) 1 1.358 .001 0% 1 2.446 .003 0% 1 0.778 0 0% 1 0.254 0 0%
Card (c) 6 40.393 .267 21% 6 5.120 .029 4% 6 28.847 .184 13% 6 11.333 .071 10%
Pr 72 0.270 .004 0% 71 0.407 .030 5% 71 0.377 .009 1% 71 0.274 .013 2%
Pc 432 1.387 .573 46% 426 0.507 .154 23% 426 1.766 .727 52% 426 0.956 .389 54%
Rc 6 0.854 .009 1% 6 0.613 .006 1% 6 0.875 .008 1% 6 0.356 .002 0%
prc, e 432 0.241 .241 19% 426 0.198 .198 30% 426 0.313 .313 23% 426 0.179 .179 25%
SC AGG SE ICS
df MS σ2 % df MS σ2 % df MS σ2 % df MS σ2 %

Participant ( p) 71 5.794 .349 43% 70 2.669 .105 13% 71 1.258 .033 6% 71 2.488 .088 10%
Rater (r) 1 6.349 .011 1% 1 0.122 0 0% 1 0.025 .00 0% 1 1.750 .001 0%
Card (c) 6 3.106 .018 2% 6 13.765 .089 11% 6 8.337 .053 10% 6 21.941 .140 15%
Pr 71 0.567 .051 6% 70 0.220 .013 2% 71 0.262 .015 3% 71 0.464 .030 3%
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014

Pc 426 0.552 .170 21% 420 1.108 .491 59% 426 0.696 .272 52% 426 1.047 .396 43%
Rc 6 0.206 0 0% 6 0.119 0 0% 6 0.194 .000 0% 6 0.928 .009 1%
prc, e 426 0.212 .212 26% 420 0.127 .127 15% 426 0.152 .152 29% 426 0.255 .255 28%

Note. MS = mean squares; σ 2 = estimated variance component; % = percentage of variance accounted for; COM = complexity of representations; AFF = affective quality of
representations; EIR = emotional investment in relationships; EIM = emotional investment in values and moral standards; SC = understanding of social causality; AGG = experience
and management of aggressive impulses; SE = self-esteem; ICS = identity and coherence of self.

rather than the ICC. The SCORS–G ratings of TAT narratives
were highly reliable and showed strong internal consistency.
To examine potential sources of variance, we estimated vari-
ance components (σ 2) with techniques from generalizability
theory (Crick & Brennan, 1982; Shavelson & Webb, 1991).
Given that all participants were rated by all raters, and that raters
were not viewed as systematically different from the universe of
raters trained to rate the SCORS–G, variance component esti-
mates were calculated using a random two-facet model in which
participants ( p) were crossed with raters (r) crossed with cards
(c). This approach estimates variance components for the uni-
verse level (i.e., the participant level), as well as for rater and for
card. Estimated variance components are also calculated for the
interactions between participant and card (σ 2pc ), participant and
2
rater (σ pr ), and rater and card (rc). Finally, an estimated vari-
ance component is calculated for the residual (σ 2prc,e ). As can
be seen in Table 4, raters were highly consistent with percentage
variance attributable to raters being 1% or less across SCORS–G
dimensions. Raters were also consistent across cards and par-
ticipants. The interaction between raters and card accounted for
very little variance (all σ 2rc accounting for 1% or less of to-
tal variance), and the interaction between rater and participant
2
was also relatively small (σ pr ranging from 0% to 6% across
SCORS–G dimensions).
Although raters tended to rate consistently across both cards
and participants, estimated variance components for that card
level and Card × Participant interaction were consistently larger.
For COM, EIR, and SE, variance component estimates were
larger for card than for the participant level (i.e., universe level),
indicating that participants’ narratives differed somewhat from
card to card. Further, the percentage of variance attributable to
the Card × Participant interaction considerably exceeded the
amount accounted for at the participant level for all SCORS–G
dimensions except AFF and SC. This suggests that participants’
relative standing on the SCORS–G dimensions, with the ex-
ception of AFF and SC dimensions, varied considerably as a
function of card. Said differently, a participant who told a story
high in COM to Card 1 might not have necessarily told a high
COM story to other cards. In general, these findings suggest that
individual reactions to the cards introduce the most variability
for several of the SCORS–G dimensions. However, these re-
sults in and of themselves do not indicate that respective cards
pull for particular types of stories. To determine if cards pull
for particular types of narratives, the mean response level for a
dimension on each respective card should be compared to the
overall mean for that dimension (see Table 5).
SCORS–G Variable Means
The means and standard deviations based on raw scores for
the eight SCORS–G dimensions across all TAT cards are pre-
sented in Table 5. Mean word count and standard deviations for
individual cards are presented in Table 6.
Stimulus Characteristics of Individual TAT Cards
Raw SCORS–G dimension scores are presented in the up-
permost portion of Table 5. To assess for individual card char-
acteristics, each participant’s response to each card was then
transformed into z scores (M = 0, SD = 1). The advantage of z
scores is that one can easily see if a card is consistently scoring
above the mean (i.e., a positive z score) or below the mean (i.e.,
a negative z score) and can easily determine the magnitude of
the difference from the mean. There are two ways of calculat-
ing z scores (across and within participant z scores) and both
are presented next. Significant results on either approach are
suggestive of card “pull.”
Across participant z scores. We calculated across partici-
pant z scores by determining the mean and standard deviation for
each SCORS–G dimension across each participant’s response
for all cards. For example, for the COM scale, the mean and
standard deviation for the sample was calculated using each
participant’s COM score for each individual card. Thus, each of
 344 STEIN ET AL.

TABLE 5.—Average raw and standardized SCORS–G dimension scores by card.

Raw Scores Averages Across Participants

Card COM AFF EIR EIM SC AGG SE ICS

1 3.62 3.52 2.90 3.64 3.41 3.62 3.90 4.69

2 3.55 3.72 3.35 3.87 3.32 3.79 3.99 4.88
3BM 3.14 2.73 2.54 3.58 3.10 3.46 3.42 3.84
4 3.42 3.35 3.23 3.54 3.27 3.37 3.75 4.54
13MF 3.21 2.47 2.75 3.06 3.06 2.96 3.44 4.05
12M 3.35 3.32 3.66 3.66 3.29 3.56 3.81 4.47
14 3.18 3.89 2.51 3.85 3.04 3.82 3.89 4.57
Average z Scores (Across Participants) by SCORS–G Dimension
Card COM AFF EIR EIM SC AGG SE ICS M SD

1 0.41 0.22 −0.10a 0.02a 0.27 0.10 0.24 0.30 0.18 0.17
2 0.28 0.41t 0.32∗ 0.33 0.16 0.31 0.38t 0.52∗∗ 0.34 0.10
3BM −0.23 −0.56∗∗ −0.43 t −0.06 −0.12 −0.10 −0.49t −0.71∗∗ −0.34 0.24
4 0.10 0.04a 0.21 −0.10 0.09a −0.20 0.00a 0.12 0.03 0.13
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014

13MF −0.19 −0.83∗∗ −0.23 −0.72∗∗ −0.17 −0.70∗∗ −0.46∗ −0.46∗ −0.47 0.26
12M 0.03a 0.02 0.60∗∗ 0.05 0.12 0.03a 0.10 0.04a 0.12 0.20
14 −0.23 0.58∗ −0.46 0.30 −0.20 0.34 0.22 0.15 0.09 0.35
F 9.62 27.32 14.93 11.30 5.50 11.50 9.89 18.60
p <.01 <.01 <.01 <.01 <.01 <.01 <.01 <.01
ŋ2 0.12 0.28 0.18 0.14 0.07 0.14 0.12 0.21
Average z Scores (Within Participant) by SCORS–G Dimension

Card COM AFF EIR EIM SC AGG SE ICS M SD

1 0.37 0.25 −0.14a −0.05 0.29 0.09a 0.11 0.28 0.15 0.18
2 0.33 0.47∗ 0.42∗∗ 0.38∗ 0.19 0.40 0.37 0.55∗ 0.39 0.10
3BM −0.35 −0.61∗∗ −0.50 −0.02a −0.13 −0.11 −0.51∗∗ −0.81∗∗ −0.38 0.28
4 0.19 0.05a 0.26 −0.04 0.08a −0.15 0.07a 0.21 0.08 0.14
13MF −0.24 −0.83∗∗ −0.22 −0.70∗∗ −0.30 −0.65∗∗ −0.46∗ −0.53∗∗ −0.49 0.23
12M 0.00a 0.10 0.67∗∗ 0.11 0.12 0.09 0.19 0.25 0.19 0.21
14 −0.28 0.65∗ −0.47 0.36∗ −0.27 0.36 0.23 0.05a 0.08 0.39
F 7.53 29.70 17.89 12.77 4.24 12.21 9.91 22.66
p <.01 <.01 <.01 <.01 <.01 <.01 <.01 <.01
ŋ2 0.10 0.30 0.21 0.16 0.06 0.15 0.13 0.25

Note. Superscript t are trends (<.10). Bold and superscript a represent the reference card. SCORS–G = Social Cognition and Object Relations Scale–Global Rating Method; COM =
complexity of representations; AFF = affective quality of representations; EIR = emotional investment in relationships; EIM = emotional investment in values and moral standards; SC
= understanding of social causality; AGG = experience and management of aggressive impulses; SE = self-esteem; ICS = identity and coherence of self.
∗
Pairwise comparison with reference comparison card is significant at the p < .05 level. ∗ ∗ Pairwise comparison with reference comparison card is significant at the p < .01 level.

the 80 participants provided four (if administered four cards) to

TABLE 6.—Means, standard deviations, and correlations between card word seven (if administered the entire protocol) COM scores. Using
count and SCORS–G variables. this distribution, each participant’s score for every individual
1 2 3BM 4 13MF 12M 14 card was transformed to a z score. Thus, if participants consis-
tently created narratives to a specific card that were above the
N 80 80 80 80 76 74 76 average for a SCORS–G dimension (e.g., COM), then this card
M 119 147 101 124 107 112 102 would receive a positive z score. These data are located in the
SD 53.10 66.66 48.06 63.81 54.03 58.01 62.59 middle portion of Table 5 for all SCORS–G dimensions.
SCORS–G
COM .61∗ .46∗ .49∗ .49∗ .52∗ .53∗ .51∗ Using these z scores, repeated-measures analyses of vari-
AFF .19 .02 −.04 −.16 −.08 −.06 −.22 ance (ANOVAs) with card as the across-participant factor were
EIR .21 .13 .05 .06 −.11 −.07 .32∗ conducted for each SCORS–G dimension. All ANOVAs were
EIM −.16 −.01 −.13 −.19 −.08 −.01 −.10 significant with effect sizes ranging from .07 (SC) to .28 (AFF).
SC .59∗ .54∗ .49∗ .63∗ .58∗ .45∗ .51∗
AGG −.00 −.17 −.11 −.22 .01 −.04 −.18
We followed these analyses up with Bonferroni-corrected pair-
SE .02 .08 −.22 −.18 −.34∗ .06 −.22 wise comparisons. Rather than consider all comparisons, we
ICS .00 −.02 −.36∗ −.10 −.35∗ −.21 −.33∗ elected to select a reference card for each SCORS–G dimen-
sion to use as a comparison. The reference card varied across
Note. SCORS–G = Social Cognition and Object Relations Scale–Global Rating SCORS–G dimensions, but it was always the card with the value
Method; COM = complexity of representations; AFF = affective quality of represen-
tations; EIR = emotional investment in relationships; EIM = emotional investment in closest to the mean (i.e., closest to zero). When cards differed
values and moral standards; SC = understanding of social causality; AGG = experience significantly from the reference card on a SCORS–G dimension,
and management of aggressive impulses; SE = self-esteem; ICS = identity and coherence
of self.
as indicated by Bonferroni-corrected pairwise comparisons, we
∗
p < .01. considered this to be evidence of card pull. Cards producing
 STIMULUS CHARACTERISTICS AND SCORS–G
SCORS–G dimension scores that significantly differ from the
reference card score are indicated in the middle portion of
Table 5.
Within participant z scores. We also calculated standard-
ized scores (i.e., z scores) within participants. The difference
between within- and across-participant z scores is that within-
participant z scores control for the effect of outliers. In this
procedure, the means and standard deviation for a SCORS–G
dimension (e.g., COM) were calculated from the individual par-
ticipant’s responses to all seven cards. Only participants with
responses to all seven cards (n = 73) were used to calculate
within-participant z scores. We then converted raw scores on
each individual card (e.g., Card 1 COM) to a z score based on
the participant’s mean and standard deviation for that SCORS–G
dimension. The bottom third of Table 5 presents these data.
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
Using the within-participant z score data, repeated-measures
ANOVAs with card as the within-subject factor were conducted
for each SCORS–G dimension. All ANOVAs were significant
with effect sizes ranging from .06 (SC) to .30 (AFF). We fol-
lowed these analyses up with Bonferroni-corrected pairwise
comparisons to determine which, if any, cards differed signif-
icantly from the reference card. As can be seen in Table 5,
effect sizes for across-participant (middle graph) and within-
participant z scores (third graph) were highly similar. The only
exceptions were for EIM for Cards 2 and 14. That is, within-
participant z scores found that these cards pulled for adaptive as-
pects of morality. Given these similarities, we discuss responses
to specific TAT cards next using only the within-participant z
scores (e.g., by TAT card and by SCORS–G dimension).
Results by TAT Card
Card 1. Card 1 did not differ significantly in the pairwise
comparison analyses for any of the SCORS–G dimensions. In
other words, this card did not exhibit significant or unique card
“pull” (M = 0.15, SD = 0.18).
Card 2. Overall, Card 2 had the highest (adaptive) mean
score across SCORS–G dimensions (M = 0.39, SD = 0.10).
This card was significantly different from the reference card for
AFF, EIR, EIM, and ICS. Narratives with more positive affective
integration (AFF) and greater emotional investment in relation-
ships (EIR) were common for this card. In addition, narratives
tended to involve characters with a higher sense of moral stan-
dards (EIM) and a more integrated sense of self (ICS). In sum,
Card 2 pulled for significantly more adaptive object relations
on multiple dimensions. In fact, although only four dimensions
reached the level of statistical significance, all SCORS–G di-
mensions scored above the mean for this card.
Card 3BM. Narratives for Card 3BM tended to be rated as
less adaptive across SCORS–G dimensions (M = –0.38, SD =
0.28). Card 3BM scored significantly lower than the reference
card for AFF, SE, and ICS. Narratives were negatively toned
and expectations of relationships were low (AFF), characters
exhibited low self-worth (e.g., inadequacy, inferiority), and their
sense of identity fluctuated widely (ICS).
Card 4. Card 4 was the most neutral of the cards studied
in terms of overall pull for positive or negative object relations
and social cognitions (M = 0.08, SD = 0.14). It did not differ
345
significantly from the reference cards for any SCORS–G vari-
ables, and ratings for this card tended to fall rather close to the
mean of all cards on all SCORS–G dimensions.
Card 13MF. Of all the cards, and as might be expected,
narratives for 13MF tended to be rated the least adaptive across
SCORS–G dimensions (M = −0.49, SD = 0.23). This card
scored significantly lower than the reference card for AFF, EIM,
AGG, SE, and ICS. Characters in these narratives were malev-
olent, harmful, angry, aggressive, and at times abusive (AGG).
There was less of an investment in morals (EIM) and moral-
ity (right and wrong) was more likely to be discussed in a less
mature fashion. Characters tended to be described in ways that
suggested low self-worth (SE) and possessing undifferentiated
sense of identities (ICS).
Card 12M. Card 12M did not consistently pull for adap-
tive or maladaptive object relations or social cognitions at the
global level (M = 0.19, SD = 0.21). However, 12M did score
significantly higher than the reference card for EIR, suggesting
that it might pull for a specific theme. That is, narratives to Card
12M tended to indicate a greater investment in relationships.
Thus, although the card doesn’t appear to pull for more negative
or positive object relations in general, it does seem to prompt
respondents to produce narratives in which they demonstrate
higher investment in relationships.
Card 14. Similar to Card 12M, Card 14 did not consis-
tently generally pull for adaptive or maladaptive scores across
SCORS–G dimensions (M = 0.08, SD = 0.39). However, unlike
Card 12M, which scored near the mean for all SCORS–G dimen-
sions, examination of the pattern of scores for Card 14 shows
greater variability, pulling for adaptive responses on some di-
mensions and maladaptive responses on others. Card 14 also
differed significantly from the reference card pulling for narra-
tives involving more emotional investment in morality (EIM)
and more adaptive expression and management of affect (AFF).
In fact, AFF scores on Card 14 were the highest of all cards
studied.
Variability Across SCORS–G Dimensions
Again, given similarities in the across-participants and within-
participants analyses, we limit our presentation of results in
this section to the within-participant results. As indicated by
the effect size (i.e., partial eta squared [ŋ2]) of .10, there were
moderate differences among cards for COM; however no card
differed significantly from the reference card (Card 12) for this
dimension.
For AFF, Card 4 was the reference card and Cards 2 ( p < .05)
and 14 ( p < .05) produced AFF scores significantly higher than
the reference card and Cards 13MF ( p < .01) and 3BM ( p <
.01) produced AFF scores significantly lower than the reference
card (Card 4). For EIR, Card 2 ( p < .01) and Card 12M ( p <
.01) produced scores significantly higher than the reference card
(Card 1). On EIM, Cards 2 ( p < .05) and 14 ( p < .05) produced
scores significantly higher than the reference card (Card 3BM),
and Card 13MF was significantly lower ( p < .01). For SC,
no card differed from the reference card (Card 4). For AGG,
Card 13MF scored significantly lower than the reference card
(Card 1). For self-esteem (SE), cards 3BM ( p < .01) and 13MF
( p < .05) were significantly lower than the reference card (Card
 346
4). For ICS, Card 2 scored significantly higher ( p < .05) than
the reference card (Card 14), and Cards 3BM ( p < .01) and
13MF ( p < .01) scored significantly lower than the reference
card. Overall, six SCORS–G variables (e.g., AFF, EIR, EIM,
AGG, SE, and ICS) evidenced significant variation across the
TAT cards. The cognitive variables, COM and SC, appeared
more robust to pull from the cards in our TAT protocol.
Word Count
We also wanted to assess whether individual cards varied in
regard to verbal productivity. To assess this, we correlated in-
dividual card count with the SCORS–G variables to examine
within-card differences (see Table 6). We used r = .3 as the
cutoff for significance to control for multiple comparisons and
spurious findings. There was a significant positive relationship
between COM and SC for Cards 1, 2, 3BM, 4, 13MF, 12M,
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
and 14. Higher verbal productivity was related to higher, more
adaptive object representations as measured by COM and SC.
Other aspects of object relations that showed significant as-
sociations with word count included ICS, SE, and EIR. That
is, there was a significant negative association between Cards
3BM, 13MF, and 14 and ICS. Higher word count was sug-
gestive of lower SCORS–G ratings on ICS. Card 13MF was
negatively related to SE. Greater verbal productivity was re-
lated to lower ratings on self-esteem. Lastly, there was a posi-
tive relationship between Card 14 and EIR; that is, greater ver-
bal productivity was associated with an increase in emotional
investment.
DISCUSSION
This study used the SCORS–G to assess the stimulus qualities
or “pull” of selected TAT cards for dimensions of object rela-
tions. Our first goal was to provide evidence for card pull using
an empirically based measure. Unique stimulus characteristics
were evidenced across as well as within cards. Stated differ-
ently, some cards produced consistent differences in the quality
of object relations and social cognition at either the global level
(i.e., across many SCORS–G dimensions) or the specific level
(i.e., on one or two dimensions). The consistency with which
some cards (e.g., Card 2, Card 3BM) scored from the mean
strongly suggests that the cards are notably contributing to how
respondents inform their narratives. In short, the cards are not
neutral and should not be viewed as “equivalent” in content.
This has potential implications for both scoring and interpreta-
tion (discussed later).
Our second goal was to assess which cards were neutral,
which pulled for more pathological or adaptive object rela-
tions at the global level, and which cards consistently produced
pathological or adaptive responses that were limited to specific
domains. Of the cards evaluated, Cards 1 and 4 appeared to
exert the least influence on respondent narratives. Neither of
these cards significantly differed from the reference card for
any SCORS–G variable and z scores were generally close to the
mean. Thus, these cards appear to be the most “neutral,” at least
in terms of the SCORS–G rating scales. One of the reasons why
Card 4 did not produce any significant relationships with the
SCORS–G variables (especially with EIR or AGG) might be
because in Card 4, there appear to be two dominant characters
with varying and at times opposing levels of investment. There-
fore, in rating these narratives, some SCORS–G variables tended
STEIN ET AL.
to be scored more in the middle. This is similar to Seedwaldt
(2006), who found different CCRT ratings depending on
whether the male or female characters were described as the
dominant character. As such, aspects of object representations
might have been missed because of this. Cards 3BM and 13MF
consistently led participants to produce narratives rated as more
pathological across several SCORS–G dimensions. In contrast,
narratives for Card 2 tended to be consistently rated as more
adaptive across SCORS–G dimensions. With regard to specific
forms of pull, Cards 12M and 14 appeared to prompt respondents
to produce narratives that affected only one or two SCORS–G
dimensions. For example, in response to Card 14 respondents
consistently produced narratives in which affect was expressed
and managed more adaptively. Thus, it is quite likely that Cards
12M and 14 do not impact as many aspects of narratives as do
Cards 3BM, 13MF, and 2. Instead, they exert a smaller impact,
influencing a more narrow range of participants’ narratives.
Our third goal was to examine how any evidence for card pull
related to previous research in this domain. Although several
aspects of our findings relate to previous research, results for
Cards 3BM and 13MF stand out most notably (Alvarado, 1994;
Aronow et al., 2001; Eron, 1950; Goldfried & Zax, 1965). For
example, Ehrenreich (1990) found that narratives to these cards
tended to possess the most primitive drives and least adaptive
defenses. Of note, Ehrenreich (1990) used a nonclinical female
sample of students attending a two year community college,
whereas our study employed a clinical sample. The consistency
in findings across diverse samples suggests that these card-
effects generalize across clinical and non-clinical settings, and
are not simply due to the presence or absence of psychopathol-
ogy and/or distress.
The literature has been mixed regarding Card 2’s stimulus
properties (e.g., Aronow et al., 2001; Eron, 1953; Goldfried &
Zax, 1965; Campus, 1976; Seedwaldt, 2006; Rapaport, Gill, &
Schafer, 1968). In this study, narratives for Card 2 were rated
highly for a number of SCORS–G dimensions, most notably
ICS. This card appeared to trigger narratives involving char-
acters with goals, ambitions, and a clear sense of direction in
life. Aronow et al. (2001), Eron (1950), and Rosenzweig and
Fleming (1949) also found that Card 2 elicited themes of per-
sonal ambition, role conflict, and achievement and aspiration,
suggesting that this finding is unlikely to be sample specific.
Similarly, Ehrenreich (1990) also found that narratives to Card
2 contained the most adaptive and mature level of defenses (as
compared to narratives for other cards). In sum, across studies,
outcome variables, and samples, Cards 3BM and 13MF pull for
more pathological global responses, whereas Card 2 triggers
more adaptive global responses. Given the number of dimen-
sions impacted in our study, the consistency of the direction
of impact, and findings from prior research, we speculate that
these cards exert a wider influence impacting multiple aspects of
respondents’ narratives and resulting in narratives that suggest
lower global function.
Whereas the global impact findings for Cards 2, 3BM, and 13
MF are most consistent with prior research, results for Card 12M
relate to Eron (1950). In this study, Card 12M appeared to trigger
the greater emotional investment and emotional intimacy and
sharing in relationships (EIR). Eron (1950), however, reported
limited “pull” for Card 12M, implying that narratives to this
card did not appear to be systematically biased or influenced by
card content. However, differences between studies might have
 STIMULUS CHARACTERISTICS AND SCORS–G
to do with the methods employed to rate narratives. Eron used a
more general, thematic approach focusing on overall quality of
narratives. In this study, a reliable and validated coding system
allowed for examination of both global and specific aspects of
narratives. At the global level, our results are similar to those
of Eron. The majority of SCORS–G variables do not appear
impacted by the content of this card. However, we did find that
this card elicits narratives higher in EIR. Thus, this card might
exhibit a more limited influence affecting only a narrow range
of object relations.
Our fourth goal was to examine the relationship among ver-
bal productivity, intelligence, and SCORS–G ratings. When
examining the relationship between verbal productivity and
SCORS–G variables, universally, higher verbal productivity was
associated with increased complexity and differentiation of self
and other (COM) and organization and coherence as well as
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
increased understanding of social behavior (SC). COM and SC
have been linked to word length in previous studies (Eudell-
Simmons et al., 2005; Stein et al., 2007; Stein et al., 2012) and
both dimensions appear to tap more “cognitive” aspects of social
functioning (Stein et al., 2012). In our sample, individual TAT
card word count was significantly positively correlated with the
Wechsler Abbreviated Scale of Intelligence Vocabulary (Wech-
sler, 1999) score (r = .41, p = .001). However, future research is
needed to further understand this complex relationship (poten-
tial link between verbal ability and SCORS–G variables: COM
and SC).
Three additional findings are notable. For Cards 3BM and
13MF, the typical relationship (i.e., higher word count re-
sults in more adaptive ratings) was reversed for some specific
SCORS–G variables. Card 3BM’s word count was negatively
related to ICS and word count was negatively related to SE for
Card 13MF. One possible explanation for why a longer narrative
on these cards was associated with more pathological ratings is
that the content of the card strongly pulls for themes or char-
acters that involve diffuse identities (Cards 3BM & 13MF) or
problems with self-esteem (Card 13MF). Thus, the more ver-
bally productive a participant is, the more these themes are
developed. Said differently, increased verbal productivity might
maximize the pull of these cards. Somewhat surprisingly, word
count was negatively related to ICS and positively related to
EIR for Card 14. With regard to EIR, there is only one character
depicted in the picture and one would expect that this would
relate to lower EIR ratings (the SCORS–G states “where only
one character is described and no relationship is depicted, code
2”). Perhaps, as verbal productivity increases, the probability of
another nondepicted person being introduced into the narrative
emerges. An area of future research would be to determine if
card pull on Cards 3BM (ICS), 13MF (SE and ICS), and 14
(EIR and ICS) might be best captured in an optimal word count
range.
Implications and Future Directions
The findings from this study primarily serve to bring the issue
of pull back into awareness. As often occurs in studies, these
results tend to raise more questions than they answer. In the
following paragraphs, we begin by discussing some key ques-
tions raised by the data. We go on to argue that, in concert with
previous research, the questions raised by our findings indicate
the need for at least two lines of future research. The first line
347
should focus on assessing how and if pull impacts predictive
validity and diagnostic accuracy of SCORS–G ratings to TAT
narratives. This line might also determine if norm-referenced
scores at the card level are needed or might clarify which spe-
cific cards or card combinations are most useful for specific
predictions (e.g., is the respondent depressed?). The second line
of research should focus on examining how personality might
interact with card pull. This is a notable implication, as currently
SCORS–G ratings tend to be averaged across cards before be-
ing compared to personality. The data reported here suggest
that personality–SCORS–G relationships might need to be ex-
amined at the individual card level. Finally, these data have some
implications regarding card selection.
Although both past and present findings suggest that some
cards strongly impact narratives, the results reported here specif-
ically raise questions regarding scoring and interpretation spe-
cific to SCORS–G ratings for TAT cards. For example, do less
adaptive SCORS–G ratings for Card 3BM indicate the same
thing as less adaptive SCORS–G ratings to Card 1 (a relatively
neutral card)? Similarly, is a response that stands in strong con-
trast to the general pull for a card meaningful? For example,
consider that Card 3BM consistently elicited more pathological
narratives. If a respondent produces a narrative to 3BM that is
high in a number of SCORS–G dimensions is that more signif-
icant than adaptive narratives to Card 2? If the answer is yes,
how should such a pattern be interpreted? Is it indicative of
particularly superb abilities in regulating emotional reactions to
distressing stimuli or is it indicative of a defensive reaction?
Are there features of the narrative that could be used to discern
a defensive reaction from an adaptive reaction?
In addition to specific interpretive considerations, these find-
ings raise questions pertaining to diagnostic utility. For exam-
ple, prior research with the SCORS–G indicates that narratives
involving a combination of painful affect and poor affect man-
agement (i.e., low AFF) paired with a sense of inadequacy and
poorly defined sense of self (i.e., low ICS) are common in de-
pressed individuals (Aronow et al., 2001; Bellak & Abrams,
1997; Holt, 1978). However, if the visual content of Cards 3BM
and 13 MF prime content that is “depressing,” resulting in nar-
ratives low in AFF and ICS, is this indicative of depression?
In contrast, are narratives low in AFF and ICS told in response
to more neutral cards (e.g., Card 1) more indicative of depres-
sion? Ultimately these are empirical questions we believe can
be examined with future research.
The first line of research suggested should focus on scor-
ing and predictive validity. Overall, these results, particularly
for cards 3BM, 13MF, and 2, might signal the need for norm-
referenced scores at the card level. Said differently, normative
or referenced-based data for all SCORS–G ratings at the card
level might be needed to control for difference in card pull
for each SCORS–G variable. Most studies tend to average raw
SCORS–G ratings across cards to calculate a participant’s score
for that dimension. This study, however, suggests that the cards
might not be “equivalent items.” Thus, averaging without cor-
recting for pull might be problematic. However, if a variety of
cards are used and pull is balanced, the need to correct for pull
might be minimized. This is ultimately an empirical question.
Specifically, research that examines how pull-corrected calcula-
tions versus raw average calculations predict a criterion variable
is needed. An initial first step in this direction, given the current
lack of card-level norms for the TAT, would be to examine how
 348
scores for “high pull” cards predict a third variable in compar-
ison to scores for “neutral” or “low pull cards.” For example,
using a dependent variable like depression, a researcher could
examine the predictive utility of relevant SCORS–G scores (e.g.,
ICS, AFF) calculated based on the average across cards that have
pull for pathological scores (e.g., Cards 3BM and 13MF), neu-
tral cards (e.g., Cards 1 and 2), and cards that pull for adaptive
scores (e.g., Card 2). Such studies might help to determine the
need for norms and might provide alternative approaches for
using TAT cards and SCORS–G ratings to assess for specific
difficulties or challenges.
The second line of research suggested focuses on person-
ality and card-pull interactions. In examining sources of vari-
ability in ratings, many SCORS–G variables exhibited strong
Participant × Card interactions. This finding is suggestive of
pull in and of itself; it also raises the possibility that the re-
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
spondent’s personality might interact with the pull of the card to
some degree. Thus, this study also suggests that future research
focusing on the relationship between personality and SCORS–G
ratings of TAT narratives might do well to focus on relationships
between personality and ratings for specific cards (as opposed
to ratings averaged across cards). Lines of research could focus
on understanding counter-pull responses (i.e., responses that
are in sharp contrast to those typically pulled by the card). For
example, do people who avoid the expression of negative af-
fect produce higher AFF scores in response to Cards 3BM and
13MF? Similarly, are individuals high in interpersonal domi-
nance or aggression more likely to produce low AGG scores in
response to Card 2? In contrast, cards that pull for a particular
type of response or emotion might be particularly well suited
at triggering chronic personality features. In other words, al-
though most individuals produce lower ICS scores in response
to Card 3BM relative to themselves, maybe respondents with
particular struggles in this domain (e.g., patients with borderline
personality disorder) exhibit the lowest scores. Research could
also focus on a more “projective” hypothesis or chronic-schema
activation hypothesis (i.e., are responses told to more neutral
cards more “projective” in nature?). For example, are individu-
als high in extroversion more likely to exhibit higher EIR in re-
sponse to Cards 1 and 4, whereas those low in extroversion score
lower?
Our data could also inform card selection for TAT research.
For example, researchers might wish to use or select different
cards depending on the specific type of object relations or affec-
tive dimensions they are interested in. In this way knowing the
stimulus properties of specific TAT cards might make TAT re-
search more productive. For example, our findings suggest that
Card 12M focuses on relational tone. Therefore research primar-
ily interested in relational dimensions might have greater suc-
cess by selecting this card. However, further research is needed
to clarify this point. This study helps elucidate the unique prop-
erties and “pull” of selected TAT cards across important aspects
of social and emotional functioning using the SCORS–G. Also,
this is one of the few studies in this area that uses an empirically
based coding system with expert raters. A last potential impact
of this study could be that this begins to spark discussions re-
garding this concept, hopefully generating additional research
questions, and thus reviving empirical research on stimulus pull.
This article has several important implications for assessment
research and practice. First, this study identified the unique stim-
ulus qualities present within a “set” of seven TAT cards. This
STEIN ET AL.
information provides insights into the utility of each card and
can aid clinicians in distinguishing whether the card’s stimu-
lus or a patient’s inner world is primarily driving (pulling for)
specific narrative themes. Knowing what themes TAT cards pull
for might help assessment psychologists recognize when pa-
tients omit or deny known stimulus characteristics. In addition,
being aware of the stimulus properties of each card can help
guard against overpathologizing (enhancing psychopathology)
of common themes. Lastly, SCORS–G data on card pull can
also help to identify spoiled and pathological responses.
Limitations
Being that we used a clinical sample, individual cards were
not rated as overly positive, which is in contrast to prior re-
search. To further clarify the distinction between card pull and
the patient’s inner world and experience, it would be beneficial
to replicate this study with a nonclinical population to examine
if more positive tones were elicited. If results were similar to this
study, this would further substantiate card pull. If similar results
do not emerge, this could still help understand which SCORS–G
variables are more and less sensitive to card pull. Either way, it is
important for future research to clarify this. Another limitation
is that we had no systematic means for examining how personal-
ity might relate to stimulus pull for cards. With the exception of
AFF, estimated variance components tended to suggest that for
all other SCORS–G dimensions a considerable amount of vari-
ance was produced by the participant–card interaction. Thus, a
participant’s personality might modulate how the participant re-
sponds to a given card. Future research using a larger sample and
simultaneously collecting personality data using an alternative
procedure (e.g., the Rorschach; self-report inventories) would
allow investigators to compare standardized scores for each card
to other personality features. This could better aid clinicians and
researchers alike in understanding patterns associated with both
deviations from the “typical” response (e.g., personality factors
related to narratives containing highly maladaptive object rela-
tions in response to Card 2) and extreme forms of the “typical”
response (e.g., highly aggressive narratives for Card 13MF).
ACKNOWLEDGMENTS
An earlier version of this article was presented at the annual
meeting of the Society for Personality Assessment, Boston, MA,
March 2011.
REFERENCES
Alvarado, N. (1994). Empirical validity of the Thematic Apperception Test.
Journal of Personality Assessment, 63, 59–79.
Aronow, E., Weiss, K. A., & Reznikoff, M. (2001). A practical guide to the
Thematic Apperception Test: The TAT in clinical practice. New York, NY:
Routledge.
Bellak, L., & Abrams, D. M. (1997). The T.A.T., C.A.T., and S.A.T. in clinical
use (6th ed.). Boston, MA: Allyn & Bacon.
Campus, N. (1976). A measure of needs to assess the stimulus characteristics
of TAT cards. Journal of Personality Assessment, 40, 248–258.
Cramer, P. (1991). The development of defense mechanisms: Theory, research
and assessment. New York, NY: Springer-Verlag.
Crick, G. E., & Brennan, R. L. (1982). GENOVA: A generalized analysis of
variance system. Dorchester, MA: University of Massachusetts at Boston,
Computer Facilities.
Ehrenreich, J. H. (1990). Quantitative studies of responses elicited by selected
TAT cards. Psychological Reports, 67, 15–18.
 STIMULUS CHARACTERISTICS AND SCORS–G
Eron, L. D. (1948). Frequencies and themes and identifications in the stories
of schizophrenic patients and non-hospitalized college students. Journal of
Consulting Psychology, 12, 387–395.
Eron, L. D. (1950). A normative study of the Thematic Apperception Test.
Psychological Monographs, 64(9), 1–47.
Eron, L. D. (1953). Responses of woman to the Thematic Apperception Test.
Journal of Consulting Psychology, 17, 269–282.
Eudell-Simmons, E. M., Stein, M. B., DeFife, J. A., & Hilsenroth, M. J. (2005).
Reliability and validity of the Social Cognition and Object Relations Scale
(SCORS) in the assessment of dream narratives. Journal of Personality As-
sessment, 85, 325–333.
Exner, J. E. (1995). The Rorschach: A Comprehensive System. Vol. 1: Basic
foundations. New York, NY: Wiley.
Goldfried, M. R., & Zax, M. (1965). The stimulus value of the TAT.
Journal of Projective Techniques and Personality Assessment, 29(1), 46–
57.
Hilsenroth, M., Stein, M., & Pinsker, J. (2007). Social Cognition and Object
Relations Scale: Global Rating Method (SCORS–G, 2nd ed.). Unpublished
Downloaded by [University of Southern Queensland] at 09:57 08 October 2014
manuscript, The Derner Institute of Advanced Psychological Studies, Adelphi
University, Garden City, NY.
Holt, R. (1978). Methods in clinical psychology (Vol. 1). New York, NY: Plenum.
Jenkins, S. R. (2008). A handbook of clinical scoring systems for the thematic
apperceptive techniques. New York, NY: Taylor & Francis.
Luborsky, L., & Crits-Christoph, P. (1998). Understanding transference: The
Core Conflictual Relationship Theme method (2nd ed). Washington, DC:
American Psychological Organization.
Murray, H. A. (1943). Manual for the Thematic Apperception Test. Cambridge,
MA: Harvard University Press.
Murstein, B. I. (1965). The stimulus. In B. Murstein (Ed.), Handbook of pro-
jective techniques. New York, NY: Basic Books.
Murstein, B. I. (1972). The thematic test and the Rorschach in predicting marital
choice. Journal of Personality Assessment, 36, 213–217.
Peterson, C. A., & Schilling, K. M. (1983). Card pull in projective testing.
Journal of Personality Assessment, 47, 265–275.
Pine, F. (1960). A manual for rating drive content in the Thematic Apperception
Test. Journal of Projective Techniques, 24, 32–45.
Pinsker, J. H., Stein, M. B., & Hilsenroth, M. J. (2007). The clinical utility
of early memories as predictors of therapeutic alliance. Psychotherapy, 44,
96–109.
Rapaport, D., Gill, M. M., & Schafer, R. (1968). Diagnostic psychological
testing (Rev. ed.). New York, NY: International Universities Press.
349
Rosenzweig, S., & Fleming, E. (1949). Apperceptive norms for the Thematic
Apperception Test: An empirical investigation. Journal of Personality, 49,
483–503.
Seedwaldt, V. (2006). Using the CCRT to interpret the TAT: Understanding
the phenomenon of “card pull” (Unpublished doctoral dissertation). Adelphi
University, Garden City, NY.
Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer.
Thousand Oaks, CA: Sage.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing
rater reliability. Psychological Bulletin, 86, 420–428.
Slavin, J. M., Stein, M. B., Pinsker-Aspen, J. H., & Hilsenroth, M. J. (2007).
Early memories from outpatients with and without a history of childhood
sexual abuse. Journal of Loss and Trauma, 12, 1–16.
Stein, M. B., Hilsenroth, M. J., Pinsker-Aspen, J. H., & Primavera, L. (2009).
Validity of the DSM–IV Axis V Global Assessment of Relational Functioning
Scale: A multimethod assessment. Journal of Nervous and Mental Disease,
197, 50–55.
Stein, M. B., Hilsenroth, M., Slavin-Mulford, J., & Pinsker, J. (2011). Social
Cognition and Object Relations Scale: Global Rating Method (SCORS–G; 4th
ed.). Unpublished manuscript, Massachusetts General Hospital and Harvard
Medical School, Boston, MA.
Stein, M. B., Pinsker-Aspen, J. H., & Hilsenroth, M. J. (2007). Borderline
pathology and the Personality Assessment Inventory (PAI): An evaluation
of criterion and concurrent validity. Journal of Personality Assessment, 88,
81–89.
Stein, M. B., Slavin-Mulford, J., Sinclair, S. J., Siefert, C. J., & Blais, M. A.
(2012). Exploring the construct validity of the Social Cognition and Object
Relations Scale in a clinical sample. Journal of Personality Assessment, 94,
533–540.
Wechsler, D. (1999). Wechsler Abbreviated Scale of Intelligence (WASI). San
Antonio, TX: Harcourt Assessment.
Westen, D. (1995). Social Cognition and Object Relations Scale: Q-sort for
projective stories (SCORS–Q). Unpublished manuscript, Department of Psy-
chiatry, The Cambridge Hospital and Harvard Medical School, Cambridge,
MA.
Westen, D., Klepser, J., Ruffins, S., Silverman, M., Lifton, N., & Boekamp, J.
(1991). Object relations in childhood and adolescence: The development of
working representations. Journal of Consulting and Clinical Psychology, 59,
400–409.
Zubin, J., Eron, L., & Schumer, F. (1965). An experimental approach to projec-
tive techniques. Chichester, UK: Wiley.