Applied Psycholinguistics (1989) 10, 357-375

Printed in the United States of America

Contexts of antonymous adjectives

WALTER G. CHARLES
Oregon State University, Corvalis

GEORGE A. MILLER
Princeton University
ADDRESS FOR CORRESPONDENCE
George A. Miller, Green Hall, Princeton University, Princeton, NJ 08544
ABSTRACT
The method of sorting is used to compare sets of adjectival contexts. Contexts of directly
antonymous adjectives are found to be highly discriminable, both with sentential and phrasal
contexts. These results are used to argue that words with different meanings normally appear in
discriminably different contexts, and that the cue for learning to associate direct antonyms is
not their substitutability, but rather their relatively frequent co-occurrence in the same sentence.

In an early attempt to provide an objective basis for psychological discussions of meaning, Deese (1965) proposed a method using distributions of
responses on word association tests in order to obtain a measure of semantic
similarity. The general idea was that two words that evoke many of the same
associative responses can be said to have similar meanings: insect and bug,
for example, would evoke many of the same associations from a population
of subjects, whereas insect and hat would not. Since two closely related
words are likely to evoke one another, an important feature of Deese's method was the assumption that the stimulus word itself should be counted as a
response by every subject. Thus, for example, everyone can be counted as
responding "insect" to insect (and "bug" to bug), so that responding "insect"
to bug (and "bug" to insect) would count as a common associative response
and so would contribute to the similarity of the two response distributions.
Given this plausible assumption, a simple computation of the intersection of
the two distributions yielded a coefficient of correlation between them,
which Deese interpreted as a measure of semantic similarity.
Although the basic assumption underlying this approach - that the meaning of a word is all the other words it makes you think of - was never
universally accepted, Deese maintained that his method could provide objective evidence for answering questions that otherwise required subjective
judgments. Antonymous adjectives provided a good example. The claim
that two adjectives are antonyms - wet and dry, for example - had previously rested solely on the subjective judgments of lexicographers. With his
measure of associative overlap, Deese (1964) was able to provide objective
1989 Cambridge University Press 0142-7164/89 $5.00 + .00

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

358

evidence to support that opinion, thus putting the whole matter on a firm
scientific base. If, for example, word associations to wet and dry are collected, it is found that each is the primary response to the other. This kind of
reciprocity was proposed as an objective definition for the semantic relation
of antonymy.
Moreover, since Deese found that all of the most frequently used English
adjectives enter into such reciprocal relations, he concluded that antonymy
must be the basic organizing principle for learning and remembering the
meanings of adjectives: "A simple pattern of organization, then, emerges
for common English adjectives. The pattern is one of contrast. . . . The
existence of such a strong pattern among the most basic adjectives suggests
that the structure of adjectives is fundamentally different from that of
nouns" (Deese, 1965, 111).
Such claims raise certain problems, as Deese was well aware. For one
thing, words that share similar associative distributions were supposed to be
similar in meaning, yet the definition of antonymy implies that antonyms
have contrasting meanings. How can the meanings of antonymous adjectives be similar and contrasting at the same time? For another, many adjectives do not seem to have antonyms. Deese himself considered the problem
of nice, which was one of the few common adjectives he studied that does
not enter into a reciprocal associative relation with any other adjective.
Such questions are not unanswerable, of course, and they should not be
allowed to obscure the importance of Deese's observation that antonymy is a
central principle organizing semantic memory for adjectives. Indeed, antonymy seems to play much the same role for adjectives that class inclusion
plays for nouns. But the assumption that word associations provide the
optimal method for studying such questions is more debatable.
Deese's general argument for the plausibility of an associative approach
rested heavily on assumptions about the learning process that is responsible
for forming word associations. Two words become associated by virtue of
sharing contexts: "the extent to which words share associative distributions
is determined by the extent to which they share contexts in ordinary discourse" (Deese, 1965, 128). Since antonymous adjectives share associative
distributions, Deese assumed that they must share contexts in ordinary discourse.
Although what it means for words to "share contexts" is open to more
than one interpretation, Deese did not pursue the question. He seems, instead, to have considered the explanation of how antonymous relations are
learned to be a special case of a more general explanation, widely discussed
at the time, for how parts of speech are learned. The linguist Fries (1952)
had proposed that two words are in the same form class if both can be
substituted into particular sentential contexts. Car and lunch, for example,
are both in the same form class because both can be inserted into the
sentential context: The
is good. For many psychologists, this proposal seemed to define what people learn when they learn the part of speech
of a new word: they learn to recognize the contexts into which it can be
inserted. Thus, the fact that people can recognize that wet and dry are both

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

359

adjectives must mean that they have learned that these words are interchangeable in particular grammatical slots. It was not necessary to pursue
the question of interchangeability into the actual contexts of antonymous
adjectives in everyday discourse.
That neglected thread is picked up in the present study. Although it may
be correct in principle to say that antonymous adjectives can be interchanged in sentential contexts, is it correct in fact? Are the sentences that
people actually use such that replacing an adjective by its antonym always
(or nearly always) produces a nonanomalous result? For example, either wet
or dry can occur in such contexts as: He wiped the surface with
a
rag, or A
climate is good for these plants. But are such
contexts typical?
This article reports on the use of the method of sorting sentential contexts, introduced by Charles (1988a, 1988b), to investigate the contexts of
antonymous adjectives. In this method, many sentential contexts for two
different words are shuffled together, and subjects are invited to sort them
out. If all the contexts of wet and dry, for example, were like the previously
mentioned contexts, subjects would have difficulty determining what the
author's choice might have been. On the other hand, if such contexts are
atypical, subjects should be able to distinguish wet contexts from dry contexts with some accuracy.
In studies with pairs of nouns, Charles found that the discriminability of
their sentential contexts decreases as the judged semantic relatedness of the
nouns increases. The limiting case of this relation would be perfect synonyms, interchangeable in all contexts, where presumably it would be impossible to tell from context alone which synonym an author had used. The
present article extends this work to adjectives, with special attention to the
kind of semantic relatedness involved in antonymy: can the finding that
words occurring in similar contexts tend to have similar meanings be extended to antonymous adjectives, which can occur in similar contexts, yet have
contrasting meanings? If two samples of sentential contexts for an antonymous pair - one sample, for example, in which wet occurred and another in
which dry occurred - were both collected haphazardly from ordinary textual
materials, would subjects be able to discriminate between them?
ANTONYMOUS ADJECTIVES

This discussion will be limited to predictive adjectives, a limitation that has

been implicit in most psycholinguistic studies of adjectives. A simple test of
whether an adjective is predictive is whether it can be used in sentences of
the following form: NP is Adj. For example, wet is predictive because the
wet mayor and the mayor is wet are both admissible, whereas former is
nonpredictive because the former mayor is admissible but the mayor is
former is not. Some adjectives can be used either way: logical is predictive in
the logical argument, but nonpredictive in the logical fallacy.
As Deese observed, associative pairing is a special feature of adjectives. In
the case of antonymous adjectives, pairing is understandable: predicative

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

360

adjectives express values of attributes, and most attributes are bipolar. There
is usually an adjective for the positive pole of an attribute and an antonymous adjective for the negative pole - where the negative is often indicated
by some such prefix as un-, as in real/unreal or common/uncommon.
What is not so easily explained is how particular words are selected
to stand in this antonymous relation. Consider an example. English has
a number of adjectives that denote wetness: wet, damp, moist, soggy,
drenched, sodden, humid all express slightly different senses of this lexical
concept. Similarly, a number of adjectives denote dryness: dry, arid,
parched, dehydrated, dessicated, thirsty, bone-dry all express modulations
of a single concept. As far as their meanings are concerned, any member of
these two sets of synonyms might have been selected as a contrasting term.
Yet there is a consensus among native speakers of English that only wet and
dry are antonyms. For example, drenched is a near-synonym of wet and
parched is a near-synonym of dry, but most people reject drenched/parched
as an antonymous pair. In short, antonymy is not a contrast between lexical
concepts - it is a contrast between associatively paired words. But how is it
learned that wet and dry have been selected as the particular pair of words to
be paired in this manner?
This situation has been noted by Gross, Fischer, and Miller (1989), who
distinguish between direct and indirect antonyms. Both direct and indirect
antonyms express conceptual contrasts, but only direct antonyms are associatively paired. For example, moist does not have a direct antonym, nor
does arid. The direct antonyms are wet/dry. Moist/dry and wet/arid are
indirect antonyms.
Using this terminology, the question becomes: How are specific words
selected as direct antonyms? Or, perhaps more answerable: how do people
learn that the direct antonymy of wet/dry is somehow different from the
same conceptual opposition when it is expressed by, say, soggy and arid?
A plausible assumption would be that this antonymous associative bond
is learned from the contexts that these words share in ordinary text and
discourse. But there are at least two different ways to characterize shared
contexts. Two different hypotheses, which can be called co-occurrence and
substitutability, might account for the development of this word-specific
associative relation. The simpler hypothesis is association through co-occurrence.
1. Co-occurrence: Two adjectives are learned as direct antonyms because
they occur together (are heard together and are spoken together) in the
same sentences more frequently than chance would allow.
The substitutability hypothesis is more complex.
2. Substitutability: Two adjectives are learned as direct antonyms because
they are interchangeable in most contexts, i.e., because any noun phrase
that can be modified by one member of the pair can also be modified by
the other.

The co-occurrence hypothesis conforms to traditional association theory,

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

361

where contiguity and frequency are assumed to be primary factors. The

substitutability hypothesis requires mediation: the association between an
adjective and its direct antonym is assumed to be mediated by the context
into which both can be substituted.
Parsimony would seem to favor the co-occurrence hypothesis, but there is
need for the substitutability hypothesis, too. How people learn that one
response can be substituted for another in particular contexts is a general
question that arises in many situations. As Ervin-Tripp (Ervin, 1961) pointed out, association theorists require a substitutability hypothesis, or something like it, because the word-association responses of adults are so often
the same part of speech as the stimulus word. Ervin-Tripp proposed an
explanation of how such learning could occur. As one theorist described it,
she assumed "that as one listens to speech, a sentence environment which
has been shared previously by two words can elicit either word as a covert
anticipation. A contiguity between these words would occur whenever there
is an error of anticipation - one word is anticipated but the other is heard"
(McNeill, 1963, 251). Using more recent terminology, one might say that
words in the same form class are associated because they compete to fill
syntactic slots (MacWhinney, 1987).
Since a theory of how people learn "what can be substituted for what"
seems to be required for other reasons, it is not unreasonable to wonder
whether that theory might also explain how people learn associative bonds
between pairs of antonymous adjectives. The argument of the present paper,
however, is that this generalization is not correct. Our experimental results
indicate that the actual sentences used in everyday discourse do not provide
contexts that can elicit either word. Although substitutability may be critical
for learning parts of speech, we need not explore the various theories of this
kind of learning in any detail, since it can have little or nothing to do with
learning the pairings of antonymous adjectives.
EXPERIMENT 1

The first two experiments not only compared the contexts of direct antonyms;
they were extended to include comparisons of the contexts of indirect antonyms as well. The reasoning was that, if the substitutability hypothesis is
correct, contexts of direct antonyms should be more similar (harder to discriminate) than contexts of indirect antonyms. Thus, the indirect antonyms,
which are semantically similar, provide a control condition against which the
discriminability of the contexts of direct antonyms can be compared.
Method

Materials. For the first study, the direct antonyms strong/weak were chosen.
The choice was constrained by several considerations: (1) both should be
predictive adjectives; (2) they should be common enough that no subjects in
the experiment could fail to know them; (3) they should satisfy Deese's
(1964) criterion that each would be the most frequent associate of the other;

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

362

(4) they should occur frequently enough in the Brown Corpus (Francis &
Kucera, 1982) that an adequate sample of sentences containing them could
be easily and objectively collected by searching that Corpus.
The indirect antonyms chosen to go with strong/weak were powerful and
faint. These were chosen to satisfy similar criteria: (1) both were predicative
adjectives; (2) they were common enough that all of our subjects would
know them; (3) they had been found (Gross et al., 1989) to be similar in
meaning to the direct antonyms; (4) the Brown Corpus has a sufficient
number of sentences containing them.
The Brown Corpus (Francis & Kucera, 1982) is available in machinereadable form on a Princeton computer. Consequently, it was a simple
matter to search this Corpus of approximately one million running words in
order to retrieve all sentences containing these four words. The smallest
number of sentences found was 25 (for faint), so 25 sentences were selected
to represent each adjective. For the other three adjectives, this selection was
made, first, by excluding all sentences in which more than one of these four
adjectives occurred and, second, by a random choice among the remaining
candidates.
Once the sentences were selected, the target adjectives were replaced by
blanks in order to convert the sentences into sentential contexts. For exampie:
Third, there were those notably Patrice Lumumba who favored a Unified
Congo with a very
central government.
It is no harder to raise big, healthy, blooming plants than
, sickly, little
things; in fact it is easier.
Large, long-range bombers can be developed which would have the capability to take off from 3,000-foot runways, but they would require
more
engines than we have today.
, ladylike sigh of relief.
One could hear a very

Each context was printed on a separate 4 x 6 inch card.

Subjects. Eight Princeton University undergraduates, all native speakers of
English, participated as paid volunteers in each of the conditions.
Procedure. Sets of contexts were compared two at a time; with four terms,
there were six pairwise comparisons of each set of adjectives. For each
comparison, a subject was given a thoroughly shuffled pack of 50 cards,
with one context printed on each card. Subjects were tested individually. No
subject was given the same set of contexts to sort more than once.
Instructions. Subjects were advised to read the contexts carefully and to try
to infer the word that was missing. They were asked to sort the contexts into
groups such that the same word was missing from all examples included in
any group. They were not told what the target words were or how many
groups to use, and they were allowed to re-sort (to change previous decisions) and to continue sorting and re-sorting as long as needed. When they

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

363

finished, they were asked to label the groups of contexts they had created
with the words they thought were missing from the contexts in each group.
Results

The data for each pairwise sorting has been summarized in tables giving the
number of times a context was correctly recognized, was confused with its
alternative, or was taken to be a context for some word other than the words
being compared.
Table 1 summarizes the results for sorting contexts for the direct antonyms, strong/weak. The rows represent the stimulus materials, and the columns represent the subjects' responses. Since there were 25 contexts per term
and 8 subjects per condition, a total of 200 judgments are available for each
set of contexts.
From Table 1 it is seen that strong contexts were recognized correctly
89.5% of the time, and weak contexts 82% of the time. Only 4.3% of the
responses confused one of these contexts with the other, with a small response bias favoring the unmarked adjective strong. The 40 instances in
which subjects labeled their groups with something other than strong or
weak are tabulated in the footnotes to Table 1; they are scattered over 8 and
12 different labels, respectively. Few of these "other" responses seem to be
closely related semantically to the target terms.
Table 2 reports the results for sorting contexts of indirect antonyms:
strong/faint, powerful/weak, and powerful/faint. From these data it is apparent that the contexts of indirect antonyms are less discriminable than
those of direct antonyms. There were fewer correct sortings of the contexts
of indirect antonyms, largely because of the increased number of "other"
responses; confusions between antonyms continued to be relatively rare.
The choice of faint as a near-synonym of weak was clearly not optimal.
When sentential contexts of faint were mixed with contexts of either strong
or powerful, the accuracy of recognition was lowest, the confusions with the
alternative contexts was highest, and the greatest number and variety of
"other" responses were elicited. The strength of the response "faint" is obviously relatively low, and in some contexts it almost never occurred to subjects that faint was the missing word:
/ wore a new double-breasted brown worsted suit with a
herringbone
design and wide lapels like a devil's ears.
With
heart and a brave smile, I endured his long absences from Chateau Belltech, his coldness, his indifference, his slights and his abuses.
To his
surprise Russ held up his hand.

Table 3 summarizes the results when the contexts to be sorted were for the
near-synonyms, strong/powerful and weak/faint. Although faint is still the
hardest to recognize, the general level of discrimination between these sets of
contexts was even poorer than between the indirect antonyms. Contexts for
strong and powerful were confused with one another more often here than

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

364

Table 1. Sorting data for direct antonyms: strong/weak

Response
Stimulus
Strong
Weak
Sum

Strong

Weak

Other

Sum

179
13
192

4
164
168

17"
23"
40

200
200
400

"ardent 4, great 4, political 3, vital 2, little 1, mixed 1, public 1, superior 1 (8).

declining 6, vital 3, inadequate 2, innocent 2, little 2, unstable 2, ardent 1, injured 1,
mere 1, normal 1, political 1, public 1 (12).

when they were tested against antonymous contexts, and faint contexts were
often grouped with those labeled "weak."
Finally, a general comment about the results of Experiment 1: the many
"other" responses elicited by these contexts seem to be largely unrelated in
meaning to the target terms, and the "other" responses unrelated to one
another. In this respect, they contrast with "other" responses obtained when
the target words were nouns (Charles, 1988a, 1988b).
Discussion

The substitutability hypothesis - that adjectives involved in a relation of

direct antonymy are set apart from other adjectives that have similar meanings because they are substitutable for one another in the same contexts implies that, in a context-sorting task, their contexts should be more difficult to distinguish from one another than from the contexts of other,
semantically similar terms. From the data presented here, it is clear that the
opposite is the case in representative usage: sentential contexts in which
either direct antonym is equally appropriate are relatively rare. This result
weakens the case for a substitutability hypothesis.
The results of Experiment 1 also show a strong and consistent bias against
the response "faint." Indeed, the overall pattern of response biases order
these four adjectives: strong> weak>powerful>faint, where the adjective
earlier in this ordering is favored over the following adjectives. In similar
studies with noun contexts, Charles (1988a) found evidence that such response biases depend on relative frequencies of occurrence. The frequencies
of these four adjectives in the Brown Corpus (Francis & Kucera, 1982) are:
strong, 198; weak, 32; powerful, 63; faint, 25, which only approximates
Charles' observation. Such correspondences are difficult to interpret, however, because different word counts disagree with one another. For example,
the corresponding frequencies in the Lorge magazine count (Thorndike &
Lorge, 1944) are 770, 276, 186, and 308, and in the Dahl count of spoken
words (Dahl, 1979) are 198, 74, 32, and 3. The only clear agreement among
these three tabulations is that strong is used most frequently.

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

365

Tkble 2. Sorting data for indirect antonyms: strong/faint, powerful/weak,

powerful/faint
Response
Stimulus
Strong
Faint
Sum

Strong
113
23
136
Powerful

Powerful
Weak
Sum

Powerful
Faint
Sum

Faint

Other

Sum

1
45
46

86"
132*

200
200

218

400

Wea

Other

100

7
107

134
139

154

Powerful

Faint

Other

0
60
60

75*
128/
203

125
12
137

95c
59 rf

200
200
400

200
200
400

"great 13, quiet 11, new 8, firm 7, big 3, staunch 3, violent 3, weak 3, democratic 2,
greater 2, growing 2, magnificent 2, mixed 2, sudden 2, talented 2, certain 1, corporate 1, dark 1, distant 1, distinctive 1, fortified 1, IBM 1, little 1, loud 1, many 1,
necessary 1, open 1, perfect 1, silent 1, unique 1, temporary 1, toned 1, true 1, united
1, wondrous 1, young 1 (36).
^distant 13, quiet 13, dark 12, great 9, small 9, loud 8, magnificent 6, weak 5, slight
4, sudden 4, dull 3, growing 3, new 3, quick 3, wide 3, certain 2, light 2, little 2,
perfect 2, shadow 2, simple 2, strange 2, wondrous 2, bad 1, beautiful 1, big 1, dizzy
1, familiar 1, figure 1, fine 1, headlight 1, heavy 1, precise 1, refined 1, silent 1, slow
1, soft 1, staunch 1, uneasy 1, unlikely 1, violent 1 (41).
^strong 50, important 9, large 9, able 3, opposing 3, grotesque 2, packed 2, small 2,
broken 1, dominant 1, influential 1, labor 1, lacking 1, leg 1, long 1, loud 1, many 1,
minor 1, muscular 1, presidential 1, sharp 1, strict 1, water 1 (23).
^strong 15, small 6, important 4, trembling 4, negative 3, able 2, large 2, light 2,
alcoholic 1, broken 1, difficult 1, distress 1, dominant 1, helpless 1, impotent 1,
injured 1, leg 1, loud 1, many 1, mere 1, more 1, my 1, non-alcoholic 1, nonviolent 1,
primitive 1, red 1, short 1, unique 1, weather 1 (29).
^strong 32, quiet 8, new 7, prominent 5, dark 3, famous 3, small 3, figure 2, important 2, staunch 2, advanced 1, clear 1, complete 1, dear 1, possible 1, reckless 1,
slight 1, thick 1(18).
/weak 22, quiet 18, small 16, slight 11, soft 7, strong 7, dark 6, deep 5, sharp 5, new
4, discernible 3, figure 3, forbidding 3, clear 2, complete 2, empty 2, false 2, pale 2,
prominent 2, cold 1, famous 1, fine 1, form 1, staunch 1, thick 1 (25).

Some of the effects of response bias can be eliminated if these data are
analyzed by signal-detection theory. In Table 1, for example, if strong is

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

366

Table 3. Sorting data for near-synonyms: strong/powerful, weak/faint

Response
Stimulus

Strong

Powerful

Other

Sum

Strong
Powerful
Sum

122
45
167

22
91
113

56"
64*
120

200
200
400

Weak

Faint

Other

Weak
Faint

122
17

Sum

139

2
22
24

76^
161rf
137

200
200
400

firm 15, united 7, new 5, good 4, respected 4, unified 3, vehement 3, major 2,

business 1, clean 1, clear 1, devious 1, protective 1, resolute 1, significant 1, solid 1,
staunch 1, top 1, unsavory 1, unsuitable 1, wide 1 (21).
*firm 10, united 5, clear 4, respected 4, large 3, muscular 3, new 3, villainous 3,
bloody 2, old 2, real 2, solid 2, clean 1, company 1, dapple 1, devious 1, firm 1, hard
1, important 1, infamous 1, influential 1, leading 1, lucid 1, major 1, quiet 1,
resolute 1, secret 1, significant 1, slight 1, stocky 1, thoughtful 1, useful 1, wide 1
(33).
^strong 8, pale 5, bright 4, weary 4, common 3, downtrodden 3, mere 3, sick 3, slight
3, active 2, normal 2, poor 2, quiet 2, shameful 2, small 2, able 1, annoying 1, boring
1, clear 1, controversial 1, damaged 1, dry 1, favorable 1, important 1, leg 1, light 1,
loud 1, mild 1, mixed 1, more 1, my 1, only 1, precise 1, sickly 1, soft 1, some 1,
striking 1, sudden 1, surprised 1, tiny 1, uneasy 1, unnecessary 1, wrong 1 (43).
Strong 26, slight 13, soft 13, bright 9, deep 9, quiet 7, dark 6, loud 5, small 5,
fleeting 4, clear 3, dry 3, pale 3, striking 3, subtle 3, sudden 3, vague 3, distinguished
2, light 2, menacing 2, mere 2, perfect 2, pleasant 2, single 2, slow 2, tiny 2, big 1,
broken 1, casual 1, childish 1, common 1, fine 1, great 1, indistinct 1, leg 1, mild 1,
only 1, own 1, nauseous 1, precise 1, reassuring 1, sick 1, sickening 1, sickly 1, still 1,
strange 1, sweet 1, the 1, triple 1, vain 1, wide 1 (51).

taken as the signal and everything else is noise, then the probability of a hit,
/7("strong" | strong), can be estimated as 179/200=0.895, and the probability of a false alarm, /(("strong" | weak), can be estimated as 13/200=0.065.
These probabilities correspond to d'=2.11. Similarly, if weak is taken as the
signal in Table 1, then the estimated probabilities for a hit and a false alarm
are 0.82 and 0.02, with d'=2.91. Thus, the average d' for direct antonyms is
2.87. When similar calculations are performed using the data of Table 2, the
average d' for indirect antonyms is 1.89. And the data of Table 3 give an
average d' for similar adjectives of 1.22. This method of analysis confirms
the conclusion that contexts of direct antonyms are easier to discriminate
than are other sets of adjectival contexts.
Finally, it should be noted that these data for adjectives extend Charles'

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

367

(1988a) observation for nouns. He found that nouns with similar

meanings tend to occur in similar contexts. Here it seems that adjectives
with similar meanings - strong/powerful and weak/faint - occur in contexts
that are harder to discriminate than are the contexts of adjectives with
contrasting meanings.
EXPERIMENT 2

Conclusions reached after an analysis of just four related adjectives may or

may not generalize. It is not feasible to test all possible pairs of direct
antonyms in this manner, of course, but at least one replication seemed
necessary. So Experiment 1 was repeated exactly, except that the direct antonyms used were public and private, with open and secret as the indirect
controls. The same constraints governed the choice of these adjectives as
governed the choice in Experiment 1.
Method

The method was the same as in Experiment 1, but new sets of 25 sentential
contexts for the adjectives public, private, open, and secret were taken from
the Brown Corpus. The principal caution required was that the sentences
containing open should use it in the appropriate sense, and not as an antonym of shut or closed. The following illustrate the kinds of sentential contexts that were constructed in this way:
Past polls of
opinion show popular favor for this policy.
The current exhibition, which remains on view through October 29, has
tapped 14 major collections and many
sources.
Mr. Black's life is an
book, so to speak, from his birth in Jackson,
Mississippi, through his basketball-playing days in L.S.U., and his attainment of a B.A. degree, which has presumably prepared him for his career
as district sales manager for Peerless Business Machines.
When I returned to make my report, the Hetman did not remember having
sent me on the
mission.
Results

The sorting data for public, private, open, and secret are summarized in
Tables 4, 5, and 6. Table 4, which corresponds to Table 1 for Experiment 1,
shows the results for the direct antonyms public/private. Public contexts
were recognized correctly 88% of the time, and private contexts 71%. The
most frequent error was to call private contexts "personal"; except for that,
"other" responses seem to differ rather widely in meaning from the targets
and from one another.
Table 5, which corresponds to Table 2 for Experiment 1, summarizes the
results for sorting contexts of the indirectly antonymous pairs of adjectives
public/secret, open/private, and open/secret. These sets of contexts were
also well discriminated, although not quite as accurately as in the case of the

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

368

Table 4. Sorting data for direct antonyms: public/private

Response
Stimulus
Public
Private
Sum

Public

Private

Other

Sum

176
11
187

4
142
146

20"
47*
67

200
200
400

"popular 4, personal 3, civic 2, all 1, civil 1, close 1, decisive 1, free 1, governing 1,

land 1, opposing 1, own 1, real 1, true 1 (14).
*personal 30, true 4, free 2, opposing 2, own 2, real 2, civic 1, different 1, ordinary 1,
popular 1, rural 1 (11).

pair of direct antonyms. Contexts for one adjective were seldom grouped
with contexts for its indirect antonyms. None of the words in this experiment
has the low response strength observed for faint in Experiment 1. However,
some subjects were unable to assess any label to some of these contexts, and
those cases are reported simply as "no response."
Table 6, which corresponds to Table 3 for Experiment 1, summarizes the
results obtained when sets of contexts for nearly synonymous adjectives,
public/open and private/secret, were compared. These contexts were surprisingly well discriminated, which suggests that not even these similar adjectives are interchangeable in sentential contexts.
Discussion

The results for these public adjectives followed the same pattern as did the
strong adjectives, although the contexts in Experiment 2 were more accurately discriminated than were those in Experiment 1. But contexts of direct
antonyms are not harder to discriminate than are the contexts of related
words, so the inference that the substitutability hypothesis is implausible is
again supported by this replication.
The response biases in Experiment 2 are not as exaggerated as in Experiment 1, but there is still a clear ordering: publioprivate>secret> open.
The frequencies of these four adjectives in the Brown Corpus (Francis &
Kucera, 1982) are, respectively, 306, 185, 46, and 242, which can be reconciled with the preferences of the subjects in this experiment if it is assumed
that the adjective open has more than one sense and that the sense used in
this experiment is not the most frequent. The Lorge magazine count (Thorndike & Lorge, 1944) and the Dahl count of spoken English (Dahl, 1979) are
not broken down by part of speech. These sources cannot be used to check
on the Francis and Kucera data because public, private, and secret are
frequently used as nouns and open is also a verb.
The results of Experiment 2 were analyzed by signal detection techniques
just as those of Experiment 1 were. From Table 4 the average d' for direct

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

369

Table 5. Sorting data for indirect antonyms: public/secret, open/private, open/secret

Response
Stimulus

Public

Secret

Public
Secret

166
17

Sum

Open
Private
Sum

Open
Secret
Sum

Other

Sum

0
98

34"
85*

200
200

183

98

119

400

Open

Private

Other

127
5

33d

200
200

132

162
170

98

400

Open

Secret

Other

136
7
143

11
153
164

53'
4(V
93

200
200
400

"much 4, new 4, the 4, massive 3, private 3, good 2, passive 2, prior 2, another 1,

competitive 1, confidential 1, diverse 1, few 1, great 1, kind 1, rest 1, special 1,
unnecessary 1 (18).
*private 26, confidential 21, new 14, the 4, great 3, military 2, prior 2, entire 2, good
2, convincing 1, heavy 1, inside 1, material 1, materialistic 1, passive 1, special 1,
sure 1, violent 1 (18).
'public 24, personal 5, strong 5, democratic 4, informal 4, violent 4, (no response) 4,
obvious 2, other 2, town 2, accepted 1, avid 1, biting 1, distracting 1, entire 1,
modern 1, interested 1, second 1, world 1 (18).
^public 11, personal 6, legal 4, different 2, real 2, respected 2, obvious 1, other 1,
radical 1, second 1, trivial 1, volunteer 1 (12).
e
heavy 11, autonomous 9, (no response) 8, public 5, democratic 4, severe 4, closed 2,
anonymous 1, display 1, editorial 1, empty 1, high 1, legal 1, loud 1, private 1, strict
1, well-kept 1 (16).
/mysterious 12, original 4, private 4, severe 3, ambitious 2, closed 2, own 2, strict 2, (no
response) 2, democratic 1, material 1, no 1, original 1, proven 1, public 1, two 1 (15).

antonyms is 2.69, from Table 5 the average for indirect antonyms is 2.40,
and from Table 6 the average for similar adjectives is 2.00. The order of
discriminability is the same as before, although the range is narrower.
In general, contexts of these public adjectives are more recognizable than
are contexts of strong adjectives. The finding that contexts of similar adjectives - public/open and private/secret - are so easily distinguishable was
unexpected, but it does not change the evidence that contextual substitutability does not explain how pairs of antonymous adjectives come to be so
strongly associated.

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

370

Table 6. Sorting data for near-synonyms: public/open, private/secret

Response
Stimulus
Public
Open
Sum

Private
Secret
Sum

Public

Open

Other

Sum

163
41
204

10
141
151

27"
18*
45

200
200
400

Private

Secret

Other

159
27
186

7
127
134

46<*

80

200
200
400

"national 7, free 6, diplomatic 3 , new 3, the 2 , active 1, agreeing 1, competitive 1,

personal 1, private 1, ( n o response) 1 (10).

*closed 2, free 2, personal 2, private 2, accusing 1, active 1, city 1, disturbing 1, first

1, national 1, new 1, radical 1, the 1, (no response) 1 (14).
public 11, real 4, closed 3, actual 2, confidential 2, free 2, special 2, broad 1, false 1,
mental 1, national 1, other 1, spelling 1, staunch 1, suburban 1 (15).
^confidential 12, special 10, public 7, stolen 6, national 3, actual 2, staunch 2, closed
1, real 1, spelling 1, suburban 1 (11).
c

EXPERIMENT 3

The results of Experiments 1 and 2 indicate that, in everyday usage, direct

antonyms are not freely interchangeable in sentential contexts. It may still be
true, however, that a noun phrase that can incorporate one adjective can also
incorporate its antonym. Disambiguation may be based on the remainder of
the sentence. When the noun phrase is incorporated into a full sentence,
substitutability may be blocked by other information outside the noun
phrase. For example, such phrasal contexts as the
school measure or
the
areas can take either public or private equally well, but when
they are incorporated in full sentential contexts, the additional information
about the situation seems to increase the likelihood of responding "public":
Mr. Delaney and Mr. O'Neill are not willing to vote on the
school
measure until the defense education bill clears the House Education and
Labor Committee.
Pictures can be taken in the
areas and when on tours.

Perhaps the results of Experiments 1 and 2 can be discounted as merely

further instances where a local ambiguity is resolved when placed in a larger
context.
Sentential contexts, therefore, may not provide a fair test of the substitutability hypothesis. In order to explore this possibility, a third sorting study
was conducted using only noun phrases as contexts.

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

371

Method

Materials. Only the contexts for the directly antonymous pairs strong/weak
and public/private were used in Experiment 3. The same sentences from the
Brown Corpus were used again, but the amount of context was reduced to
merely the noun phrases containing these adjectives: a determiner, if there
was one; any other adjectives, if there were any; and the noun that the target
adjectives modified. This way of constructing phrasal contexts meant that
often only two or three words of context were available to the subjects. For
example:
a very

central government
, sickly, little things
the
eye
many
sources

As before, each context was printed on a separate 4x6 inch card.

Subjects. Eight Princeton University undergraduates, all native speakers of
English, participated as paid volunteers.
Procedure. The method of sorting contexts was continued, but instead of
pairwise comparisons of different sets of contexts, subjects were given all
four sets of phrasal contexts simultaneously. Thus, each subject was presented with a thoroughly shuffled pack of 100 cards to sort. Subjects were tested
individually.
Instructions. Unlike Experiments 1 and 2, subjects in Experiment 3 were
explicitly told the four terms under which contexts should be sorted. They
were asked to read the phrasal contexts carefully and to sort each card under
one of the four adjectives, strong, weak, public, private, whichever fitted
best into that context. Since the four possible responses were known to the
subjects in advance, "other" responses are eliminated, and when subjects
finished they were not asked to label the groups they had created.
Results

The results of Experiment 3 are summarized in Table 7. As expected, there

are far more false alarms when only phrasal contexts are available. In particular, contexts of weak were sorted under the heading strong almost as often
as under weak, thus creating a large response bias in favor of the unmarked
adjective. But the other three adjectives were better recognized. And contexts of the strong/weak pair were not often mistaken for contexts of the
public/private pair, although strong and public contexts were sometimes
confused. Overall, however, subjects were able to discriminate among the
contexts reasonably well: 58 % of the contexts were sorted correctly.

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

372

Table 7. Sorting data for phrasal contexts: strong, weak, public, private

Response
Stimulus
Strong
Weak
Public
Private
Sum

Strong

Weak

Public

Private

Sum

132
86
33
19
270

29
96
11
11
147

27
8
113
47
195

12
10
43
123
188

200
200
200
200
800

Discussion

The contexts of direct antonyms became less discriminable when the amount
of context was reduced from a sentence to a noun phrase, which confirms
Charles' (1988a) results when he reduced the amount of context for nouns.
Although discriminability is reduced, it does not fail completely. An approximate signal-detection analysis can be performed, as follows. For example, using the data for strong, the probability of a hit can be estimated as
132/200=0.66, and the probability of a false alarm as (86+33 + 19)/600=
0.23; then d'=lA5. Following this method, the values of d' for weak,
public, and private are 1.32, 1.26, and 1.53, respectively. These values of d'
are inflated, of course, by the relative ease of discriminating strong/weak
from public/private contexts. A more conservative approximation for strong
would use 86/200=0.43 to estimate the false alarm probability, in which case
rf'=0.59. Following this method, the values of d' for weak/public, and
private are 1.01, 0.89, and 1.08, respectively.
The source of this discriminability becomes clearer when individual phrasal contexts are examined. For example, hints and compulsions are strong,
not weak. Wheezes and sickly little things, on the other hand, are more
likely to be weak than strong. Similarly, public goes with a better
spirited, whereas private goes with the ficimage and honest and
tional
eye or a
hospital nurse. Some of these collocations
are almost compound nouns. The point, however, is that it is simply not true
that any noun phrase that can take one adjective can equally well take its
direct antonym. Hence, even though substitutability might provide an important cue for learning which contrasting adjectives to pair as direct antonyms it is still not an attractive hypothesis.
Table 7 shows considerable confusion between strong and public contexts.
This result may say something about the importance of marking. Presumably, the unmarked terms, strong and public, are more salient semantically,
are governed by fewer restrictions on their use, and are learned earlier by
children (Clark & Chase, 1972). Alternatively, this result may say something
about the importance of frequency of use. Unmarked terms tend to be used
more frequently (Greenberg, 1966), presumably because there are more dif-

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

373

ferent contexts in which they can occur. Or, to put it the other way around,
words that are used infrequently tend to be used only in specific contexts.
GENERAL DISCUSSION

The results of this series of experiments using the method of sorting contexts
confirm for adjective contexts some of the general observations made by
Charles (1988a) when he used the method to study noun contexts:
1. The method of sorting contexts is robust and gives intuitively plausible
answers to questions about linguistic contexts.
2. Words with similar meanings tend to occur in similar contexts.
3. Response biases in sorting are largely attributable to differences in the
frequency of use of the terms involved.
4. The greater the amount of context provided, the more discriminable the
contexts will be.

Such results encourage further explorations using this method.

The results also cast doubt on substitutability as an important cue for
learning to pair direct antonyms. The substitutability hypothesis may (or
may not) be required for an associative account of learning syntactic categories, but it is simply not available as an explanation for learning pairs of
antonymous adjectives.
Showing that substitutability is an improbable explanation does not
prove, of course, that antonymous pairing is learned as a consequence of cooccurrence. At the anecdotal level, however, while making up the stimulus
sets we found five sentences in the Brown Corpus where strong and weak
both occurred. For example: Strong hands act; weak hands react and . . .
for the purpose of understanding strong and weak points of an individual.
But no sentences were found in which the indirect antonyms strong and
faint, powerful and weak, or powerful and faint co-occurred. Similarly, we
found 17 sentences in which public and private were both used, whereas the
indirect antonyms were used together far less often: public/secret occurred
once, private/open twice, and open/secret once.
These observations are suggestive, but these samples from the Brown
Corpus are too small to disconfirm a reasonable null hypothesis. An analysis of a larger corpus would be required in order to determine whether
antonymous adjectives are regularly used together in the same sentences
more frequently than would be expected by chance.
The following data seem relevant, however. The adjectives big, little,
large, and small occur with sufficient frequency in the Brown Corpus to give
a more reliable estimate of their relative frequencies of co-occurrence. Moreover, these four adjectives illustrate the specificity of antonymous pairing.
Although big and large are closely similar in meaning, as are little and small,
they are generally paired off antonymously as big/little and large/small. Big/
small does not sound quite right; large/little sounds definitely wrong. Although the conceptual opposition is the same, the associative pairings are

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

374

Table 8. Frequencies of occurrence and co-occurrence: big, little,

large, small

Big

Little
Large
Small

Big

Little

Large

316

12
273

(1.6)
(2.0)
(2.9)

(1.7)
(2.5)

3
347

(3.2)

Small
4
3
26
504

not. If co-occurrence in the Brown Corpus could be shown for these adjective pairs, it should serve to discount any proposal that antonymous pairing
reflects some kind of conceptual or semantic generalization.
The results of a search for sentences containing co-occurrences of these
four adjectives in the Brown Corpus are shown in Table 8. The diagonal cells
of Table 8 indicate the number of sentences that contain each of the adjectives big, little, large, and small. The cells above the diagonal give the
numbers of sentences that contain both the row and column adjectives.
Since there are approximately 54,700 sentences in the Corpus, it is possible
to estimate the number of co-ocurrences that would be expected if the
adjectives occurred independently. These estimated values are shown in parentheses below the diagonal of Table 8.
Table 8 shows that, on the average, the pairs big/little and large/small cooccur more often than chance would predict, and more than three times as
often as do big/small and large/little. Since this outcome corresponds to the
intuitions of most people for whom English is the first language, it seems
reasonable to suppose that the antonymous pairing - the "clang" association
between direct antonyms - is a consequence of frequently perceiving and
using these words together in the same syntactic structures.
Finally, it should be emphasized that this conclusion in favor of the cooccurrence hypothesis applies only to learning antonymous associations. It
says nothing one way or the other about conditions required to learn the
syntactic categories of words. It is hoped that one consequence of the
present research will be to resolve a long-standing confusion between these
two kinds of learning.
ACKNOWLEDGMENTS
The preparation of this paper was supported in part by Army Research Institute
Contract MDA 903-86-K-02242, in part by Office of Naval Research Contract
N00014-86-K-0492, and in part by a grant from the James S. McDonnell Foundation. The opinions expressed are those of the authors, however, and are not the
responsibility of these sponsors. We are indebted to Dr. Slava Katz for the tabulation
in Table 8.

Applied Psycholinguistics 10:3

Charles & Miller: Antonymous adjectives

375

REFERENCES
Charles, W. G. (1988a). Categorization of contextual information. Unpublished doctoral dissertation, Princeton University.
(1988b). The categorization of sentential contexts. Journal of Psycholinguistic Research,
77,403-411.
Clark, H. H., & Chase, W. G. (1972). On the process of comparing sentences against pictures.
Cognitive Psychology, 3, 472-517.
Dahl, H. (1979). Word frequencies of spoken American English. Detroit, MI: Gale Research.
Deese, J. (1964). The associative structure of some common English adjectives. Journal of
Verbal Learning and Verbal Behavior, 3, 347-357.
(1965). The structure of associations in language and thought. Baltimore: Johns Hopkins
Press.
Ervin, S. M. (1961). Changes with age in the verbal determinants of word-association. American Journal of Psychology, 74, 361-372.
Francis, W. N., & KuCera, H. (1982). Frequency analysis of English usage: Lexicon and
grammar. Boston: Houghton-Mifflin.
Fries, C. C. (1952). The structure of English: An introduction to the construction of English
sentences. New York: Harcourt Brace.
Greenberg, J. H. (1966). Language universals. In T. A. Sebeok (Ed.), Current trends in linguistics. Volume III: Theoretical foundations. The Hague: Mouton.
Gross, D., Fischer, U, & Miller, G. A. (1989). Antonymy and the representation of adjectival
meanings. Journal of Memory and Language, 28, 92-106.
MacWhinney, B. (1987). The competition model. In B. MacWhinney (Ed.), Mechanisms of
language acquisition. Hillsdale, NJ: Erlbaum.
McNeill, D. (1963). The origin of associations within the same grammatical class. Journal of
Verbal Learning and Verbal Behavior, 2, 250-262.
Thorndike, E. L., & Lorge, I. (1944). The teacher's word book of 30,000 words. New York:
Teachers College Press.