Journal of Experimental Psychology:

Learning, Memory, and Cognition

2001, Vol. 27, No. 6, 1487-1490
Copyright 2001 by the American Psychological Association, Inc.
0278-7393/O1/S5.00 DOI: 10.1037*0278-7393.27.6.1487
Are You Sure You Forgot? Feeling of Knowing in Directed Forgetting
Ali I. Tekcan and Melis Akturk
Bogazici University
One significant issue in metamemory is how variables increasing memorability affect metamemory.
Previous research has produced inconsistent results. The effect of directed forgetting on the magnitude
and accuracy of feeling-of-knowing (FOK) judgments was investigated. Participants were presented with
word pairs, some to be remembered and some to be forgotten, and then were asked to recall all target
words regardless of initial instructions. For unrecalled items, they were asked to give FOK judgments
about performance in a future memory task: a cued stem-completion task (Experiment 1) or a recognition
test (Experiment 2). This encoding manipulation increased both the memory performance and the
magnitude of FOK judgments. However, no such effect on the accuracy of FOK judgments was observed.
The main purpose of this study was to address the effects of
encoding conditions on feeling-of-knowing (FOK) judgments. A
few studies have investigated this issue with episodic memory
tasks, such as paired-associate learning (e.g., Lupker, Harbluk, &
Patrick, 1991; Nelson, Leonesio, Shimamura, Landwehr, & Na-
rens, 1982; Schacter, 1983). The main question in these studies has
been how the strength of encoding relates to metamemory.
In studies with paired-associated learning, it has been consis-
tently shown that the degree or the depth of initial encoding leads
to an increase in object-level memory performance, that is,
in recall and recognition (e.g., Carroll & Nelson, 1993; Lupker
et al., 1991; Nelson et al., 1982; Schacter, 1983; Schwartz &
Metcalfe, 1992). However, the effects of such manipulations on
metamemory measures have been less clear. In studies on FOK
judgments, there are two metamemory measures of interest: the
magnitude of FOK judgments regarding performance in a future
criterion task that is generally less challenging than free recall and
the accuracy of these judgments in predicting the actual perfor-
mance in that task. Most studies have consistently demonstrated an
effect on the magnitude but not the accuracy of these judgments.
Using a levels-of-processing manipulation, Lupker et al. (1991)
found that deeper processing led to better performance in the initial
cued-recall task and in the criterion task (cued stem-completion
task). Intensity and accuracy of FOK judgments were also higher
for items that received deeper processing. In another experiment,
Lupker et al. (1991) obtained the same pattern of results with
study-time manipulation.
Similarly, Nelson et al. (1982) and Carroll and Nelson (1993)
showed that overlearning influenced measures of memory and
metamemory in the same manner. For instance, Nelson et al.
(1982) found that intensity as well as accuracy of FOK judgments
was higher for items that had been overlearned 1 week before.
However, results from some studies (Schacter, 1983; Schacter and
Ali 1. Tekcan and Melis Akturk, Department of Psychology, Bogazici
University, Istanbul, Turkey.
We thank Berivan Ece for her help in collection and analysis of data.
Correspondence concerning this article should be addressed to Ali t.
Tekcan, Department of Psychology, Bogazici University, Bebek-80815,
Istanbul, Turkey. Electronic mail may be sent to tekcanal@boun.edu.tr.
Worling, 1985) contradicted these findings. In both of these stud-
ies, increased study time for cue-target pairs (1.5 vs. 5 s) led to an
increase in the frequency of positive FOK judgments but had no
such effect on accuracy.
Thus, there is abundant evidence that encoding manipulations
have similar effects on explicit object-level measures of memory
and FOK intensity for better-studied materials (for an exception,
see Schwartz & Metcalfe, 1992). However, results regarding the
effects of encoding manipulations on FOK accuracy are
inconsistent.
The purpose of this study was to provide further data on the
effects of encoding conditions on the magnitude and accuracy of
FOK judgments by use of another well-established paradigm, the
directed-forgetting procedure. Lupker et al. (1991) suggested that
one reason why Schacter (1983) failed to find an effect of in-
creased study time on FOK accuracy may be that such a difference
may emerge only if the encoding manipulations lead to a sizable
difference in initial task performance. Moreover, as Carroll and
Nelson (1993) demonstrated, effects of overlearning become ap-
parent when the manipulation is done within subjects. Thus, we
reasoned that directed forgetting was an appropriate manipulation
for addressing the effect of initial learning, especially with regard
to changes in the magnitude and accuracy of FOK judgments.
In a typical directed-forgetting study with the item method,
participants are presented with a list of words, some of which they
are instructed to remember and some of which they are instructed
to forget. Then, when they are instructed to remember both the
to-be-remembered (TBR) and the to-be-forgotten (TBF) items,
recall level for TBR items is higher (Basden, Basden, & Gargano,
1993; Bjork and Woodward, 1973; MacLeod, 1989; MacLeod,
1999; Mungan & Peynircioglu, 1999). There is agreement that
when the item method is used, the TBR-TBF difference stems
from selective rehearsal of the TBR items (Basden et al., 1993;
Johnson, 1994; MacLeod, 1999).
Experiment 1
In Experiment 1, we explored how the magnitude and the
accuracy of FOK judgments may change as a function of a
directed-forgetting procedure, a manipulation that has been shown
to consistently increase the memorability of items.
1487
1488
TEKCAN AND AKTORK
Method
Participants. A total of 80 Bogazici University students participated in
the experiment for extra credit in their introductory psychology course.
Data from 12 participants were eliminated because the gamma correlation,
used to gauge FOK accuracy, could not be calculated for these data. The
data reported here are based on 68 participants (43 women and 25 men).
The mean age of the participants was 19.9 years (SD = 1.49).
Design and procedure. The independent variable (instructions to for-
get or remember) was manipulated as a within-subject variable. Partici-
pants were tested individually. The instructions followed those of MacLeod
(1989). Each participant was presented with 60 word pairs followed by
instructions to either remember or forget a specific word pair. Participants
were told that the letter R indicated that they would have to remember that
word and the letter F indicated that they could forget that word because it
would not be on the test. They were given a practice session with a word
list consisting of six pairs.
In the experiment, participants were shown each word pair for 1 s and
then received instructions (either to remember or forget) for 3 s. In the test
phase, participants were asked to remember the target word for each cue
word regardless of the initial instructions. For each target word they failed
to remember, they were asked to make a FOK judgment by thinking about
the following question: "Would you be able to find the correct word if the
first two letters of the target word were provided?" They were asked to
indicate their FOK judgment on a scale from 1 to 6, where 1 indicated
certainly no and 6 indicated certainly yes. After participants completed the
cued-recall task and FOK judgments, they were given the cued stem-
completion task, in which all the cue words as well as the first two letters
of each target word were given. Participants were asked to remember as
many of the target words as possible. Participants worked at their own pace
during the initial cued-recall and cued stem-completion tasks.
Materials. A total of 120 words were selected from a word-frequency
database in Turkish, constructed by basically following die KuJera-Francis
(1967) methodology (Goz, in press). All were high-frequency nouns (100
or more per million) with a length of five or six letters. Sixty word pairs
were formed by randomly matching the words. Each word pair was typed
in 18-point uppercase letters on 7- by 12-cm index cards. Separate cards
were prepared for the instructions (Remember or Forget), and Turkish
equivalents of the letters FFFFF and RRRRR were typed on cards.
Two more sets of cards were prepared, one for the cued-recall test and
one for the cued stem-completion test. Each card for the cued-recall test
had the cue word followed by a dash. The cards for the stem-completion
task were the same except that after the dash, the first two letters of the
target word were also provided. The numbers of letters of the second word
which were missing were shown with lines. The orders of the word pairs
were different in the three sets.
The study list consisted of 60 word pairs. The first and the last four of
the pairs served as buffers. Instructions to remember and forget were
randomly assigned to the remaining 52 word pairs, leading to 26 TBR
and 26 TBF word pairs. The instruction status of the words was counter-
balanced so that each word pair was TBR for half of the participants and
TBF for the remaining half. Buffer word pairs were always followed by
instructions to remember.
Results
Two experimenters carried out data collection. Since the pre-
sentation during the study was done manually, we checked
whether there were any experimenter effects. There was no dif-
ference between the data collected by these two experimenters on
any of the memory or metamemory measures.
The means and standard errors of the means for different mea-
sures are given in Table 1.
Table 1
Memory and Feeling-of-Knowing (FOK) Measures for
To-Be-Remembered (TBR) and To-Be-Forgotten (TBF)
Words in Experiments 1 and 2
Parameter
Recall
Median FOK
Overall stem completion
Conditional stem completion
Gamma
Recall
Median FOK
Overall recognition
Conditional recognition
Gamma
M
Experiment
.38
2.93
.51
.25
.21
Experiment
.32
3.30
.77
.67
.19
Type
TBR
SEM
1
.03
0.18
.02
.02
.07
2
.02
0.19
.02
.02
.07
of words
M
.10
2.09
.22
.17
.21
.07
2.33
.59
.57
.23
TBF
SEM
.01
0.14
.02
.01
.06
.02
0.16
.02
.02
.05
Note. Overall stem completion and overall recognition refer to perfor-
mance in the criterion test for all items, whereas conditional stem comple-
tion and conditional recognition refer to performance for nonrecalled items
in the initial recall test. Recall, stem completion, and recognition are
measured in proportions. FOK is measured on a 6-point scale. Gamma
correlation can take on values between - 1 and 1.
Recall. A clear directed-forgetting effect was observed on
recall performance; participants remembered more TBR words
than TBF words, ?(67) = 13.30, p < .0001.
FOK ratings. Median FOK ratings for each participant were
calculated for the TBR and TBF items separately. Overall, partic-
ipants gave higher FOK ratings for TBR words than for TBF
words, t(61) = 6.28, p < .0001. It is also worth noting that median
FOK ratings for both types of items were moderate (2.09 and 2.93
on a 6-point scale).
Cued stem-completion test. Participants were provided with
each cue word as well as the first two letters of the corresponding
target word. The effect of encoding instructions on the criterion
performance was analyzed in two ways. First, we looked at overall
stem-completion performance regardless of whether the words
were recalled or not in the initial cued-recall test. There was a clear
effect of directed forgetting; a higher proportion of TBR words
than of TBF words was completed, ;(67) = 13.13, p < .001,
reflecting a difference closely resembling that in the initial recall
test. Second, when we looked at stem-completion performance
conditional upon nonrecall in the initial task, a higher proportion of
TBR words than of TBF words again was completed successfully,
f(67) = 4.36, p < .001.
FOK accuracy. We used Goodman and Kruskal's gamma as
the measure of FOK accuracy (Nelson, 1984; see also Nelson,
1996, and Wright, 1996, in response to Schraw, 1995). The gamma
values for TBR and TBF words were virtually identical,
f(67) = 0.01, p > .10. In addition, although both of the gamma
correlations were above chance in predicting criterion perfor-
mance, 1(67) = 2.95 for TBF words and 2.81 for TBR words, p <
.01 for both, the gamma values were quite low for both types of
FEELING OF KNOWING IN DIRECTED FORGETTING
1489
items, as is generally the case in paired-associate learning (e.g.,
Lupker et al., 1991; Schwartz & Metcalfe, 1992).
Experiment 2
The lack of a difference in FOK accuracy between TBR and
TBF words in Experiment 1 might have been partly attributable to
the nature of the criterion test used. To address this possibility, we
replicated Experiment 1 by using a recognition test as the criterion
task.
Method
Participants. A total of 73 Bogazici University students participated in
the experiment for extra credit in their introductory psychology course.
Data from 7 participants were eliminated because the gamma correlation
could not be calculated for these data. The mean age of the remaining 66
participants (47 women and 19 men) was 19.58 years (SD = 1.30).
Materials, procedure, and design. The 60 word pairs used in Experi-
ment 1 were used in this experiment as well. The only change was that the
criterion task in this experiment was a forced-choice recognition test with
five alternatives comprising the correct target, two semantically related
lures, and two phonologically related lures.
Results
The means and standard errors of the means for different mea-
sures are given in Table 1.
Recall. There was a reliable directed-forgetting effect; partic-
ipants recalled a higher proportion of TBR words than of TBF
words, f(65) = 10.89, p < .001.
FOK ratings. Participants gave higher FOK ratings for unre-
called TBR words than for unrecalled TBF words. This difference
was significant, f(65) = 4.91, p < .001.
Recognition test. Overall, more TBR words than TBF words
were recognized correctly, f(65) = 10.08, p < .001. When the
same analysis was carried out conditionalized upon nonrecall in
the initial recall test, a higher proportion of TBR words than of
TBF words again was recognized, t{65) = 4.50, p < .001.
FOK accuracy. There was no difference between the gamma
correlations for TBR words and TBF words, f (65) = 0.43, p > . 10.
Although gamma values for both types of words were again low,
they were above chance in predicting recognition performance,
K65) = 4.36 for TBR items and 2.66 for TBF items, p < .01 for
both.
General Discussion
We showed that directed-forgetting manipulation influenced
the intensity but not the accuracy of FOK judgments. The
finding that an initial encoding manipulation affected the mag-
nitude of FOK is consistent with previous literature (e.g., Lup-
ker et al., 1991; Nelson et al., 1982) as well as with the two
frameworks attempting to account for how FOK judgments are
made. The accessibility account (Koriat, 1993), which suggests
that FOK judgments are based on the amount of partial infor-
mation and speed with which that information comes to mind,
would predict higher FOK judgments for TBR items than for
TBF items because more partial information would become
available for better-learned information. The cue-familiarity
account (e.g., Reder & Ritter, 1992; Schwartz & Metcalfe,
1992), which suggests that FOK judgments are based on how
familiar the cue is, would predict the same pattern because the
TBR cue words would be more familiar than the TBF cue words
as a result of the encoding manipulation.
The main factor for differences in FOK accuracy as a function
of encoding manipulations in some previous studies was that the
accuracy was virtually nil for less-well-encoded items and above
chance for better-encoded items (Carroll and Nelson, 1993; Lupker
et al., 1991; Nelson et al., 1982). Our results, along with those of
Schacter (1983) and Schacter and Worling (1985), do not support
this pattern. One potential reason may be that encoding manipu-
lations need to have a large effect on initial memory performance
for FOK accuracy to be affected (Koriat, 1993; Lupker et al.,
1991). Although this reason is a potential factor, our findings
appear to eliminate this explanation because the directed-
forgetting instructions had a strong effect on initial cued-recall as
well as criterion performance tests. It is obviously still possible
that the TBR-TBF difference in our study did not exceed some
unidentified threshold. However, the amount of difference in var-
ious memory measures in this study seems similar to those ob-
tained in studies that reported differences in FOK accuracy (e.g.,
Lupker et al., 1991).
References
Basden, B. H., Basden, D. R., & Gargano, G. J. (1993). Directed forgetting
in implicit and explicit memory tests: A comparison of methods. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 19,
603-616.
Bjork, R. A., & Woodward, A. E., Jr. (1973). Directed forgetting of
individual words in free recall. Journal of Experimental Psychology, 99,
22-27.
Carroll, M., & Nelson, T. O. (1993). Effect of overleaming on the feeling
of knowing is more detectable in within-subject than in between-subject
designs. American Journal of Psychology, 105, 227-235.
GOz, 1. (in press). Yazih Turkge'nin kelime sikhgi sozliigu [Word frequency
dictionary of written Turkish]. Ankara, Turkey: Turk Dil Kurumu.
Johnson, H. M. (1994). Processes of successful intentional forgetting.
Psychological Bulletin, 116, 274-292.
Koriat, A. (1993). How do we know that we know? The accessibility model
of the feeling of knowing. Psychological Review, 100, 609639.
Kuera, H., & Francis, W. N. (1967). Computational analysis of present-
day American English. Providence, RI: Brown University Press.
Lupker, S. J., Harbluk, J. L., & Patrick, A. S. (1991). Memory for things
forgotten. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 17, 897-907.
MacLeod, C. M. (1989). Directed forgetting affects both direct and indirect
tests of memory. Journal of Experimental Psychology: Learning, Mem-
ory, and Cognition, 15, 13-21.
MacLeod, C. M. (1999). The item and list methods of directed forgetting:
Test differences and the role of demand characteristics. Psychonomic
Bulletin and Review, 6, 123-129.
Mungan, E., & Peynircioglu, Z. F. (1999). Unutmaya yonlendirmenin
bellek iizerinde niteliksel etkileri [Qualitative effects of directed forget-
ting on memory]. Turk Psikoloji Dergisi, 14, 1-12.
Nelson, T. O. (1984). A comparison of current measures of the accuracy of
feeling-of-knowing predictions. Psychological Bulletin, 95, 109133.
Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting
performance on one item relative to another item, not of the absolute
performance on an individual item: Comment on Schraw (1995). Ap-
plied Cognitive Psychology, 10, 257-260.
Nelson, T. O., Leonesio, R. J., Shimamura, A. P., Landwehr, R. F., &
1490
TEKCAN AND AKTURK
Narens, L. (1982). Overlearning and the feeling of knowing. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 8, 279-
288.
Reder, L. M., & Ritter, F. E. (1992). What determines initial feeling of
knowing? Familiarity with question terms, not with the answer. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 18,
435-451.
Schacter, D. L. (1983). Feeling of knowing in episodic memory. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 9, 39-54.
Schacter, D. L., & Worling, J. R. (1985). Attribute information and the
feeling-of-knowing. Canadian Journal of Psychology, 39, 467475.
Schraw, G. (1995). Measures of feeling-of-knowing accuracy: A new look
at an old problem. Applied Cognitive Psychology, 9, 321-332.
Schwartz, B. L., & Metcalfe, J. (1992). Cue familiarity but not target
retrievability enhances feeling-of-knowing judgments. Journal of Exper-
imental Psychology: Learning, Memory, and Cognition, 18, 10741083.
Wright, D. B. (1996). Measuring feeling of knowing: Comment on Schraw
(1995). Applied Cognitive Psychology, 10, 261-268.
Received February 23, 2001
Revision received April 6, 2001
Accepted May 21, 2001