Kuhn (1993) - Informal Reasoning

Connecting Scientific and Informal Reasoning
Author(s): Deanna Kuhn

Source: Merrill-Palmer Quarterly (1982-), Vol. 39, No. 1, Invitational Issue: The Development
of Rationality and Critical Thinking (January 1993), pp. 74-103
Published by: Wayne State University Press
Stable URL: http://www.jstor.org/stable/23087301 .
Accessed: 06/11/2014 12:08
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
Wayne State University Press is collaborating with JSTOR to digitize, preserve and extend access to Merrill-
Palmer Quarterly (1982-).
http://www.jstor.org
This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

All use subject to JSTOR Terms and Conditions
Merrill-Palmer Quarterly, Vol. 39, No. I
Connecting Scientific and Informal Reasoning
Deanna Kuhn
Columbia University
Current research on both informal reasoning and scientific reasoning ought to bear
directly on efforts to understand and promote critical thinking, and yet these two
bodies of work have proceeded largely independently of one another and neither has
been linked to the older, more extensive literature on critical thinking. A way is
offered here of conceptualizing and examining informal reasoning that links it to
scientific reasoning, and makes it clear how both are central to critical thinking. Both
informal and scientific reasoning entail the evidence-based justification of beliefs, or
the coordination of theory and evidence. Foundational abilities that lie at the heart of
both types of reasoning are the ability to recognize the possible falsehood of a theory,
and the identification of evidence capable of disconfirming it. Identification of ex
tended precursors to these abilities suggests that their development is both gradual
and multifaceted.
What is critical thinking and why is it important? Answers to these

questions, I have claimed on a number of occasions (Kuhn, 1986, 1988,
1990a, 1990b), need to be informed by an empirical data base regarding
the kinds of reasoning (both valid and faulty) that children and adults, in
fact, engage in during the course of their own thinking. The new and
rapidly growing research area of informal reasoning (Voss, Perkins, &

Segal, 1991) holds the promise of providing this crucial data base. To
date, however, few links have been established between this new litera
ture and the older, more extensive body of literature dealing with critical
thinking.
Scientific reasoning (Klahr & Dunbar, 1988; Klahr, Fay, & Dunbar,
1991; Kuhn, Amsel, & O'Loughlin, 1988; Schäuble, 1990; Schäuble,
Klopfer, & Raghavan, 1991 ) is another growing research area that appears
to have much to do with critical thinking, but, again, few links are evi
dent. In this article, a way of conceptualizing and examining informal
reasoning is offered that enables identification of connections between it
Correspondence may be sent to Deanna Kuhn, Department of Psychology, Teachers

College, Columbia University, New York, NY 10027.
Merrill-Palmer Quarterly, January 1993, Vol. 39, No. 1, pp. 74-103.
Copyright © 1993 by Wayne State University Press, Detroit, Michigan 48202
74

Reasoning 75
and more traditional scientific reasoning, and, in turn, clarifies how both
are central to critical thinking.
INFORMAL REASONING AS ARGUMENT
Researchers have become hesitant to judge people's thinking abili

ties by their performance on problems that a psychologist conceives of
and presents to them. Instead, it increasingly is claimed that we should
seek to identify the problems that arise in people's own lives and the kinds
of thinking they have developed to deal with them. Those who have
undertaken the empirical study of so-called "practical," or real-world
intelligence have thus far focused largely on examining the thinking that
people do in their work lives (Perlmutter, Kaplan, & Nyquist, 1990; Scrib
ner, 1986; Sternberg & Wagner, 1986; Wagner & Sternberg, 1985,
1986). This has been a quite sensible starting point. Work is a major,
often challenging part of most people's lives, and hence one in which
people's best, most complex thinking is likely to be observed. Yet, for
many, perhaps most, people, work may not be the arena in which the
ability to think well ends up mattering the most. In this section, I describe
the investigation of a nonwork-related kind of real-world thinking. It is a
kind of thinking, I claim, that plays a fundamental role in people's lives
and lies at the heart of what we mean, or should mean, by real-world
intelligence.
It is social, rather than cognitive, psychologists who have come clos
est to this kind of thinking, in their long ongoing study of attitudes and
opinions. Traditionally, however, attitudes have been conceived of sim
ply as points on a unidimensional continuum, with research attention
focused on the variables that influence movement in one direction or the
other on that continuum. This focus, in fact, parallels the conception that
surrounds the whole opinion survey and poll-taking enterprise in our
culture more broadly. Is one for or against abortion? Is the President doing
a good job? Social psychologists now know that there is much more to an
attitude than a point on a continuum, and yet, as any politician trying to
get elected is aware, the point-on-a-continuum conception continues to

dominate our culture. It is a paradox that as a society we expend as much
money and effortas we do to assess exactly what people think on a whole
range of social issues and devote so little to understanding why they think
that way—what the thinking is that underlies their professed opinions.
Implicit in all of the opinion surveying is the assumption that the views
people hold are reasoned views—that a process of weighing pros and
cons, or positive and negative evidence, has at least at some level of

76 MERRILL-PALMER QUARTERLY
consciousness gone into an opinion for or against abortion or gun control.
Otherwise, the opinions that people profess, and that we undertake to
assess with such precision, seem scarcely worthy of the attention. And yet
we have little real evidence to tell us that this is the case.
To what extent, then, does a process of rational argument underlie the
beliefs people hold and the opinions they espouse? It is this very basic sense
of thinking well that I have addressed in research on argumentive reasoning
about everyday issues (Kuhn, 1991 ). It is in argument that we are likely to
find the most significant way in which higher-order thinking and reasoning
figure in the lives of average people. Thinking as argument is implicated in
all of the beliefs people hold, the judgments they make, and the conclu
sions they come to. Itarises every time a significant decision must be made.
Hence, it is at the heart of what we should be interested in and concerned
about in examining people's thinking. I will not undertake to debate here
the historical reasons that psychologists have paid relatively little attention
to such an important kind of thinking, tending instead to approach thinking
largely as problem solving. I will simply note, using the psychologist's
terminology, that thinking of the sort I am referring to is open-ended, ill
structured, and deeply embedded in a rich, complex knowledge base. As
such, it threatens to be intractable. A major goal of our research has been to
develop a way of conceiving of such thinking that makes it more approach

able, namely, thinking as argument.
Rhetorical and Dialogic Argument
To examine thinking as argument, a distinction must be established

first, and then an identity, between two kinds of arguments. The dictio
nary definition of an argument in the first sense is "a course of reasoning
aimed at demonstrating the truth or falsehood of something." An argu

ment in this sense is referred to here as a rhetorical argument. More
commonly, an argument is thought of in its second sense, as a dialogue

between two people who hold opposing views. Each offers justification
for his or her own view, and, at least in a skilled argument, each attempts
to rebut the other's view by means of counterargument. An argument in
this sense is referred to here as a dialogic argument.

Although connections are rarely made between these two kinds of
arguments, they in fact bear a close relationship to one another in terms of
certain fundamental cognitive skills entailed in both. In a dialogic argu

ment, at a minimum one must recognize an opposition between two
assertions: that, on surface appearance at least, both are not correct. One
must then relate supporting and refuting evidence to each of the asser
tions, and, ideally, if the argument is to move toward resolution, one

Reasoning 77
must be able to weigh supporting and refuting evidence in an integrative

evaluation of the relative merit of the opposing views.
What is less often noted is that these same skills are entailed, in fact,
in a rhetorical argument in more implicit form, although the rhetorical
argument may appear on the surface to be less complex. An argument in
support of an assertion is an empty, indeed superfluous, argument unless
one can conceive of an alternative to what is being asserted (i.e., an
opposing assertion). Once two or more contrasting assertions are in
place, cognitively speaking, the further challenge poses itself of relat

ing evidence to them. Presumably, it is a weighing of positive and nega
tive evidence that has led one to espouse the favored assertion over its
alternatives. Indeed, it is just such a weighing process that is implicit

when one speaks of a reasoned argument. Thus, any reasoned argument
in support of an assertion implicitly contains a full dialogic argument.
The identity between rhetorical and dialogic arguments provides a
framework for exploring the nature of the less externally observally rhetori
cal argument. Are the elements of the dialogic argument evident when we
probe the thinking underlying people's beliefs and opinions? And is the
presence or absence of these elements revealing of the quality of people's
thinking?
Design of the Investigation
To investigate these questions, we asked people their opinions, or,

more formally, their causal theories, on three topics: (1) What causes
prisoners to return to a life of crime after they are released? (2) What
causes children to fail in school? (3) What causes unemployment? These
topics were chosen as ones that people have occasion to think and talk
about. They are also ones about which people are able and willing to
make causal inferences without a large base of technical knowledge.
They nevertheless involve phenomena of which the true causal structure
is complex and uncertain.
Following the framework of the dialogic argument, subjects were
asked for evidence to support their theories and then probed regarding
alternative theories, counterarguments, and rebuttals. A number of ques
tions also were asked that addressed subjects' epistemologica! reflections

on their own thinking: For example, about certainty ("How sure are you
about what causes ?"); alternative viewpoints ("Could more than
one point of view be right about this?"); and expertise ("Do experts know
for sure what causes ?"). At a second session, we presented some
evidence of our own that related to two of the topics and asked subjects to
interpret it.

The 160 subjects who participated in this research were selected to
represent average people across the life span, beginning with adolescents
(ninth graders), and including young adults in their 20s, middle adults in
their 40s, and older adults in their 60s. Within each age group, as well as
among males and females, two different education levels were included,
generally those who had high school versus at least some college educa
tion (these differences were prospective among the adolescent group).
We also included a group of experts of three different types: experienced
parole officers, regarded as having domain expertise in the return to crime
topic; experienced teachers, regarded as having domain expertise in the
school failure topic; and philosophers (specifically, PhD candidates work

ing on their dissertations in the Columbia University philosophy depart
ment), who were regarded as having expertise in reasoning itself.
Evidence
Although we will not dwell on the theories themselves, one impor

tant note about them is that people tended to hold these theories with
certainty. From one half to three quarters (across topics) claimed that they
were sure or very sure that their theories were correct. So, in eliciting their
theories, people were asked questions to which they felt they knew the
answers. Rather than the theories themselves, our interest was in the
arguments that people offer to support their theories. What kind of evi
dence could we expect lay people with no special knowledge or interest
in these topics to offer?

In fact, for each topic, roughly 40% (averaged across topics) of the
subjects offered what we classified as genuine evidence. Such genuine
evidence is, by no means, evidence that is conclusive, nor even compel
ling, nor even necessarily very convincing evidence. Rather, it is simply
evidence that is (a) differentiated from the theory, which we will see is an
important criterion, and (b) bears on its correctness. In Table 1 are summa
rized the different kinds of genuine evidence observed, with excerpts from
the protocols to illustrate what these arguments are like in the subjects' own
language. About half refer to covariation (i.e., variation in the alleged

cause corresponds to variation in the outcome), but some other kinds of
reasoning familiar from the experimental literature on reasoning appear as

well, such as counterfactual reasoning, discounting, and analogy.
More important to focus on here are the remaining 60% of re
sponses, roughly, that were classified as unsuccessful, beginning with

those in a category we called nonevidence. Nonevidence takes a number
of forms, but the most striking is that in which the phenomenon itself is
taken as sufficient evidence that it is produced by the cause which the

Reasoning 79
Table 1. Genuine Evidence
1. Covariation evidence
la. Correspondence. Evi (20Cms) [family problems] Well, if someone makes
dence notes simply a cor a study of cases of students where failures, drop
respondence between an outs . . . students who drop out of school . . . and
tecedent and outcome. sees where they have family problems, perhaps
that would be solid evidence to prove what I be
lieve.
1 b. Covariation. The idea of (60Cfs) llack of parental support] Let's say the read
covariation becomes ex ing scores that are being published right now, and
plicit, and the evidence it somehow has a reflection on the [geographical]
incorporates both com area they are in. The good ones are being in areas
parison and quantifica where parents take an interest, and the bad ones
tion. appear where there are many single-parent fami
lies, where perhaps they don't even have a home.
(TCmc) [unemployment] You could probably take
a survey and find out the percentage of people who
get jobs who have been convicts. I'm sure it's very
low.
1c. Correlated change. (TCfs) [drugs] I guess people who have drugs and
Change in the outcome might not be doing so great in school, and so if
co-occurs with change in they knew they took drugs they could see what hap
the antecedent. pened when they weren't on drugs and, you know,
prove it or whatever.
2. Evidence external to the causal
sequence itself
2a. Positive (40Nfc) (bad environment] It is not uncommon to
hear that when someone is arrested that they have
had previous charges in the same place for a simi
lar crime. [What does this show?] I think it does
show that it is in fact true that it's the environment.
2b. Negative (counterfactual) (40Nmc) Iprison's failure to rehabilitate] Their
habit patterns. All outward appearances in every
case that I have known or even heard of has not
changed. If they had had their head properly
shrunk, everything about them would have
changed, from their haircut right on through.
3. Indirect evidence
3a. Analogy (particular to par (20Cmc) [return to a familiar way of life] I think if
ticular) you look at it in terms of, well, an occupation,
whether crime could be an occupation or not . . .
most people generally stay in an occupation their
whole life, and it's very hard for them to change.
3b. Assumption (general to (20Cmc) [desire for material rewards] People that
particular) don't have things . . . I mean, everybody wants
things, especially in this country. This isa consumer
society. People wantto accumulate things, and
that's not true of everywhere. People just want basic
things, and ifyou don't have them, and you don't
know how to get them, then you can take them.

MERRILL-PALMER QUARTERLY
Table 1. Continued
3c. Discounting (elimination (40Nfc) [unemployment] Well, I don't believe that

of alternatives) anyone is just born that way . . . that they like
crime, that they like going to prison. So if they
could get a good job, I don't believe anyone would
turn to crime.
3d. Partial discounting (dis (40Cfs) [family background] The background of the
counting of alternative parent is something important, because you find
factor at one level or end some very intelligent children with IQs of 140. And
of its range of operation yet they have no family with background. You put
only) them in a very expensive school, they cannot sus
tain themselves.
Note. Symbols in parentheses refer to subject's age group (T, 20, 40, or 60), education
level (N, Noncollege; C, College), sex (m or f), and topic (s, school; c, crime). The sub
ject's theory is indicated in brackets.
subject advocates. For example, Joe, a subject in his 20s, for the school
failure topic indicates poor nutrition as the cause. His response to our
question of "What evidence would you give to show this?" was: "The
points that they get in school. The grades that they get in school
show. . . . [What would that showP] That they are lacking something in
their body. That the kids who were failing lack something in their body."
So, in other words, the very fact of the outcome presumes the cause.
Joe's response is clearly an example of nonevidence. Consider an

other response that is somewhat harder to classify. Kathryn, a teenager,
offers the theory that prisoners return to crime because they prefer life in
prison.
[How do you know that this is the cause?] I think because if they commit
crime they're getting attention. They'll be—the prisoner, you know, in
prison—they'll be taken care of, they'll be given food and all this, and they get
attention. They come out and everybody, you know ... he was a prisoner, so
that they stay away from him; they're scared and everything. So they decide
that the only way they're going to be, that they'll have attention, or they're
going to be cared for, is if they're in prison.
[If you were trying to convince someone else that your view is right, what
evidence would you give to try to show this?] The evidence I would give is that
when they are in prison, they're secure. They're sure that, you know, no one's
going to hurt them. Well, they're not sure no one's going to hurt them, but,
you know, they know they're secure. They've got a place to eat, a place to
sleep. But if they come out into the world and, you know, with unemployment
and people not wanting to take anybody in that was an ex-con or something,
when people reject them because of their past, they're sure to go back into the
crime just to go back to their jail cell and stay in it.

Reasoning 81
[Can you be very specific, and tell me some particular facts you could
mention to try to convince the person?] Well, some facts could be that when
they're in there, they'd want to make friends with other cons and stuff like that,
and when they're out here, they've got to start all over again, and it's real hard
for people who committed a mistake, for other people to accept that they've
paid for it and everything. And then when they're out here people reject them
and they look at them, you know, like they're scared of them. They don't want
to stay in the world if they think everywhere they go people are going to be
looking at them and feeling, you know, real insecure when they're around. So
they'd rather be where people, you know, they're all the same.
How should Kathryn's response be characterized? Is this evidence

that she is offering for her theory? We classified it as pseudoevidence,
which we define simply as a scenario, or script, depicting how the phe
nomenon might occur. The defining characteristic that distinguishes
pseudoevidence from genuine evidence is that pseudoevidence cannot
be sharply differentiated from the theory itself. Hence, responses to
"What causes X?" do not differ sharply from responses to "How do you
know that this is the cause?" or "What evidence can you give to show that
this is the cause?" In Kathryn's case, she makes an intuitively convincing
case for the plausibility of the cause she specified leading to the outcome,
yet without providing any genuine evidence that this cause is in operation
in instances of the phenomenon. Her own words establish that, for her,
the function of evidence is to establish such plausibility: "The evidence I
would give is that when they are in prison they're secure." This evidence
does not establish that "preference for prison life" is the cause of the
phenomenon; rather, it enhances its plausibility as a possible cause.
At its most minimal, pseudoevidence simply illustrates the causal
sequence. At its best, it enhances the plausibility of the causal sequence,

as Kathryn's pseudoevidence does. In establishing plausibility, pseudo
evidence scripts bear on causal mechanism, and there is much in the
causal reasoning literature to suggest mechanism as a perfectly appropri
ate means of causal explanation (Amsel, Langer, & Loutzenhiser, 1991;
Antaki, 1988; Hilton, 1988). Might not pseudoevidence and genuine
evidence be regarded simply, then, as alternative explanatory styles?
The difference is a much more important one than that of style. In the
causal reasoning and attribution literature, subjects are asked to identify
the causal factor in a specific past instance, making the criterion of plausi
bility a relevant one: Could this cause have produced the outcome? In the
present context, in contrast, subjects are asked to justify the assertion that,
in general, X is the cause of outcome Y. In this context, genuine evidence
is superior to pseudoevidence, on the grounds that it is more definitive.

What is the basis for this claim? First, plausibility is neither a necessary
nor a sufficient condition for the correctness of a causal theory. Often in
the history of science, causal theories that initially appear very implausi
ble later have been proven correct (Einhorn & Hogarth, 1986). Likewise,
highly plausible theories have been disconfirmed. Furthermore, a causal
relation between two factors can be demonstrated in the absence of any
plausible theory connecting them as, for example, when a substance is
found beneficial in treating a disease in the absence of an understanding

of how it achieves its effect.
"Good" pseudoevidence, then, might heighten our interest in testing
a causal theory (by enhancing plausibility), but it cannot tell us whether
the theory is correct. In fact, because pseudoevidence can never conflict
with a theory, it cannot really be considered evidence at all. Instead, it
should be regarded as part of the theory itself. In proposing their theories,
it is reasonable to assume that all the subjects envisioned some mecha
nism whereby the alleged cause produces its effects. When, in offering
pseudoevidence, they elaborate their description of this mechanism, they
are elaborating the theory, not providing evidence that bears on its cor
rectness. Again, even the most plausible theories can be wrong.
Alternative Theories
Can subjects who offer only pseudoevidence envision the possibility

that this is not what happens? This is the next question explored by asking
subjects to generate an alternative theory. In response to the question ("A
person whose view is different from yours—what might they say is the
cause?"), some subjects generate an alternative theory without difficulty.
Others generate what appears to be an alternative but then immediately
agree with it ("That could be part of it, too"), in effect incorporating the
alternative cause into their own theory. Such subjects are unable to con
ceive of anything that is not a cause. Other subjects try unsuccessfully to
generate an alternative, producing something very like their own theory.

Still other subjects simply decline: "I don't know what someone else
would say. I have no idea"; or, "I don't know what they would say. I'd
really have to get someone else's point of view. 'Cause I imagine my
thoughts run in this direction and that's about it"; or, "I don't know what
they might say is the reason. I don't think I'm wrong"; or, significantly,
the hypothetical other's view is simply assimilated to one's own, as in "I
think they'll say the same thing I'd say. I think that the majority think the
way I do."
The percentage of subjects who are able to generate alternative theo
ries averaged across topics is about 60%, higher than the 40% who

Reasoning 83
generate genuine evidence. But, importantly, there is a significant associa

tion between the two, one that makes clearer the meaning of pseudo
evidence. In not generating alternatives, those subjects who rely on
pseudoevidence do not call upon this pseudoevidence to perform the
very function it cannot, that is, to address the correctness of a theory,
relative to all the others with which (if the subject conceived of the
possibility) it could compete.
Thus, subjects who generate neither genuine evidence nor alterna
tive theories take their theories for granted, simply as statements about the
way the world is. The theories are not reflected on as objects of cognition,
that is, as claims needing to be evaluated in the light of alternatives, as
well as evidence.
Counterarguments
To evaluate a theory against alternatives implies that it could be true

or false (i.e., indicates an acceptance of its falsifiability). It is the critical
issue of falsifiability, or disconfirmation, that requires the study of coun
terarguments ("What could someone say to show that you were wrong?").
Do subjects comprehend the evidence that would falsify their theory,
were they to encounter it? The success rate here is about 50% (see Table 2
for a summary of the various forms of successful counterarguments), and
again we focus on unsuccessful responses.

The most common reason that attempted counterarguments are un
successful is that they leave the original causal theory in place: They do
not constitute an argument against it. Dean, for example, theorizes that
lack of parental support is the cause of school failure. He offers the
following counterargument to this theory:
Well, the parents can use a lot of arguments that I am wrong. They would say
they try their best under the circumstances naturally because they claim first
comes to make a living, to sustain their house, and taking care of the children.
But maybe they haven't got the opportunity to give them as much as they
would like to give them. But that's a lot of cause of failure a lot of times. [What
evidence might this person give to try to show that you were wrong?] The
evidence would be that they try their best. They try to do everything that is
possible but sometimes it is just impossible to do it.
Clearly, Dean is addressing the causes of his alleged cause, not offering
an argument against its causal status. Possibly, subjects like Dean take the
correctness of their theory for granted to the extent that they can only
envision questioning why the causal antecedent exists, not whether it
exists or plays a causal role.

Table 2. Successful Counterarguments
1. Noncovariation arguments
la. Arguments against causal (60Nms) [lack oí family support] He could possibly
sufficiency. Antecedent is point out to me that here is a kid who succeeded
present and outcome fails while the family was let's say drunkards or sepa
to occur. rated and there was no inspiration at home.
lb. Arguments against causal (40Cms) Ilack of family support] I think they would
necessity. Antecedent is have to prove it with hard fact, that would show
absent and outcome still that not just one child but a large cross section of
occurs. the population was failing and, you know, the par
ents were very heavily involved.
1c. Arguments against causal (40Cms) Ifamily problems] They could point out ex
sufficiency and necessity amples, specific examples that particular children
failed and their family situation was very . . . what
we consider a strong, stable family situation. They
could also conversely point out a child doing well
but yet the family is fragmented.
2. Discounting arguments
2a. Full discounting. Exis (60Cms) (poor educational system] The fact that
tence of the antecedent is they are really consistently introducing and carry
denied. ing out a practical program, an intelligent program,
a progressive program in the schools.
2b. Partial discounting. The (TNms) [negat/ve attitude toward school] He would
antecedent is denied for show me some other students who have these . . .
some subset of cases. have good marks, and ask them why do you study
hard. And they will say because the school is
good.
Note. See Table 1 note for abbreviations key.
Other subjects claim to be open to the possibility of counterargu

ments but are unable to envision any. Bill provides an example:
Well, they would have to come up with a different viewpoint on the whole
subject. I don't know what they could say. [Could someone prove that you
were wrong?] Oh, sure. I'm flexible. [Is there any fact or evidence which, if it
were true, would show your view to be wrong?] Sure. [What would it be?] I
have no idea.
Some of those subjects who do attempt a counterargument simply

offer an alternative theory as a counterargument (e.g., "They would say
it's not the parents, it's the school that causes kids to fail"). Such coun
terarguments, of course, have the very serious deficiency of leaving the
original theory unexamined.
The most significant form of unsuccessful counterargument does
address the theory but does not focus on the evidence that would be

Reasoning 85
critical in evaluating it. The following subject, Harriett, for example,

believes that stress is the cause of return to crime. She offers the follow
ing counterargument:
I have no [what they could

idea say]. I don't know how they could say that
there is no
stress [here] in New York. I can't imagine. Even sitting on the
bus. . . . Why isn't the bus moving faster? I'm looking at my watch and saying
this man is sitting in the office waiting for me, and this was stress. It was all
built up in me. [What evidence might this person give to try to show that you
were wrong?] They couldn't, unless they live in a dream world. [Could some
one prove that you were wrong?] If they can prove it, I'd love to know it. No, I
don't think so.
Harriett makes an inferential error like that found in much of the experi
mental reasoning literature (see Baron, 1988, for review), reasoning
based on positive cases (of antecedent and outcome). It is, of course, the
absence of negative cases (stress without return to crime or return to crime
without stress), not the presence of positive ones, that are critical in
establishing support for a theory.

Despite the critical role that falsifying cases play in examining a
theory, our subjects often not only fail to consider them, like Harriett, but
show considerable resistance to the very idea. As one of them put it,
rather plaintively, "If I knew the evidence that I'm wrong, I wouldn't say
what I'm saying."
Other subjects are even more adamant that there can be no coun
terarguments to their theories. Their responses are especially interesting

as they offer some sense—better, in fact, than that provided by the sup
porting evidence itself—of how subjects regard their own theories, by
suggesting exactly what about the theory a subject sees as uncontestable.
Marilyn provides a good example. Her theory centers on family problems

as the cause of school failure. In response to the request for a coun
terargument, she says:
They will never prove me wrong. I stand firm. I am a parent. I have two children,
and they're not going to prove me wrong. [What evidence might this person give
to try to show that you were wrong?] I'm not really interested. All I have are the
results that I put into it, and it's worked thus far, and I'm not changing my stance.
And they're not going to prove anything to me, because if they do, to me they're
just very narrow people who want things just exactly the way they want. It's not
going to work out that way. [Could someone prove that you were wrong?] No,
they're not going to prove a darn thing to me. This is my career, [/s there any fact
or evidence which, if it were true, would show your view to be wrong?] No. I'm
a parent. I've lived it. Absolutely. My experiences may not coincide with some
body else's. Maybe their lifewasa lot easier. So they will be speaking from their
experience. But don't step on my turf.

Marilyn exhibits an ownership over her theory that undermines its inde
pendent existence, rendering it uncontestable. In such cases, the argu

ment or point of view "belongs" to the subject, and, as such, is not
available for independent examination. To challenge the theory is to

challenge the subject's own self.
Rebuttals
Rebuttals ("What could you say to show that the other person was
wrong?") are critical because they complete the structure of argument,
integrating argument and counterargument (or original and alternative
theory). Only 25% of subjects (averaged across topics) achieve an integra

tive rebuttal. (See Table 3 for a summary of the various forms of successful
integrative rebuttals.) Others offer a more limited form of rebuttal, in

which they simply offer a counterargument to an alternative theory,
hence leaving the original theory unexamined. And some simply argue by
assertion (e.g., "If they said it's the school, I'd say no, it's the parents"),
leaving both original and alternative theories unexamined. Still others
simply decline, like Lois, who says: "I don't think I'd even try. [Why not?]
He wants to believe it, that's fine. I'm not argumentative."
Epistemologica! Theories
As Lois's reply hints, underlying subjects' argumentive reasoning are

implicit theories of knowledge and knowing. The major findings regard
ing subjects' epistemologica! theories are summarized in Table 4. They

resemble those of others who have investigated naive epistemologica!
theories, beginning with Perry's (1970) pioneering work (Kitchener &
Fischer, 1990; Kramer & Woodruff, 1986). What is perhaps most interest
ing about the findings is the rich insight they provide into subjects' episte
mologica! beliefs from just the three questions shown at the top of Table 4
("Do experts know for sure what the cause is?" "Would it be possible for
experts to find out for sure if they studied this problem long and carefully
enough?" and "How sure are you of your view, compared to an expert?").
(Assignment to epistemologica! level was based only on these questions

about expertise, with the questions about proof, shown in the fourth
column, merely corroborative.) About 50% of the subjects (averaged
across topics) are classified as absolutists, who regard knowledge as cer
tain and accumulative, a rather remarkable percentage in view of the
topics involved: Fully half of an adolescent and adult population think
that complex questions, like why prisoners become repeat offenders, can
be answered with complete certainty. Another 35% (roughly) are multi
plets, or relativists, who note that experts disagree and go on to claim that

Reasoning 87
Table 3. Integrative Rebuttals
1. Rebuttals of counterarguments
1a. Qualitative rebuttal. The (40Cms) [poor home environment] [What evidence
counterargument is rebut might this person give to try to show that you were
ted on qualitative wrong!1] Evidence that shows that their early envi
grounds that undermine ronment was not the way I said it was in cases of
its force, thereby restor failures; for instance, that the people were wealthy
ing force to the original or that the situation was other than the way I said
theory. it. But I think there are subtle forms of deprivation.
The fact that somebody is wealthy does not mean
that they pay attention to their children or set a
good example.
lb. Quantitative rebuttal. (40Cfc) [return to same environment] Every once in
The instances that consti a while you hear of someone who really did better
tute the counterargument themselves while they were in prison. They went to
are alleged to be of low school or they wrote a book or whatever, but that's
frequency. one in a million, is my guess.
2. Rebuttals involving alternative
theories
2a. Arguments against causal (40Cms) [lack of motivation] {alternative theory:
sufficiency of the alterna "enticements" of earning money and taking drugs]
tive theory. The alterna In terms of what you might call enticements of life,
tive cause is not sufficient I would revert back to my basic premise that if you
to produce the outcome, have the desire for learning and value learning,
as long as the original then even if you might to some degree take up the
cause is absent. other enticements, you'd still maintain enough in
terest in learning to not fail.
2b. Arguments against causal (40Cfc) [lack of motivation] [alternative theory:
necessity of the alterna "joining gangs" and poor home life] Well, I could
tive theory. The outcome show that there are students who are failing and
occurs in the absence of they're not in gangs and they have a fine home life,
the alternative cause, as but they just don't seem to try for some reason.
long as the original cause
is present.
2c. Arguments against rela (60Nmc) [innate character] [alternative theory:
tive importance of the al poor environment] While environment plays an im
ternative theory. The al portant part, I think even more important is the im
ternative cause may be portance of that innate quality of goodness.
contributory but to a
lesser extent than the
original cause.
2d. Arguments that attempt (60Nmc) [lack of economic opportunity] [alterna
to reconcile original and tive theory: "antisocial" personality] I've heard ar
alternative theories. Origi guments where they said certain people are basi
nal and alternative cally antisocial. But there again you come right
causes are linked into a back to what made them antisocial, and you come
single causal chain. back with they never had the real opportunities to
get into the mainstream of today.
Note. See Table 1 note for abbreviations key.

5
"β ai c φ
o o , υ ζ >· o)
- p o) j3? — «ι o -cl
q. o φ α
03
υ ob i «2
i * u 3
5
o £ o d c ιλ ς/ί
ra o o (λ 2
-. φ ■ c _φ
U -X ¡i 3 =t 5
c c ra 03
fp '!= -C
φ ζ -q il
■vt V o -q -
txj -q S &-s
ra ; >- — > τ3
φ. 5-1
> _q "o
φ o -
al 2 o : d .£· o φ> § o ™ sε
■£ 9- o £ c <λ *ιλ_q φ s
φ φ ι/>
o o 03 qj e _q 3 -C
o c o 03 <j
-* > o φ
' S c l/) _q è s c
> o 13 v- os £
φ
L >. 03 q "ό ιλ q. % 00
- TD o) os" υ -ϋ o
: o ~ o
03 ^ o i _q Lo £ η
Jd ·- -c
*■'
• 00 "αϊ
m -q § i s
^ α;
2
o q
* a» c
φ > o
ró -σ
j_ v)
2 "5
Theories § s o c03 -o a;
. _c
oo
o! m ε ζ
o -εϊ
ζ (λ — 00
d
i- .3
"ιλ"o
o > s j= o ~o · g s
χ χ ï ^ 5,
Epistemologica!
ρ . ~o
4.
a¿ £ ^ 00
â e 3? ~o Bp §
-q q.
v. φ ο ε
Table χ
o £ "6
-
^.e ^
e dj
12 ,o 1/503
3
- ° -^ ~ -2 ορ ν
s u > "o ζ -g
»o o ο .e
im q. s o o -C
£ -c φ
o
c '·££ „\ 75 *
-ili 03 > cl) (d
-c r o u ^ -C
c c . a> oo
§ £ φ o-* >* "c
cl· cl· "o -^ = ír, n,
a —t3
χ 3 e
cl· e * cu o
o ζ φ i ü -q
o q- o
Q ,ο y = § ¡¡
c « 1λ u .1s
-q
o
^ ^
._ v,
-o >.
00 cu — oo cu _>>
-σ υ a "o >
*_ φ *s C o
1= £ qj "ö
_2 o += .9- o ^ φ
ο c ^
£ <5 ε "5 ^ 5 3 2l

"Ο OJ
ο
Ο > I §
οο e
ν -E — o J s.
OI οι 5 _s;
Ε ? 3 a~
α> .2 * "O -g
c ^
0) Û0 Π3 3
-Q c O) c CL
3 o c ra l/J
O i 0J
o δ £ ^ oc
c c
o E I >· 2
O QJ "5 »C c t/1
5 2? ra 5
"Β Vît JD Ρ aj o ra >, <υ
ο a tJ
U ™ </Γ .tn v2 C >--° ra
Ω. c "u (Λ le 12 3
Ο o E CL· ra - O
g1 —
.E >
Ε Έ U ¡I 'c U > ΙΛ u c 4
ο ee ra o O > </) erara
'α. £ "3· χ 'o.
ο ο o a E le > ■£ ■£
OJ <υ
— 1/1
=3 O
- ra ai s "S
"C u Q. >
CD CD c ra -Q C
ra d»
Q. _Q aj
X a3 CD 3 u E E
* 13 CÛ Z5 ra
E - -
° ^ OJ „3s
S * E (L· S-8 "g
-Q ra
OC m "O
§ ?C =
3 (p
"O £
QJ c ra <s>
■£ s-= O t>
C S ra <u 3 2 o '.tí 'u
00 O) >
J <Λ - ra ra ra °-í o -2
C 1/1· Q. -
-a -E T3
í c~
δ — 0)
— α> E 3 . c E ·=
i/l _Q
<υ -* O α» ra tu
ra ra DO > r
E
u -σ I râ
t: £ — =
o Ü> ra ζ tc _2 rä SP
Ci E u "ζ: ra
Ol
- 0) ra .«£> _c -Q
g>¡. >- ._ β .c ™ ° t;
■σ u Q- a; Ξ α; *ί δ =
00 ci </> .. (
in -C οι (1) i_ ¿i o
c o
ra "Ό >~ o 00 , , ~ O -S 5 -s ra
c υ
■c"¿ £ I 'οο E >
~ ο 2 I> c S 5
c te oc í ra E .2
D </> O ¿ α) — C _ ΟΟ C
Q. QJ ° αϊ *J
X O S O 'o -γ; "5
di 00 BP I ? - § ^ra ?-s > οο y Q
<υ
— o *00 Ρ 3 _:
= u c . ü -i* 'ra
O) CU 1c
cu
cu w 5 V c 3^ . m ra
.—. a»
U/ >- -X
sv
m S¿ o
iE -El.o P¿ Ό
— CL· _Q
E ~ -J
-C
-7-5
Q. -Q
U c "■ ~σ >■ ra
OJ αι -
δ O
o .E ^ =>S?
» Ooo 3
-τ
</) g »s <N .£ Ο oo >
ra S ,
^ o. ra 00 _a>
"O -Q
<Ü 9¿ »S s
-o? -
:§ ! ν S "O a>
c <L·
ra . «-O
1 c §
_Ξ ai O
ra u +3 o
2

nothing is therefore certain and, from there, that all opinions are of equal
validity. Everyone has a right to their opinion, multiplists maintain, and

hence all opinions are equally right. Both absolutists and multiplists thus
leave the knowing process out of their judgments. Only 15% of subjects
fall into the evaluative category, in which knowing is regarded as a pro
cess that entails thinking, evaluation, and argument.
This epistemologica! naïveté may be a critical factor in accounting
for the limited argumentive reasoning ability that people display. People
must see the point, the value, of argument ifthey are to engage in it. Ifone
accepts the absolutist view of knowledge as entirely certain and accumu
lative, or the multiplist view of knowledge as entirely subjective and

based only on the desires and tastes of the knower, argument becomes
superfluous. Without an epistemologica! understanding of their value, the

incentive to develop and practice the argumentive skills examined here
may be lacking.
Evaluation of Evidence
Finally, the evidence presented to subjects was of two types, which

we referred to as underdetermined and overdetermined. Underdeter
mined evidence in effect simply restates the phenomenon in the context
of a specific instance, with few clues as to its cause. The underdetermined
evidence for the crime topic is:
Pete Johnson is someone who has spent a good portion of his adult life in
prison. He was first convicted of a crime at age 14, when he took part in the
theft of a newspaper stand. He began serving his first prison sentence at age
18, after being convicted on several charges of auto theft and robbery. He
remained in a medium-security state prison until the age of 20. After he was
released on parole, he returned to live with his mother in the same neighbor
hood where he had grown up and began to look for a job. After three months
out of jail, he took part in the robbery of a grocery store. He was caught and
convicted and returned to prison. Since then, Pete has served three more
prison sentences for different crimes, with only brief periods out of prison
between sentences.
The overdetermined evidence for the crime topic is (with order of the
second and fourth paragraphs varied across subjects):
A study was done of 25 prisoners who were about to be released from
prison. All had served more than one prison sentence; some were in prison for
the third or fourth time. All had been in prison for the past three years or
longer, mostly for crimes of armed robbery.

A social worker investigated the prisoners' life histories. All had unhappy

Reasoning 91
early lives with many personal and family problems. None had good school
records. They tended to be uninterested in school, to do poorly, and to drop
out without
finishing. Almost all became involved in crime at an early age.
A government official did a study of their prison life. The prison was badly
overcrowded; each prisoner shared a cell with two or three others. Because of
crowded conditions, prisoners were able to have periods of exercise and out
door recreation only infrequently. No prisoner received job training.
Another social worker followed their lives outside prison during the six
months following their release. The majority had been unable to find jobs
since they had been released. Some applied to training programs, but there
were long waiting lists with only a few openings. Many had not found suitable
housing.
I will not take the space here to discuss the evidence evaluation
results in detail and will simply summarize their significance by noting

that subjects commonly assimilated both kinds of evidence to their own
theories (see Kuhn, 1991, for a full report). "This pretty much goes along
with my own view," was the prototypical response, and, again, subjects
expressed high certainty regarding these evaluations. Ifevidence is simply
assimilated to a theory, any ability to evaluate its bearing on the theory is
lost, of course. More broadly, with this loss comes loss of the ability to
maintain a differentiation between what derives from one's own thought
and what derives from external sources, and hence control of the interac
tion of theories and evidence in one's own thinking (Kuhn, 1989).
Results Across Subject Croups and Topics
Results across subject groups in this research are easily summarized

because they are consistent across all of the skills that were examined.
Skill levels show no significant differences by sex or by age group, but
consistent and sizeable differences by education group, at every age
level. For example, from 16% to 29% of all noncollege subjects (across
topics) generate genuine evidence, compared to 53% to 66% of college
subjects. These results suggest that it is not chronological age, but some
very broad kinds of experience associated with education that figure most
prominently in the wide variability in observed performance. (See Kuhn,
1991, for a full report of the results and for furtherdiscussion of the role of
education in the development of these skills.)
The other very important result is the significant, although by no
means total, generality of skills across the three topics. For each of the
major skills examined (evidence generation, alternative theories, coun
terarguments, and so on), although many subjects exhibit a skill on some
topics and not others, the numbers of subjects exhibiting the skill for all

topics or no topics are significantly greater than would be expected by

chance, if performance across topics were independent. This outcome is,
of course, critical because it suggests that we have identified forms of
thinking that transcend the particular content in terms of which they are
expressed. However imperfectly, we are tapping something about the
way that people think, above and beyond exactly what they are thinking
about. The expertise results support this conclusion. The philosophers
reasoned well overall, as we expected, but the domain expertise of the
others did not influence reasoning ability. Parole officers reasoned no

better about the crime topic than they did about the other topics, nor did
teachers reason better about the school topic.
LINKING SCIENTIFIC AND ARGUMENTIVE THINKING
In the research described in the previous section, we explored what
is rational or scientific about the thinking that ordinary people do. In

research on scientific thinking more traditionally conceived (Kuhn et al.,
1988; Kuhn, Schäuble, & Garcia-Mila, in press; Schäuble, 1990), ordi
nary people are asked to think in scientific ways. Like scientists, subjects
are asked to investigate a domain in which multiple variables interact and
to draw conclusions about the causal relations that exist there. In the
computerized racecar domain designed by Schäuble (1988, 1990), for

example, the subject constructs cars whose features (such as color, en
gine size, presence or absence of a tail fin) can vary and uses them in road
tests that the subject designs and carries out. In a microgenetic design, the
subject investigates this microworld in repeated sessions extending over a
period of weeks. The subject's task is to discover which features of the

cars make a difference in the speed they travel along a racetrack.
The development of scientific thinking traditionally has been concep
tualized as the development of operations or strategies of thought that
operate in a more or less domain-general manner (Inhelder & Piaget,

1958). Alternatively, and more recently, it has been conceptualized as
domain-specific conceptual change (Carey, 1986; Keil, 1984). In our

research on scientific thinking, we have focused on the specific theories
that a subject holds within a content domain, without foregoing the
search for strategic change in the ways in which the subject brings new
evidence to bear on these theories (Kuhn, 1989). As in the informal
reasoning domain, then, the problem is one of the coordination of theo
ries with new evidence bearing on them. As the experimental sessions
progress, the process of theory revision can be observed, as well as the
evidence generation and evidence interpretation strategies that subjects

employ.

Reasoning 93
The difficulties that preadolescent (and often adult) subjects exhibit

as scientists indicate that the challenges parallel those that we have
identified in the informal reasoning domain. First, the subject must have
the ability to reflect on his or her own theory as an object of cognition,
to an extent sufficient to recognize that it could be wrong. Second, the
subject needs to recognize evidence that could disconfirm the theory.

We can again focus on the prototypical case in which a subject theo
rizes a causal relation between two factors (e.g., the presence/absence

of a muffler and the cars' speed). To discover that the theory is wrong
and the muffler has no effect, the subject must conceive of the possibil
ity that the theory could be wrong and then generate and interpret
evidence that disconfirms it.
Subjects whose scientific thinking is the most rudimentary transform
the task goal into one of producing an outcome (e.g., the fastest car). By
middle childhood, however, most subjects have at least a tentative grasp
of the concept that one can demonstrate a set of causal relations that
exist among features of the cars and outcomes. These relations, how
ever, are more strongly theory-driven than evidence-driven. The subject

does not conceive of the possibility that theorized relations are wrong
and does not generate evidence that could disconfirm them.
The classic inferential error that subjects consistently make is the one
Inhelder and Piaget (1958) labeled false inclusion; the co-occurrence of
antecedent and outcome is taken as evidence that the antecedent is caus
ally implicated in the outcome (despite the presence of additional covari

ates). Hence, the subject who believes the mufflerto be causal takes the
co-occurrence of the muffler's presence and a fast speed as evidence of
the theory's correctness. When asked to justify their inferences, such
subjects are likely to mix theory-based and evidence-based justifications

indifferently; both point to the correctness of the theory in the subject's
mind and hence one is as good as the other in justifying it.
In the upper part of Table 5, this form of reasoning is illustrated in
parallel in the scientific and informal argumentive domains. In both

cases, evidence (of co-occurrence) serves as an illustrative instance of
what the subject knows to be correct. In the science domain, Car 2 is

shown in parentheses, as it typically plays only an incidental part in the
subject's reasoning.
To ever transcend this form of reasoning (and many subjects, in both
domains, never do), one must master exclusion reasoning, illustrated in
the lower part of Table 5 for both domains. Exclusion (inferring the ab
sence of a causal relationship) is essential to effective scientific reasoning
because it allows one to eliminate factors from consideration (and thereby
simplify analysis). An absence of exclusion is similarly a barrier to effec

Table 5. Illustration of False Inclusion and Valid Exclusion Reasoning in

Scientific and Argumentive Contexts
Argument
Science Theoretical Belief: Family Prob
Theoretical Belief: Muffler > Speed lems > School Performance
False Inclusion
Car 1 Muffler Fast
Large wheels
[Car 2 No muffler Slow] IHow do you know that family
Small wheels problems are the cause of chil
dren doing poorly in school!1] My
neighbor has always had marital
problems and she and her hus
band finally got a divorce and her
child has done very poorly in
school.
[What have you found out?] The muffler
makes a difference. [How do you know?] Be
cause car 1 has a muffler and it goes fast.
¡What about the wheels—do they have any
thing to do with it?] No, they just go around
to make the car move.
Valid Exclusion
Car 1 Muffler Fast
Large wheels
Car 2 No muffler Fast [What evidence would show that
Large wheels you were wrong?] If you looked at
families who have problems and
found that their children did just
as well in school as children
whose families don't have prob
lems.
[What have you found out?] The muffler
makes no difference because here you have it
and here you don't and the car still goes fast.
tive informal reasoning; subjects in the argument research, as noted previ

ously, often were unable to conceive of anything that was not a cause,
taking a "that could be part of it too" stance with regard to all possibilities.
Exclusion poses more of a challenge than inclusion (inferringthe pres
ence of a causal effect) for several reasons. First, and most fundamentally,
is the domination of affirmation over negation—the presence of some
thing is more salient than its absence—and, for this reason, both scientific
and lay theories pertain more often to the presence than the absence of
causal relations. Second, the belief that a factor is irrelevant often leads

Reasoning 95
subjects to ignore it. In so doing, they forego the possibility of encountering

disconfirming evidence and, hence, ever revising this belief.
Third, the ways in which uncontrolled evidence (of antecedent
outcome correspondences) can be used as the basis for inclusion do not
work for exclusion. As illustrated in the upper part of Table 5, inclusion is

often (although unjustifiably) based on a single instance (of co-occurrence
of antecedent and outcome). Although such reasoning is sometimes ob
served in our studies of scientific investigation, it is harder for a subject to
draw the conclusion that a causal effect is absent based on a single
instance, without at least some implicit comparison.
When inferences are based on multiple instances, a similar asymme
try between inclusion and exclusion arises. After a number of (uncon
trolled) multivariable antecedent-outcome instances have been gener
ated, a subject may make an inductive inference (e.g., "The cars with big
engines go faster than those with small ones"). Such inferences are some
times correct, but they are particularly prone to belief bias, because
subjects are remembering antecedents and outcomes and can do so selec
tively. In any case, a comparable strategy is not as readily available in the

case of exclusion (e.g., noting that cars that lack mufflers tend overall to
be no faster or slower than those with mufflers). Again, the bias is toward
noting the presence, not the absence, of effects. A valid exclusion infer
ence can be made based on just two instances, as illustrated in the lower
part of Table 5, but it requires that other variables be held constant so that
they do not contribute their own effects and invalidate the comparison.
Yet another and perhaps the most important reason that exclusion
poses a formidable challenge is that it competes with the everpresent

and more compelling temptation of false inclusion. The lure of false
inclusion is particularly great because these inferences are so often theo
retically motivated. The subject who believes that the muffler helps the
car to go fast, for example, needs only to observe a car having a muffler
that goes fast and it becomes very difficult to resist the inference that the
muffler at least contributed to the car's speed and to attend to the pattern
of evidence over multiple instances that would demonstrate the muffler's
noncausal role.
Often, in our microgenetic studies, such an insight is achieved, only

to be lost again. Randy, for example, starts out with the incorrect belief
that the muffler affects speed. After a long period during which he con
ducts experiments that are not capable of disconfirming his belief, at the
sixth session he designs and correctly interprets a valid experiment show
ing that a car with and a car without a muffleryield the same speed. "No,
the muffler doesn't matter," he concludes. "I just had a feeling it might
help to push it along." But he then goes on to comment on a third car, one

that achieved the maximum speed and happened to have a muffler. Here,
"
Randy says, "The muffler might have helped 'just a little bit.'
In the logbooks that were provided for subjects to record informa
tion, in contrast to the professional scientists and college students whom
we observed, who for the most part systematically recorded particular
constellations of features and their observed outcomes, in 11-year-old
Jamie's logbook we find the assertions that "With big wheels [the car] will
go slower because it takes more time for the wheels to go around" and "A
tailfin would make a difference because it has more weight for the wheels
to turn around." Like our subjects in the argument research who generate
pseudoevidence in support of their theories, these children confuse evi

dence with a plausible theory that can assimilate it.
In both the scientific reasoning context described here and the argu
mentive reasoning context described earlier, what children or adults need
to be able to do is to distance themselves from their own beliefs to a
sufficient degree to be able to evaluate them, as objects of cognition. In
other words, they must have the capacity and the disposition to think
about their own thought. This is metacognition in the most fundamental
and arguably most significant of the many ways in which this term has
been used in recent years. The traditional scientific hypothesis-testing
strategies of investigation and inference indeed need to be acquired as
important tools, but our observations of children's struggles as amateur
scientists have led us to the conclusion that the major challenge they face
is not one of acquiring correct experimentation strategies but of develop
ing the ability to coordinate their existing theories with new evidence they
generate, in an explicit, conscious, and controlled way; in other words,
again, to think about their own thought.

Becoming a competent scientist, accordingly, is more than a matter
of acquiring formal hypothesis-testing strategies applicable to any kind of
content. Attention to subjects' conceptions of the content of what they are
reasoning about is largely absent in traditional studies of scientific reason
ing and casts much of this research in a new light. For example, the
failure to control variables traditionally has been interpreted as a failure to
attend to these variables and therefore to recognize the possibility that left
uncontrolled they may exert their own effects on outcome. Our observa
tions, however, suggest a different interpretation. Rather than under

attend to these variables, subjects more often overattend to them, in ways
that are driven by their theoretical beliefs (Kuhn, 1989).
Conceptualizing an individual's task in scientific reasoning (and in
deed in inductive reasoning in general) as a task of coordinating two
entities—or problem spaces, in Klahr and Dunbar's (1988) terms—rather
than simply the application of a set of strategies to varying content, makes

Reasoning 97
the task a much richer, more complex one. Coordination implies recipro
cal adjustment. Not only must evidence serve as the basis for evaluating
and possibly revising theories, but theories influence the direction and
form of investigation. This holds true for professional as well as intuitive
scientists. The difference between the two lies in the degree of control
that is attained of the coordination process (Kuhn, 1989).
DEVELOPING REASONING COMPETENCY
Development of the competencies that have been sketched here
clearly occurs only gradually. A number of precursors can be identified,

ones that, in fact, extend to as early in life as infancy. By the end of the
firstyear of life, infants have begun to make causal inferences based on
the juxtaposition of an antecedent and outcome. These are initially fo
cused on the effects of the child's own actions but later extend to actions
and outcomes that operate independently of the child. Later, these infer
ences are based on a more extended evidence base of multiple observa
tions and are formalized into explicit knowledge of which the child is
consciously aware and able to verbalize. The early roots and ubiquitous
nature of the covariation-to-causality inference suggest that by preado
lescence it has already been overlearned to an extent that causes it to
remain a strong competitor to the less error-prone forms of inference that
are beginning to develop.

Precursors to valid inclusion and exclusion inference include the
earliest concepts of comparison, firstin the case of individuals (Can I run
faster than my brother?) and later groups of individuals (Do the girls in the
class run faster than the boys?). Once the concept of a fair comparison is
considered (What ifthe boys wore running shoes and the girls did not?), it
remains only to formalize the comparison into the framework of a con
trolled test of relations between variables (gender and running speed), a

transition explored by Case (1974) in an interesting way.
A second set of precursors to valid inference involves recognition of
indeterminacy, due to multiple possibilities, explored in a line of research

beginning with the studies by Pieraut-Le Bonniec (1980). (See Acredolo &
O'Connor, 1991, or Byrnes & Beilin, 1991, for review.) In a recent study,
Sodian, Zaitchik, and Carey (1991) have given this line of research an
intriguing new perspective by casting it in the framework of scientific
reasoning (specifically, hypothesis testing). They showed that, when
given the choice, many firstand second graders preferred a determinate
to an indeterminate test of a very simple (two-alternative) hypothesis.
They posed first and second grade subjects a problem in which some

children wanted to find out if a mouse in their house was a large or small
one, by placing food in a box overnight. Two boxes were available, one
with a large opening (able to accommodate a large or small mouse) and

one with a small opening (big enough for only the small mouse to pass
through). The subject was asked which of the two boxes the children
should put food in, to find out if the mouse was large or small. Sodian et
al. report that Π of 20 firstgraders and 12 of 14 second graders chose the
small-opening box in response to this question, indicating both consider
able competence and considerable development in this age range.
As a problem in scientific reasoning, this problem is, of course,
considerably simpler than the prototypical scientific reasoning problems
that have been the focus of discussion here. The theory involves only first
order categorization (large or small mouse), rather than second-order
causal relations among categories or features. The situation was deliber
ately designed so that subjects would be unlikely to have any prior beliefs
regarding the theory, and the subject did not have to generate or evaluate
evidence over multiple instances. Most important, however, the subject

did not have to generate or evaluate evidence to disconfirm a theory. To
answer correctly, the subject needed only to recognize that one choice
yielded a determinate solution and the other did not: The large-opening
box confirms both the small-mouse and large-mouse categorization,
whereas the small-opening box confirms only the small-mouse categoriza
tion. Subjects' justifications for their choices were consistent with this
confirmation logic (e.g., "If the food is gone, this tells them it's a small
mouse"). (It is of course possible to cast this problem in the framework of
disconfirmation; that is, the small-opening test is capable of discon
firming a large-mouse theory, but the point is that subjects need not do so
to answer correctly.)
The performance reported by Sodian et al., nonetheless, demon
strates the onset by age 6 to 7 years of the ability, in at least some
contexts, to recognize alternative possibilities and hence indeterminacy,

and this is clearly an important precursor of the competencies examined
here, as we have seen. Many subjects, in both the scientific and argu
mentive reasoning contexts, are precluded from reasoning more effec
tively because they never envision the possibility of alternatives to the

state of affairs depicted by their own theories.
Sodian et al. (1991) note further that the hypothesis-testing compe
tence displayed by their subjects reflects at least a primitive differentia
tion of theory and evidence, in the sense that the hypothesis is differenti
ated from the evidence that will test it. In fact, rudimentary forms of
theory-evidence coordination are apparent at much earlier ages. Imag

ine a 2-year-old who calls her parents into her bedroom one night with

Reasoning 99
the claim that it is a ghost in her closet that is the cause of a soft
"whooshing" noise that is keeping her awake. This child understands as
well as her parents that opening the closet door will provide the evi
dence capable of disconfirming this causal hypothesis (even though she
does not understand the logic of disconfirmation in any formal or reflec
tive sense). In the next few years, this 2-year-old will develop the under
standing of false belief that has been the object of attention of the new
theory-of-mind research (Feldman, 1992; Perner, 1991; Wellman, 1990).
Yet, for many years, and most likely well into and even throughout her
adult life, she will exhibit difficulty in bringing evidence to bear on
her own beliefs in a way that reflects clear differentiation between the
implications of evidence and what she believes to be true.
Identification of these extended precursors to the reasoning com
petencies examined here underscores the fact that the development in
question is both gradual and multifaceted. In our microgenetic studies we

see various individual facets or skills appear, disappear, and reappear
repeatedly, as they very gradually begin to reinforce one another and
become consolidated (Kuhn, 1989; Kuhn et al., in press; Siegler & Crow
ley, 1991). Yet, as Kunda's (1990) review of a wide range of research on
adults' "motivated" reasoning makes clear, as do our own studies of
adults' reasoning in both scientific and informal contexts, this develop

ment is one that occurs not once but many times over, as people encoun
ter new contexts, challenges, and motives for believing what they do.
CONCLUSION
An attempt has been made here to establish a parallel between the

evidence-based justification of beliefs (the coordination of theory and
evidence) in informal reasoning and in more traditional scientific reason

ing contexts. In both cases, the essence of what we have examined is the
ability to justify what one claims to be true.
Exploration, and perhaps problem solving, are also part of the enter
prise of formal science, as reflected in Reichenbach's (1938) classical

distinction between the contexts of discovery and justification. Yet, in
seeking to redress epistemologists' traditional disinterest in the context of
discovery (Gholson, Shadish, Neimeyer, & Houts, 1989), psychologists
interested in the thinking of scientists may have made the opposite error,
focusing on the creative insights achieved by the lone scientist, to the
exclusion of the social exchange through which ideas are articulated,
questioned, clarified, defended, elaborated, and indeed very often arise
in the firstplace. This analyzing and weighing process of argument is, in

interiorized form, almost certainly a core aspect of what goes on in the
private thought of the individual scientist as well.
A characterization of science as argument allows us to establish
links between the scientific thinking of the child, the adolescent or

adult, and the professional scientist. Argumentive reasoning, we have
seen, figures in the activities of each of them. Both scientific and infor
mal theories must be regarded as possible states of affairs that are sub
ject to confirmation or disconfirmation by evidence. This recognition

entails awareness of the theory, that is, the ability to reflect on it as an
object of cognition. This reflective capacity makes it possible to evaluate
the theory, by bringing evidence to bear on it. Such evidence, likewise,
must be contemplated as an object of cognition, distinct from the
theory, such that relations between the two can be constructed and the
evidence becomes capable of disconfirming the theory. I have under

taken here to show that these two abilities—the ability to recognize the
possible falsehood of a theory and the identification of evidence capable
of disconfirming it—are foundational abilities that lie at the heart of both
informal and scientific reasoning.
These abilities also lie at the heart of critical thinking, which similarly
can be regarded, at the most global level, as the ability to justify what one
claims to be true. The linking of scientific and informal reasoning under
taken here contributes to a badly needed integration of the diverse re
search literature that bears on critical thinking. Informal, scientific, and

formal logical thinking have all been studied largely in isolation from one
another (Galotti, 1989). By linking them, we extend the scope and hence
significance of all of these forms of thought.
Although it is not possible to explore them in any detail in the present
context, the links established here have numerous educational and practi
cal implications. In the case of science education, to take just one exam
ple, the typical approach to getting students to see the relevance of sci
ence has been to try to connect the content of science to phenomena

familiar to them. An ultimately more powerful approach may be to con
nect the process of science to thinking processes that figure in ordinary
people's lives.
The research that has been described here contributes to establishing
a base of empirical knowledge about how people think, a knowledge
base that I claimed at the outset is essential to efforts to both understand
and promote critical thinking. We want people to think critically because
we believe that it will contribute to the quality and fulfillment of their
lives, both individually and collectively as a society. The challenge that
confronts those who study critical thinking, then, is to identify and better
understand the kinds of thinking that can fulfill this function.

Reasoning
REFERENCES
ACREDOLO, C., & O'CONNOR, J. (1991). On the difficulty of detecting cognitive uncer
tainty. Human Development, 34, 204—223.
AMSEL, E., LANCER, R., & LOUTZENHISER, L. (1991). Causal inquiry and the acquisition
of expertise in law and psychology. In R. Sternberg & P. Frensch (Eds.), Complex
problem solving: Principles and mechanisms. Hillsdale, N): Erlbaum.
ANTAKI, C. (1988). Structures of belief and justification. In C. Antaki (Ed.), Analysing

everyday explanation. London: Sage.
BARON, J. (1988). Thinking and deciding. New York: Cambridge University Press.
BYRNES, J., & BEILIN, H. (1991). The cognitive basis of uncertainty. Human Development,
34, 189-203.
CAREY, S. (1986). Cognitive science and science education. American Psychologist, 4,

1123-1130.
CASE, R. (1974). Structures and strictures: Some functional limitations on the course of
cognitive growth. Cognitive Psychology, 6, 544-573.
EINHORN, J., & HOGARTH, R. (1986). Judging probable cause. Psychological Bulletin,
99,3-19.
FELDMAN, C. (1992). The new theory of theory of mind. Human Development, 35,
107-117.
GALOTTI, K. (1989). Approaches to studying formal and everyday reasoning. Psychological

Bulletin, 105, 331-351.
GHOLSON, B., SHADISH, W., NEIMEYER, R., & HOUTS, A. (Eds.). (1989). Psychology of
science: Contributions to metascience. New York: Cambridge University Press.
HILTON, D. J. (1988). Logic and causal attribution. In D. J. Hilton (Ed.); Contemporary

science and natural explanation. New York: New York University Press.
INHELDER, B., & PIAGET, J. (1958). The growth of logical thinking from childhood to
adolescence. New York: Basic.
KEIL, F. (1984). Mechanisms in cognitive development and the structure of knowledge. In

R. Sternberg (Ed.), Mechanisms of cognitive development. New York: Freeman.
KITCHENER, K., & FISCHER, K. (1990). A skill approach to the development of reflective
thinking. In D. Kuhn (Ed.), Developmental perspectives on teaching and learning think
ing skills. Contributions to human development (Vol. 21). Basel: Karger.
KLAHR, D., & DUNBAR, K. (1988). Dual space search during scientific reasoning. Cogni
tive Science, 12, 1-48.
KLAHR, D., FAY, Α., & DUNBAR, K. (1991). Developmental differences in experimental
heuristics. Unpublished manuscript, Carnegie-Mellon University, Pittsburgh, PA.
KRAMER, D., & WOODRUFF, D. (1986). Relativistic and dialectical thought in three adult
age groups. Human Development, 29, 280-290.
KUHN, D. (1986). Education for thinking. Teachers College Record, 87, 495-512.
KUHN, D. (1988). Making cognitive development research relevant to education. In W.

Damon (Ed.), Child development today and tomorrow. San Francisco: Jossey-Bass.

KUHN, D. (1989). Children and adults as intuitive scientists. Psychological Review, 96,
674-689.
KUHN, D. (1990a). Education for thinking: What can psychology contribute? In M.

Schwebel, C. Maher, & N. Fagley (Eds.), Promoting cognitive growth over the life span.
Hillsdale, NJ: Erlbaum.
KUHN, D. (Ed.). (1990b). Developmental perspectives on teaching and learning thinking

skills. Contributions to human development (Vol. 21). Basel: Karger.
KUHN, D. (1991). The skills of argument. New York: Cambridge University Press.
KUHN, D., AMSEL, E., & O'LOUGHLIN, M. (1988). The development of scientific thinking
skills. Orlando, FL: Academic Press.
KUHN, D., SCHÄUBLE, L., & GARCIA-MILA, M. (in press). Cross-domain development of
scientific reasoning. Cognition and Instruction.
KU Ν DA, Ζ. (1990). The case for motivated reasoning. Psychological Bulletin, 108,480-498.
PERLMUTTER, M., KAPLAN, M., & NYQUIST, L. (1990). Development of adaptive compe
tence in adulthood. Human Development, 33, 185-197.
PERNER, J. (1991). Understanding the representational mind. Cambridge, MA: MIT Press.
PERRY, W. (1970). Forms of intellectual and ethical development in the college years. New
York: Holt, Rinehart & Winston.
PIERAUT-Le BONNIEC, G. (1980). The development of modal reasoning. New York: Aca
demic Press.
REICHENBACH, H. (1938). Experience and prediction. Chicago: University of Chicago

Press.
SCHÄUBLE, L. (1988). Processes of constructive learning and inference in children: Under

standing the causal structure of a microworld. Unpublished doctoral dissertation, Teach
ers College, Columbia University.
SCHÄUBLE, L. (1990). Belief revision in children: The role of prior knowledge and strate
gies for generating evidence, journal of Experimental Child Psychology, 49, 31-57.
SCHÄUBLE, L„ KLOPFER, L., & RAGHAVAN, K. (1991). Students' transition from an

engineering model to a science model of experimentation, journal of Research in
Science Teaching, 28 (9), 859-882.
SCRIBNER, S. (1986). Thinking in action: Some characteristics of practical thought. In R.

Sternberg & R. Wagner (Eds.), Practical intelligence: Nature and origins of competence
in the everyday world. New York: Cambridge University Press.
SIEGLER, R., & CROWLEY, K. (1991). The microgenetic method: A direct means for study
ing cognitive development. American Psychologist, 46, 606-620.
SODIAN, B., ZAITCHIK, D., & CAREY, S. (1991). Young children's differentiation of hypo
thetical beliefs from evidence. Child Development, 62, 753-766.
STERNBERG, R., & WAGNER, R. (Eds.). (1986). Practical intelligence: Nature and origins
of competence in the everyday world. New York: Cambridge University Press.
VOSS, J., PERKINS, D., & SEGAL, J. (1991). Informal reasoning and education. Hillsdale,
NJ: Erlbaum.

Reasoning
WAGNER, R., & STERNBERG, R. (1985). Practical intelligence in real-world pursuits: The
role of tacit knowledge, journal of Personality and Social Psychology, 49, 436-458.
WAGNER, R., & STERNBERG, R. (1986). Tacit knowledge and intelligence in the everyday
world. In R. Sternberg & R. Wagner (Eds.), Practical intelligence: Nature and origins of
competence in the everyday world. New York: Cambridge University Press.
WELLMAN, H. (1990). The child's theory of mind. Cambridge, MA: MIT Press.


Kuhn (1993) - Informal Reasoning

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Kuhn (1993) - Informal Reasoning

Caricato da

Copyright:

Formati disponibili

Connecting Scientific and Informal Reasoning

Author(s): Deanna Kuhn

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

Connecting Scientific and Informal Reasoning

What is critical thinking and why is it important? Answers to these

rapidly growing research area of informal reasoning (Voss, Perkins, &

Correspondence may be sent to Deanna Kuhn, Department of Psychology, Teachers

Copyright © 1993 by Wayne State University Press, Detroit, Michigan 48202

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

INFORMAL REASONING AS ARGUMENT

Researchers have become hesitant to judge people's thinking abili

get elected is aware, the point-on-a-continuum conception continues to

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

consciousness gone into an opinion for or against abortion or gun control.

Otherwise, the opinions that people profess, and that we undertake to

develop a way of conceiving of such thinking that makes it more approach

Rhetorical and Dialogic Argument

To examine thinking as argument, a distinction must be established

nary definition of an argument in the first sense is "a course of reasoning

aimed at demonstrating the truth or falsehood of something." An argu

commonly, an argument is thought of in its second sense, as a dialogue

this sense is referred to here as a dialogic argument.

certain fundamental cognitive skills entailed in both. In a dialogic argu

tions, and, ideally, if the argument is to move toward resolution, one

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

must be able to weigh supporting and refuting evidence in an integrative

argument may appear on the surface to be less complex. An argument in

support of an assertion is an empty, indeed superfluous, argument unless

one can conceive of an alternative to what is being asserted (i.e., an

opposing assertion). Once two or more contrasting assertions are in

place, cognitively speaking, the further challenge poses itself of relat

alternatives. Indeed, it is just such a weighing process that is implicit

Design of the Investigation

To investigate these questions, we asked people their opinions, or,

Following the framework of the dialogic argument, subjects were

tions also were asked that addressed subjects' epistemologica! reflections

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

The 160 subjects who participated in this research were selected to

topic; experienced teachers, regarded as having domain expertise in the

school failure topic; and philosophers (specifically, PhD candidates work

Although we will not dwell on the theories themselves, one impor

dence could we expect lay people with no special knowledge or interest

in these topics to offer?

ling, nor even necessarily very convincing evidence. Rather, it is simply

language. About half refer to covariation (i.e., variation in the alleged

reasoning familiar from the experimental literature on reasoning appear as

sponses, roughly, that were classified as unsuccessful, beginning with

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

Table 1. Genuine Evidence

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

3c. Discounting (elimination (40Nfc) [unemployment] Well, I don't believe that

Joe's response is clearly an example of nonevidence. Consider an

going to be cared for, is if they're in prison.

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

How should Kathryn's response be characterized? Is this evidence

sequence. At its best, it enhances the plausibility of the causal sequence,

This content downloaded from 70.90.173.73 on Thu, 6 Nov 2014 12:08:25 PM

plausible theory connecting them as, for example, when a substance is

found beneficial in treating a disease in the absence of an understanding

rectness. Again, even the most plausible theories can be wrong.

Can subjects who offer only pseudoevidence envision the possibility

ceive of anything that is not a cause. Other subjects try unsuccessfully to

generate an alternative, producing something very like their own theory.