Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/
info/about/policies/terms.jsp
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content
in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship.
For more information about JSTOR, please contact support@jstor.org.
Sage Publications, Inc. and American Educational Research Association are collaborating with JSTOR to digitize, preserve
and extend access to Educational Researcher.
http://www.jstor.org
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions
Can
We
Test
Validly
for
Critical
Thinking?
STEPHENP. NORRIS
This paperfocusesfirst on two questions:(a) Is criticalthinkinggeneralizable?
and (b)
Whatis a criticalthinkingdisposition?It is arguedthat the controversial
natureof these
questionslimitstheabilitytojudgethe validityof criticalthinkingtesting. Followingthis
discussion,the paperoutlinesa seriesof proceduresthat can help increasethe validity
of onefundamentalaspectof criticalthinkingtesting-multiple-choicetestingof credibility judgment.The proceduresrely on verbalreportsof examinees'thinkingon items to
that multiplegain directevidenceon the reasonsfor theiranswerchoices.It is recognized
choicetestscannottestall important
critical
but
that
aspectsof
thinking,
improvingmultiplechoicetestswheretheyareapplicable
canhaveimportant
andscientificimplications.
practical
21
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions
EDUCATIONAL RESEARCHER
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions
23
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions
EDUCATIONAL RESEARCHER
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions
according to the evidence in their verbal reports but reach the keyed answers-that is, examinees do not follow
criteriaof criticalthinking,but, for sheer
reasons, arriveat
extra-critical-thinking
the keyed responses.
The best modifications to make in
view of any of these conditions cannot
be determined by a set of rules. Item
writing requirescreativityand informed
judgment, and, though items must be
constrainedby data, the data underdetermine their exact nature. Generally,
many differentitems can meet the same
constraints.
Assessment of the Approach
At the end of the above procedure, we
would have direct evidence from verbal reports that examinees who respond in accord with the key tend to
use the credibilityjudgment criteriabeing tested, and examinees who respond
contraryto the key tend not to use the
criteria.Thatis, we would be able to say
that in the context chosen, where we
know the criteria apply, use of the
criteriais what determines differences
in scores.
Until the generalizabilityissue in its
various guises is resolved, however, we
should not make inferences about examinees' use of the criteriain other contexts. A series of credibility judgment
tests with their items cast in various
contexts, each designed using the previously described procedure, could be
used to help test the various generalizability hypotheses. In this manner, making higher quality tests for
restricted ranges of contexts could improve theorizing which in turn could
lead to more valid critical thinking
testing.
The recommended verbal reporting
approach still requires considerableexploration.First,it is not known whether
the use of verbalreportsof thinkingcan
help with the development of tests of
critical thinking dispositions. It seems
worth exploring whether interviewing
examinees is a way to understand the
beliefs and values that underlie the
critical thinking abilities they use, but
it is not clear how tests of criticalthinking dispositions might look.
Second, the methodology as described is not designed for improving
criticalthinking testing in the context of
real-world problems. Typically in such
problems, several criticalthinking abilities have to be orchestrated,judgments
must be made about how to apply ab-
25
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions
Note
This paper is based upon research supported
by the Social Sciences and Humanities Research
Council of Canada, Grant Nos. 418-81-0781,
410-83-0697, and 410-85-0587. I thank Linda
Phillips and two anonymous reviewers for
helpful comments on a previous draft.
References
Baron, J. (1985). Rationalityand intelligence.Cambridge: Cambridge University Press.
Bereiter, C., & Scardamalia, M. (1987). An attainable version of high literacy:Approaches
to teaching higher-order skills in reading
and writing. CurriculumInquiry,17(1), 9-30.
Ennis, R.H. (1981). Rational thinking and
educational practice. In J. F. Soltis (Ed.),
Philosophyof education(80th Yearbook of the
National Society for the Study of Education,
Vol. 1, pp. 143-183). Chicago: National
Society for the Study of Education.
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed
research. EducationalResearcher,18(3), 4-10.
Ennis, R. H., & Millman, J. (1985a). Cornell
Critical Thinking Test Level X. Pacific
Grove, CA: Midwest.
Ennis, R. H., & Millman, J. (1985b). Manual:
CornellCriticalThinkingTests.Pacific Grove,
CA: Midwest.
Ennis, R. H., & Norris, S. P. (in press). Critical
thinking evaluation: Status, issues, needs.
In J. Algina & S. M. Legg (Eds.), Cognitive
assessmentof languageand mathoutcomes.Norwood, NJ: Ablex.
Glaser, R. (1984). Education and thinking: The
role of knowledge. AmericanPsychologist,39,
93-104.
Haney, W., & Scott, L. (1987). Talking with
children about tests: An exploratory study
of test item ambiguity. In R. O. Freedle &
R. P. Duran (Eds.), Cognitiveand linguistic
analysesof testperformance
(pp. 298-368). Norwood, NJ: Ablex.
McPeck, J. (1981). Criticalthinkingand education.
New York: St. Martin's.
National Assessment of Educational Progress.
(1985). Thereadingreportcard:Progresstoward
26
EDUCATIONAL
RESEARCHER
This content downloaded from 103.229.202.168 on Wed, 25 Nov 2015 03:30:47 UTC
All use subject to JSTOR Terms and Conditions