EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

CRITICAL THINKING

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or
by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no
expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No
liability is assumed for incidental or consequential damages in connection with or arising out of information
contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in
rendering legal, medical or any other professional services.
 EDUCATION IN A COMPETITIVE
AND GLOBALIZING WORLD

Additional books in this series can be found on Nova’s website

under the Series tab.

Additional E-books in this series can be found on Nova’s website

under the E-books tab.

PERSPECTIVES ON COGNITIVE PSYCHOLOGY

Additional books in this series can be found on Nova’s website

under the Series tab.

Additional E-books in this series can be found on Nova’s website

under the E-books tab.
EDUCATION IN
I A COMPET
TITIVE AND GLOBALIZING
G WORLD

CRITICA
R AL THIINKING
G

CHR HER P. HORVATH

RISTOPH H
AND
JAMESS M. FOR
RTE
EDITORS

Nova Scien
nce Publisheers, Inc.
N York
New
Copyright © 2011 by Nova Science Publishers, Inc.

All rights reserved. No part of this book may be reproduced, stored in a retrieval system or
transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical
photocopying, recording or otherwise without the written permission of the Publisher.

For permission to use material from this book please contact us:
Telephone 631-231-7269; Fax 631-231-8175
Web Site: http://www.novapublishers.com

NOTICE TO THE READER

The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or
implied warranty of any kind and assumes no responsibility for any errors or omissions. No
liability is assumed for incidental or consequential damages in connection with or arising out of
information contained in this book. The Publisher shall not be liable for any special,
consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or
reliance upon, this material. Any parts of this book based on government reports are so indicated
and copyright is claimed for those parts to the extent applicable to compilations of such works.

Independent verification should be sought for any data, advice or recommendations contained in
this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage
to persons or property arising from any methods, products, instructions, ideas or otherwise
contained in this publication.

This publication is designed to provide accurate and authoritative information with regard to the
subject matter covered herein. It is sold with the clear understanding that the Publisher is not
engaged in rendering legal or any other professional services. If legal or any other expert
assistance is required, the services of a competent person should be sought. FROM A
DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE
AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS.

Additional color graphics may be available in the e-book version of this book.

Library of Congress Cataloging-in-Publication Data

Critical thinking / editors, Christopher P. Horvath and James M. Forte.

p. cm.
Includes index.
ISBN 978-1-62081-702-5 (eBook)
1. Critical thinking. I. Horvath, Christopher P. II. Forte, James M.
BF441.C735 2011
160--dc22
2011012599

Published by Nova Science Publishers, Inc. © New York

 CONTENTS
Preface vii 
Chapter 1 Rules for Reasoning Revisited: Toward a Scientific Conception
of Critical Thinking 1 
D. Alan Bensley 
Chapter 2 Critical Thinking and Systems Thinking: Towards a Critical
Literacy for Systems Thinking in Practice 37 
Martin Reynolds 
Chapter 3 Developing Critical Thinking through Probability Models 69 
Einav Aizikovitsh-Udi 
Chapter 4 The Promotion of Critical Thinking Skills through Argument
Mapping 97 
Christopher Dwyer, Michael Hogan and Ian Stewart 
Chapter 5 Beyond GDP? Towards a New System of Social Accounts 123 
Frédéric Lebaron 
Chapter 6 A Four-Component Instructional Model for Teacher
Training in Critical-Thinking Instruction: Its Effectiveness
and Influential Factors 141 
Yu-chu Yeh 
Chapter 7 Crucial Connections: An Exploration of Critical Thinking
and Scholarly Writing 159 
Roisin Donnelly and Marian Fitzmaurice 
Chapter 8 How Can Critical Thinking Be Recognized in the Classroom? 175 
Patrícia Albergaria-Almeida, José Joaquim Cristino Teixeira-Dias
and Mariana Martinho 
Index 189 
 PREFACE
In reflective problem solving and thoughtful decision making using critical thinking one
considers evidence, the context of judgment, the relevant criteria for making the judgment
well, the applicable methods or techniques for forming the judgment, and the applicable
theoretical constructs for understanding the problem and the question at hand. In this book,
the authors present topical research in the study of critical thinking. Topics discussed include
developing critical thinking through probability models; the promotion of critical thinking
skills through argument mapping; an instructional model for teacher training in critical
thinking; advanced academic literacy and critical thinking and critical thinking and higher
education.
Chapter 1 – Critical thinking (CT) has been widely acclaimed as an important educational
outcome, yet numerous problems remain concerning how to conceptualize it. A review of the
literature in this chapter reveals that many authors agree that CT involves skills and
dispositions related to good reasoning, but they often disagree about which specific skills and
dispositions are needed and in how these are organized. Closer inspection of other terms
commonly associated with CT such as “evaluation,” “analysis,” and “higher order thinking,”
reveals that they, too, are poorly specified and add little to the meaning of CT. The lack of
clarity and specificity of CT skills, dispositions, and other terms impedes the instruction,
assessment, and scientific study of CT and is only beginning to be addressed through
empirical research. Although research reviewed in this chapter shows that explicit infusion of
CT into subject matter instruction is more effective than less explicit, immersion and
traditional approaches, it also reveals the need for greater specificity in identifying the
principles and other antecedent conditions that improve CT. To address these deficiencies, a
rule-based approach to defining, instructing, and assessing CT is proposed. The approach
operationally defines CT skill as the appropriate use of relevant rules and procedures for
reasoning in a discourse context. It further assumes that CT is a motivated process involving
dispositions and self-regulation in the service of belief formation and revision.
Chapter 2 – Rather than exploring one tradition of systems thinking – CST - this chapter
explores the notion of contemporary systems thinking as being implicitly critical. An
argument will be made that the need for what might be called a ‘systems literacy’ reflects a
need for the original critical idea of systems. The basis of such a literacy is a proposed
framework of systems thinking in practice based on revised ideas of boundary critique (Ulrich
and Reynolds, 2010). After describing what this critical literacy in systems thinking in
practice looks like and entails, the question of how the critical kernel emerged amongst
contemporary systems thinking in practice approaches is examined. This section traces the
viii Chritopher P. Horvatha nd James M. Forte

influence of critical thinking traditions on systems thinking. Finally, some views are offered
on why attention to the critical literacy of systems thinking in practice is significant to a
contemporary world beset with complex issues of change and uncertainty.
Chapter 3 – In light of the importance of developing critical thinking, and given the
scarcity of research on critical thinking in mathematics studies in the broader context of
higher-order thinking skills, authors have carried out a research that examined how teaching
strategies oriented towards developing higher-order thinking skills influenced the students’
critical thinking abilities. The guiding rationale of the work was that such teaching can foster
the students’ skills of and dispositions towards critical thinking. In this research, a primary
attempt has been made to examine the relations between education for critical thinking and
mathematics studies through examining teaching and learning critical thinking according to
the infusion approach, which combines critical thinking and mathematical content
(“Probability in Daily Life” learning unit).The main contribution of this work and the
innovations it is expected to introduce lie in elucidating the connection between critical
thinking and the study of mathematics and creating insights into the mechanisms of critical
thinking development, and its place and importance in the study of mathematics, in spite of
the uncertainty whether critical thinking skills acquired in studying one field will necessarily
be applied by students in other fields, referred to as “the transfer problem.” In this way it will
be possible to strengthen the status of mathematics studies in imparting higher-order thinking
skills in various frameworks, in parallel with and beyond the formal program of studies. The
purpose of this research is to examine how and to what extent it is possible to develop critical
thinking by means of the learning unit “Probability in Daily Life” using the infusion
approach. The research questions that guided it are: (1) To what extent does the study of
“Probability in Daily Life” in the infusion approach contribute to the development of critical
thinking dispositions? (2) To what extent does the study of “Probability in Daily Life” in the
infusion approach contribute to the development of critical thinking abilities? (3) What are
the processes of construction of critical thinking skills (e.g., identifying variables, postponing
judgment, referring to sources, searching for alternatives) during the study of the “Probability
in Daily Life” learning unit in the infusion approach? The present research involved nine
groups of gifted and high-achieving mathematics students in eleventh grade from all the
social groups and strata of Israeli society. The students studied the learning unit “Probability
in Daily Life” modified by the researchers to include critical thinking teaching in the infusion
approach. The students were then tested in two critical thinking tests, CCTDI and the Cornell
Critical Thinking Test, the results of which were statistically analyzed, and also selectively
interviewed, with subsequent qualitative analysis of the interviews and lesson transcripts.
Thus the research combines quantitative and qualitative methods.
The research findings can be summed up in the following categories: (i) In all three
iterations of the experimental teaching, a moderate improvement was detected in the critical
thinking dispositions of all experimental groups. (ii) Throughout these iterations, a moderate
improvement was also detected in the students' critical thinking abilities. (iii) Teaching
critical thinking contributed to the construction and use of these skills in the framework of
mathematics. Thus, when teachers consistently emphasize critical thinking skills, the students
are more likely to succeed in the subject of mathematics. (iv) This research did not detect a
clear-cut distinction between the critical thinking abilities and dispositions of excellent and
average mathematics students. That is, no direct correlation has been found between the
development of mathematical knowledge and the development of critical thinking. On the
 Preface ix

basis of these findings, the following recommendations for further research can be made: (1)
A more comprehensive examination of the processes of critical thinking: to what extent could
the students describe, orally and in writing, the processes of thinking, activate them and apply
the thinking skills they studied on the procedural and meta-cognitive level? Did they make an
informed use of terms and strategies of higher-order thinking, including critical thinking? In
other words, it should be examined what use the research participants make of the “language
of thinking,” or, in the words of Costa and Marzano, “do they speak thinking?” (Costa &
Marzano, in Harpaz, 1997). Developing such a language involves, on the part of the teacher,
such skills as using precise vocabulary, presenting critical questions, presenting data rather
than answers, aspiring for exactness, giving directions, and developing meta-cognition. (2)
Examination of the attitudes and perceptions of education students in colleges for teacher
training, practicing teachers and researchers of mathematical education with regard to
teaching that develops critical thinking in mathematics; evaluation of these students’ and
professionals’ critical thinking functions in teaching, learning, and research. (3) Teaching
“Probability in Daily Life” and conducting the same research among all the strata of the
students’ population and not only among those who study mathematics at the higher level.
It definitely seems that in the last decade, there has been a rapidly growing awareness of
the importance of promoting the development of thinking skills in the Israeli educational
system, and the system has been making considerable progress towards integrating the
curriculum learning materials that contribute to the development of higher-order thinking
skills. In 1994, the Ministry of Education recognized thinking skills as a distinct subject of
studies. This recognition lead to the establishment of a Subject Committee for Thinking
Skills, which is in charge of consolidating appropriate didactic materials, as is the case with
the rest of the academic subjects in the school system. The complex and ceaselessly changing
contemporary reality, which requires independent decision-making on a daily basis, makes it
extremely important to impart to students the ability to think critically. Critical thinking is
needed in every field of activity, as it allows the individual to deal with reality in a
reasonable, mature and independent way (Lipmann, 1991). The need for developing critical
thinking in different disciplines is anchored in the ideals of education for democracy, as our
freedom to think about and criticize the reality and society in which authors live is a form of
expression of our autonomy as individuals. Today this idea is even more vital, because of the
growing need to be capable of engaging in inquiry and evaluation based on rational
considerations regarding the various messages authors are exposed to in different areas of life
(Feuerstein, 2002, Perkins, 1992, Swartz, 1992). In the field of education, mathematics has
traditionally been considered a branch of knowledge particularly suited to the teaching and
learning of higher-order thinking skills, such as critical thinking. Mathematics curricula all
over the world, including Israel, identify the acquisition of these skills as one of their goals.
The idea that mathematics is a discipline suited to teaching critical thinking also appears in
the research literature. However, in spite of this assumption, very few empirical studies to
date have engaged with the question of whether the study of mathematics indeed develops or
even requires this mode of thinking. The answer to this question is far from being clear. The
present research tackles precisely this basic question, “Is it possible to develop critical
thinking in the framework of mathematics studies?”
Chapter 4 – Argument mapping is a method of visually diagramming arguments using a
'box and arrow' format with the aim of simplifying the reading of an argument structure and
facilitating the assimilation of core statements and relations. The current chapter presents the
x Chritopher P. Horvatha nd James M. Forte

findings of a controlled trial in which argument mapping training was compared with
hierarchical outline training as techniques for teaching critical thinking skills. Eighty-one
undergraduate psychology students were allocated to one of three groups: an argument
mapping group, an outlining group, or a control group and were tested on critical thinking
before and after an 8-week intervention period. Results revealed that students in the argument
mapping group scored higher than the control group at post-test on the critical thinking skills
of evaluation and inductive reasoning. Students in the outlining group scored significantly
higher than those in the control group on tests of analysis and inductive reasoning. There were
no significant performance differences at post-test between those in the argument mapping
group and those in the hierarchical summary group. Results are discussed in light of research
and theory on best practice in the cultivation of critical thinking.
Chapter 5 – The limits of the Gross Domestic Product (GDP), most of them known for a
long time and sources of an already flourishing [6] scientific literature, are the starting point
of this observation: centred on production, this indicator describes the incomes and their
evolution increasingly poorly, in particular due to the globalisation of economies; "defensive"
expenditures such as the reconstruction of pollution-damaged environment or prison expenses
are then considered as contributions to wealth; households' production, which is non-
monetary to a vast extent, is not taken into account; the production of the non-merchant
sector, assessed by the production costs, is poorly measured; its calculation relies on various
by partial "inputations" (a housing expenses is attributed to owner-households); it is based on
the notion of average and not of that of variance.
Chapter 6 – This study investigated the effectiveness of a training course in critical-
thinking instruction with an emphasis on four components that are most likely to bring about
teachers’ improvement in personal teaching efficacy and teacher behaviors during the
training. Eighty-two preservice teachers participated in a 16-week training session in this
study. Based on both qualitative and quantitative analyses, the findings suggest that providing
guided practice and generating reflective teaching are crucial to the successfulness of a
teacher training program and that a training course in critical-thinking instruction does, in
fact, produce more lasting effects if it simultaneously imparts professional knowledge, raises
personal teaching efficacy as well as heightens reflective teaching. At the same time, it is
found that professional knowledge and field practices are indeed decisive in teachers’ overall
improvement in personal teaching efficacy and teacher behaviors during teacher training.
Chapter 7 – Academic writing in the context of producing quality research articles is
something which all academics engage in and there is evidence of increased attention to
supporting the development of the writing and subsequent output of academics and research
students. However, while scholarly writing is learnt in complex ways, critical thinking is an
intrinsic part of such writing, and is highly valued across all the academic disciplines and
indeed is a high priority on both employability and citizenship agendas. However, in practice
the teaching of critical thinking is difficult and there is a lack of discussion about what it
means within the context of the writing process. This study describes a pedagogic
intervention with a group of academic staff to support the participants not only to explore
critical thinking in their own writing, but also to consider in depth how they would apply this
learning to their work with students in higher education. Within the context of an academic
writing module on a postgraduate programme for academic staff in higher education, an
action research approach was used with participants to improve their understanding of the
role of critical thinking in the academic writing process. The data suggests that the pedagogic
 Preface xi

intervention resulted in greater confidence in terms of participants’ critical writing skills and
also supported them to help their own students in the academic writing process. An
exploratory model is proposed for critical academic writing encompassing a series of
scaffolded in-class activities, virtual peer learning, and tutor feedback – culminating in the
publication and dissemination of individual practice-based educational research.
Chapter 8 – A crucial goal of Higher Education is to support students in developing their
ability to think critically. However, “critical thinking” can be an ambiguous expression. In
this chapter authors intend to clarify its meaning, as well as to characterise the most
significant indicator of critical thinking: higher-order questioning, as well as the relationship
between critical thinking and active learning. Authors also propose to shed light on the
teaching approaches which seem to inhibit students’ critical skills and to present teaching and
learning strategies to enhance critical thinking. Finally, authors will describe the curriculum
of a first year chemistry course at a Portuguese university aimed at fostering students’ critical
thinking through the encouragement of quality questioning. Authors will describe in detail the
teaching, learning and assessment strategies designed and implemented in this course.
In: Critical Thinking ISBN: 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 1-36 2011 Nova Science Publishers, Inc.

Chapter 1

RULES FOR REASONING REVISITED: TOWARD A

SCIENTIFIC CONCEPTION OF CRITICAL THINKING

D. Alan Bensley
Frostburg State University
Frostburg, Maryland, USA

ABSTRACT
Critical thinking (CT) has been widely acclaimed as an important educational
outcome, yet numerous problems remain concerning how to conceptualize it. A review of
the literature in this chapter reveals that many authors agree that CT involves skills and
dispositions related to good reasoning, but they often disagree about which specific skills
and dispositions are needed and in how these are organized. Closer inspection of other
terms commonly associated with CT such as “evaluation,” “analysis,” and “higher order
thinking,” reveals that they, too, are poorly specified and add little to the meaning of CT.
The lack of clarity and specificity of CT skills, dispositions, and other terms impedes the
instruction, assessment, and scientific study of CT and is only beginning to be addressed
through empirical research. Although research reviewed in this chapter shows that
explicit infusion of CT into subject matter instruction is more effective than less explicit,
immersion and traditional approaches, it also reveals the need for greater specificity in
identifying the principles and other antecedent conditions that improve CT. To address
these deficiencies, a rule-based approach to defining, instructing, and assessing CT is
proposed. The approach operationally defines CT skill as the appropriate use of relevant
rules and procedures for reasoning in a discourse context. It further assumes that CT is a
motivated process involving dispositions and self-regulation in the service of belief
formation and revision.

Keywords: critical thinking, disposition, metacognition, rules, skills.

2 D. Alan Bensley

INTRODUCTION
Most educators agree that improvement of critical thinking (CT) is an important
educational objective, but they often disagree on exactly what CT is and on how to teach and
assess it. In a way, CT is like Little Orphan Annie--people want to adopt her, but they are not
sure who she is or where she came from. An attempt to discover the origins of CT reveals that
multiple disciplines have contributed to this rich construct, but it has remained poorly defined
by many who use the term. The review that follows focuses primarily on the contributions
from philosophy, education, and psychology that have arguably made the most important,
unique contributions. It is not surprising that various disciplines have used different
terminology and approaches in their discussions of CT. What is more surprising is that while
many view CT as a legitimate construct, it remains a loose conglomeration of concepts
without a clear claim to scientific legitimacy. This has impeded progress in the scientific
study of CT, in general, and in the determination of what are the most effective ways to teach
and assess it, in particular.
Accordingly, the purpose of this chapter is to examine an approach that could help refine
the conceptualization of CT in ways that not only improve its operational definition as a
scientific construct but also facilitate its instruction and assessment. To better understand the
need for this approach, discussion begins with an examination of disagreements and problems
in conceptualizing CT. Then, discussion moves to how different approaches conceptualize CT
instruction and assessment, followed by a review of research on what are the most effective
approaches to teaching CT. This leads to a discussion of how using both general and
discipline-specific CT rules could provide a framework for aligning CT instruction with
assessment and promote the scientific study of CT. In particular, the approach can facilitate
the explicit instruction of CT principles, methods, and concepts infused into subject matter
instruction, an approach that empirical research has already shown is effective.

PROBLEMS IN CONCEPTUALIZING CT
Definitions of CT abound (e.g., Beyer, 1995; Ennis, 1987; Facione, 1990a; Fisher &
Scriven, 1997; Halpern, 2003, Kurfiss, 1988; Lipman, 1991; Moon, 2008; Paul, 1993). After
more than three decades of intense discussions about the nature of CT, disagreements about
some of its important attributes continue despite persistent complaints from those working in
various disciplines about the need to refine its conceptualization (e.g., Bailin, Case, Coombs,
& Daniels, 1999a; Bensley, 2009; Cody, 2006; Halonen, 1995; Johnson, 1992; Morgan, 1995;
Petris, 2004; Riddell, 2007; Williams & Worth, 2001; Yanchar, Slife, & Warne, 2008). As the
following review suggests, CT remains a construct in transition, in need of further integration
of concepts from philosophy, education, psychology, and other disciplines.
The early beginnings of CT, before it was CT and before there were disciplines, can be
traced to early Greek language and philosophy. According to The New Oxford American
Dictionary, the word “critical” derived from the Greek words, kritikos meaning “discerning”
originally from krinein meaning “judge, decide”. The modern word “criterion” came from
kriterion or “means of judging” (McKean, 2005, p. 451). The connection of these ancient
words to modern views is apparent in the idea that CT involves the use of criteria in making
 Rules for Reasoning Revisited 3

reasoned judgments (Beyer, 1995; Lipman, 1991) and in the modern emphasis in the CT
literature on the use of appropriate methods to reason well and make good judgments (e.g.,
Bensley, 1998; Halpern, 2007; Paul,1993).
Arguably, the first substantial philosophical contribution to CT came from the Greek
philosopher Socrates (469-399 B.C.) who developed a special kind of dialogue that used
reasoning to examine opinions, beliefs, and authoritative statements. Socrates emphasized
reflection on the quality of belief and thinking, an important aspect of CT found in
contemporary philosophical views (e.g., Ennis, 1987; Paul, 1993). Plato (428?-347 B.C.)
who recorded the work of his teacher Socrates further developed the rational approach,
seeking to find the truth behind appearances (Paul, 1993). In some ways, Plato exerted even
more influence through his training of Aristotle who played a pivotal role in the development
of reasoning and science. Aristotle (384-322 B.C.) invented syllogistic reasoning, helped
develop the dialectic, began treating scientific observation as evidence, discussed the
relationship between knowledge within oneself versus from other sources, and applied his
ideas about reasoning to the practical problems of good citizenship and the improvement of
reasoning. Of particular relevance to the arguments advanced in this chapter, Aristotle was
the first to extract and formulate rules for correct reasoning from discourse and observations
of thinking (Ryle, 1949). Like Aristotle, logicians have continued to derive logical rules from
analysis of the practice of argumentation and criticism.
Over the next several centuries, philosophers and rhetoricians applied many of Aristotle’s
ideas to improve public speaking, legal procedures, and reasoning in general. Although
skeptics later criticized Aristotle for being unduly optimistic about the use of reason to arrive
at truth, Aristotle’s emphasis on the “art of thinking” influenced numerous books on the use
and improvement of reasoning beginning with Cicero and Quintilian and extending to the
development of logic textbooks. For example, the Port-Royal Logic credited to Antoine
Arnauld and published in 1662 both extended and challenged parts of Aristotle’s approach.
Arnauld’s textbook incorporated the assessment of probability, a concern about the attitudes
related to good reasoning, and an attempt to balance skepticism with gullibility (Kennedy,
2004). His clearly written book used rules of reasoning to support the practice of thinking in
the way he believed people naturally think and was widely used in the Eighteenth and
Nineteenth Centuries.
In the Nineteenth Century, some instructors and authors of logic textbooks like James
McCosh viewed part of their mission to make logic accessible to beginning college students
and help them pursue the practical goal of applying logic. Philosophers in Great Britain had
revived interest in Aristotle, continuing a tradition that valued rhetoric and the art of thinking
(Kennedy, 2004). This approach also continued to assume that teaching students to reason
well in a logic course would help them reason well about many different topics, consistent
with the doctrine of formal discipline, a view originating with Plato, but developed by
Medieval scholars (Smith, Langston, & Nisbett, 1992). The doctrine of formal discipline
assumes that learning one challenging subject like Latin will help students improve their
reasoning in other subjects, that is, that general rules for reasoning acquired in one
challenging subject will readily transfer to learning of another subject. But increasingly,
knowledge in the sciences was becoming specialized; and distinct academic disciplines arose.
By the early part of the Twentieth Century, research by E. L. Thorndike, one of the founders
of educational psychology, had seriously challenged the doctrine of formal discipline and the
4 D. Alan Bensley

idea that rules for reasoning acquired in one discipline would readily transfer to another
(Thorndike, 1906; Thorndike & Woodworth, 1901).
Another trend in the Nineteenth Century that influenced the development of CT was an
increasing emphasis on the autonomy of individual thinkers. Immanuel Kant (1724-1804)
questioned the 2,000 year-old distinction between the knowledge one acquires through
personal experience and the knowledge acquired through testimony and other sources
external to the individual (Kennedy, 2004). He concluded that they did not differ in kind and
that both were subjectively experienced. This view tends to move much responsibility for
evaluating the quality of experience, testimony, and other sources of information to the
individual. The contemporary CT goal of educating students to be autonomous, self-regulated
learners and thinkers probably owes much to endorsement of these philosophical ideas in
Britain and the US.
At the beginning of the Twentieth Century, authors working in different areas helped
develop the concept of CT. In his influential 1906 book, Folkways, the sociologist William
Graham Sumner discussed the importance of critical habits of thought in education. He
decried the orthodoxy that develops when educators promote the codes, standards, and
fashions of their group resulting in endorsement of popular opinions that maintain “broad
fallacies, half-truths, and glib generalizations” (Sumner, 1940, p. 631).
More influential, however, was the work of John Dewey, the student of pragmatist and
philosopher, Charles Sanders Peirce. Like Peirce, Dewey viewed scientific inquiry as a good
model for thinking and its improvement. Dewey (1910) introduced the term “reflective
thinking” to emphasize the importance of actively reflecting on the quality of beliefs,
knowledge, and other products of thinking. His ideas about thinking resonate in recent
conceptions of CT despite the fact that he never used the term “critical thinking,” per se.
Dewey made clear that reflective thinking should be integrated into content learning in the
schools and emphasized the importance of fostering certain attitudes or what many today call
CT dispositions. The influence of Dewey is apparent in one of the most commonly cited
definitions of CT in use today offered by philosopher, Robert Ennis, who defines CT as
“reasonable, reflective thinking focused on deciding what to believe or do” (Ennis, p. 10,
1987). Dewey may also be the source of the view that the classroom should be a place for a
community of inquiry (Lipman, 1991) that has influenced some efforts to reform the
classroom along CT lines. His emphasis on the importance of maintaining student interest and
encouraging students to reflect on the content they study are still emphasized (Bensley, 2010;
Kurfiss, 1988).
Educator Edward Glaser followed up Dewey’s early work by beginning efforts to assess
CT and demonstrating empirically that instruction could improve CT skill. Like Dewey,
Glaser argued that CT involves certain attitudes (Glaser, 1941). The development of the
Watson-Glaser Critical Thinking Appraisal marked the beginning of efforts to assess CT with
standard instruments, primarily evaluating argument analysis skills applied to everyday
questions and situations. In the years that followed, the skills aspect of CT came to dominate
efforts to teach and assess it (Bailin et. al., 1999a). Indeed, many experts view CT as a skill or
having a skill component (e.g., Beyer, 1995; Ennis, 1987; Facione, 1990a; Fisher & Scriven,
1997; Halpern, 2003, Kurfiss, 1988; Lipman, 1991; McPeck, 1990; Paul, 1993; Seigel, 1988).
Given the longstanding influence of philosophy and rhetoric on discourse in both
scholarly and professional settings, it is not surprising that the skills most associated with CT
are reasoning skills. Many of the same experts who define CT as involving skill also maintain
 Rules for Reasoning Revisited 5

that thinking critically involves good reasoning, reasoned judgment, or taking a rational
approach (e.g., Beyer, 1995; Ennis, 1987; Facione, 1990a; Fisher & Scriven, 1997; Halpern,
1998, Kurfiss, 1988; Lipman, 1991; McPeck, 1990; Paul, 1993; Seigel, 1988). The
development of the informal logic movement in philosophy commonly associated with CT
has focused on teaching people to reason well and to identify fallacies in natural language
discourse (Eemeren et al., 1996). This admittedly selective review of the definitions of CT
authorities suggests that most agree that CT involves certain skills, especially skills for
reasoning well.
The focus on good reasoning in CT comes from a longstanding tradition in philosophy
and rhetoric on the art and practice of good thinking. Many philosophers view rules of logic
as normative in the sense that they are rules that should be followed for a person to be rational
or reasonable. In this view, thinking is judged in terms of how well it reaches certain
standards or criteria for what is considered sound or good reasoning. Reflecting on the quality
of thinking in relation to criteria and standards is important to self-correction in CT (Lipman,
1991). The normative view of good reasoning often tends toward a prescriptive emphasis
when philosophers and educators recommend students follow certain principles and rules to
produce thinking that meets prescribed standards. In contrast, psychologists usually take a
descriptive approach when scientifically studying how people think, sometimes investigating
how well people can use the rules of reasoning (Galotti, 1989) and documenting their
thinking errors in relation to norms. Other psychologists, however, take a more prescriptive
approach to rules, studying how teaching of rules might be used to improve CT skills.
Clearly, many philosophers, psychologists, and educators agree that acquiring reasoning
skills is important to CT, but disagreements multiply when those defining CT seek to identify
specific skills that should be part of a critical thinker’s skill set (Bailin et al., 1999a).
Taxonomies of CT skills often list many of the same skills, such as argument analysis skills,
clarifying/defining the question, and assessing the credibility of sources (e.g., Ennis, 1987;
Facione, 1990a; Halpern, 1998; 2007). The same taxonomies also show many disagreements
about other skills they specify. For example, besides argument analysis skills, Halpern
(1998) has listed skills such as “skills in thinking as hypothesis testing” and skills in thinking
about “likelihood and uncertainty” (Halpern, 1998, p. 452). In a more recent, comprehensive
taxonomy for a book on critical thinking in psychology, she has listed specific skills such as
“how to develop an awareness of biases in memory”, “how to isolate and control variables in
order to make strong causal claims”, and “how to use graphs, diagrams, hierarchical trees,
matrices, and models as solution aids” to problem solving (Halpern, 2007, pp. 6-7). In
contrast, Ennis (1987, pp. 14-15) listed “making value judgments,” “identifying and handling
equivocation,” and “employing and reacting to fallacy labels” not listed by Halpern. In
contrast to both Ennis and Halpern, a Delphi panel of CT experts assembled by the American
Philosophical Association (APA) did not specifically list any of these as skills.
To some extent, differences are likely due to the disciplinary focus of the authors, i.e.
Halpern is a psychologist and Ennis is a philosopher. In her latest taxonomy, Halpern (2007)
has included skills for problem solving, decision making, and creativity, putting these in the
generic terminology of cognitive psychology. In contrast, Ennis (1987) listed no skills for
creativity although he has acknowledged a connection between CT and creativity, as did the
APA Delphi panel (Facione, 1990a). Although the APA panel, composed mostly of
philosophers, acknowledged that CT is related to problem solving, decision making, and
creative thinking, like Ennis, they focused on argumentation and reasoning skills (Facione,
6 D. Alan Bensley

1990a). This raises additional questions about the relationships between reasoning, problem
solving, decision making, creative thinking, and CT, all of which are generic labels,
themselves.
At an even more general level, experts use different terms to refer to skill. For example,
the APA’s panel of experts referred to evaluation and other skills as cognitive skills (Facione,
1990a). In contrast, Ennis (1987) used the term “ability” to refer to what many, and perhaps
even he, would sometimes call skills. The distinction is not necessarily trivial. Skill
sometimes refers to an ability acquired through learning and practice, and ability is often
associated with a basic intellectual aptitude. Clarification of such terminological differences
could help advance the scientific study of CT.
Halpern (2007) has observed that a variety of groupings of CT skills can be made
depending on the context and reason for the grouping. Although Halpern has constructed her
taxonomies with cognitive psychology in mind, no author of a taxonomy has used empirical
methods to determine which skills are listed and how they are organized. This is related to the
question of how general are CT skills. It further raises the question of whether CT skills
might actually be composed of many subskills or a large set of microskills that vary across
disciplines, tasks, and other contexts. Moreover, as Hayes (1985) noted, many different
thinking strategies may be used to do thinking tasks; and it is not clear how thinking strategy
use is related to CT skill acquisition, further complicating efforts to scientifically study CT
skills.
Another important way CT has lacked clarity and specification has been its persistent
association with “higher order thinking” (e.g., Barak & Dori, 2009; Bissell & Lemons, 2006).
The cognitive processes or skills that constitute higher order thinking typically go unspecified
and often generally refer to problem solving, decision making, reasoning, and creative
thinking. Similarly, educators have often associated CT with higher levels of Bloom’s
Taxonomy such as evaluation, application, and synthesis. This appeal to Bloom’s Taxonomy
is misdirected because Bloom’s taxonomy is a scheme for describing kinds of assessment
objectives at different levels, and does not address the problem of specifying CT skill as an
antecedent condition. As noted by Ennis (1987), the vague use of higher order thinking and
Bloom’s Taxonomy offers too little guidance for how to approach CT and insufficient criteria
for judging whether CT has been displayed.
Some authors have attempted to elaborate the psychological basis of higher order
thinking and the levels of Bloom’s Taxonomy. Halpern (2007) has stated that “higher order
cognitive skills are relatively complex, require judgment, analysis, and synthesis; and are not
applied in a rote or mechanical manner” (Halpern, 2007, p. 6). More specific correspondences
between Bloom’s levels and research on learning and cognition have been developed by
Anderson and Krathwohl (2001) and Marzano (2001). These revised taxonomies focus on
cognitive categories within the knowledge domain, and are proposed to help teachers develop
learning and skill objectives. Neither, however, has been developed according to a cognitive
psychological theory or clear line of research. Although they should be lauded for seeking to
be consistent with psychological research, both need further corroboration from research
studies (Moseley et al., 2005). As with Bloom’s original taxonomy, it would be a mistake to
assume that the newer taxonomies have identified kinds of cognitive processes that produce
CT.
In general, the lack of specificity in the language used to discuss CT is a problem for the
scientific study of CT. Scientific progress is made as scientists begin with a phenomenon
 Rules for Reasoning Revisited 7

described in natural language terms, such as the term “reasoning,” and then operationally
define it in terms of observable behaviors that can be measured, such as the appropriate use of
certain rules for reasoning. The use of measurement procedures allows cognitive and
educational researchers to refine the meaning and precision of terms.
In contrast, some using Bloom’s Taxonomy generalize from vague terms originally used
to describe kinds of assessment objectives such as evaluation, analysis, and synthesis to the
construct of CT as a kind of thinking. For them, tasks requiring analysis or evaluation are
prima facie CT tasks. Assuming that such tasks call forth CT does little to insure the construct
validity of CT measures. It is better to begin with a clear conception or theory of the construct
and then develop tests and measures designed to assess it. The assumption that evaluation
tasks elicit CT runs into trouble when an evaluative judgment is affectively-based and does
not require reasoning such as evaluating which of two models of car is better based on a
criterion of personal preference. Likewise, an analytical judgment that involves only a simple
categorical distinction, such as analyzing American cars into subordinate categories of
“Fords” and “Chevrolets” would require little thought and conscious effort for adults.
Psychological conceptions of CT often assume it is effortful thinking (e.g., Halpern, 1998);
and reasoning likely involves conscious attention (DeWall, Baumeister & Masicampo, 2008).
If “evaluation” is to be a scientifically useful CT term, it should be further specified in
the context of performance of actual reasoning tasks and then empirically studied. Keeley and
Browne (1986) examined an evaluative level task by asking 37 college seniors from different
majors to “critically evaluate” a 550-word essay on the value of attending college. Keeley and
Browne operationally defined CT at the evaluative level as the ability to detect a number of
reasoning problems on the essay task. Although many of the seniors successfully posed
questions about sampling and logic, many also missed other reasoning problems and failed to
identify ambiguous language and assumptions in the essay. These results may suggest that the
students differed from Keeley and Browne in their concept of critical evaluation and what it
entails. This may have prevented students from using CT skills they possessed. Alternately,
students may not have possessed the requisite skills. While this research has provided a good
starting point as a means to add empirical meaning to what skills are involved in critical
evaluation in the context of a specific CT task, it also highlights the need for clearer
specification of CT skills. Future studies should decouple students’ CT reasoning skills from
their knowledge of their skills and when to use them. Unless a vague term like “evaluation” is
further defined and studied as a cognitive operation or process that produces CT and is not
just defined as a general label for what people are presumed to do during task execution, it
adds little to the scientific meaning of CT.
This points to a more general kind of conceptual error commonly made in viewing CT as
a skill. As Bailin et al., (1999a) have noted, CT skill is often conceived as an outcome
indicating the degree to which a person meets standards in performance on a thinking task.
While this normative move is useful in anchoring CT performance to standards and criteria, it
can lead to a category error when it is assumed that performance reflects an actual cognitive
ability or process. Many experts describe CT as a cognitive ability (Facione, 1990a), but few
have actually studied it as such; for an exception see the research of Stanovich and his
colleagues (e.g., Stanovich & West, 1997; Toplak, & Stanovich, 2002). Nor have educational
tests been developed that can isolate and diagnose specific cognitive problems with reasoning
(Leighton & Gierl, 2007).
8 D. Alan Bensley

Many authors would agree that acquiring reasoning skills is important to CT, but some
have objected that there has been too great an emphasis on logic and reasoning at the expense
of other important aspects (Walters, 1994). In particular, they have argued that knowledge,
belief, and dispositions have been neglected. In the tradition of Dewey and Glaser, many now
agree that CT involves having certain dispositions that may be conducive to thinking
critically (e.g., Bailin et al., 1999a; Beyer, 1995; Ennis, 1987; Facione, 1990a; Halpern, 1998,
Kurfiss, 1988; Lipman, 1991; McPeck, 1990; Paul, 1994; Perkins & Salomon, 1993; Seigel,
1992). CT dispositions are broadly viewed as individual differences in attitudes, traits, habits
of mind, and cognitive style related to CT. Studies of college students have shown that CT
skills and dispositions are distinguishable, and both contribute to CT (Clifford, Boufal, &
Kurtz, 2004; Taube, 1997).
As with CT skills, despite considerable agreement that dispositions are important to CT,
authors disagree more when they try to identify specific dispositions. For example, many
endorse open-mindedness (e.g., Ennis, 1987; Halpern, 1998; Paul, 1993) while fewer mention
fair-mindedness (e.g., Paul, 1993; Fisher, 1991). Some identify skepticism (e.g., Beyer, 1995;
McPeck, 1981) and others flexibility in thinking (Halpern, 1998; Stanovich & West, 1997).
Ennis (1987) has identified 14 dispositions in his taxonomy such as “seeking reasons,” using
one’s critical thinking abilities” and “being open-minded” (Ennis, 1987, p. 12). In contrast,
Halpern (1998) has offered five including “open-mindedness” or “flexibility,” “habitual use
of plans…,” “willingness to engage in and persist at a complex task,” willingness to abandon
nonproductive strategies in an attempt to self-correct,” and an “awareness of the social
realities that need to be overcome…” (Halpern, 1998, p. 452).
Based on the work of the APA expert panel and factor analysis of scores on the
California Critical Thinking Dispositions Inventory, Facione and Facione identified seven
dispositions including “open-mindedness,” “truth-seeking,” “CT self-confidence,”
“inquisitiveness,” “maturity,” “analyticity,” and “systematicity” (Facione & Facione, 1992,
pp. 2-3). Although factor analysis obtained seven general dispositions, this may not be a
complete set of CT dispositions, given that which factors are identified in a factor analysis
depends upon which items have been entered into the analysis. It is unclear whether a
complete set of general CT dispositions can be identified because of limitations to the
psychometric approach and because CT dispositions may be context-specific.
In this regard, Siegel (1992) raised the fundamental question of whether CT dispositions
are generalizable. As was true almost two decades ago when Siegel (1992) posed this
question, little research has been conducted on whether CT dispositions generalize to
different tasks and situations. Although Siegel tentatively concluded in favor of
generalizability, examination of the lives of some great critical thinkers suggests that the
generalizability of CT dispositions is limited (Bensley, 2006). For example, Alfred Russel
Wallace, a scientist with prodigious CT skills who discovered natural selection along with
Charles Darwin, also believed in the implausible claim of spiritualists that the dead could be
contacted through spirit mediums. When presented with clear evidence that mediums were
using legerdemain and trickery to produce the appearance of contact with spirits, Wallace
continued to believe in spiritualism. He seems to have been disposed to use his CT skills in
most of his scientific writings but was not disposed to fair-mindedly evaluate the evidence
concerning spiritualism and theistic claims. Although Wallace was willing to apply the idea
of natural selection to all non-human species, he was not disposed to entertain the logical
implication that humans evolved through natural selection, maintaining to the end that
 Rules for Reasoning Revisited 9

humans had undergone some sort of special creation or evolution, consistent with his theistic
beliefs.
Given that CT dispositions may not generalize to all situations or contexts, it is important
to identify the dispositions involved in performing specific reasoning tasks in particular
contexts. For example, one might hypothesize that the disposition, fair-mindedness, would be
necessary to critically evaluate the evidence on both sides of an argument (Paul, 1993; Fisher,
1991). Fair-mindedness might be especially important when strong beliefs are held. A
different disposition, systematicity, or the tendency to be diligent and take a systematic,
organized approach to inquiry might be needed more for an extensive and comprehensive
analysis of a question. For belief revision, open-mindedness and flexibility in thinking might
be especially important (Stanovich & West, 1997). People who are not open-minded may be
reluctant to consider a claim contrary to a strongly held belief and if they are not flexible in
their thinking will not revise that belief to be consistent with good evidence that refutes their
belief.
This last example highlights another objection to the traditional emphasis on CT skills,
that is, that it neglects the formation and revision of belief, an important function of CT. In
this regard, Paul (1993) has distinguished CT in the “strong sense” from CT in the “weak
sense.” Weak-sense critical thinkers have adequate skills for reasoning but do not use their
skills to revise mistaken, unsupported beliefs or may even use those skills to bolster or
promote their entrenched, mistaken beliefs. In contrast, critical thinkers in the strong sense
use their CT skills to develop their beliefs and revise them when a fair evaluation of the
relevant evidence shows their beliefs are mistaken. Changing one’s belief requires, not only
CT skill but also openness to alternative points of view and fair-mindedness in evaluating all
the relevant evidence.
This distinction between weak-sense and strong-sense critical thinkers is useful because
people have often been shown to evaluate evidence biased in the direction of their prior
beliefs (e.g., Lord, Ross, & Lepper, 1979; Kardash & Scholes, 1996; Stanovich & West,
2008). Lord, et al., (1979) had participants read mixed research evidence on the questions of
capital punishment and rate the quality of the evidence. They found that participants often
maintained their initial belief on the question, sometimes even increasing the strength of their
belief, despite good evidence contradicting it. Belief perseveration in the face of contradictory
evidence suggests that people find it very difficult to escape the biasing effects of prior belief.
Several studies suggest that CT dispositions are needed to escape the biasing effects of
prior belief. In one study, Stanovich and West (1997) had 349 college student participants
first rate their endorsement of several propositions. Then, participants completed measures of
CT dispositions for flexible thinking, open-mindedness, and other dispositions. Following
this, they evaluated several everyday arguments on the same propositions that they had
previously rated. Independently, several experts rated the strength of the arguments to
determine argument quality and later to infer evaluation ability. To measure cognitive ability,
Stanovich and West used scores on the SAT and a vocabulary test that served as a proxy for
an intelligence test.
Stanovich and West (1997) conducted a series of regression analyses of students’
argument quality scores that took into account their prior belief ratings to predict each
individual student’s argument quality with the beta weight used as a measure of their ability
to reason independently of their prior belief. They found that these scores were reliably
related to cognitive ability and a composite CT disposition measure called actively open-
10 D. Alan Bensley

minded thinking. After partialing out cognitive ability, actively open-minded thinking was a
significant predictor of the ability to evaluate the arguments, suggesting a unique contribution
of CT disposition.
Recently, Bensley, Spero, Kennedy, Murtagh, and Bernhardt (2010) investigated how
individual differences in CT skills and dispositions are related to belief revision. They
presented students with a literature review discussing whether people use only 10% of their
brains, a question about which students are likely to have a prior belief and interest. Students
rated their belief in the 10% myth before and after reading a literature review that presented
evidence on both sides of the question but which contained much high quality evidence
clearly refuting the 10% myth. To test their critical reading skill, students also took a critical
reading test about the review. To assess their CT disposition, participants completed measures
of flexible thinking, open-mindedness, and intellectual engagement. Consistent with previous
research, students generally did not revise their belief after reading the literature review and
completing the test. However, those students who scored better on the critical reading test
also tended to rate their belief in the 10% myth significantly lower after reading and taking
the CT test. Likewise, students with higher scores on a composite measure of CT disposition
tended to revise their belief more. These results suggest that people do not generally revise
their their beliefs in response to disconfirming evidence, but are more likely to do so when
they have more CT skill and are more disposed to think critically.
Taken together, the review of the CT disposition literature suggests that, not only do
many CT experts recognize the importance of both CT skills and dispositions but empirical
studies have supported the contribution of both. Clearly, more research is needed on the
relationship between CT skills and specific dispositions and on how specific CT skills and
dispositions contribute to belief revision, an important function of CT. Perhaps the most
important implication is that a more complete scientific approach to the study of CT should
include assessment of both CT skills and dispositions in the same individuals.
It seems doubtful that a person with CT skills, even one disposed to use those skills,
would think critically if he or she was not also aware of the need to use a specific skill in a
particular situation. From a psychological perspective, CT is a deliberate and purposeful
cognitive activity (Halpern, 1998) that involves regulation of one’s own thinking and
behavior to meet certain standards. A common theme among psychologists and educators is
that a person’s thinking should be self-regulated. Likewise, philosophers who emphasize
thinking as a reflective activity assume that such thinking involves active consideration of the
quality of one’s knowledge and beliefs with the goal of holding beliefs that are consistent
with reasonable evaluation of the relevant evidence (e.g. Dewey, 1910; Ennis, 1987: Paul,
1993). Along these same lines, Paul (1993) has described CT as thinking about thinking.
The psychological term that has come closest to capturing all of these ideas is
metacognition, originally proposed by Flavell (1976). Metacognition refers to the knowledge
a person has of his or her own cognitive processes and products and has become increasingly
associated with the ability to self-regulate cognition. A connection between metacognition
and critical thinking has been proposed many times (e.g., Halpern, 1998; McGuinness, 1990;
Swartz, 1989) but has only occasionally been tested, (e.g., Hanley, 1995; Ku & Ho, 2010).
See Mosely et al., (2005) and Tarricone (2011) for reviews of theories connecting
metacognition with CT.
The two components of metacognition most important to self-regulation in CT are
monitoring and control. Metacognitive monitoring involves observations and judgments a
 Rules for Reasoning Revisited 11

person makes about his or her own progress toward some cognitive goal. Accurate monitoring
is necessary for making appropriate decisions about how to adjust learning, thinking, or other
cognitive activities to reach the goal, that is, for the regulation and control of cognitive
processing. For example, knowing how well one is thinking on a task can help a person
decide if the right thinking strategy is being used or whether changes are needed to improve
thinking.
Based on this reasoning, it might be expected that people who are unaware of what they
do not know on a test will not modify their approach to improve their knowledge and skill.
Kruger, Dunning, and their colleagues have extensively studied self-monitoring of test
performance on reasoning and a variety of other tests (Dunning, Johnson, Ehrlinger, &
Kruger, 2003). They have consistently found that those people who performed better on
various tests were also more accurate in estimating their performance on those tests. Dividing
test takers into quartiles based on their performance, they found that those in the lowest
quartile who did the worst on a test tended to greatly overestimate their scores. As test
performance improved, people became increasingly accurate in estimating their actual scores.
Interestingly, although test takers in the top quartile tended to be much better calibrated and
did not overestimate their scores as did poorer performers, they instead tended to slightly
underestimate how well they did.
To account for these results, Kruger and Dunning (1999) have proposed that poor
performers are doubly cursed. Not only do they lack the knowledge and skill of better
performers, but their lack of knowledge and skill robs them of the ability to know that they
lack knowledge and skill; hence, they are also more deficient in accurately monitoring their
test performance.
Few studies have directly examined the relationship between CT skill and test monitoring
accuracy, but Kruger and Dunning (1999) did examine reasoning test performance and
students’ ability to accurately estimate their test performance in two experiments. In one, they
examined the monitoring accuracy of 45 college students who took a logical reasoning test
they constructed from LSAT test preparation questions. As before, they found that those who
did the worst on the test greatly overestimated their performance on the test while the best
performers were better calibrated but tended to underestimate how well they did.
In a follow-up study of disjunctive reasoning performance on the Wason selection task
(Wason, 1966), Kruger and Dunning tested whether improving skill on the selection task
would produce better self-monitoring in a group trained to do the selection task as compared
to a control group not receiving the special training. In the first phase of their study before
training, participants showed the usual pattern of estimation errors. The poorest performers
greatly overestimated the number of selection problem questions they answered correctly
while those who performed better were better calibrated, and the best performers
underestimated their scores. In the second phase, Kruger and Dunning randomly assigned half
of the students to a group that received a deductive reasoning training packet adapted from
Cheng, Holyoak, Nisbett, and Oliver (1986) while the other half did a filler task. After
training, they had all participants look at their original tests again and re-estimate how many
they had correctly answered. This time, the accuracy scores of the poor performers in the
training group rose to nearly the expert levels of the best performers with calibration
increasing as performance increased. Monitoring accuracy of the untrained group showed no
improvement. Kruger and Dunning (1998) interpreted their results as suggesting that the
training improved reasoning skill on the Wason selection task and this also improved self-
12 D. Alan Bensley

monitoring of that skill. When poorer performers were trained, they acquired knowledge that
helped them determine which questions were correctly answered and which were not,
resulting in improved calibration.
These results suggest that self-monitoring may be involved in the regulation of CT and
that researchers seeking to comprehensively study CT should examine it and other
metacognitive variables along with CT skills and dispositions. Understanding the contribution
of metacognition also has implications for questions related to how best to teach CT so that
knowledge and skills acquired through instruction transfer to other contexts, especially real-
world applications. It has been suggested that improving metacognitive skill promotes
transfer of CT (Billing, 2007; Halpern, 1998). As discussed next in the review of instructional
practices that promote CT, practices that call attention to CT principles, rules, skills, and
dispositions externalized in the discourse of thinking may help students monitor their
thinking.

APPROACHES TO TEACHING CT
It should be clear from the previous discussion that CT is not just a kind of thinking that
goes on inside the head but is externalized in various kinds of discourse and contexts. CT is
good thinking found in formal dialogues and exchanges in scholarly forums, courtrooms,
professional discussion of problems, in everyday discussions of political questions, and in
textbooks and classrooms. Indeed, those who teach and assess CT in various disciplines look
for evidence of it in the use of certain principles, rules, procedures, language, and other
externalized products of thinking. Given the complexity of CT and the importance of teaching
it well in different disciplines and different kinds of discourse, the task seems daunting. Are
there normative rules, principles, standards, and criteria that should be taught to students in all
disciplines? How should these be integrated into the discourse of instruction and assessment?

What is the Best Way to Teach Critical Thinking?

Ennis (1989) has provided a useful scheme for classifying approaches to teaching CT in
terms of whether or not CT skills and dispositions are made explicit and how CT instruction
is related to subject matter instruction. Using these two dimensions, Ennis (1989) classified
approaches into four different types (general, immersion, infusion, and mixed). The general
approach focuses instruction on explicitly teaching principles for thinking, usually separate
from regular course content instruction and sometimes in abstract form as in a formal logic
course. A second approach called “immersion” does not make rules or principles of thinking
explicit but instead relies on intense, thoughtful exposure or immersion to CT in subject
matter. Like the general approach, a third approach called “infusion” employs explicit
instruction of rules, principles, and knowledge related to thinking skills, but students receive
this explicit instruction as they study relevant subject matter and are encouraged to think
deeply about it. Finally, the mixed approach combines explicit teaching of CT rules and
principles as a separate thread of instruction with either immersion or infusion.
 Rules for Reasoning Revisited 13

RESEARCH ON APPROACHES TO TEACHING CRITICAL THINKING

Very few studies have empirically compared the different approaches (e.g., Angeli &
Valanides, 2009; Valanides & Angeli, 2005). In one study Angeli and Velanides (2009)
compared students taught with the general, immersion, and infusion approaches on their
ability to write a CT discussion of an ill-defined issue. Before instruction, students took the
California Critical Thinking Skills Test (CCTST) of Facione (1990b) to assess individual
differences in CT skill. Instruction in the general, infusion, immersion, and control groups
involved having all students discuss in dyads an essay on the influence of the media on
society. Only students in the general and infusion groups got brief lectures that explicitly
discussed five different CT skills. The infusion group received guided instruction in the use of
the skills while the immersion group was engaged in Socratic questioning about the essay
without any explicit mention of the five skills. The control group received no explicit
instruction in skills or relevant guided instruction but instead simply prepared an outline of
the essay. After instruction, students received a different essay on whether drugs should be
legalized and worked collaboratively in their dyads to develop an outline presenting their
joint position on the question. Angeli and Velanides developed a CT rubric to compare the
outlines of the four groups. Statistically controlling for students’ initial CCTST performance,
they found that the infusion and immersion groups performed significantly better than the
control group on the outline task using the rubric to score the outlines. Although both the
infusion and immersion groups were significantly better than the control group and had large
effect sizes, the infusion group had the largest effect size.
More extensive support for the advantage of infusion over immersion came from a large
meta-analysis of many CT studies examining the effect sizes of general, immersion, infusion,
and mixed approaches (Abrami et al., 2008). Abrami and co-workers found that the effect
size for the infusion approach was larger than that of either the immersion or general
approaches. However, the mixed approach combining infusion with explicit instruction of CT
as a separate thread in the course had the largest effect size of the four (Abrami et al., 2008).
The results of this meta-analysis suggest that instructors should design courses in which CT is
explicitly taught as a separate thread of instruction and infused into course content instruction.
Abrami et al., (2008) obtained findings related to pedagogical grounding that might also
be attributed, at least in part, to explicit instruction. Specifically, they compared studies on
how much training in CT the instructor received, how extensive were observations made
relating course activities to skill development, how detailed was the description of the course
curriculum in relation to CT skill objectives, and when CT was simply listed as a course
objective with no supporting information. They obtained by far the largest effect size for the
studies in which instructors had more CT training followed by those with more extensive
observations while those studies that merely mentioned CT among course objectives had the
smallest effect size. It seems plausible that those instructors with more CT training and who
made more extensive observations related to skill development would be in much better
position to make CT skills and principles explicit; however, this interpretation awaits direct
testing.
It should be noted that the comparison of instructional components in studies is
inherently difficult because they often lack precise specification. Moreover, the approaches
taken by instructors may not fit neatly into categories such as the four types of CT
14 D. Alan Bensley

instructional approaches Ennis (1987) proposed. Unfortunately, the meta-analysis of Abrami

et al, (2008) did not permit very elaborate characterization of either “explicit” or “infusion”.
Consistent with the suggestions of Abrami et al., future studies should seek to clearly identify
the instructional elements that promote CT, especially the principles that should be made
explicit.
Instructors can make CT principles explicit in several other ways not identified by
Abrami et al, (2008) or Ennis (1989). In general, explicit instruction is any kind of
instructional communication that draws attention to the use of a principle or rule, making it
more likely to be accessed, reflected upon, and deliberately used. Besides explicitly stating
CT goals and objectives, CT discourse may include special problems and questions that
illustrate certain rules, strategies, and procedures. These can be solved problems or writing
samples that model and annotate correct use of CT rules for argumentation. Assignments may
also require students to find the structure and use of rules in problems (Halpern, 1998).
Initially, when students are first learning several new CT rules, instructors can make rules
explicit and organize the rules through use of scaffolding structures (Rosenshine & Guenther,
1992) such as the tables described by Bensley (2010) for organizing rules and standards of
evidence used in psychological discourse. Likewise, instructors may provide rubrics that
highlight important criteria and rules for producing good written products or other
performances (Facione & Facione, 2009). Other methods for explicit instruction involve
coaching that guides application of and reflection on use of rules. In general, the effects of
explicit rule provision should be enhanced when students practice using the rules and then are
given feedback about their use in CT assignments and assessments.
As described later, these suggestions for making CT rules explicit can be readily adapted
to the infusion of CT into course content and work. Yet, effective instruction that explicitly
infuses CT rules and principles into course content requires many instructional decisions
regarding sequencing, placement, and language use in the infusion of specific CT rules into
various forms of classroom discourse. Although detailed discussion of these issues goes
beyond the scope of this chapter, the following review of some studies from psychology
illustrates how the explicit infusion of CT rules into courses can produce substantial increases
in CT skill, replicating and extending this basic finding from Abrami et al., (2008). These
include studies in which students received explicit CT rule instruction infused into their
courses that improved argument analysis skills (e.g., Bensley et al., 2010; Nieto & Saiz, 2008;
Solon, 2007) critical reading skills (Bensley & Haynes, 1995), and skills for analyzing the
quality of research (Penningroth, Despain, & Gray, 2007).
In two of these studies, explicit infusion produced gains on the Cornell Test of Critical
Thinking-Form Z, (CTCT) a general CT test assessing argument analysis skill (Nieto & Saiz,
2008; Solon, 2007). Solon compared two very similar general psychology classes, one
receiving explicit infusion of CT into their course work and the other control class treated the
same but not receiving the CT instruction. The CT class received explicit instruction of
general logical rules like modus tollens, wrote argumentative essays, read from the CT
textbook by Halpern (2003), and completed exercises accompanying it. The CT class, not
only did significantly better than the control class on the CTCT but also performed as well on
a test of general psychology. These results suggest that explicit infusion of rules and other CT
materials into course instruction was effective in efficiently improving general CT skills
without sacrificing acquisition of regular course content. In another well–controlled study,
 Rules for Reasoning Revisited 15

Nieto and Saiz found that the structural training of Halpern (1998) with practice exercises and
feedback also produced significant gains on the CTCT and on their own reasoning test.
Other studies, focusing less on general CT skills, have used explicit CT instruction
infused into psychology course work to demonstrate improvement in CT skills related to
psychology (e.g., Bensley et al., 2010; Bensley & Haynes, 1995; Penningroth, Despain, &
Gray, 2007). In one of these, Penningroth, Despain, and Gray (2007) examined how rules and
principles for thinking critically about the quality of psychological research and information
could be infused into a course on problem solving in psychology. Penningroth et al. used CT
principles for evaluating psychological information from Smith (2003), explicitly covering
each rule with subject matter that illustrated its application as students covered chapters from
a CT textbook by Stanovich (2004). Students practiced applying the principles to evaluate
psychological studies of varying quality. After instruction, students in the problem solving
course performed significantly better than those in a comparable class on a test developed by
Lawson (1999) and his colleagues that assessed correct identification of violations of the rules
in different psychological examples.
Bensley and Haynes (1995) used explicit CT instruction and infusion to teach students to
critically evaluate the quality of evidence presented in literature reviews on psychological
questions. One introductory psychology class was explicitly taught CT rules for identifying
strengths and weaknesses of different kinds of evidence using criteria and standards of
evidence found in Bensley (1998). The CT class also wrote a short literature review on a
question based on prewriting that organized the kinds of evidence to be presented in their
review. A second introductory psychology class, using the same textbook but taught by a
different instructor, received no explicit CT instruction. After instruction, the CT-infused
class correctly organized and labeled more evidence from a literature review they read than
the control class when both were compared to their pre-test scores on the same task. On a
second task, all students wrote a generic outline for a hypothetical CT paper. The outlines of
students in the CT class used more appropriate CT language as judged by CT experts than the
outlines constructed by the control class.
A common component of the studies just reviewed was that explicit CT instruction
involved the teaching of rules, criteria, and principles for reasoning that were contextualized
in the discourse of the subject. Often, instruction in those studies contained components of
effective, direct, and guided instruction. This instruction provided students with models
demonstrating the use of rules in relevant subject matter, gave them practice in the form of
exercises focused on assessing skills, and then gave them feedback. In these respects, it also
resembled approaches recommended by Angelo (1995) and Beyer (1997) that contain direct
instructional components. Although direct instruction has been shown to be highly effective
in teaching subject matter (Walberg, 2006), some have objected that it is ineffective in
teaching higher-order cognitive skills, as Doyle (1983) noted.
To investigate whether CT rules could be directly taught in a psychology course,
Bensley, Crowe, Bernhardt, Buckner, and Allman (2010) used a mixed approach called direct
infusion (DI). The DI approach incorporated explicit teaching of CT rules embedded into
subject matter instruction as well as other components of effective instruction including
guided instruction (Mayer, 2004), direct instruction (Walberg, 2006) and feedback from
formative assessments (Black & Wiliam, 1998). Specifically, instruction targeted and made
rules for argument analysis explicit, teaching those rules and principles infused into
psychology subject matter instruction, guiding instruction through exercises that provide
16 D. Alan Bensley

practice in using the rules followed by feedback, and then formative assessments for
evaluating skill acquisition with additional feedback.
To test the effectiveness of DI on acquisition of argument analysis skills, Bensley et al.,
(2010) compared research methods classes receiving and not receiving DI of argument
analysis rules into their classes on an argument analysis test administered before and after
instruction. This test called Analyzing Psychological Statements (APS) served as a
summative assessment of skill acquisition, measuring the ability to correctly apply
argumentation rules in everyday and psychology-related situations. It included items
assessing the ability to recognize kinds of evidence, evaluate different kinds of evidence,
distinguish arguments from non-arguments, and find assumptions in examples. APS items
sampled the same skills taught to the CT group but differed in content from examples used in
instruction and earlier formative assessments.
Bensley et al. found that the CT-infused research methods class showed significantly
greater gains on the APS than students in traditional research methods not receiving explicit
CT instruction. Supporting the conclusion that it was DI that led to the CT-infused group’s
gains on the APS was the similarity of the groups in overall academic experience, GPA, SAT,
and CT disposition; however, differences in instructor and textbook across the research
methods classes, weaken this conclusion.
Testing under better controlled conditions, Bensley and Spero (2011) assessed the
effectiveness of directly infusing CT skills for both argument analysis and critical reading in
which the same instructor taught a CT-infused class and two control classes all using the
same textbook. Specifically, one class received explicit instruction of CT rules for argument
analysis and critical reading directly infused into course work. A second control class
received memory improvement instruction, and a third received traditional instruction
focused on content knowledge acquisition. Before instruction, all three classes were pretested
with the APS and a new critical reading test (CRT) that had them critically analyze a
literature review on a psychological question. After each test, students estimated their score,
assessing their ability to metacognitively monitor their test performance. After instruction at
the end of the semester, all three groups were post-tested with the same measures. Planned
comparisons revealed that the CT-infused class showed significantly greater gains than the
two control classes on both the APS and the CRT. Also after instruction, the CT–infused class
showed significantly greater gains in their metacognitive monitoring accuracy on the APS
than the two control classes. These results replicated under better controlled conditions the
findings of Bensley et al., (2010) and extended them to show that DI was effective in
improving critical reading. DI may also have improved monitoring, suggesting a relationship
between acquisition of argument analysis and metacognitive skills.
In summary, many studies have shown that explicitly teaching CT, infused into subject
matter instruction, is more effective than immersion (Abrami et al, 2008). In particular, one
kind of explicit infusion called direct infusion that incorporates components of guided and
direct instruction has been shown to be more effective than traditional instruction (e.g.,
Bensley & Spero, 2011; Bensley et al., 2010; Nieto & Saiz, 2008; Penningroth, Despain, &
Gray, 2003; Solon, 2007). Still, questions remain about how best to explicitly infuse CT rules
and principles, especially regarding the nature of the rules and principles to be taught. One
question with important implications for the effectiveness of instruction is whether CT rules,
skills, and dispositions are general or specific to subject or context.
 Rules for Reasoning Revisited 17

THE GENERALITY-SPECIFICITY DEBATE

Stephen Norris (1992) has provided useful distinctions for approaching the problem of
the generalizability of CT. He has contended that CT is generalizable when it has satisfied
four conditions. The first condition concerns whether CT can be shown to be a kind of
“thinking-in-general” that can be abstracted from the objects and particulars of thought. The
second is the extent to which at least some commonality in CT is found across topics,
subjects, and fields. The third concerns whether CT contributes a significant fund of resources
for dealing effectively with each of these various fields, subjects, and concerns” (Norris,
1992, p. 1). Finally, the fourth concerns the extent to which CT transfers.
Examining the four conditions from the perspective of rules of reasoning can help
provide an empirical basis to testing whether conditions have been met for at least the last
three conditions. Regarding the first condition, it makes sense to view CT as a kind of
thinking-in-general inasmuch as it is a form of applied reasoning that can be found and used
in many different kinds of discourse. Philosophers have analytically extracted many rules for
reasoning that exist abstractly as sets of normative rules and principles for a kind of thinking-
in-general and that can be further applied in different subjects. Regarding the second
condition, some of the same general rules and principles of reasoning associated with CT can
be found across topics, subjects, and fields suggesting that they are needed in common. The
third condition that CT contributes a significant fund of resources to effectively deal with
various fields and subjects has been at least partially met. In response to complaints that
traditional education has not helped students learn to think effectively, educators have
developed many CT instructional, assessment, and other materials for different subject areas.
Some of these materials have been general in that students from different subject areas have
benefitted from them. Empirical research has often supported the effectiveness of this
approach as compared to traditional approaches. Finally, the fourth condition that CT skills
transfer has received some empirical support. As will be discussed shortly, learning to apply
rules and principles of CT (many of them general) in certain subject areas, discourse types,
and contexts has transferred to ones that differ to some extent; but transfer has often been
obtained only under very special conditions (Billing, 2007).
For purposes of our discussion of rules of reasoning, the positions on the generality-
specificity debate can be contrasted as two extremes. Those holding the generalist view
maintain that learning a set of general rules helps a person think in different disciplines
regardless of the content or context. In contrast, specifists maintain that CT skills and rules
are context-bound or specific to subject matter. Advocating an early specifist position,
Toulmin (1958) argued that different disciplines have their own discipline-specific criteria
and rules for evaluating evidence. Refining the specifist position, McPeck (1981; 1990)
argued that all CT is inextricably associated with specific content and skills in a domain and
so learning the rules for one domain helps little to think in another discipline, that is, they do
not transfer.
Considerable research on subject or domain specificity has supported the specifist
position. It is plausible that epistemological differences in disciplines could impair transfer of
CT from one to another. Analyzing discourse from different disciplines, Moore (2004) found
that the features of argumentative discourse in three disciplines differed in object, content,
and referent of evaluation, suggesting subject specificity in disciplinary discourse. Another
18 D. Alan Bensley

study by Renaud and Murray (2008) found that when psychology students studying with
general CT test questions were compared to those studying with subject-specific CT
questions, the students showed greater improvement on subject-specific CT test questions
than on general CT questions.
Consistent with Thorndike’s earlier research, Newell (1980) found little transfer of
reasoning skill. Lehman and Nisbett (1990) found that the reasoning skills of college students
from different majors were discipline-specific. Also, the reasoning skills of college students
showed limited generality across different tasks for measuring CT, even when those tasks
might be expected to tap the same kind of reasoning (Toplak & Stanovich, 2002). In general,
many studies reviewed by Detterman (1992) and Barnett and Ceci (2002) have found that
learning to think and solve problems in one subject often showed little transfer to other
subjects and problems, supporting the domain specificity of learning and thinking. Taken
together, these findings provide considerable converging support for the specifist position. In
general, poor transfer may be due to a failure to access appropriate rules and patterns of
thinking needed to think in a different subject area or context from the one in which they were
acquired (Lockhart, 1992).
Supporting the generalist position, Fong, Krantz, and Nisbett (1986) found that teaching
students general statistical rules such as the “law of large numbers” was effective in helping
them learn how to think about everyday problems and that the training transferred outside of
the classroom. It should be noted, however, that this demonstration was for the use of a single
rule, the law of large numbers, which may be more readily acquired than some other rules
(Fong & Nisbett, 1991; Lehman,; & Nisbett, 1990). Other recent research has further
challenged domain specificity by showing more generality in thinking processes than many
cognitive psychologists had previously assumed (Halford & Andrews, 2007). Schunn and
Anderson (1999) found that psychologists used more domain-general rules in scientific
reasoning tasks than did bright psychology students. Halpern (1998) has documented other
cases of transfer of CT consistent with the view that students who are taught CT will apply
their skills outside of the classroom.
The considerable evidence supporting each side of the generalist-specifist debate suggests
that the debate needs to be rethought. In one recent attempt at resolution, Davies (2006) has
argued that the dilemma of choosing between the two positions is based on the fallacy of the
false alternative. Instead, CT involves the use of both general skills and other discipline-
specific skills and modes of thinking. Davies further argued that an infusion approach could
serve to combine the two approaches.
In another attempt to resolve the discrepancies in the research on teaching general rules
and domain or context-dependent thinking, Perkins and Salomon (1989) proposed that both
general rules and the specific context must be taken into account. They explained the
difficulty students have with general rules for thinking as due to not knowing how to use the
rules in specific contexts. Similarly, Lockhart (1992) argued that CT is like pattern
recognition and thinkers must acquire the ability to use cues that signal appropriate thinking
strategies. More recently, Billings (2007) concluded from a review of the literature on transfer
that CT instruction was more likely to promote transfer when abstract principles were coupled
with examples. Finally, a review of research on discovery learning by Mayer (2004) found
little evidence that students readily induce thinking rules when learning content and do not
efficiently acquire them without guidance. Taken together, these results further support the
 Rules for Reasoning Revisited 19

hypothesis that rules should be taught explicitly and infused into course material so that
students learn to recognize the patterns of rule use in various contexts.

ANALYSIS OF PROBLEMS IN CONCEPTUALIZING CT

Difficulties in conceptualizing CT reviewed in this chapter can be classified as presenting
two different, but inter-related sets of problems. The first, more fundamental set of problems
concerns the lack of definitional clarity and specificity in CT terms. The second set of
problems is related to the tendency not to specify the purpose of CT when conceptualizing it.
Regarding the definitional problem, the review showed widespread agreement on more
general, high-level terms such as “skill,” “disposition,” “reasoning,” “analysis,” and
“evaluation,” but less agreement when more specific identification of terms was attempted.
Disagreements about specific skills often seemed to be discipline- and task-specific. Except
for some psychometric studies, researchers have not conducted much systematic, empirical
research to identify either specific skills or dispositions. Further systematic task analysis and
empirical study of the steps involved in performance of more precisely defined evaluation and
analysis tasks is needed.
Principles of CT also lack specification in a variety of ways that could impact CT
instruction and assessment. Although many CT authorities recommend teaching CT
principles, what constitutes a principle remains unclear. For example, CT principles might be
logical rules, normative statements for how to approach questions, recommendations for
analyzing and evaluating certain kinds of informal arguments, strategies for responding to
persuasive communications, important concepts underlying CT, or some combination of
these. Scriven (2003) has noted that principles often take a variety of forms as commonsense
prescriptions in informal logic. Although variability in the description of principles is to be
expected, a failure to specify the principle appropriate for a particular context could reduce
transfer of learning from one subject to another. This would likely pose problems for
metacognitive control of strategy use and self-regulation, in general. It may, therefore, be
useful to identify the most felicitous expressions of principles likely to promote acquisition
and transfer of CT skills in particular situations. More research is needed to examine how to
promote the acquisition and transfer of specific CT rules and the role metacognition plays.
A related problem in specifying CT rules and principles concerns whether they are
general or specific. Although CT instruction has been commonly assumed to be at least
somewhat generalizable, little attention has been paid to whether principles and rules are
general or specific. Because it is not known how the generality of CT rules affects the
acquisition and transfer of CT skills, the generality of rules should be studied in the context of
particular tasks.
Whenever higher-order hypothetical constructs such as critical thinking or evaluation
remain ill-defined, it may be useful to adopt a more bottom-up, atomistic approach to defining
them, as has been successfully employed in other sciences. This involves using lower-level
components or features to define a higher-level concept. Because, at a fundamental level, CT
involves the appropriate use of criteria, standards, and logical rules in a particular context,
these common low-level components could be specified and help define the higher-order
reasoning skills. Another way to refine an operational definition of a higher-order cognitive
20 D. Alan Bensley

construct is to make an aspect of its definition more explicit (Williams, 1999). Using CT rules
to operationally define CT in thinking tasks could provide the means to make CT terms more
explicit in both instructional and assessment research and to refine the CT construct from the
bottom up.
Another specification problem impeding the scientific study of CT skills is that skills are
typically defined as outcomes. As with other mental constructs, this can be a problem when
an outcome like performance on a CT test is treated as a proxy for CT as a mental construct
and when the conditions that produce the CT outcome are not well understood (De Houwer,
2011). Although defining skills in terms of standards-based outcomes is useful for goal-
setting and assessment, cognitive psychological explanations require that CT also be defined
in terms of well-defined procedures, strategies, and cognitive processes that produce the
skillful performance in observed outcomes. Williams (1999) has clearly discussed how to
operationalize CT in terms of observable behaviors and measured outcomes, but much more
work needs to be done on how to specify antecedent conditions that produce CT.
As mentioned before, the second set of problems in conceptualizing CT occurs when the
purpose and goals underlying the use of CT skills and dispositions are not viewed as
components of a purposeful, self-regulated process. The goal of any CT episode is, of course,
to reason well. This presumes that some outcomes of thinking are better than others. For
example, in constructing an argument, reasoning to a conclusion that is well-supported would
be a better outcome than reasoning to a poorly-supported conclusion. The purpose, therefore,
is to produce well-reasoned conclusions and beliefs that follow from these conclusions. This
is captured in the commonly cited definition by Ennis (1987, p. 10) that CT is “reasonable,
reflective thinking focused on deciding what to believe or do.” Likewise, the purpose of
revising faulty beliefs to be consistent with well-reasoned conclusions is closely related to
self-correction in CT (Lipman, 1991) and to CT in the strong sense (Paul, 1993.
This suggests that modeling CT as a goal-oriented and self-regulated process would help
solve problems in conceptualizing it that do not neglect its function. To revise and correct
beliefs, thinkers must, not only have good CT skills and the dispositions to use those skills
but also be able to accurately monitor their thinking outcomes to insure that they their beliefs
have been revised appropriately. This process requires self-regulation because to reach this
goal and follow normative rules thinkers must monitor and strategically adjust the
deployment of thinking strategies in ways that change the states and outputs of the system.
Therefore, CT research should examine how people monitor the outcomes of their use of CT
skills and dispositions as they think about evidence that should lead them to revise their
beliefs. In the next section, I propose an approach designed to solve conceptual problems in
specifying CT skills that treats CT as a motivated, self-regulated process involving
dispositions and metacognitive skills in the service of belief revision.

CONCEPTUALIZING CRITICAL THINKING FOR INSTRUCTION

AND ASSESSMENT

To help solve the previously mentioned problems, I propose that CT rules be identified
and contextualized within discourse to provide a general framework for the instruction,
assessment, and scientific study of CT. Embedding CT into natural language discourse for
 Rules for Reasoning Revisited 21

instructional purposes has been shown to be an effective approach (Kuhn, 1991; Lipman,
Sharp, & Oscanyan, 1980). By constructing instructional and assessment materials in relation
to CT rules, rule use can be manipulated as part of instructional discourse while
measurements of the ability to apply such rules in various thinking tasks can serve as
dependent variables. More technically, this allows better alignment of CT instruction with
assessment (Bensley & Murtagh, 2011).
Many psychologists have used abstract rules of reasoning as a way to scientifically study
the acquisition and use of reasoning skills (Galotti, 1989). Most often, rules and procedures
have been used to study deductive and formal reasoning (e.g., Braine, Reiser, & Rumain,
1984; Osherson, 1975; Ripps, 1988) but also in the study of informal and inductive reasoning
(Lehman & Nisbett, 1990) in problem solving research (Newell & Simon, 1972) and in
research on thinking, in general (Lovett & Anderson, 2005). In many other studies,
judgments using heuristics or rules of thumb have been compared to judgments made using
normative rules (Gilovich, Griffin, & Kahneman, 2002).
It should be noted at the outset that the rule-based approach to CT proposed here does not
assume the psychological validity of any particular rules unlike many of the cognitive
theories just cited (e.g., O’Brien, 2004; Smith, Langston, & Nisbett, 1993). The proposed
rules are not part of mental procedures or cognitive representations of arguments or other
parts of reasoning. They are external representations found in the discourse of thinking in
which arguments are made, analyzed, and evaluated. Their representational status is,
therefore, externalized with respect to cognition when cognition is viewed as an internal
mental process. An important advantage of external CT rules is that they are the directly
observable representations of rules used in argumentative discourse. As such, they can readily
be made explicit for purposes of instruction, assessment, and objective/subjective inspection.
External rules for reasoning at their most basic level are propositional statements for how
to relate symbols together in ways that express elementary, logical relations. They may be
rules for expressing formal logical relations, but more often in CT and informal logic they are
associated with reasoning under uncertainty and defeasible reasoning (Pollock, 1987). As
such, these rules often serve the heuristic function of helping to move the thinker towards a
tentative conclusion. They are the kind of rules commonly used in everyday, legal, scientific,
and disciplinary thinking. Examples of simple, general rules expressed in English might be “a
reason provides support for a claim” or “an anecdote or example provides relatively weak
support for arguments.” For scientific thinking, a somewhat general rule might be “true
experiments can allow for causal inferences.” In the disciplinary thinking of neuroscience,
“data from brain imaging studies provide better evidence about the localization of brain
damage than case studies of brain damage.” As seen in these examples, external rules may be
general and express how premises are logically related to conclusions or how to value certain
kinds of commonly used evidence, or they may be specific rules for making judgments in a
discipline. Rules appear to vary in reference, generality, and complexity across kinds of
discourse but also in their expression within discourse communities.
As Ennis (1992) noted almost two decades ago, more research that analyzes and
compares the rules of different kinds of discourse is needed. This is important because
criteria, standards, and rules acquire their normative status when a community of experts have
studied their use and found them to be reliable and have come to tentatively accept them in
their CT discourse. As should become clear from the discussion that follows, rules of
reasoning acquire their status as rules, not simply when philosophers have codifed abstract,
22 D. Alan Bensley

logical relations between symbols but also though an epistemological route. Especially in
highly specialized, disciplinary thinking, rules are based on norms derived from what experts
in the field have come to acknowledge as the best available knowledge. Further study of the
use and derivation of rules is needed to help determine how to best express different rules for
instruction and assessment within and across different forms of CT discourse.
CT rules are often expressed in linguistic form and share some features with rules of
language. The formation of arguments in discourse is largely accomplished through the
manipulation of language that follows certain semantic and syntactic rules. Intuitively, we
often experience our thoughts as a kind of inner speech, and some like McPeck (1990) have
argued that thinking is largely language use. Although showing some overlap, the rules of
reasoning are not the same as the rules of language (Smith, Langston, & Nisbett, 1993); and
the overt form that language takes cannot be what the mind uses to reason (Gleitman &
Papafragou, 2005; Jackendoff, 1996). Still, external rules of reasoning are likely to share
more features with natural language than do mental representations of reasoning rules. This
makes language a good tool for externalizing, observing, and manipulating thought.
Learning the rules for using certain indicator terms such as the rule that “therefore”
signals a conclusion should follow can help students improve their CT (Bensley & Haynes,
1995; Paul, 1984); however, doing so is not sufficient for CT to occur (Fawkes, 2003). Rules
for using the language of argumentation are just one set of external rules of reasoning, and
other non-linguistic ways of representing CT rules as well as more complex rules may be
needed for specific applications.
Expressed in natural language, simple rules of reasoning can be combined to form more
complex argumentation structures. Along these lines, some informal logicians have developed
“argumentation schemes” that serve a heuristic function in argumentation tasks (Walton,
Reid, & Macagno, 2008). Argument schemes are forms of argument that serve as logical
tools for analyzing scientific, legal, and other informal arguments found in natural language
discourse. Argumentation schemes have a strong connection with the use of fallacies. For
example, “argument from waste” or what is more often called “argument from sunk costs” is
an argumentation scheme used to justify continuance of an apparently failing enterprise such
as the Viet Nam War in its later years. Formerly, logicians would have considered argument
schemes like argument from sunk costs to be fallacies. Now, from the perspective of
argumentation schemes, they are considered by many to be a kind of defeasible
argumentation strategy. They are still viewed as a weak strategy, but they sometimes lead to
a sound conclusion. Importantly, they provide the occasion to pose critical questions. CT
rules of reasoning are like argumentation schemes in that both can serve as heuristic strategies
in analysis of discourse, but CT rules are more variable in form. Simple ones are elemental,
yet they can be combined and elaborated to form more complex argumentation structures like
argumentation schemes.
Another example of the refinement of the rules of reasoning is the informal logic
developed by Stephen Toulmin to deal with certain problems in inductive logic (Toulmin,
1958). To accommodate specific rules required for thinking in the sciences and other
disciplines, Toulmin refined and elaborated the specification of premises used in inductive
reasoning. He distinguished premises as serving three different functions in argumentation,
statements serving as grounds, warrants, or backing (Toulmin, Rieke, & Janek, 1979).
Grounds serve as the foundation for support of a claim exemplified in common knowledge,
personal testimony, statistical or factual data, and experimental or research findings. Warrants
 Rules for Reasoning Revisited 23

are statements that connect the grounds to claims as being reliable and providing genuine,
solid support. Warrants essentially authorize the grounds as relevant to the claim and permit
the movement from claim to conclusion. Warrants are exemplified in laws, principles, legal
rulings, formulas, rules of thumb, theories, and various authoritative statements. Backing
provides the reasons and other information that back a warrant, supporting why it should be
accepted as reliable. Just as different warrants tend to be specific to different fields, so do the
different kinds of backing tend to be specific for different warrants. For example, the backing
for a law passed by Congress might be a Supreme Court ruling upholding it while a scientific
law would be backed by its replication in many different experiments in many different
laboratories.
In contrast to Toulmin’s analysis of kinds of premises used in induction, Bensley (1998;
2010) has taken a rule-based approach to evaluate the quality of scientific and non-scientific
kinds of evidence used in inductive arguments in psychology. Initially, some common
general rules were adopted for inductive and causal reasoning such as the rule that
conclusions should be consistent with relevant evidence and the corollary heuristic rule that in
complex inductive arguments the side supported by the most, high quality evidence is favored
when drawing a tentative conclusion. To evaluate the quality of evidence in psychological
inductive arguments, other fairly general rules of CT were based on discussions found in CT
books in psychology, (e.g., Stanovich, 2004; Wade & Tavris, 2006; Zechmeister & Johnson,
1992) and various psychology textbooks on research methodology. One such rule, that well-
controlled research studies provide stronger evidence than poorly controlled ones generally
applies to psychological and other scientific arguments but is not relevant to CT in other
disciplines such as art.
Tables 1 and 2 organize these rules for common kinds of evidence used in psychological
discourse and can serve as standards of evidence. Table 1 shows various kinds of non-
scientific evidence commonly used in everyday psychological discussions, and Table 2
contains various research methods used as evidence in more formal psychological discussions
such as literature reviews. The tables list strengths and weaknesses for each kind of evidence
that serve as heuristic rules for judging the quality of evidence typically provided by each
kind of evidence when used in support of a psychological claim.
In Toulmin’s analytic scheme, the strengths and weaknesses serve as rules for evaluating
the quality of backing each kind of evidence provides for hypotheses and theories (the
warrants). They can also be applied to help evaluate the individual instances of evidence (the
grounds) offered to support particular claims. For example, strong conclusions related to
Freud’s psychoanalytic theory are usually not warranted because Freudian theory has been
backed almost exclusively by relatively weak case study evidence. Likewise, the grounds
provided to support claims invoking Freudian theory are often based on case study data that
provide weak support. In contrast, much stronger conclusions related to cognitive behavior
theory may be warranted because cognitive behavior theory and treatments of certain
disorders have been backed by many well-controlled, randomized trial experiments. As Table
2 shows, well-controlled experiments tend to provide much better support. Consequently, a
behavior therapist invoking cognitive behavior theory who has also systematically
manipulated treatment conditions and made careful observations of the effects on behavior
would have better grounds to warrant a conclusion or diagnosis based on cognitive behavior
theory.
24 D. Alan Bensley

An important fact about disciplinary thinking is that psychology and other fields develop
their own specific rules for reasoning in their discourse. The rule that functional magnetic
resonance imaging (fMRI) data provides better support than PETscan data because of its
superior spatial resolution in scanning the brain is a more specific rule for evaluating evidence
from arguments in cognitive neuroscience than the rules in Table 2. As disciplines accumulate
knowledge, the rules for evaluating arguments and what counts as high quality evidence
related to that knowledge become more specific. Because these rules depend on scientific
findings, themselves, they tend to remain unsettled until they have been well studied and
accepted by experts in the field. For example, although many psychologists now treat
findings from (fMRI) studies as high quality evidence that neural activity in specific brain
areas causes certain mental events, this view remains controversial given that the theory for
how brain states are related to cognitive states is far from settled (Miller, 2010).

Table 1. Strengths and Weaknesses of Non-scientific Sources and Kinds of Evidence

Approach Strengths Weaknesses

Commonsense Belief
Informal beliefs and folk
theories of mind
commonly assumed to be
true
-is a view shared by many, not
just a few people.
-is familiar and appeals to
everyday experience.
-is not based on careful,
systematic observation.
-may be biased by cultural
and social influences.
-often goes untested.

Anecdote -can vividly illustrate an -is not based on careful,

Story or example, often ability, trait, behavior, or systematic observation.
biographical, used to situation. -may be unique, not repeatable,
support a claim -provides a “real-world” and cannot be generalized to
example. large groups.

Personal Experience
Reports of one’s own
experience often in the
form of testimonials and
introspective self-reports
-tells what a person may be
feeling, experiencing, or aware
of at the time.
-is compelling and easily
identified with.
-is often subjective and biased.
-may be unreliable because
people are often unaware of
the real reasons for their
behaviors and experiences.

Statement of Authority
Statement made by a
person or group assumed to
have special knowledge or
expertise
-may be true and useful when
the authority has relevant
knowledge or expertise.
-is convenient because
acquiring one’s own
knowledge and expertise takes
a lot of time.
-is misleading when presumed
authority does not have or
pretends to have special
knowledge or expertise.
-may be biased.

Reprinted from Bensley, D. A. (2010). A brief guide to teaching and assessing critical thinking in
psychology, APS Observer, 23, 49-53.
 Rules for Reasoning Revisited 25

Table 2. Strengths and Weaknesses of Scientific Research Methods/Designs Used as

Sources of Evidence

Method/Design Strengths Weaknesses

Case Study -provides much information
Detailed description of about one person.
one or a few subjects -may inform about a person
with special or rare abilities,
knowledge, or characteristics.
-may be unique and hard to
replicate.
-may not generalize to other
people.
-cannot show cause and effect.

Naturalistic Observation -allows observations to be -allows little control of

Observations of behavior readily generalized to real extraneous variables.
made in the field or world. -cannot test treatments.
natural environment -can be a source of hypotheses. -cannot show cause and effect.

Survey research -allows economical collection -may have problems of self

A method like a of much data. reports such as dishonesty,
questionnaire that allows -allows for study of many forgetting, and misrepre-
many questions to be different questions at once. sentation of self.
asked -may involve biased sampling.

Correlational Study -allows researcher to calculate -does not allow random assign-
A method for finding a the strength and direction of ment of participants or much
quantitative relationship relation between variables. control of subject variables.
between variables -can use it to make predictions. -cannot test treatments.
-cannot show cause and effect.

Quasi-experiment -allows comparison of -does not allow random assign-

A method for comparing treatments. ment of participants or much
treatment conditions -allows some control of control of subject variables.
without random extraneous variables. -Cannot show cause and
assignment effect.

True Experiment -allows true manipulation -cannot manipulate and test

A method for comparing of treatment conditions. some variables.
treatment conditions in -allows random assignment and -may control variables and
which variables can be much control of extraneous conditions so much that they
controlled through random variables. become artificial and
assignment -can show cause and effect. unlike “real world” cnditions.

Adapted from Bensley, D. A. (2010). A brief guide to teaching and assessing critical thinking in
psychology, APS Observer, 23, 49-53.

In the language of Toulmin, the theoretical underpinnings of fMRI and cognitive

neuroscience do not provide sufficient backing to warrant the conclusion that brain activity in
a certain brain region as measured by fMRI causes some mental state. In contrast, rules of
evidence like those in Table 2 that are based on established scientific methodology and
26 D. Alan Bensley

practices are more settled and tend to be more general. For example, scientists tend to
evaluate evidence provided by true experiments more highly than from correlational and
quasi-experimental studies because most scientists agree that true experiments can help show
causation while the other two methods cannot.
Finally, although a rule-based model shows promise in helping promote understanding of
CT skill acquisition, the research reviewed earlier strongly suggests that other psychological
variables must be involved in belief revision. A comprehensive psychological model should
include motivational and affective variables in addition to cognitive variables. To further
explicate the rule-based approach, I describe a tentative proto-model that includes
dispositional, affective, and metacognitive variables in addition to cognitive variables. For
models that include some of these same variables, see Halpern (1998) and Perkins, Jay, and
Tishman, (1993). The proto-model is offered as a part of the rule-based framework to guide
research on CT and does not make strong assumptions about how CT skills, dispositions,
motives, and metacognitions are related.
At the cognitive level, it is expected that people internalize external rules through explicit
instruction or through inference in the analysis of discourse. The internal representations of
external rules may or may not resemble those found in previous research on use of rules and
heuristics. This is a worthy question for future research in the tradition of the heuristics and
biases literature that may be aided by clear specification of the external rules people use, but
the proposal of external rules does not require this assumption. It is assumed that people have
somewhat limited knowledge of their own rule use, skill levels, and dispositions; but they can
improve their ability to accurately monitor use of such rules when rules are made explicit and
their use is practiced.
At the motivational level, CT is a thinking process that can be affected by the goals,
dispositions, and effort of the thinker. Rules promulgated by disciplines and thinking cultures
can prescriptively direct thinking toward the goal of revising beliefs to be consistent with
critical evaluation of relevant evidence. The attitudes and dispositional approaches a person
adopts can also affect movement towards CT goals. In terms of approach achievement
motivation, Bartels, Mangun-Jackson, and Ryan (2010) found that need for achievement
predicted cognitive strategies such as CT and that fear of failure was negatively associated
with CT in terms of avoidance motivation. Other research on motivated reasoning suggests
that the kind of processing people engage in can affect their tendency to more readily accept
the validity of information consistent with their preference than information that is
inconsistent (Kunda, 1990). More recently, Ditto (2010) has argued that the amount of
processing a thinker engages in can explain much of motivated reasoning by assuming, for
example, that people engage in more extensive search for alternative information when
information is inconsistent with a conclusion they prefer. It may be that motivation can affect
CT in a variety of ways that are not yet well understood.
At the affective level, information related to belief is often affectively-toned, and
emotions associated with beliefs can dispose a person to approach information and thinking
tasks in particular ways, usually in the direction of maintaining personal beliefs. Blanchette
and Richards (2004) have found that participants asked to reason with emotional versus
neutral statements were more likely to draw invalid inferences with emotional statements. In
another study of college students, Oaksford, et al. (1996) induced them to be in either
positive, negative, or neutral moods and tested them on Wason’s selection task. Participants
in both the positive and negative mood conditions showed a greater confirmatory bias on the
 Rules for Reasoning Revisited 27

task than those in a control mood condition. In contrast, interest might be expected to
enhance effort and the quality of thinking performance.
It seems hard to escape the conclusion that at a fundamental level CT is a psychological
construct. In psychological terms, it seems to be a process involving cognitive skills,
background knowledge, dispositions, motives, emotions, and metacognitive factors. Although
some philosophers, (e.g., Facione, 1990; Paul, 1993) and psychologists (e.g., Halpern, 1998;
Stanovich, 2008) have theorized that multiple variables are involved, research to develop
comprehensive, empirically-based models of CT is in only its beginning stages. The rule-
based proto-model proposed here is an effort to provide a framework to move scientific
research on CT in a more productive direction, one that views CT as a multivariate process
that is related to the use of normative rules contextualized in the discourse of CT tasks. Until
the inter-relationships of these variables are understood in relation to the use of external rules,
a complete, functional understanding of how CT occurs will not be possible.

CONCLUSION
The literature reviewed in this chapter on the conceptualization of CT revealed problems
with the clear specification of CT skills and dispositions and the need to identify the
antecedent conditions that produce CT. While many authorities agree that CT involves skills
and dispositions related to good reasoning, many disagreements persist about which specific
skills and dispositions are involved. The lack of clarity and specificity in CT skills,
dispositions, and other terms has impeded the instruction, assessment, and scientific study of
CT. Review of other research on CT instruction showed that explicit teaching of CT infused
into subject matter instruction was more effective than less explicit traditional and immersion
approaches, but it also revealed that greater specificity was needed to identify the principles
leading to improvement of CT.
To address the need for greater specificity and clarity in conceptualizing CT, a rule-based
approach to defining, instructing, and assessing CT was proposed. This approach treats CT
skill as the appropriate use of rules and procedures for reasoning that can be infused into
relevant discourse and then assessed as an outcome. This conception allows CT to be viewed
in the context of individual difference and/or instructional conditions that produce appropriate
CT rule use as an outcome. For instruction of skills, the rule-based approach employs the
explicit instruction of CT rules, contextualizing them in subject matter instruction. In
assessment, instruction is aligned with assessments measuring the appropriate use of relevant
rules contextualized in subject matter differing in content from that used in instruction.
Improved specification of rules can promote both the experimental manipulation of rules in
discourse as instructional components and the construction of tests and measures for assessing
the effectiveness of CT instructional strategies and skill acquisition that can be varied to test
for transfer. External CT rules explicity operationalized in instruction and assessment allow
for a more systematic mapping between input and output conditions and a functional
approach to the psychological study of CT that can promote scientific progress in studying it
(De Houwer, 2011)
One particular implementation of this rule-based approach called direct infusion has
further specified the conditions of explicit infusion of CT. It incorporates effective
28 D. Alan Bensley

instructional practices such as explicit instruction of rules for analyzing psychological

arguments and critical reading along with practice exercises, assessments, and feedback to
guide skill acquisition. The direct infusion approach and the rule-based framework, in
general, also allow the study of CT dispositions and metacognitive skills along with CT rule
acquisition. Results of a new study by Bensley and Spero (2011) investigating direct infusion
of CT rules into psychology subject matter instruction showed that DI promoted CT skill
acquisition and increased metacognitive monitoring skill. This study illustrates how a rule-
based approach using Tables 1 and 2 provided a way to systematically align instruction and
assessment and track the gains due to direct infusion.
The second set of problems in conceptualizing CT concerns the tendency to focus on CT
skills as outcomes instead of viewing CT as a goal-oriented process that also involves
dispositions and metacognition in the service of belief revision. An advantage of the CT rule-
based approach is that besides CT skills assessment, the contribution of CT dispositions and
metacognitive skills can be also measured as people use rules to perform a CT task.
Viewing CT as a multivariate process with the goal of the self-correction of thinking in
accordance with appropriate rule use can allow for a more complete modeling of belief
revision and other changes in thinking resulting from instruction and experience. However,
the encouraging results of the two new studies reported, although consistent with earlier
research, should be considered preliminary and in need of replication.
It should not surprise us that some of the new studies reviewed in this chapter would
reveal CT to be a complex kind of thinking, multivariate in appearance. Yet, this complexity
should not discourage us. Taking a multivariate, process approach to CT should improve our
ability to identify antecedent conditions that predict CT as opposed to measuring it as a single
outcome presumed to be a proxy for CT skill. Finding relationships among the many
variables thought to be related to CT may turn the complexity of CT into its strength,
providing it with a unique identity among constructs used to explain thinking. In this way,
what has often been described as the art of thinking might increasingly become the science of
thinking and its improvement. By viewing CT as involving rule use, research in psychology,
philosophy, education, and other disciplines might be better coordinated to promote the
scientific study of CT. Ultimately, this might help integrate what has appeared to be a
complex set of inter-related terms into a useful model of a legitimate, although complex,
scientific construct.

REFERENCES
Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., &
Zhang, D. (2008). Instructional interventions affecting critical thinking skills and
dispositions: A stage 1 meta-analysis. Review of Educational Research, 4, 1102-1134.
Anderson, L. W., & Krathwohl, D. R. (eds.). (2001). A taxonomy for learning, teaching and
assessing: A revision of Bloom’s taxonomy of educational objectives. New York:
Longman.
Angelo, T. A. (1995). Classroom assessment for critical thinking. Teaching of Psychology,
22, 6-7.
 Rules for Reasoning Revisited 29

Angeli, C. & Velanides, N. (2009). Instructional effects on critical thinking performance on

ill-defined issues. Learning and Instruction, 19, 322-334.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999a). Common misconceptions of
critical thinking. Journal of Curriculum Studies, 31, 269-283.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999b). Conceptualizing critical
thinking. Journal of Curriculum Studies, 31, 285-302.
Barak, M. & Dori, Y. J. (2009). Enhancing higher order thinking skills among inservice
science teachers via embedded assessment. Journal of Science Teacher Education, 20,
459-474.
Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn? A
taxonomy for far transfer. Psychological Bulletin, 128, 612-637.
Bartels, J. M., Mangun-Jackson, S., & Ryan, J. J. (2010). Dispositional approach-avodiance
achievement motivation and cognitive self-regulated learning: The mediation of
achievement goals. Individual Differences Research, 8, 97-110.
Bensley, D. A. (1998). Critical thinking in psychology: A unified skills approach. Pacific
Grove, CA: Brooks/Cole.
Bensley, D. A. (2006). Why great thinkers sometimes fail to think critically. Skeptical
Inquirer, 30, 47-52.
Bensley, D. A. (2008). The critical thinking in psychology test battery. Unpublished
manuscript.
Bensley, D. A. (2009). Thinking about cognition: An Applied approach. Unpublished
manuscript.
Bensley, D. A. (2009). Thinking critically about critical thinking approaches: Comment on
Yancher, Slife, and Warne (2008). Review of General Psychology, 13, 275-277.
Bensley, D. A. (2010). A brief guide to teaching and assessing critical thinking in
psychology. APS Observer, 23, 49-53.
Bensley, D. A., Crowe, D. S., Bernhardt, P., Buckner, C., & Allman, A. L. (2010). Teaching
and assessing critical thinking skills for argument analysis in psychology. Teaching of
Psychology, 37, 91-96.
Bensley, D. A., & Haynes, C. (1995). The acquisition of general purpose strategic knowledge
for argumentation. Teaching of Psychology, 22, 41-45.
Bensley, D. A., & Murtagh, M. P. (2011). Taking a scientific approach to critical thinking
assessment: Guidelines for psychology programs. Manuscript submitted for publication.
Bensley, D.A., & Spero, R. (2011, May). Improving critical thinking and metacognitive
monitoring skills: A study of the direct infusion approach. Poster presented at the 23rd
annual meeting of the Association for Psychological Science, Washington, DC.
Bensley, D.A., & Spero, R., Kennedy, E. Murtagh, M., & Bernhardt, P. (2010, May).
Individual differences in belief revision and critical thinking skills and dispositions.
Poster presented at the 22nd annual meeting of the Association for Psychological Science.
Boston, MA.
Beyer, B. K. (1995). Critical thinking. Fastback (vol. 385, pp. 7-33). Bloomington, IN: Phi
Delta Kappa Educational Foundation.
Beyer, B. K. (1997). Improving student thinking: A comprehensive approach. Boston: Allyn
& Bacon.
Billing, D. (2007). Teaching for transfer of core/key skills in higher education: Cognitive
skills. Higher Education, 53, 483-516.
30 D. Alan Bensley

Bissell, A. N., & Lemons, P. P. (2006). A new metjod for assessing critical thinking in the
classroom, Bioscience, 56, 66-72.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in
Education, 5, 7-73.
Blanchette, I, & Richards, A. (2004). Reasoning about emotional and neutral materials: Is
logic affected by emotion? Psychological Science, 15, 745-752.
Braine, M. D., Reiser, B. J., & Rumain, B. (1984). Some empirical justification for a theory
of natural propositional logic. In G. H. Bower (Ed.), The psychology of learning and
motivation (vol. 18, pp. 313-371). New York: Academic Press.
Cheng, P. W., Holyoak, K. J., Nisbett, R. E., & Oliver, L. M. (1986). Pragmatic versus
syntactic approaches to training deductive reasoning. Cognitive Psychology, 18, 293-328.
Clifford, J. S., Boufal, M. M., & Kurtz, J. E.(2004). Personality traits and critical thinking
skills in college students: Empirical tests of a two-factor theory. Assessment, 11, 169-176.
Cody, D. F. (2006). Critical thoughts on critical thinking. The Journal of Academic
Librarianship, 32, 403-407.
Davies, W. M. (2006). An ‘infusion’ approach to critical thinking: Moore on the critical
thinking debate. Higher Education Research & Development, 25, 179-173.
Detterman, D. K. (1993). The case for the prosecution: Transfer as an epiphenomenon. In D.
K. Detterman & R. J. Sternberg (Eds.), Transfer on trial: Intelligence, cognition, and
instruction. (pp. 1-24). Norwood, NJ: Ablex.
Ditto, P. H. (2009). Passion, reason, and necessity: A quantity of processing view of
motivated reasoning. In T Bain & J. Fernandez (Eds.), Delusion and Self-deception.
(pp. 23-52). New York: Psychology Press.
De Houwer, J. (2011). Why the cognitive approach in psychology would profit from a
functional approach and vice versa. Perspectives on Psychological Science, 6, 202-209.
DeWall, C. N., Baumeister, R. F. & Masicampo, E. J. (2008). Evidence that logical reasoning
depends on conscious processing. Consciousness & Cognition, 17, 628-645.
Dewey, J. (1910). How we think. New York: Columbia University Teachers College Press.
Doyle, W. (1983). Academic work. Review of Educational Research, 53, 159-199.
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize
their own incompetence. Current Directions in Psychological Science, 12, 83-87.
Eemeren, F H. van, Grootendorst, R., Henkemans, F. H., Blair, J. A., Johnson, R. H., Krabbe,
E. C. and Zarefsky, D. (1996). Fundamentals of argumentation theory: A handbook of
historical backgrounds and contemporary developments. Mahwah, NJ: Erlbaum.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron
& R. F. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). New
York: Freeman.
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed
research. Educational Researcher, 18, 4-10.
Ennis, R. H. (1992). Clarifications and directions for research. In S. P. Norris (Ed.). The
generalizability of critical thinking: Multiple perspectives on an educational ideal. (pp.
17-37). New York: Teachers College Press.
Facione, P. A. (1990a). Critical thinking: A statement of expert consensus for purposes of
educational assessment and instruction: Research findings and recommendations (The
Delphi Report). Prepared for the Committee on Pre-College Philosophy of the American
Philosophical Association. ERIC ED 315 423.
 Rules for Reasoning Revisited 31

Facione, P. A. (1990b). The California Critical Thinking Skills Test (CCTST) test manual.
Millbrae, CA: California Academic Press.
Facione, P. A., & Facione, N. C. (1992). The California Critical Thinking Disposition
Inventory (CCTDI) Test Manual. Millbrae, CA: California Academic Press.
Facione, P. A., & Facione, N. C. (2009). Holistic critical thinking scoring rubric. Millbrae,
CA: California Academic Press. Retrieved July 9, 2010 from
www.insightassessment.com/ pdf_files/RUBRIC%20HCTSR.pdf
Fawkes, D. (2003). Reliance on indicator terms is not critical thinking. Inquiry: Critical
Thinking, 21, 7-14.
Fisher, A. (1991). Testing fairmindedness. Informal Logic, 13, 31-35.
Fisher, A. & Scriven, M. (1997). Critical thinking: Its definition and assessment. CA:
Edgepress.
Flavell, J. (1976). Metacognitive aspects of problem solving. In. L.B. Resnick (Ed.), The
nature of intelligence. (pp. 231-235). Hillsdale, NJ: Erlbaum.
Fong, G. T., D. H., Krantz, & Nisbett, R. E. (1986). The effects of stiatistial training on
thinking about everyday problems. Cognitive Psychology, 18, 253-292.
Fong, G. T. & Nisbett, R. E. (1991). Immediate and delayed transfer of training effects in
statistical reasoning. Journal of Experimental Psychology, 120, 34-45.
Galotti, K. M. (1989). Approaches to studying formal and informal reasoning. Psychological
Bulletin. 105, 331-351.
Gilovich, T., Griffin, D., & Kahneman, D. (Eds.) (2002). Heuristics and biases: The
psychology of intuitive judgment. Cambridge, UK: Cambridge University Press.
Glaser, E. M. (1941). An experiment in the development of critical thinking. New York:
Bureau of Publications, Teachers College, Columbia University.
Gleitman, L. & Papafragou, A. (2007). Language and thought. In. K. J. Holyoak & R. J.
Morrison. (Eds.), The Cambridge handbook of thinking and reasoning. (pp. 633-661).
Cambridge, UK: Cambridge University Press.
Halford, G. S. & Andrews, G. (2007). Domain general processes in higher cognition:
Analogical reasoning, schema induction, and capacity limitations. In M. J. Roberts (ed.),
Integrating the mind: Domain general versus domain processes in higher cognition. (pp.
213-232). New York: Psychology Press.
Halonen, J. (1995). Demystifying critical thinking. Teaching of Psychology, 22, 75-81.
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Dispositions,
skills, structure training, and metacognitive monitoring. American Psychologist, 53, 449-
455.
Halpern, D. F. (2003). Thought and knowledge: An introduction to critical thinking. (4th ed).
Mahwah, NJ: Erlbaum.
Halpern, D. F. (2007). The nature and nurture of critical thinking. In R. Sternberg, H.
Roediger, & D. Halpern (Eds.), Critical thinking in psychology (pp. 1-14). Cambridge,
UK: Cambridge University Press.
Hanley, G. L. (1995). Teaching critical thinking: Focusing on metacognitive skills and
problem solving. Teaching of Psychology, 22, 68-72.
Hayes, J. R. (1985). Three problems in teaching general skills. In J. Segal, S. Chipman, & R.
Glaser (Eds.), Thinking and Learning (vol. 2). (pp. 391-405) Hillsdale, NJ: Erlbaum.
Jackendoff, R. (1996). How language helps us think. Pragmatics & Cognition, 4, 1-34.
32 D. Alan Bensley

Johnson, R. H. (1992). The problem of defining critical thinking. In S. P. Norris (Ed.), The
generalizability of critical thinking: Multiple perspectives on an educational ideal. (pp.
38-53). New York: Teachers College Press.
Kardash, C. M. & Scholes, R. J. (1996). Effects of pre-existing beliefs, epistemological
beliefs, and need for cognition on interpretation of controversial issues. Journal of
Educational Psychology, 88, 260-271.
Keeley, S. M. & Browne, N. M. (1986). How college seniors operationalize critical thinking
behavior. College Student Journal, 20, 389-395.
Kennedy, R. (2004). A history of reasonableness: Testimony and authority in the art of
thinking. Rochester, NY: University of Rochester Press.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in
recognizing one’s own incompetence lead to inflated self-assessments. Journal of
Personality and Social Psychology, 77, 1127-1134.
Ku, K. Y., & Ho, I. T. (2010). Metacognitive strategies that enhance critical thinking.
Metacognition Learning, 5, 251-267.
Kuhn, D. (1991). The skills of argument. Cambridge, UK: Cambridge University Press.
Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 3, 480-498.
Kurfiss, J. G. (1988). Critical thinking: Theory, research, practice, and possibilities.
Washington, DC: ERIC Clearinghouse on Higher Education.
Lawson, T. J. (1999). Assessing psychological critical thinking as an outcome for psychology
majors. Teaching of Psychology, 26, 207-209.
Lehman, D. R., & Nisbett, R. E. (1990). A longitudinal study of the effects of undergraduate
training on reasoning. Developmental Psychology, 26, 952-960.
Leighton, J. P., & Gierl, M. J. (2007). Defining and evaluating models of cognition used in
educational measurement to make inferences about examinees’ thinking processes.
Educational Measures: Issues and Practices, 26, 3-16.
Lipman, M. (1991). Thinking in education. Cambridge, UK: Cambridge University Press.
Lipman, M., Sharp, A. M., & Oscanyan, F. S. (1980). Philosophy in the classroom (2nd ed.).
Philadelphia: Temple University Press.
Lockhart, R. L. (1992). The role of conceptual access in the transfer of thinking skills. In S. P.
Norris (Ed.), The generalizability of critical thinking: Multiple perspectives on an
educational ideal. (pp. 54-65). New York: Teachers College Press.
Lord, C. R., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization:
The effects of prior theories on subsequently considered evidence. Journal of Personality
and Social Psychology, 37, 2098-2019.
Lovett, M. C. & Anderson, J. R. (2005). Thinking as a production system. In K, J. Holyoak &
R. J. Morrison. (Eds.), The Cambridge handbook of thinking and reasoning. (pp. 401-
429). Cambridge, UK: Cambridge University Press.
Marzano, R. J. (2001). Designing a new taxonomy of educational objectives. Thousand Oaks,
CA: Corwin Press.
McGuinness, C. (1990). Talking about thinking: The role of metacognition in teaching
thinking. In K. J. Gilhooly, M. T. Keane, R. H. Logie, & G. Erdos (Eds.), Lines of
thinking. (vol. 2). ( pp. 301-312). New York: John Wiley.
McKean, E. (2005). The New Oxford American Dictionary (2nd ed.). New York: Oxford
University Press.
McPeck, J. E. (1981). Critical thinking and education. New York, NY: St. Maritn’s Press.
 Rules for Reasoning Revisited 33

McPeck, J. E. (1990). Teaching critical thinking: Dialogue and dialectic. New York:
Routledge.
Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? The
case for guided methods of instruction. American Psychologist, 59, 14-19.
Miller, G. A. (2010). Mistreating psychology in the decades of the brain. Perspectives on
Psychological Science, 5, 716-743.
Moon, J. (2008). Critical thinking: An exploration of theory and practice. London, UK:
Routledge.
Morgan, W. R. (1995). Critical thinking-what does that mean?: Searching for a definition of a
crucial intellectual process. Journal of College Science Teaching, 24, 336-340.
Mosely, D., Baumfield, V., Elliott, J., Gregson, M., Higgins, S., Miller, J., et al. (2005).
Frameworks for thinking: A handbook for teaching and learning. Cambridge, UK:
Cambridge University Press.
Moore, T. (2004). The critical thinking debate: How general are general thinking skills?
Higher Education: Research and Development, 23, 3-18.
Newell, A. (1980). One final word. In D. T. Tuma & F. Reif (Eds.), Problem solving and
education: Issues in teaching and research. Hillsdale, NJ: Erlbaum.
Newell, A. & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-
Hall.
Nieto, A. M., & Saiz, C. (2008). Evaluation of Halpern’s “structural component” for
improving critical thinking. The Spanish Journal of Psychology, 11(1), 266-274.
Norris, S. P. (1992). Introduction: The generalizability question. In S. P. Norris (Ed.), The
generalizability of critical thinking: Multiple perspectives on an educational ideal. (pp.
38-53). New York: Teachers College Press.
Oaksford, M., Morris, F., Grainger, B., & Williams, J. M. (1996). Mood, reasoning, and
central executive processes. Journal of Expermental Psychology: Learning, Memory, &
Cognition, 22, 476-492.
O’Brien, D. P). (2004). Mental-logic theory: What it proposeds and reasons to take this
proposal seriously. In J. Leighton & R. Sternberg (Ed.), The nature of reasoning (pp.
205-233). Cambridge, UK: Cambridge University Press.
Osherson, D. N. (1975). Logic and models of logical thinking. In R. Falmagne (Ed.),
Reasoning, representation and process in children and adults (pp. 81-92). Hillsdale, NJ:
Erlbaum.
Paul, R. (1984). Critical thinking: Fundamental to education for a free society. Educational
Leadership, 42, 4-14.
Paul, R. (1993). Critical thinking: Fundamental to education for a free society. Santa Rosa,
CA: Foundation for Critical Thinking.
Penningroth, S. L., Despain, L. H., & Gray, M. J. (2007). A course designed to improve
psychological critical thinking. Teaching of Psychology, 34, 153-157.
Perkins, D. N. & Salomon, G. (1989). Are cognitive skills context-bound? Educational
Researcher, 18, 16-25.
Perkins, D. N., Jay, E., & Tishman, S. (1993). Beyond abilities: A dispositional theory of
thinking. Merrill-Palmer Quarterly, 39, 1-21.
Petris, K. (2004). Critical thinking: An extended definition. Education, 124, 461-466.
Riddell, T. (2007). Critical assumptions: Thinking critically about critical thinking. Journal of
Nursing Education, 46, 121-126.
34 D. Alan Bensley

Ripps, L. J. (1988). Deduction. In R. J. Sternberg & E. E. Smith (Eds.), The psychology of

human thought (pp 116-152). Cambridge, UK: Cambridge University Press.
Renaud, R. D. & Murray, H. G. (2008). A comparison of a subject-specific and a general
measure of critical thinking. Thinking Skills and Creativity, 3, 85-93.
Rosenshine, B. & Guenther, J. (1992). Using scaffolds for teaching higher level cognitive
strategies. In J. Keefe & H. Walberg (Eds.), Teaching for thinking. (pp. 35-47). Reston,
VA: National Association of Secondary School Principals.
Ryle, G. (1949). The concept of mind. New York: Barnes & Noble.
Schunn, C. D. & Anderson, J. R. (1999). The generality/specificity of expertise in scientific
reasoning. Cognitive Science, 23, 337-370.
Scriven, M. (2003). The philosophy of critical thinking and informal logic. In D. Fasco (Ed.),
Critical thinking and reasoning: Curlent research, theory, and practice. (pp. 21-45).
Cskill, NJ: Hampton Press.
Siegel, H. (1988). Educating reason: Rationality, critical thinking and education. London:
Routledge.
Siegel, H. (1992). The generalizability of critical thinking skills, dispositions, and
epistemology. In S. P. Norris (Ed.), The generalizability of critical thinking: Multiple
perspectives on an educational ideal. (pp. 97-108). New York: Teachers College Press.
Smith, E. E., Langston, C., & Nisbett, R. E. (1993). The case for rules in reasoning. In R. E.
Nisbett (ed.), Rules for reasoning. (pp. 361-401). Hillsdale, NJ: Erlbaum.
Smith, R. A. (2002). Challenging your preconceptions: Thinking critically about psychology.
(2nd ed.). Belmont, CA: Wadsworth.
Solon, T. (2007). Generic critical thinking infusion and course content learning in
introductory psychology. Journal of Instructional Psychology, 34, 972-987.
Stanovich, K. E. (2004). How to think straight about psychology. (7th ed.). Boston: Pearson.
Stanovich, K. E. (2008). Individual differences in reasoning. In J. E. Adler & L. J. Ripps
(eds.), Reasoning: Studies of human inference and its foundations (pp. 414-436). New
York: Cambridg, UK: Cambridge University Press.
Stanovich & West, R. F. (1997). Reasoning independently of prior belief and individual
differences in actively open-minded thinking. Journal of Educational Psychology, 89,
342-357.
Stanovich & West, R. F. (2008). On the failure of cognitive ability to predict myside and one-
sided thinking biases. Thinking and Reasoning, 14, 129-167.
Sumner, W. G. (1940). Folkways: A study of the sociological importance of usages, manners,
customs, mores, and morals. Boston: Ginn and Company.
Swartz, R. J. (1989). Making good thinking stick: The role of metacognition, extended
practice, and teacher modeling in the teaching of thinking. In D. M. Topping, D. C.
Crowell, & V. N. Kobayashi (Eds.), Thinking across cultures. (pp. 417-436). Hillsdale,
NJ: Erlbaum.
Tarricone, P. (2011). The taxonomy of metacognition. New York: Psychology Press.
Thorndike, E. L. (1906). Principles of Teaching. New York: A. G. Siegler.
Thorndike, E. L. & Woodworth, R. S. (1901). The influence of improvement in one mental
function upon the efficiency of other functions. Psychological Review, 8, 247-261.
Taube, K. (1997). Critical thinking ability and disposition as factors of performance on a
written critical thinking test. Journal of General Education, 46, 129-164.
Toulmin, S. E. (1958). The uses of argument. New York: Cambridge University Press.
 Rules for Reasoning Revisited 35

Toulmin, S. E., Rieke, R. & Janik, A. (1979). An introduction to reasoning. New York:
MacMillan.
Toplak, M. E., & Stanovich, K. E. (2002). The domain specificity and generality of
disjunctive reasoning: Searching for a generalizable critical thinking skill. Journal of
Educational Psychology, 94, 197-209.
Velanides, N. & Angeli, C. (2005). Effects of instruction on changes in epistemological
beliefs. Contmporary Educational Psychology, 30, 314-330.
Wade, C., & Tavris, C. (2005). Invitation to psychology. (3rd ed.). Upper Saddle River, NJ:
Prentice Hall.
Walberg, H. J. (2006). Improving educational productivity: A review of extant research. In R.
F. Subotnik & H. J. Walberg (Eds.), The scientific basis of educational productivity (pp.
103-159). Greenwich, CT: Information Age.
Walters, K. S. (1994). Critical thinking, rationality, and the vulcanization of students. In K. S.
Walters (ed.). Re-thinking reason: New perspectives in critical thinking. Albany, NY:
State University of New York.
Walton, D., Reid, C., & Macagno, F. (2008). Argumentation schemes. Cambridge, UK:
Cambridge University Press.
Wason. P.W. (1966). Reasoning. In B. M. Foss (Ed.), New horizons in psychology. (pp. 135-
151). Baltimore: Penguin Books.
Williams, R. L. (1999). Operational definitions and assessment of higher-order cognitive
constructs. Educational Psychology Review, 11, 411-427.
Williams, R.L. & Worth, S. L. (2001). The relationship of critical thinking to success in
college. Inquiry: Critical Thinking across the Disciplines, 21, 5- 16.
Yanchar, S. C., Slife, B. D., & Warne, R. (2008). Critical thinking as disciplinary practice.
Review of General Psychology, 12, 265-281.
Zechmeister, E. B. & Johnson, J. E. (1992). Critical thinking: A functional approach. Pacific
Grove, CA: Brooks-Cole.

ACKNOWLEDGMENT AND DEDICATION

I dedicate this chapter to the memory of my friend and colleague, Dr. Ronald Reed, who
helped me get started in critical thinking and introduced me to the work of Matthew Lipman.
In: Critical Thinking ISBN: 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 37-67 2011 Nova Science Publishers, Inc.

Chapter 2

CRITICAL THINKING AND SYSTEMS THINKING:

TOWARDS A CRITICAL LITERACY FOR SYSTEMS
THINKING IN PRACTICE

Martin Reynolds*
Communication and Systems Department, The Open University, Walton Hall
Milton Keynes MK7 6AA UK

1. INTRODUCTION
“The core aspects of systems thinking are gaining a bigger picture (going up a level of
abstraction) and appreciating other people’s perspectives” (Chapman 2004 p. 14)

This simple distinction made by Jake Chapman builds upon a distinction made by
Richard Bawden in identifying two transitions implicit in systems thinking; one, towards
holism, and another towards pluralism (Bawden 1998). The transitions speak of two worlds.
One, the holistic ontological real-world ‘universe’ of interdependent elements, encapsulating
complex interrelationships. Two, an epistemological socially constructed world of
‘multiverse’ (cf. Maturana and Poerksen 2004 p.38), encapsulating differing perspectives on
reality. The two worlds are of course abstractions – ways of framing. This act of framing
itself constitutes a third distinct critical world. This is a world where boundaries inevitably
need to be made and questioned on the inevitable limitations on (i) framing reality (limits on
being holistically ‘universe’), and (ii) framing engagement with reality (limits on being
pluralistically ‘multiverse’). Whilst striving towards aspirations of holism and pluralism, this
third critical dimension confers a peculiar sense of grounding, purposefulness and
responsibility in systems thinking.
As Werner Ulrich shows in his seminal work Critical Heuristics of Social Planning
(1983), the systems ‘idea’ as a philosophical tool can be traced back to Immanuel Kant’s
*
Contact Details:
Tel: +44 (0)1908 654894 (work)
Email: m.d.reynolds@open.ac.uk
Website: http://sites.google.com/site/jjntest1/Home/people/martin-reynolds-1
38 Martin Reynolds

1784 enlightenment treatise - Critique of Pure Reason. Kant uses the systems idea as an
holistic concept:
“ - a whole which is prior to the determinate knowledge of the parts and which contains
the conditions that determine a priori for every part its position and relation to the other parts.
This idea accordingly postulates a complete unity in the knowledge obtained by the
understanding, by which this knowledge is to be not a mere contingent aggregate, but a system
connected according to necessary laws” (Immanuel Kant, 1784, quoted in Ulrich, 1983 p.223;
Ulrich’s italics)

The systems idea as originally formulated by Kant is an abstract holistic principle used as
a means for understanding ‘the real world’. The Kantian critical interpretation of the systems
idea rests on the principle that it is simply not possible to have comprehensive knowledge. In
the subsequent treatise Kant “undertakes the dialectical task of making reason reflect upon its
own limitations” (Ulrich, 1983:269). Critique of Pure Reason formulates an original critical
interpretation to the systems idea. In course, the critique provides a fundamental and enduring
epistemological challenge to classical comprehensive rationalism; a rationalism that assumes
the possibility of comprehensive knowledge and understanding.
Werner Ulrich’s critical systems heuristics is often regarded as an example of one
particular tradition of systems thinking called critical systems thinking (CST).
The positivist interpretation of systems science came under critical review during the
1970s most notably through the works of C. West Churchman (1970) and Russell Ackoff
(1974) in America and Peter Checkland (1979) in England (original papers reproduced in
Flood and Jackson, 1991a). Checkland built on Churchman’s critique, defining an alternative
“soft” systems tradition, and naming the former as the “hard” systems tradition. Around the
same time Werner Ulrich was also building on Churchman’s work, subsequently contributing
to what later came to be referred to as a “critical” systems tradition (cf. Flood & Jackson,
1991a). Contemporary systems approaches can then be classified as ‘hard’, ‘soft’, or
‘critical’, according to the degree to which they maintain Kant’s original critical sense of the
systems idea (see Figure 1).
Rather than exploring one tradition of systems thinking – CST - this chapter explores the
notion of contemporary systems thinking as being implicitly critical. An argument will be
made that the need for what might be called a ‘systems literacy’ reflects a need for the
original critical idea of systems. The basis of such a literacy is a proposed framework of
systems thinking in practice based on revised ideas of boundary critique (Ulrich and
Reynolds, 2010). After describing what this critical literacy in systems thinking in practice
looks like and entails, the question of how the critical kernel emerged amongst contemporary
systems thinking in practice approaches is examined. This section traces the influence of
critical thinking traditions on systems thinking. Finally, some views are offered on why
attention to the critical literacy of systems thinking in practice is significant to a contemporary
world beset with complex issues of change and uncertainty.
 Critical Thinking and Systems Thinking 39

Systems ‘Type’ Selected Systems Approaches

& philosophical origins*

Hard Systems
Ontology: realism • general systems theory (Bertalanfy, 1940)
Epistemology: positivism • classical ‘mechanistic’ cybernetics (Ashby, 1956)
Intention: control • operations research (Churchman, Ackoff & Arnoff, 1957)
• systems engineering (Hall, 1962)
• socio-technical systems (Trist et al., 1963)
• RAND-systems analysis (Optner, 1965)
• system dynamics (Forrester, 1961; 1971; Meadows et al.,
1972; 1992; Senge, 1990)
• organic cybernetics (Beer, 1979; Varela et al., 1974)
Soft Systems
Ontology: nominalism • Inquiring systems design (Churchman, 1971)
Epistemology: constructivist • soft systems methodology (Checkland, 1981)
interpretivism • strategic assumption surface testing (Mason & Mitroff, 1981)
Intention: appreciation • interactive management (Ackoff, 1981)
• cognitive mapping and strategic options development analysis
(Eden and Ackermann, 1988)

Critical Systems
Ontology: nominalism • critical systems heuristics (Ulrich, 1983)
Epistemology constructivist/ • system of systems methodologies (Jackson & Keys, 1984)
critical idealism • community operational research (Rosenhead, 1984)
emancipation • liberating systems theory (Flood, 1990)
Intention: • interpretive systemology (Fuenmayor, 1991)
• total systems intervention (Jackson & Flood, 1991)
• systemic intervention (Midgley, 2000)

* Glossary of Terminology

Ontology (assumptions about the nature of ‘things’ or ‘being’).

realism: ‘real world’ is made up of systems.
nominalism: systems are means of re-presenting (naming) phenomena of the real world.

Epistemology (assumptions of knowledge generation).

positivism: validity based on ‘objective’ scientific method of gathering empirical facts.
constructivism: knowledge is socially constructed
interpretivism: validity based upon ‘subjective’ interpretations (multiple realities) of phenomena.
critical idealism: phenomena (maps), as distinct from noumena (objects), are imbued with human
purpose and must lay open their perspective and purpose for critical reflection

Intention (primary pledge or human purpose embodied in systems approach).

control: enables technical mastery over natural and social entities.
appreciation: enables furthering communication and understanding between different groups.
emancipation: enables freedom from coercive material and ideological forces.
Adapted from Reynolds and Holwell, 2010 p. 10.

Figure 1. An overview of systems approaches and their philosophical origins .

40 Martin Reynolds

2. WHAT IS CRITICAL IN SYSTEMS THINKING IN PRACTICE?

“Systems literacy is not just about measurement. The learning journey up the ladder of
complexity—from quarks, to atoms, to molecules, to organisms, to ecosystems—will be made
using judgment as much as instruments. Simulations about key scientific ideas and
visualizations of complex knowledge can attract attention—but the best learning takes place
when groups of people interact physically and perceptually with scientific knowledge, and
with each other, in a critical spirit. The point of systems literacy is to enable collaborative
action, to develop a shared vision of where we want to be.” (Thackara, 2005)

“Clear systems thinking is one of the basic literacies of the modern world” commented
Geoff Mulgan – a senior government advisor in the UK government’s Cabinet Office during
the 1990s… “not least because it offers unexpected insights that are not amenable to common
sense” (Mulgan, 1997). Our common sense understanding of situations is continually and
inadvertently shaped by our actions or practice. In the same way effective systems thinking is
a literacy that is continually being informed, moulded and (re)shaped by ongoing practice. It
is this interplay between conceptual tools and practice that resonates with the idea of systems
thinking in practice as an important development in critical thinking.
The name - systems thinking in practice - suggests an important interplay between
understanding and practice; systems thinking continually being informed, moulded and
(re)shaped by ongoing practice. Systems thinking in practice deals with what might be
considered to be the critical literacy required to be competent practitioners in supporting real
world decision-making. The name provides a continual reminder of the important interplay
between understanding and practice.

Systems thinking in practice involves stepping back from messy situations of complexity,
change, and uncertainty, and clarifying key interrelationships and perspectives on the
situation. It further requires engaging with multiple often contrasting perspectives amongst
stakeholders involved with and affected by the situation so as to best direct responsible
joined-up thinking with action to bring about morally justifiable improvements.

The above definition encapsulates three generalized purposeful orientations of systems

thinking in practice (Reynolds & Howell, 2010 p.17):

1) Making sense of, or simplifying (in understanding), relationships between different

entities associated with a complex situation. The prime intention is not to get some
thorough comprehensive knowledge of situations, but rather to acquire a better
appreciation of wider dynamics in order to improve the situation.
2) Surfacing and engaging (through practice) contrasting perspectives associated with
complex situations. The prime intention here is not to embrace all perspectives on a
predetermined problem so as to solve the problem, but rather to allow for
possibilities in reshaping a problem-situation for improved possibilities of resolution.
3) Exploring and reconciling (with responsibility) ethical issues and power relations,
both expressions of boundary issues associated with inevitable partial understandings
of a situation and partiality amongst different stakeholders. The intention here is to
gently disrupt, unsettle and thereby provoke new systems thinking.
 Critical Thinking and Systems Thinking 41

An effective systems approach to managing real world complex situations embodies all
three aspects of systems thinking in practice.
The criticality of systems thinking in practice can be expressed in terms of promoting
continual and meaningful conversation. The ‘conversation’ works at two levels. One is an
expression of boundary reflection, a conversation between our conceptual constructs of real
world realities – constructs called ‘systems’ - and the actual realities being addressed. The
other is an expression of boundary discourse, a conversation between people involved with
and affected by the systems used to construct and engage with reality (Ulrich and Reynolds,
2010). Whereas boundary reflection is a conversation influenced by conventional critical
systems thinking, boundary discourse is influenced also by traditions of social learning. Both
conversations constitute what might be referred to as boundary critique, a triadic interplay
between judgements of ‘fact’, value judgements, and boundary judgements, underpinning
systems thinking in practice. Ulrich describes this interplay as an ‘eternal triangle’:

“Thinking through the triangle means to consider each of its corners in the light of the
other two. For example, what new facts become relevant if we expand the boundaries of the
reference system or modify our value judgments? How do our valuations look if we consider
new facts that refer to a modified reference system? In what way may our reference system
fail to do justice to the perspective of different stakeholder groups? Any claim that does not
reflect on the underpinning ‘triangle’ of boundary judgments, judgments of facts, and value
judgments, risks claiming too much, by not disclosing its built-in selectivity” (Ulrich 2003
p.334)

Boundary critique can be described in terms of activities underpinning a framework of

systems thinking in practice; constituting what has been referred to as an overall critical
systems framework (Reynolds 2008a). The framework is supported by three (sub)frameworks
respectively – framework for understanding (fwU), framework for practice (fwP), and a
framework for responsibility (fwR) - The activities of boundary critique involve continual
revising of boundary judgements (systems thinking) with judgements of ‘fact’ (observing)
and value judgements (evaluating) (see Figure 2).

Adapted from Reynolds 2008a p.386.

Figure 2. Critical systems framework illustrating systems thinking in practice activities.

42 Martin Reynolds

In developing this into a broader heuristic for systems thinking in practice, three
complementary entities can be added: firstly, real-world contexts of change and uncertainty
associated with a framework for understanding; secondly, people or practitioners involved
with making change associated with a framework for practice; and thirdly, the ideas and
concepts – including systems - as tools for effecting change associated with a framework for
responsibility.
What is critical in systems thinking in practice is: (i) an appreciation that complex
realities, despite good intentions with complexity sciences, can never be holistically
comprehended; (ii) an acknowledgement that any perspective on a situation is laden with
values that inhibit any sense of neutral engagement; and (iii) an awareness of the limitations
of systems design in the light of (i) and (ii).

3. HOW SYSTEMS THINKING BECAME CRITICAL

Systems science emerged in the 1940s and 1950s in response to clear problems of
military logistics generated during World War Two. Whilst the problems addressed by
variants of systems science (systems engineering, system dynamics, systems analysis and
operations research) might be highly complicated in terms of involving many variables, the
problem situation could nevertheless be well defined; that is, the methods would serve what
Jackson and Keys (1983) would call a clearly defined “unitary purpose”. A war-time
consensus is likely to generate broad and common objectives.
Systems science relates to the transformation in practice in order to address issues of
interrelatedness and pluralism in perspectivs. In the late 1950s, operations research (OR -
outside of America the term used is ‘operational’ research), previously used as a means of
controlling non-human (hardware) variables in weapon, computer, or space systems, was
applied to the more challenging task of addressing organisational, community and societal
problems. The increasingly pluralistic demands of the post-war/post-colonial era, provided
new challenges for systems science techniques. In systems science, the response was soft
systems thinking and critical systems thinking (see Figure 1).
The emergence of ‘soft’ and ‘critical’ systems approaches since the 1980s can best be
understood as an epistemological challenge to the systems idea as described by Ulrich:

“The systems idea as we understand it does not presuppose that we can know ‘the whole
system’, but only that we can undertake a critical effort to reflect on the inevitable lack of
comprehensiveness in our understanding and design for (social) systems. Thus the systems
idea, if we do not scientistically misunderstand it, challenges us to make transparent to
ourselves and to others the normative implications of our systems concepts and designs”
(Ulrich, 1983 p.21).

During the Second World War the ‘systems idea’ acquired currency as an operational
tool. Systems engineering and OR transformed the holistic idea into a methodological tool for
controlling variables within the context of clearly pre-defined instrumental action. At the
same time ‘systems science’ and Bertalanffy’s ‘general systems theory’ translated the
epistemologically critical interpretation of Kant’s systems idea into an ontological realist
concept of systems as actual comprehensive representations of ‘real world’ phenomena. This
 Critical Thinking and Systems Thinking 43

now pervades everyday language reference to the ‘education system’, ‘health system’ or
‘legal system’ etc. Systems as concepts assumed a social ‘factual’ status. The application of
OR to organisational and societal problems further entrenched systems thinking into a
positivist epistemological framework associated with the narrow instrumental purposes of
gathering empirical ‘facts’.
In seeking to identify more clearly just what the critical systems idea is and its relevance
to contemporary systems thinking in practice literacy, three dimensions of analysis are used in
critiquing each of the hard and soft systems traditions. Each of the three dimensions -
contexts, practitioners, and systems as conceptual constructs - correspond to the systems
thinking in practice framework introduced earlier (Figure 2).

3.1. Critique of the ‘Hard’ Systems Tradition

Context Matters
From the 1950s, Churchman and Ackoff were amongst the pioneers applying principles
of operations research to organisational management and wider societal issues (Churchman
et.al., 1957). Ackoff (1981) following the lead of Churchman’s critique of OR, expands on
how approaches to problem-solving based upon assumptions of consensual, unitary purpose
could not be applied as a tool for most social systems design, characterised more often by
complexity and conflict. Complexity is not of the type that arises from some innate but
growing complexity factor in institutional dynamics, but is rather derived from taking account
of conflicting human perceptions and interests involved in applied systems dynamics (cf.
Ellis, 1995). The American Apollo Space Programme provides perhaps the best example of
overcoming considerable technical complexity using ‘hard’ systems (operations) research in
achieving the goal of putting a man on the moon. In allowing technology to be mobilised for
such a unitary purpose, instrumental questions of ‘how’ resources might be deployed to
achieve the objective dominated, first, practical questions concerning ‘what’ resources were
actually available, and second, ethical questions of ‘why’ resource use needed to be
prioritised for this mission. The powerful instrumental techniques of OR and other ‘hard’
system variants were effectively allowed free rein.
In contrast, Churchman and Ackoff could document the failures of OR in contexts where
the purpose and purposefulness were ill-defined because of conflicting interests amongst
different actors. On one level OR developed academically with the pursuit of ‘modelling’
techniques “ - a study of the delights of algorithms; nuances of game theory; fascinating but
irrelevant things that can happen in queues” (Churchman in 1979 p. 50). In instances where
the instrumental force of techniques assumes an authority of its own, effectively by-passing
issues of purpose, the ‘irrelevance’ of approaches might be transformed into a more intrusive
form - what has been termed a “cybernetic technocratism” (Flood, 1990), or where, in terms
of social planning, there is a propensity for the means to define the ends (Ulrich 1988a).

Practitioner Matters
Hard systems thinking in social science is characterised by a positivist epistemology
wherein the systems being examined assume the status of real world objects - a realist
ontology - in much the same way that natural ‘objective’ science based on empiricism treats
its subject-matter (Churchman, 1970, in Flood and Jackson 1991a). Checkland (1981),
44 Martin Reynolds

Jackson (1985), Flood and Jackson (1991b) and Ellis (1995) draw attention to Burrell and
Morgan’s 1974 typology of four social science paradigms - subjectivist “radical humanism”
and “interpretivism”, and objectivist “radical structuralism” and “functionalism” - in
signalling the correspondence between the “functionalist paradigm” and hard systems
thinking. The paradigm claims that sociological ‘models’, represent actual ‘systems’
constituents of the real world which, once conceptually formulated can then be legitimately
engineered. There appears to be general agreement that the enduring legacy of perceiving
‘systems’ in terms of a realist ontology - existing outside of human purposefulness - was
strongly influenced by Bertalanffy’s ‘general systems theory’ in the 1940s (Checkland, 1991;
Jackson, 1990; Flood & Ulrich, 1991). Such theoretical underpinning promotes systems
practice as an essentially regulatory function.

Systems Change Matters

The technical bias and positivist theoretical underpinnings of hard systems thinking lends
itself to a perspective of systems as homeostatic, ‘closed’, and with an equilibrium to be
maintained. The task of systems practice from the ‘hard’ perspective, as Oliga (1990)
observes, is to ensure the ‘stability’ of such systems. Control is the management intention
which systems practice is seen to serve. In narrowly focusing on goals and instrumental
‘means’ for achieving the goals, rather than allowing for questioning the systems’ built in
objectives (purpose) and underlying interests (purposefulness), social consequences of
systems design are ignored (Checkland, 1981). The result of this orientation towards control
is to maintain or even accentuate existing power relations implicit in the system (Flood &
Ulrich 1991).
Checkland (1978) points out how the ‘hard’ systems tradition promotes the ‘neutral’
‘value-free’ image of the systems analyst. Implicitly, therefore, it would appear that
institutional conservatism relates as much to stabilising the institutional practice of systems
analysis (authority of the experts) as with stabilising institutional practice of client
organisations being served by systems analysis.

Towards Soft Systems Approaches

From the late 1970s the three dimensions of critique were translated into two distinct
systems’ approaches; firstly, a soft systems approach that focused primarily on developing an
epistemological challenge in the practitioner domain, and secondly, a critical systems
approach which focused increasingly more on the political challenge generated in the systems
change dimension. The soft systems ‘interpretivist’ approach attempts to; (a) provide the
basis for techniques to address pluralist purposes, (b) adopt the principle of ‘multiple
realities’ as an anti-positivist (interpretivist) theoretical framework, and (c) promote
institutional change (Jackson, 1991a; Flood and Ulrich, 1990). Amongst the soft systems
approaches to emerge in the 1980s (see Figure 1), Checkland’s (1981) soft systems
methodology (SSM), developed with colleagues at Lancaster University, is one of the best
known and most enduring (case studies are particularly well documented in Checkland and
Scholes, 1990).
The success of SSM might be attributed to the detailed development of methodological
procedures for addressing complex human-based problem situations. These are commonly
represented as an iterative learning cycle with seven stages of enquiry (Figure 3) although
Checkland himself more recently describes it in terms of two parallel streams of enquiry.
 Critical Thinking and Systems Thinking 45

Stage 1 The `problem situation' unstructured

The remit of intervention is identified, providing the site sometimes referred to as ‘the mess’
Stage 2 The problem-situation expressed
Ordinarily carried out through a rich picture; a tool used for brainstorming ideas of the problem
situation usually with the active collaboration of those involved with the system. Designed to
generate the main issues (e.g. conflicts) and tasks (e.g. communication links, social or institutional
norms and roles) of the system.
Stage 3 Root definitions of relevant systems
Systems deemed relevant to the problem situation are first conceptually identified through isolation
of ‘problem themes’, named and provided with concise root definitions; formulated around
fundamental systems questions encapsulated in the mnemonic CATWOE:

Customers/clients : beneficiaries of ‘T’

Actors/agents: those who would do ‘T’; ‘experts'
Transformation (‘T’): the purpose of the system
Weltanschauung (worldview): value-informed view which makes ‘T’ meaningful
Owner(s): decision makers who control conditions of the system
Environment: ‘constraining’ elements outside the system
Stage 4 Conceptual modelling
To encapsulate the key activities which the system must undertake in order to fulfil the requirements
of the root definition. System models are expressions of ideal forms of organised activities. Derived
from the root definition rather than rich picture. Used for pro-active interrogative and analytical
work, rather than a contrivance at representing ‘reality’.
Stage 5: Comparative analysis
Conceptual constructs (models) compared with the ‘realities of the mess’. Carried out both
monologically, comparing conceptual models with the ‘rich picture’ formulated in Stage 2, and/or,
more preferably, dialogically, in a process of re-presenting models back to those involved with the
system in order gain critical feedback.
Stage 6: Debate changes
The critique emerging from Stage 5 is used as the basis for a debate amongst those involved with
the system, concerning the desirability and feasibility of future changes
Stage 7: Action to improve the situation
Implementing changes agreed upon during the Stage 6 debate or reverting back to Stage 3.
Source: adapted from Checkland, 1981.

Figure 3. Stages of Soft Systems Methodology.

46 Martin Reynolds

A distinction between a ‘logic-based stream of analysis’ (represented by the 7 stage core

method) and a ‘stream of cultural analysis’, articulated later by Checkland and Scholes
(1990), precludes any assumption of a linear flow between these seven stages. The ‘cultural
analysis’ constituent of the framework, reinforces the essential iterative nature of SSM. It is a
particularly important constituent to stages 1 and 2 of the core method, providing a dialogical
means towards defining relevant systems in stage 3.

3.2. Critique of ‘Soft’ Systems

Context Matters
Jackson (1982) questioned the ambitions of soft systems thinking as expressed in the
work of Churchman, Ackoff, and Checkland, arguing that the domain of application for soft
systems remains, as with hard systems, restricted. Jackson observed that soft systems can
only work in conditions which allows for “genuine” debate amongst those involved. Despite
many years of documented practical success there would appear also to be circumstances that
mitigate against its successful deployment:

“SSM continues to be employed uncritically in problem situations where the mobilisation

of differentiated power resources by different interest groups makes genuine participation
impossible” (Jackson, 1990 p. 362).

Flood and Jackson (1991b) further argue that SSM practitioners often compound this
problem by advocating for SSM a status as a ‘meta-methodology’ incorporating hard systems
as a constituent part of SSM.

Practitioner Matters
Jackson’s critique of soft systems approaches are rooted in the assertion that interpretivist
theory offers nothing in relation to theorising about institutional change... “It is surprising to
find that at the moment no genuinely interpretive systems theory exists... Such a theory would
have to probe the systemic nature of interpretations individuals employ in constructing the
social world” (Jackson, 1982 p.18). John Oliga (1988), whilst acknowledging the
achievement of soft systems approaches in having made an ontological break with
empiricism, rejecting realist assumptions of there being an ‘objective’ world of social facts,
suggests that soft systems practitioners continue to assume attaining ‘objective knowledge’ at
the level of theory. In pursuing this point, Oliga distinguishes between ‘naturalistic’ and
‘historic’ hermeneutics; suggesting that it is the former which effectively objectifies others’
realities. The validity of SSM is based upon respect for the point of view and aims of all the
‘stakeholders’. However, Oliga argues that because SSM practitioners neglect the influence
of social structural factors on the formation, maintenance and (it might be added) articulation
of worldviews – a term introduced by Churchman (1971) in translation of the more rich
German term, Weltanschauung - they perpetuate ontological realism at the theoretical level
despite having made an ontological break with empiricism at the methodological level.
The effect, referred to as “epistemological impoverishment” (Flood & Ulrich, 1991 p.
306), is to produce a theoretical stance which is relativistic and without external
legitimisation. Similarly, Jackson (1991a) argues, change can only be understood in SSM via
 Critical Thinking and Systems Thinking 47

the processes of communication facilitating mutual understanding between the involved

actors. This is epistemologically ‘impoverished’ through not being able to appreciate the
‘effects of material conditions’ and the incidence of ‘false consciousness’ on peoples’
worldviews (Flood & Ulrich, 1991).

Systems Change Matters

“What I hear Habermas arguing is that the debate at stages 5 and 6 of the soft systems
methodology will be inhibited by society’s structure. I think that it is in the nature of society
that this will be so.” (Checkland, 1981 p. 283).

“Recommendations of soft systems thinking remain “regulative” because no attempt is

made to ensure that the conditions for “genuine” debate are provided” (Jackson, 1991b p.133).

SSM does not discriminate between worldviews. According to Jackson, the methodology
is therefore ‘ideologically naive’ since no attempt is made to relate worldviews to social
relations of power effecting incidences of false consciousness.

Put another way; “...there are no explicit directives in the theory that aim to prevent the
approach from being expert driven” (Flood & Ulrich 1991p. 198). An authority is implicitly
established by virtue of there being no explicit reference to theory (cf. Sangren, 1988).

Oliga (1990) maps out an architecture of power and ideology in relation to different
systems practice, and examines how underlying assumptions inform institutional perspectives
of ‘stability’ (social control) and ‘change’ (social transformation). Whilst hard systems
thinking (as a function of positivism) typically neglects the subjective domain of
‘worldviews’ and focuses on relations of power as constituents of systems regulation, soft
systems typically neglect the relations of power determining ideology. Oliga argues that by
focusing upon either one or the other in the ‘power/ideology’ dialectic, both ‘hard’ and ‘soft’
traditions implicitly share a conservative view of using systems practice for social control;
that is, maintaining social order in the face of actual or potential conflicts. What may appear
to be institutional liberalisation, as professed by some soft systems practitioners, turns out to
be ideological conservatism (a point made by Flood & Ulrich, 1991); ideology removed from
relations of power.

Towards Critical Systems Approaches

To summarise the critique so far, contextual concerns have questioned the
(in)appropriateness of systems approaches in application to diverse and often conflicting
human purposes. Practitioner concerns have generated debate over the principles behind the
social construction of knowledge in relation to the ‘systems idea’. Since the 1940s with
Bertalanfy’s realist ontological interpretation of ‘systems’, considerable confusion between
the ontological and epistemological conceptualisations of systems have beleaguered the
systems literature (Flood & Ulrich, 1990:186; Checkland, 1991p. 26). Finally, systems and
institutional concerns over social/political stability and change have raised fundamental issues
concerning power and ideology in relation to systems practice.
48 Martin Reynolds

At the same time that Peter Checkland was establishing soft systems methodology,
Werner Ulrich (1983) was formulating critical systems heuristics (CSH). Both addressed
problems of applying ‘hard’ systems practice - particularly as manifest in operations research
- to social affairs. Both acknowledge significant influence from Churchman (1971) who first
introduced the idea, derived from his mentor Edgar A. Singer, that social systems are
teleological or, in more common language, purposeful.
Whereas in soft system practice the original systems idea (as a human construct) was
thought best to be reinvented through new terminology - “wholismic thinking” (Ackoff,
1981) or “holonic thinking” (Checkland, 1991) - abandoning the term ‘systems thinking’ to
its realist ontological colonisation (ibid p. 27), Churchman and Ulrich significantly retain the
idea of a system bounded by “whole system judgements”; constructs imbued with human
intentionality. The use of the term “systemic” thinking and practice encapsulates and retains
the meaning of the original epistemological intention of the systems idea.
In the following sub-section concepts and ideas derived from Churchman and Ulrich are
traced, providing a further understanding of the idea of a critical dimension to systems
thinking in practice.

3.3. Emergence of Boundary Critique

Context Matters
Churchman’s characterisation of purposeful systems dealt initially with only those
involved in the systems design. Nine conditions that must be fulfilled for a system (S) to
demonstrate purposefulness were identified (derived from the philosophy of Immanuel Kant).
The conditions are reproduced in summary below (adapted from Churchman, 1971 p. 43)

1) S is teleological
2) S has a measure of performance
3) There is a client whose interests are served by S
4) S has teleological components which coproduce the measure of performance of S
5) S has an environment (both social and ecological)
6) S has a decision maker who can produce changes in the measure of performance of
S’s components and hence changes in the measure of performance of S
7) S has a designer who influences the decision maker
8) The designer aims to maximise S’s value to the client
9) There is a built in guarantee that the purpose of S defined by the designer’s notion of
the measure of performance can be achieved and secured

Churchman (1979 p. 79) later reordered these nine conditions into three groups of three
categories; each group corresponding with a particular social role - client, decision maker,
and planner. Each category is associated with two allied categories which Ulrich later termed
role specific concerns and key problems. Reynolds later renamed these three category groups
in terms of stakeholders, stakes and stakeholding issues (Reynolds, 2007). Ulrich also
identified each category group with a term reflecting the primary source of influence -
motivation, control, and expertise - for client, decision maker, and planner (“designer”)
respectively (Ulrich, 1983 p. 250) (see Table 1).
 Critical Thinking and Systems Thinking 49

Table 1. Categories of ‘Involved’ in a Purposeful System’s Design

Churchman’s 1971 Churchman’s 1979 three groups of Ulrich’s 1983 sources of

nine conditions for three categories for a purposeful influence informing a
a purposeful system system purposeful system

Group 1
condition 3. social role: client sources of motivation: whose
condition 1. role specific concerns: purpose purposes are served?
condition 2. key problems: measure of performance

Group 2
condition 6. social role: decision maker sources of control: who has
condition 4. role specific concerns: components the power to decide?
condition 5. key problems: environment

Group 3
condition 7. social role: planner/designer sources of expertise: who
condition 8. role specific concerns: implementation has the know-how?
condition 9. key problems: guarantor
Adapted from Ulrich, 1983 pp. 245-250.

Churchman (1979 p.80) suggests a role for those affected by systems design, and
provides a self-reflective description of an additional three categories that centre around the
role of systems philosopher; along with the two related categories, the enemies of the systems
approach and significance. It is Ulrich (1983) however who systematically distinguishes
between those involved in a system’s design and those affected by a systems design so as to
define the latter role more concisely for social systems planning. The category of those
affected by, but not involved in, systems design are designated by Ulrich as being the witness;
those who in practical discourse will argue the case of the affected (ibid p. 252). The role
specific concerns of the witness are conceptualised as those of emancipation; liberation from
oppressive material conditions and false consciousness.

“... it [emancipation] reminds us that social mapping and design is not merely a matter of
instrumental orientation toward some purpose (as functionalistic “systems science” seems to
assume), but that for socially rational planning it is essential that the planner initiate a process
of emancipatory self-reflection on the part of the affected” (Ulrich, 1983 p. 257; original
italics).

The final ‘key problem’ category represents the possibilities of a conflict in worldviews -
“different visions of what social reality and human life in it ought to be” (ibid) - between the
involved and the affected. Consequently the “source of influence” for this category group is
defined as the source of legitimisation. Table 2 summarises the twelve “critical-heuristic
categories.”
50 Martin Reynolds

Table 2. Critical-Heuristic Categories

Categories Dimensions of intentionality

1 Client?. Purpose? (role)
2 Measure of (concerns) Sources of
3 improvement? (problems) motivation The purposeful
4 Decision maker? (role) System of concern
5 Components? (concerns) Sources of Those involved (or context
6 Environment? (problems) control
7 Planner? (role) sources of of application)
8 Expertise? (concerns) expertise on which depends
9 Guarantor? (problems) the meaning of
10 Witness? (role) sources of Those affected ‘improvement’
11 Emancipation? (concerns) legitimisation
12 Worldviews? (problems)
Adapted from Ulrich, 1983:258; 1993 p.595 and 1996 p.43.

Checkland (1981) similarly uses Churchman’s nine conditions of a purposeful system as

a basis for formulating the CATWOE mnemonic (client, actor, transformation, worldview,
owner, and environment; see Figure 2). It has been argued that Checkland uses the conditions
in a functionalist manner by accepting them as given, though defined differently by different
stakeholders. Consequently, Ulrich (1983 p. 247, footnote 11), for example, argues that SSM
contributes (albeit unintentionally) to systems maintenance. In contrast, Churchman and
Ulrich engage in challenging the conditions by asking for each condition what ‘is’ the actual
situation in juxtaposition to what ‘ought’ to be the ideal situation. This engagement with the
ethics of conditions - encouraging an ‘is’/’ought’ dialogue - and Ulrich’s subsequent
development of the categories associated with the ‘affected’ (Churchman’s “systems’
enemies”) in juxtaposition with the ‘involved’ - is a key distinction between Checkland’s
SSM and the critical systems approach.
Churchman (1979) in The Systems Approach and Its Enemies describes his purposeful
systems inquiry as a process of unfolding, though refrains from providing any precise
definition. In examining his own twelve categories of ‘whole systems judgements’ belonging
to a purposeful system, Churchman states:

“I’ll be more interested in their process of unfolding rather than in their

definitions...(and).. in explaining the unfolding of meaning, I’ll use imagery, and specifically
the imagery of striving-force and the opposite, passive-helplessness” (Churchman, 1979 p.
80).

Unfolding as a dialectical process comes closest to any form of definition offered. Ulrich
also appreciates Churchman’s process of unfolding as a dialectic:

“We call this dialectical interplay between planners (“systems rationality”) and witnesses
(lived social practice) the process of unfolding. The process of unfolding is intended to
represent our critical, and practicable, solution to the problem of practical discourse” (Ulrich,
1983 p. 266, original italics).
 Critical Thinking and Systems Thinking 51

Ulrich is describing here what in later years he would refer to as constituting the last of
three steps of unfolding: first, the mapping out of the twelve categories (roles, concerns,
problems) in the form of “boundary questions”; second, contrasting “actual with ideal”
mapping; and third, promoting “stakeholder participation” (Ulrich, 1988a pp. 423-425).
The first step is analogous Singer’s holistic concept of sweeping in (Churchman, 1979
p.78; Ulrich, 1988a p. 423). In CSH terms this exercise is undertaken with precise guidelines
associated with what is termed the “social mapping” of the twelve boundary questions
(‘roles’, ‘role-concerns’, and ‘key problems’) associated with sources of motivation; control,
knowledge, and legitimacy; each source of influence (Table 2).1
Whilst sweeping in conjures up an endless quest for comprehensiveness, the process of
unfolding has been described as “the critical counterpart to the sweep-in process” (ibid). The
critical idea of the sweep-in concept is to increase the awareness and understanding of
systems’ dimensions and concerns from various perspectives. This is undertaken through the
subsequent two steps of unfolding: firstly, through subjecting each of the twelve boundary
questions to an “is” and an “ought” mode, and secondly, through subjecting the systems
design (as created by those involved) to a wider democratic process “in which the affected
citizens emancipate themselves from the premises and promises of experts” (Ulrich, 1983 p.
263, my italics).
The second and third ‘steps’ of unfolding as described by Ulrich prompt questions
regarding the underlying practitioner underpinnings and systems implications of CSH.

Practitioner Matters
A feature of Ulrich’s CSH is the attempt to marry the ideas from Habermasian critical
social theory with the concerns of systems practice so as to effect an alternative practical
approach to enquiry (Ulrich, 1983:106-166). The approach builds upon Churchman’s ideas of
systems practice as a dialectical pursuit; though Ulrich more precisely associates the
dialectics of systems practice with Habermas’ pragmatistic model of rational discourse and
communicative action (ibid pp. 240-243). The model challenges the means/end dichotomy
prevalent in positivist approaches manifest in what Habermas refers to as decisionistic and
technocratic models of social enquiry. Decisionistic models separate the privilege of choosing
between ‘ends’, seen as located in the domain of politics, from the value free ‘means’, located
in the domain of expertise. Technocratic models reverse the primacy of the politician over the
expert implicit in the decisionistic model, and suggests instead that the political process in
decision making is simply a “stopgap in a still imperfect rationalisation of power, in which
the initiative has in any case passed to scientific analysis and technical planning” (Habermas,
1971, quoted in Ulrich, 1983 p. 75). Both models assume the separability of means and ends.
The pragmatistic model in contrast identifies a dialectic between means or expertise,
representing questions of fact, and decision making about ends or politics, representing
questions of value:

1
Ulrich distinguishes between social mapping and social design: “If the task is to determine... actual social reality,
i.e., the problem situation, we speak of “mapping”; if the task is to determine (“make real”) future social
reality, we speak of “design”” (Ulrich, 1983 p. 242). Social design corresponds with the SSM principle of
conceptual modelling.
52 Martin Reynolds

“The dialectical or “pragmatistic” model thus requires a model of rational discourse

between experts and political agencies, a model that can guarantee an adequate translation of
practical needs into technical questions, and of technical answers into practical decisions. The
basic requirement for such a discourse is that it be public; a second necessary requirement is
that it be “free from oppression”, that is , not subject to external sources of systematic
distortion” (Ulrich, 1983 p. 78; original italics).

The “oppression” and “distortion” referred to by Ulrich can be understood in terms of a

second theoretical construct offered by Habermas known as the knowledge-constitutive
interest theory (Habermas, 1971). The theory is based upon the anthropological premise of
there being two fundamental forms of human activity, work (or ‘labour’) and interaction
(‘language’ or ‘communication’). Each activity is associated with a particular interest. Work
is associated with a technical interest in the prediction and control of natural and social
affairs. Interaction is associated with a practical interest in fostering mutual human
understanding. In order to realise the full potential of these two human activities - that is,
having labour free from ‘materialistic’ and economic constraints and demands, and
communication free from distortion brought about by ‘false consciousness’ - Habermas
postulates a third emancipatory interest. This interest ensures freedom from coercion. The
three constitutive interests are invariant though complementary, and are underpinned by three
equally invariant though complementary ‘rationalities’ which are referred to respectively as
instrumental, strategic and communicative (Figure 4).

Basis of Human Knowledge Constitutive Interests

Interest & Associated Rationalities
“Work” technical interest in prediction and control of natural and social affairs
instrumental rationality (labour)
success depends upon technical mastery over social and natural
processes

“Interaction” practical interest in fostering mutual understanding

strategic rationality (human interaction)
success depends upon practical mastery over ensuring mutual
understanding

“Power/Authority” emancipatory interest in being free from coercion

communicative rationality (authority relations)
success depends upon being free from coercion imposed by power
relations
Adapted from Habermas (1971) and Oliga (1988).

Figure 4. Habermas’ Taxonomy of Knowledge-Constituent Interests.

Ulrich’s application of Habermas’ pragmatistic model reinforces Churchman’s idea of

purposeful systems through making explicit the importance of including a dimension of
human intentionality (Ulrich, 1983 p. 237). Returning to Burrell and Morgan’s framework of
social theory paradigms it is evident that CSH does not fit readily into any of the four
‘paradigms’. CSH might be seen rather in terms of proposing radical change through
operating a dialectic between the structuralism in the objective domain, as depicted by the
 Critical Thinking and Systems Thinking 53

deliberations of the ‘involved’, and humanism in the subjective domain, as depicted by the
counter-deliberations offered by the ‘affected’.
In defining an emancipatory interest, constitutive theory provides an element of purpose
to the pragmatistic model of inquiry through dialogue between those “involved” and those
“affected”:

“... the idea of the “emancipatory” interest is to combine the “technical” interest in
instrumental control with the “practical “ interest in mutual understanding, so as to emancipate
the inquirer from the seemingly objective (because unreflected) constraints produced by the
former, technical, interest” (Ulrich, 1983:63).

From a theoretical standpoint, systems practice has undergone what Flood (in Flood &
Ulrich, 1990) has identified as the two significant epistemological breaks: the first, manifest
in particular through Checkland’s SSM, has brought in the interpretivist idea of systems
thinking where systems are considered as epistemological conceptual tools rather than
ontological real world entities; the second, manifest through Ulrich’s CSH, has highlighted
the critical interpretivist divide between the (systems) rationale of the involved in contrast to
the (social) rationale of the affected. Both ‘breaks’ are associated with a constructivist
epistemology. The challenge remains with reconciling these two “epistemological breaks”
with constitutive interest theory. With the first break the challenge is to clearly demarcate the
relevance of constitutive interests to the epistemological idea of systems, in the same way - as
discussed elsewhere (Reynolds, 1998) - as the ontological relevance of constitutive interests
has been made relevant in systems practice; that is, alignments of ‘hard’/technical,
‘soft’/practical, and ‘critical’/emancipatory (cf. Ulrich, 1988b pp. 150-151, Table IV; Oliga,
1988 p. 92; Flood and Jackson, 1991b pp. 324-325) .
The second epistemological break bears on the Habermasian ‘emancipatory’ project more
directly by challenging systems practice to define more clearly the task of emancipating the
‘affected’ from the “premises and promises of involved experts” (Ulrich, 1983 p. 308). From
a political or institutional standpoint, systems inquiry has shifted from an ontological concern
regarding the factors necessary to control and maintain an object system, to a more explicit
epistemological concern with systemic practice to bring about social transformation.

Systems Change Matters

The interaction between the involved and the affected corresponds at an institutional level
with Oliga’s interpretation of the interaction between power and ideology (Oliga, 1990). To
recap, Oliga contends that the failure of systems practice to engage in the dialectic between
power and ideology has effectively reinforced measures of social control and stability rather
than facilitating social transformation and change. In hard systems approaches this failure is
brought about by a neglect of the ‘ideological’ constituent (dismissing ‘subjective’) whilst in
soft systems approaches the failure is due to neglecting issues of ‘power’ (thereby implicitly
and uncritically accepting ‘false consciousness’). Oliga argues that forces of change can only
take effect when issues of power and ideology are both “doubted” through critical
engagement between those involved with and those affected by systems practice.
Through its focus on interaction between the involved and the affected, CSH has an
implicit radical agenda of institutional transformation. Although the methodology cannot
claim to replace efforts to achieve institutional democratisation, it has a core purpose of
54 Martin Reynolds

ensuring that systems are not expert driven but are open to social critique and thereby open to
radical change (cf. Ulrich, 1988b p. 159).
The “polemical employment of boundary judgements” is the term given by Ulrich (1983
p. 313) to the heuristic tool for enabling lay citizens or their representatives (‘witnesses’) to
question the ‘premises and promises’ of the planners. According to Kant’s critical ideal of
reason “... no standpoint, not even the most comprehensive systems approach, is ever
sufficient in itself to validate its own implications” (Ulrich, 1988b p. 157). Those ‘affected’
by a system can theoretically question the premises of experts in a polemical manner without
assuming any expertise of their own. As Ulrich points out, polemic...

“entails no positive validity claims and hence requires neither theoretical knowledge nor
any other kind of special expertise or “competence”. A polemical argument is advanced
merely in hypothetical fashion, to show the dogmatic character of the opponent’s (“the
expert’s”) pretension of knowledge” (Ulrich, 1983 p. 305).

The practical limitations are centred around achieving meaningful dialogue between the
‘involved’ and the ‘affected’ in circumstances where the former might be an unwilling player
and the latter has little effective means of expression. Flood and Ulrich (1990 p. 201)
maintain that “... it pays careful and explicit attention not to presuppose that those in control
of “decision power” are willing to take account of the views and interests of those affected,
but only that they are interested in making their own views and interests appear to be
defensible on rational grounds”
The question raised by Jackson (1985) as to why CSH should suppose that the powerful
should take account of the views and interests of those affected but not involved is addressed
by Flood and Ulrich:

“As a rule, the powerful ... seek to conceal their specific private interests behind some
facade of common interest, of generally acceptable norms or “objective necessities”. A critical
approach, although it cannot “force” the powerful to take account of the less powerful, can at
least unveil this facade of rationality and objectivity which is so characteristic of the strategic
action of powerful vested interests in present-day “interest group liberalism” ...[Polemical
employment of boundary judgements] pays careful and explicit attention not to presuppose
that those in control of “decision power” are willing to take account of the views and interests
of those affected, but only that they are interested in making their own views and interests
appear to be defensible on rational grounds” (Flood and Ulrich, 1990 p. 201).

Those in power have an interest in justifying the status quo through recourse to
objectifying their authority (with the implicit intention of rendering harmless their position).
Ulrich (1988b p. 158) argues that through such strategies the powerful leave themselves open
to possible exposure and challenge given the application of appropriate social critique enabled
by “democratically secured institutional arrangements”. Ulrich later suggested possible areas
for innovation:

“...a critically heuristic training for citizens... I believe that the systems idea...might
become important as a “countervailing power” to face the steadily growing influence of
expertise in our society, namely, by something like a generally available “expertise of
laypeople in dealing critically with expertise”. ... I think of new arenas of participatory
 Critical Thinking and Systems Thinking 55

conflict resolution such as “planning cells” and “citizen reports on technological projects”, i.e.
, institutional arrangements within which citizens, together with experts and designers, can
train themselves in critically heuristic debate.” (Ulrich, 1993:608, original italics).

The suggestions here were later developed into what Ulrich (2003) described as critical
reflective practice, as against another parallel tradition of CST referred to as critical pluralism
(cf. Mingers, 1997; Jackson, 1999).

3.4. Critical Reflective Practice and Critical Pluralism

In the same way that Checkland’s SSM gained prominence within the tradition of soft
systems approaches, Ulrich’s CSH emerged and developed prominence during the same
period within a critical systems thinking tradition. CST was named in the mid-1980s, and was
later given expression with two significant publications: the journal Systems Practice, first
published in 1988 and renamed in 1998 as Systemic Practice and Action Research; and
secondly, a compilation text in 1991 entitled Critical Systems Thinking: Directed Readings
(Flood and Jackson, 1991a). The Centre for Systems Studies at Hull University has played a
leading role in promoting CST through encouraging research and publications.
A particular variant of CST promoted at Hull is one based on promoting methodological
and theoretical pluralism. The dominant expression of this is total systems intervention (TSI)
– a methodology for drawing different methods together through a three-fold process of (i)
creatively exploring problematic situations, (ii) choosing an appropriate systems approach,
and (iii) implementing it (Flood and Jackson, 1991b). The emphasis on pluralism has been
championed in particular by Mike Jackson. TSI builds on an earlier categorisation of systems
methodologies (Jackson and Keys, 1984) called system of systems methodologies (SOSM).
SOSM provides a matrix for classifying systems methods on two dimensions: one, the level
of complexity of the problem situation (simple or complex), and the other, the degree of
shared purpose amongst participant stakeholders (unitary, pluralist, or coercive relationships).
It is this latter dimension that draws on the hard, soft, critical typology using metaphors as
guiding principles – mechanic for the ‘hard’, living organism for the ‘soft’ and the metaphor
of prison for the ‘critical’ situations. The classification yields a six celled matrix as illustrated
in Table 3. Each cell defines a problem situation which then invites particular suitable
systems methods (some examples are given).
The two dimensions of situations are helpful in delineating the two aspects of systems
thinking described above. The simple/ complex dimension relates to levels of interrelatedness
and interdependencies, and the unitary/ pluralist/ coercive dimension relates to levels of
engagement with multiple perspectives. Again such a model has been helpful in prompting
systems practitioners to think more clearly about the nature of the problem situation – the
‘mess’ – in a simplified manner. It has helped with the appreciation that different systems
methods might complement each other and indeed complement other approaches used for
similar problem situations.
56 Martin Reynolds

Table 3. System of systems methodologies

Participants
Unitary Pluralist Coercive
‘hard’ systems based ‘soft’ systems based ‘critical’ systems
on mechanistic on organic metaphor based on prison
metaphor metaphor
‘Systems’ Simple Simple unitary: e.g. Simple pluralist: e.g. Simple coercive:
i.e., systems engineering Strategic assumption e.g., critical systems
problem surfacing and testing heuristics
situations Complex Complex unitary: Complex pluralist: Complex coercive:
e.g., system e.g. soft systems (non available!)
dynamics, viable methodology
systems model
Adapted from Jackson, 2000 p.359.

There are two difficulties with TSI. First, that a problem situation can somehow be easily
identified as constituting one of the six ‘problem situation’ types by an expert practitioner
seems to deny possibilities of there being underlying contrasting perspectives on the situation
amongst different stakeholders. What may appear to be simple or unitary from one ‘expert’
perspective can often actually be quite coercive from the perspective of other stakeholders
associated with the situation. Second, there is an underpinning difficulty in the pigeon-holing
of particular systems approaches as being only suitable for specific types of situation. Firstly,
there may be different opinions on where different systems approaches ‘fit’ based upon actual
experiences of using the approach. A study of 30 key systems thinkers as practitioners (rather
than focusing on methods associated with them) reveals the rich and diverse experiential
background of using different systems approaches (Ramage and Shipp, 2009). Secondly, such
pigeon-holing takes away the potential for systems approaches to themselves adapt and
develop through different contexts of use amongst different users. A revised account of five
systems approaches – system dynamics, viable systems model, strategic options development
and analysis, soft systems methodology, and critical systems heuristics – drawn from various
philosophical traditions suggests that their respective robustness over 30 years of use derives
from their adaptability by different users in different contexts of use (Reynolds and Howell,
2010). Whilst systems approaches may well have derived from particular paradigmatic
traditions of either functionalism, interpretivism or critical social theory, this does not imply
that they remain fixed in this tradition (ibid p.296). So whilst sociological paradigms may be
helpful in understanding the origins of particular systems approaches, they are less helpful in
theorizing and steering practice for developing methodologies based on mixed methods (Zhu,
2011).
Ulrich contrasts TSI, which he views as constituting a ‘shallow’ form of
complementarism, with what he calls a ‘deeper complementarism’ offered by boundary
critique underpinning critical reflective practice (Ulrich, 2003). This deeper sense of
methodological complementarism does not privilege multiple methods per se but neither does
it alienate or devalue any particular method. Rather than suggesting that there are appropriate
methods for different predefined contexts, there is an acknowledgement of value given to any
professional practice but that any such practice can benefit from a reflection on how it deals
 Critical Thinking and Systems Thinking 57

with judgments of ‘fact’, value judgements and boundary judgements. Boundary critique can
therefore complement any methodology as a reflective tool. This deeper sense of
complementarism using boundary critique at the level of methodology resonates with the
theory of communicative action (Habermas, 1984) and the deeper sense in using Habermas’
three knowledge constitutive interests at the level of theory in a more integral fashion for
critical systems thinking (Reynolds, 2002). Traditions of American pragmatism with the
writings of Charles Peirce, William James, and John Dewey, also provide more helpful
theorizing frameworks for systems practice (Ulrich, 2006; Zhu, 2011).
Ulrich and Reynolds (2010) have further delineated two forms of boundary critique –
boundary reflection and boundary discourse. Boundary reflection corresponds to a framework
for understanding. Boundary discourse corresponds to a framework for practice. Together
they contribute towards the framework for systems thinking in practice introduced earlier
(Figure 2). The next section examines the implications of this critical systems framework for
contemporary critical thinking.

4. IMPLICATIONS OF A ‘CRITICAL’ SYSTEMS THINKING IN PRACTICE

The previous section provides a framework for understanding the emergence of
contemporary systems thinking in practice. But how might this inform a framework for
practice for engaging with different perspectives and a framework for responsibility in
enacting systems thinking in practice from a critical perspective?
From a contemporary ‘soft’ and ‘critical’ systems perspective, systems are regarded as
conceptual constructs enabling an interdisciplinary space for purposeful conversation across
disciplines and for exploring possibilities for creative change. In the context of international
development discourse I have called this a ‘creative space’ (Reynolds, 2008b). If systems
thinking in practice provides such a potentially powerful agent of change, what is that may
inhibit such change? In the practical domain of engaging with different perspectives, the fear
for change is manifest in the traps of uncritical thinking that pervade our everyday practices.
Aligned with these traps is an unclear use of language around systems thinking. What
precisely is meant by the terms systemic, systematic and system and how might such terms be
more meaningfully incorporated in to a critical systems literacy?

4.1 Three traps of thinking in practice

Three particular traps of thinking in practice can be highlighted (Reynolds and Holwell,
2010, pp. 5-6; pp. 301-303). Each trap is associated with an uncritical focus on each of the
three pillars of a framework for systems thinking in practice introduced earlier (Figure 2) –
(i) silo problem-solving (fixing situations) representing the trap of reductionism, (ii) people
management in (fixing people) representing the trap of dogmatism, and (iii) systems
obsession (with fixed ideas) representing the trap of fetishism (with expressions of uncritical
holism and pluralism). Each of these traps can then be aligned with relevant systems ideas
associated with managing change.
58 Martin Reynolds

Trap 1: Silo Problem-Solving: Towards Anticipating Systemic Change

“We are most comfortable working in silos – our own, independent function or
department, our own industry or agency, our own sector (business, government, civil)… Our
structures hardwire us into silos, reinforcing independent rather than interdependent habits –
even matrix organizations are still fixed within topic-specific domains, and the fact that non-
governmental or non-profit agencies are segregated by function (education, healthcare,
housing, etc.) creates the same silo mindsets” (Huston 2007 p.46).

The conventional functionalist systems idea of organisation – a whole consisting of

related parts contributing to a particular function - has contributed considerably to a
reification of this type of silo thinking. Organisations are typically organised with
departmental terms of reference carrying clearly defined remits for employees. The idea is
neat, easy to work with in terms of providing some assurance of certainty, or at least lack of
ambiguity, and most importantly, as suggested above, comfortable. Comfort is conventionally
drawn from some basic (mis)understanding about organisations working as self-contained
functional systems, the output of which is unquestionably some ‘good’ for the wider
community. It pervades many impressions of organisations whether small and simple or large
and complex. The UK National Health Service (NHS) for example was likened to a super-
tanker by a British Government Minister in the 1980s. The analogy conjures up not just
slowness and difficulty in being re-directed, but that there are discrete parts with particular
functions all contributing towards an ultimate destination. The image is very much in contrast
to many complexity theorists such as the Noble laureate, Ilya Prigogine (1997), who claims
that no static system can exist. Organisations like the health service are inherently
unpredictable because they continually change from within due to the changing dynamics of
interrelated parts. This is what is meant by systemic change.
A systemic issue comprises complexity, uncertainty, interdependencies and controversy
involving a wide range of variables requiring resolution. A technical problem on the other
hand bounded by a fixed bounded silo occupies the more comfortable domain, amenable to a
solution, usually provided by a traditional ‘expert’. Characteristics of issues are troublesome!
They can sometimes distract from getting things done. But can they be ignored?
The trap of silo thinking is based upon the idea that such issues can be ignored. It is
associated with reductionism. A critical perspective on systems acknowledges that, to use a
famous systems adage, a system is merely a map of a situation or territory, not to be confused
with the actual territory. Real world complexities represent something that exists outside of
any one conceptualisation of context. The real world complexity provides the site for
systemic change. In terms of a systems literacy, the tension between system and situation
might be appreciated in terms of a conversation. The distinction between thinking about
systems and systems thinking is helpful in clearing ground between systems thinking and
related disciplines associated with systems sciences (e.g., complexity and chaos theory). It
respects rather than struggles against two different perceptions of ‘systems’: one, as with
systems thinking, an epistemological construct; the other, as with systems sciences, more an
ontological entity.
A key underplayed intent of systems thinking associated with systemic change is to make
simple the complex web of interrelationships and interdependencies in a transparent (and
thereby questionable) manner. In short, systems thinking about systemic change involves a
 Critical Thinking and Systems Thinking 59

continual conversation between ‘systems’ and ‘situations’; a tension expressed through the
act of making simple the complex – a tension that invites more an artistic rather than
scientific literacy. This is not to deny the importance of a scientific literacy promoting more
detailed understanding in terms of, say, evolutionary science, chaos theory and complexity
sciences, but the craft of systems thinking is primarily geared towards making manageable the
complex. The task involves using a language that is accessible to all stakeholders.

Trap 2: Fixing People: Towards Purposeful Systematic Change

Glendower:
I can call spirits from the vasty deep.

Hotspur:
Why, so can I, or so can any man; but will they come when you do call for them?

From William Shakespeare (Henry IV Pt.1 Act III Scene 1)

"A systems approach begins when first you see the world through the eyes of another"
(Churchman 1968 p.231).

The Shakespeare quotation was used as an introduction to the British House of Commons
Environment, Food and Rural Affairs Committee (March, 2003) report: The Water
Framework Directive (WFD): Fourth Report of Session 2002-03 Volume 1 p.5. HC130-1. Its
purpose was to highlight the problem and inadequacy of conventional approaches towards
environmental management through control-orientated fiscal and regulatory measures.

The trap of ‘fixing people’ into pre-designed purposes – ‘purposive management’ – is

based upon the misguided behaviourist idea that different purposes from different
perspectives can be moulded into a consensual purpose. The story of failure in organizational
change projects, and the argument for WFD, in contrast, suggests alternative strategies based
upon working with people/ stakeholders rather than working on them. The trap here is related
to the trap of dogmatism. Systemic failure in many situations can often be associated with the
dogmatic disregard of other perspectives that inform the situation.
The literacy called for requires not just simplifying realities for individual comprehension
but making sense of realities for mutual understanding amongst stakeholders involved in a
situation in order to foster shared practice. This second aspect of a systems literacy speaks to
the human dimension of intervention. As such it speaks of systematic change; change directed
by human agents. The term ‘systematic’ relates to an inevitable requirement of orderliness.
Our means of communication through language and discourse requires levels of
systematisation to a greater or lesser extent so as to generate some sense of mutual
understanding.
The systems literacy relates to two significant intervention theories. In 1960 Douglas
McGregor published The Human Side of Management, in which he introduced the concept of
Theory X and Theory Y styles of management. Theory X is the conventional mechanistic
style of top-down management treating people as having no responsibility, preferring to be
told what to do and to have decisions made for them. Theory Y conversely assumes a more
constructive role:
60 Martin Reynolds

“It was quite an article of faith, this Theory Y. Everything about the structure of
corporations went against it: the perks and power structure of the hierarchy, the labour
relations tradition, the curricula at most business school…, and all those devices like
performance appraisals, that measured one person against another. Adopting Theory Y would
mean giving up both the stick (threatening to fire people) and the carrot (bribing them or being
paternalistic). Without those two weapons, what leverage did a manager have? Only the
ability to spark other people’s involvement and commitment, by giving them the opportunities
to do good work—hardly a strong incentive by conventional standards” (Kleiner 1996 p. 46).

The second idea of social learning is situated in planning theory. John Friedman (1987)
describes social learning as the third of four traditions informing planning - the other three
being ‘social reform’, ‘policy analysis’ and ‘social mobilization’. He contrasts social learning
with the more control-oriented tradition of policy analysis which, he claims:
"is a form of anticipatory decision-making, a cognitive process that uses technical reason
to explore and evaluate possible courses of action.... Social learning, on the other hand, begins
and ends with action, that is, with purposeful activity… It is the essential wisdom of the social
learning tradition that practice and learning are construed as correlative processes, so that one
process necessarily implies the other." (Friedman 1987 p. 181).

Social learning, like Theory Y, invokes a proactive engagement amongst stakeholders in

systematically managing change. The idea moves away from implementation modelled on
hierarchical notions of working on people – restructuring, reconfiguring, re-engineering – and
then dealing with inevitable subsequent resistance amongst stakeholders, towards a more
collective notion of working with people – stakeholding development. The notion of social
learning builds on the importance of nurturing the tension between changing practice and
understanding between stakeholders (Blackmore et al. 2007; Reynolds 2008b). The learning
here is collaborative (hence ‘social’) involving multiple stakeholders including professional
experts, and the action is concerted, again involving multiple stakeholders. The notion of
concerted action is captured in the metaphor of an orchestra, with multiple individual players
doing different things, though all contributing towards some hopefully harmonious output.
Conventional systematic change is purposive. This involves a linear application of tools
to serve a prescribed purpose. In contrast, purposeful systematic change involves use of
language, amongst other tools, for iterating on better revised goals based on improved
understanding and better practice.

Trap 3: Maintaining Systems or ‘Systems’ Obsession: Toward Meaningful Systems

Change

“To a man with a hammer, everything looks like a nail” (Mark Twain).

This familiar mantra provides a reminder that our tools and models, including systems
frameworks as systems tools, can often be sub-consciously overpowering in determining how
we approach issues. But what about its counterpart? Continually adopting ‘new’ systems
runs the risk of elevating the notion of ‘system’ to a fetish status; celebrating the very notion
of system as being the panacea for crises. Systems are often referred to in association with
new developments – miraculous ways of doing things.
 Critical Thinking and Systems Thinking 61

The trap of systems maintenance, or being obsessive with the tools we construct, lies in
reifying and privileging the ‘system’ - whether it’s old or new – as though it has some
existence and worth outside of the user and some status beyond its context of use in enabling
change. McGregor’s Theory X depicting a conventional model of management hierarchically
imposed and indiscriminately applied across all parts of an organisation, regarding
stakeholders as objects rather than subjects, is perhaps the most pervasive example of an
implicit system - a conceptual model - resilient to change. It is a pervasive way of thinking
that continues to hold a widespread grip on management practice. There are many other
‘systems’ that similarly entrap our understanding and practice. A generic term for these is
‘business as usual’ (BAU). Examples include the annual cycles of organisational planning,
target setting, budgeting, the development of performance indicators and performance related
pay incentives etc. BAU models maintain existing ‘systems’ principally because of a fear for
change. But the fear is not evenly distributed amongst all stakeholders. Some fear change
more than others simply because the system works in a partial manner. The system works for
some and not for others.
All systems are partial. They are necessarily partial – or selective – in the dual sense of (i)
representing only a section rather than the whole of the total universe of considerations, and
(ii) serving some parties - or interests - better than others (Ulrich 2002 p. 41). In other
words, no proposal, no decision, no action, no methodology, no approach, no system can get a
total grip on the situation (as a framework for understanding) nor get it right for everyone (as
a framework for practice) (Reynolds, 2008a).
Drawing on the quotation from Chapman at the beginning of this chapter, the two
dimensions of partiality respond to the two transitions implicit in systems thinking about
systems change; one, towards holism, and another towards pluralism. Given the partiality of
systems a third critical dimension is required where systems boundaries inevitably need to be
made and questioned on the inevitable limitations of being holistic and pluralistic.
Frameworks are used widely as a means of providing some overriding shape and
guidance towards recommended action. Different systems approaches can be considered as
frameworks. Systems are more detailed expositions of a framework. The relationship between
a framework and systems is analogous to that between policy and plans. Whereas policy
provides an overall guidance structure, individual plans around projects and programmes
might be considered as expressions of policy. Indeed the term policy framework is often used
to describe the wider setting of planning initiatives on projects and programmes. As the name
implies, framework has two interrelated parts; one, a cognitive or conceptual device – a frame
of reference which, two, enables work through systems (plans, projects, programmes etc.)
The trap of fetishism signals responsibility in systems thinking in two dimensions – one
towards understanding and another towards practice. First, with respect to understanding,
there is an imperative to continually ask questions of ‘systems’; to appreciate them as
judgements of fact rather than matters of fact. For example, when confronted with arguments
of an iniquitous ‘economic system’ generating continual social and ecological
impoverishment, or an ‘education system’ that systematically continues to marginalise
particular sectors of our community, systems practitioners have a responsibility to create
space for, and help support the framing of, better systems, rather than perpetuating the myth
that these are some God-given realities that we need to simply live with.
Second, with respect to practice, the sense of responsibility here lies with our humility in
systems design; in avoiding inclinations to fetishise systems. Geoffrey Vickers cautioned
62 Martin Reynolds

against over-enthusiasm in the models (that is, systems) that we generate (Vickers 1987). We
can often live by the dictate of models rather than, as should be, the changing realities in
which the models are applied and which ought to further shape or indeed make redundant
such models.

4.2 Towards a critical systems literacy

The risk of systems obsession is akin to moralism. Humberto Maturana makes a relevant
point distinguishing between being moralistic and ethical. Moralists, he suggests, “lack
awareness of their own responsibility. People acting as moralists do not see their fellow
human beings because they are completely occupied by the upholding of rules and
imperatives; that is a particular systems design. They know with certainty what to be done
and how everybody else has to behave” (Maturana & Poerksen 2004 p.207). Being ethical, in
contrast requires giving legitimacy to people, and particularly those who may disagree with
the rules. Using our own form of systems literacy, systems boundaries (the domain of systems
change) are subject to systematic changes invoked by the designers and users of systems, and
systemic changes invoked by those subject to the use of systems. There is here a triadic
interplay between three perpetual factors – systems with their boundaries, people and their
values, and real world entities and events in the factual domain. The relationship between
them can be expressed in terms of either an entrapped vicious circle or a liberating virtuous
cycle. For example, in terms of vicious circles, the Mark Twain quote might be seen in terms
of a hammer (a system’s tool), the hammerer (systematic people), and the hammered
(systemic events). The analogy of the UK National Health Service to a super tanker ship
might be interpreted in terms of a steer (the system or definitive plan of direction), a steerer
(systematic pilots or experts), and the steered (passive passengers).

Table 4. Features of a critical systems literacy

Type of Location of Primary Risks or Some key

change change intent traps vocabulary
Systemic Complex Make simple & Seeing a mess as simple Complexity
realities or manageable the problem-solving i.e., Feedback
situation complex web of reductionist thinking rather Emergence
realities for than as improvement Uncertainty
improving situations resolution. Autonomy
Systematic Stakeholders Developing mutual Fixing people as objects for Perspectives
understanding and purposive endeavours rather Praxis
shared practice than as purposeful subjects. Learning
Stakeholding

Systems Conceptual Improvement of Complacency and obsession Judgements

worlds situations with ‘systems’ e.g., as Boundaries
and emancipation holistic devices, rather than Reframing
through reflective as temporary pragmatic Critique
practice constructs
 Critical Thinking and Systems Thinking 63

The three types of trap noted above represent responses to particular types of well-
founded anxiety and fear with managing complex issues. There is the continual fear of
systemic uncertainty in unforeseen events and unintended consequences, the fear of losing or
even reinforcing excessive systematic control, and the fear of change in systems; an undue
ultimate optimism in old or new systems. Table 4 summarises these traps in terms of
contributing towards a critical literacy of systems thinking in practice.
A key intent of systems thinking associated with systems change is to continually
question boundaries of our conceptual constructs with a primary focus on improving the
situation. That is, with a focus on steering good systemic change.

5. SUMMARY
In an online analysis contrasting the source of scholarly publications in which the terms
‘critical thinking’ and ‘systems thinking’ were mentioned, 88% of the papers in which the
term critical thinking appeared were in social sciences, arts and humanities, whereas for
systems thinking 48% were found in literature from those fields with the remainder being
dispersed across a range of other disciplines including business, engineering, maths, and
different biophysical sciences (Cabrera, 2006). As noted by Cabrera, this would suggest that
systems thinking appears to have significant currency over and above critical thinking in
fostering a greater engagement of interdisciplinarity.
The argument put forward in this chapter though is that systems thinking is indeed
interdisciplinary, but coupled with more explicit attention to critical thinking, systems
thinking provides for a transdisciplinary engagement; one that transcends conventional
disciplinary silos. The critical literacy embodied in such transciplinarity is manifest in a
framework of systems thinking in practice. The notion of systems thinking in practice derives
from a critical systems perspective constituting three activities associated with three entities –
(i) a framework for understanding complex interrelationships in the real world context of
change and uncertainty, (ii) a framework for practice when engaging with different
perspectives amongst people involved and affected in the contexts of interest, and (iii) a
composite framework for responsibility acknowledging the limiting and integral features of
framing understanding and framing practice in the conceptual world of ideas and tools. The
framework appreciates (multi)disciplinary efforts towards framing an understanding of
interrelationships and interdependencies of complex realities in the real world. The
framework practically engages with multiple perspectives in endeavours of interdisciplinarity
towards framing some sense of mutual understanding across different disciplines and
perspectives. And most importantly the framework transcends disciplines through both (i)
boundary reflection – checking on the partiality of understanding judgements of ‘fact’
through any one disciplinary framework – and (ii) boundary discourse – checking against the
partiality of value judgments that inevitably inform any inquiry from any disciplinary or
indeed interdisciplinary perspective. The transdisciplinary framework acts as a framework for
responsibility. Together the three frameworks working together constitute a framework of
systems thinking in practice.
Whereas a systems literacy involving systemic and systematic change provides a
language to mediate between the mess of real world situations, and the systems (including
64 Martin Reynolds

methods, methodologies, approaches) used to deal with them, a critical systems literacy
involving in addition, systems change, provides a language to mediate between systems ideas
developed amongst systems practitioners and established thinking and practice associated
with different professional traditions. The critical literacy refers to all approaches, whether
traditionally systems based or belonging to other traditions of professional practices. It is in
the practice of using them whilst being aware of the inevitable traps – reductionism (silo
problem-fixing), dogmatism (fixing people), and systems fetishism associated with holism
and pluralism (fixed systems) – that enables a critical systems literacy.

REFERENCES
Ackoff, R. (1974). Redesigning the Future: a systems approach to societal problems. New
York, John Wiley.
Ackoff, R. (1981). Creating the Corporate Future: plan or be planned for. New York, John
Wiley.
Ashby, W. R. (1956). An Introduction to Cybernetics. London, Chapman & Hall.
Bawden, R. J. (1998). "The Community Challenge: The Learning Response " New Horizons
(Journal of the World Education Foundation Australia), 99 (October): 40-59.
Beer, S (1979), The Heart of Enterprise, John Wiley, London and New York.
Beer, S (1985), Diagnosing the System for Organizations, John Wiley London and New
York.
Bertalanfy, L. V. (1956). "General Systems Theory." General Systems Yearbook, 1: 1-10.
Blackmore, C. et al. (2007). "Social Learning: an alternative policy instrument for managing
in the context of Europe's water " Environmental Science and Policy (Special Issue)
10(6).
Cabrera, D. (2006) Systems Thinking Thesis. Cornell University, Ithaca, NY
Chapman, J. (2004). System Failure: why governments must learn to think differently.
London, Demos.
Checkland, P. (1978). "The Origins and Nature of 'Hard' Systems Thinking." Journal of
Applied Systems Analysis, 5: 99-110.
Checkland, P. (1981). Systems Thinking Systems Practice. Chichester, John Wiley.
Checkland, P. (2000). "Peter Checkland at 70: A Review of Soft Systems Thinking." Systems
Research and Behavioural Science, 17(1): 11-58.
Checkland, P. and J. Scholes (1990). Soft Systems Methodology in Action. Chichester, John
Wiley.
Checkland, P. and S. Holwell (1997). Information, Systems and Information Systems
Chichester, John Wiley.
Churchman, C. W. (1968). The Systems Approach. New York, Dell.
Churchman, C. W. (1970). "Operations Research as a Profession." Management Science 17:
B37-53.
Churchman, C. W. (1971). The Design of Inquiring Systems: basic concepts of systems and
organizations. New York, Basic Books.
Churchman, C. W. (1979). The Systems Approach and its Enemies. New York, Basic Books.
 Critical Thinking and Systems Thinking 65

Churchman, C. W., and R. Ackoff, et al. (1957). Introduction to Operations Research. New
York, John Wiley.
Eden, C. and F. Ackermann (1988). Making Strategy: the journey of strategic management.
London, Sage
Ellis, K. (1995). Critical Considerations in the Development of Systems Thinking and
Practice. Systems Practice, 8(2), 199-214.
Flood, R. L. (1990). Liberating Systems Theory: towards critical systems thinking. Human
Relations, 43.
Flood, R. L., & Jackson, M. C. (Eds.). (1991a). Critical Systems Thinking: Directed
Readings. Chichester: John Wiley & Sons.
Flood, R. L., & Jackson, M. C. (1991b). Total Systems Intervention: A Practical Face to
Critical Systems Thinking. In R. L. Flood & M. C. Jackson (Eds.), Critical Systems
Thinking. Chichester: John Wiley.
Flood, R. L., & Ulrich, W. (1990). Conversations on Critical Systems Thinking. In R. L.
Flodd & M. C. Jackson (Eds.), Critical Systems Thinking: Directed Readings. Chichester:
Wiley.
Forrester, J. W. (1961). Industrial Dynamics. Cambridge MA, Wright-Allen Press.
Forrester, J. W. (1971). World Dynamics. Cambridge Mass., Wright & Allen.
Friedman, J. 1987. Planning in the Public Domain: from Knowledge to Action. New York:
Picnceton University Press.
Fuenmayor, R. (1991). Between Systems Thinking and Systems Practise. Critical Systems
Thinking: Directed Readings. R. L. Flood and M. C. Jackson. Chichester, Wiley.
Habermas, J. (1971). Knowledge and Human Interests. London, Heinemann.
Habermas, J. (1984). The Theory of Communicative Action Volumes 1 and 2. Cambridge,
Polity Press.
Hall, A. D. (1962). A Methodology for Systems Engineering. New York, Van Nostrand.
Huston, T. (2007). Inside Out: Stories and Methods for Generating Collective Will to Create
the Future We Want. Cambridge, MA, Society for Organizational Learning.
Jackson, M. (1982). "The Nature of Soft Systems Thinking: the work of Churchman, Ackoff
and Checkland." Journal of Applied Systems Analysis, 9: 17-28.
Jackson, M. (1985). The Itinerary of a Critical Approach. Review of Ulrich's 'Critical
Heuristics of Social Planning'. Journal of the Operational Research Society, 36, 878-881.
Jackson, M. C. (1990). The Critical Kernal in Modern Systems Thinking. Systems Practice,
3(4), 357-364.
Jackson, M. C.(1991a). Modernism, post-modernism and contemporary systems thinking. In
R. L. Flood & M. C. Jackson (Eds.), Critical Systems Thinking: Directed Readings (pp.
287-302). Chichester: John Wiley.
Jackson, M. C. (1991b). The Origins and Nature of Critical Systems Thinking. Systems
Practice, 4(2), 131-148.
Jackson, M. C. 1999. "Towards Coherent Pluralism in Management Science." Journal of the
Operational Research Society 50:12-22.
Jackson, M. C. 2000. Systems Approaches to Management. London: Kluwer
Academic/Plenum Publishers.
Jackson, M. and P. Keys (1984). “Towards a System of Systems Methodologies”. Critical
Systems Thinking: Directed Readings. R. L. Flood and M. C. Jackson. Chichester, John
Wiley.
66 Martin Reynolds

Kleiner, A. (1996). The Age of Heretics: Heroes, Outlaws, and the Forerunners of Corporate
Change. New York, Currency (Doubleday).
Mason, R. O. and I. I. Mitroff (1981). Challenging Strategic Planning Assumptions: Theory,
Cases and Techniques. New York, John Wiley.
Maturana, H. and Poerksen (2004). From Being to Doing: The Origins of the Biology of
Cognition. Heidelberg, Germany, Carl-Auer Verlag.
Meadows, D. H., D. L. Meadows, et al. (1972). The Limits to Growth: a Report for the Club
of Rome's Project on the Predicament of Mankind. London, Earthscan.
Meadows, D. H., D. L. Meadows, et al. (1992). Beyond the Limits of Growth. Post Mills,
Chelsea Green.
Midgley, G. 1992. "Pluralism and the Legitimation of Systems Science." Systems Practice
5:147-172.
Midgley, G. (2000). Systemic Intervention: Philosophy, Methodology and Practice. New
York, Kluwer/Plenum.
Midgley, G. (Ed.) (2003), Systems Thinking (4 vols.). London: Sage.
Mingers, J. 1997. "Towards Critical Pluralism." Pp. 407-440 in Multimethodology: The
Theory and Practice of Combining Management Science Methodologies, edited by J.
Mingers and A. Gill. Chichester: John Wiley.
Mulgan, G (1997) Life After Politics: New Thinking for the Twenty First Century Fontana
Press, London.
Oliga, J. C. (1988). Methodological Foundations of Systems Methodologies. Systems
Practice, 1(1), 87-109.
Oliga, J. C. (1990). Power - ideology matrix in social systems control. In R. L. Flood & M. C.
Jackson (Eds.), Critical Systems Thinking: Directed Readings (pp. 269-286). Chichester:
John Wiley.
Optner, S. L. (1965). Systems Analysis for Business & Industrial Problem Solving. New
York, Prentice-Hall.
Prigogine, Ilya. 1997. End of Certainty. New York: The Free Press.
Ramage, M. and K. Shipp (2009). Systems Thinkers. London, Springer.
Reynolds, M. (1998). "Unfolding" Natural Resource Information Systems: fieldwork in
Botswana. Systems Practice and Action Research, 11(2), 127-152.
Reynolds, M. (2002). "In defence of knowledge constitutive interests. A comment on 'What is
this thing called CST?' (Midgley, 1996)." Journal of the Operational Research Society,
53(10): 1162-1164.
Reynolds, M. (2004). "Churchman and Maturana: Enriching the Notion of Self-Organization
for Social Design." Systemic Practice and Action Research, 17(6): 539-556.
Reynolds, M. 2007. "Evaluation based on critical systems heuristics." Pp. 101-122 in Systems
Concepts in Evaluation: An Expert Anthology, edited by Bob Williams and Iraj Imam.
Point Reyes CA, USA EdgePress.
Reynolds, M. (2008a). "Getting a grip: A Critical Systems Framework for Corporate
Responsibility." Systems Research and Behavioural Science, 25(3): 383-395.
Reynolds, M. (2008b). "Reframing expert support for development management." Journal of
International Development, 20: 768-782.
Reynolds, M, and S Holwell (Eds.). (2010). Systems Approaches to Managing Change: a
practical guide. London, Springer
 Critical Thinking and Systems Thinking 67

Rosenhead, J. (1989). Rational Analysis for a Problematic World: Problem Structuring

Methods for Complexity, Uncertainty and Conflict. Chichester, John Wiley.
Sangren, P. S. (1988). Rhetoric and the authority of ethnography: "postmodernism" and the
social reproduction of texts. Current Anthropology, 29(3), 405-435.
Senge, P. (1990). The Fifth Discipline. New York, Currency Doubleday.
Thackara, J. (2005) In the Bubble: Designing for a Complex World. MIT Press ISBN 0-262-
20157-7 (extract from chapter 8 found on http://www.thackara.com/inthebubble/toc.html)
Trist, E.L., Higgin, G.W., Murray, H., and Pollock, A.B. (1963) Organisational Choice,
Tavistock, London.
Ulrich, W. (1983). Critical Heuristics of Social Planning: a new approach to practical
philosophy. Stuttgart (Chichester), Haupt (John Wiley - paperback version).
Ulrich, W. (1988a). Churchman's "Process of Unfolding" - Its Significance for Policy
Analysis and Evaluation. Systems Practice, 1(4), 415-428.
Ulrich, W. (1988b). Systems Thinking, Systems Practice, and Practical Philosophy: a
programme of research. Systems Practice, 1(2), 137-163.
Ulrich, W. (1993). Some Difficulties With Holistic Thinking. Systems Practice, 6(6), 583-
608.
Ulrich, W. (1996). A Primer to Critical Systems Heuristics for Action Researchers. Hull:
University of Hull.
Ulrich, W. (2003). "Beyond Methodology Choice: critical systems thinking as critically
systemic discourse." Journal of the Operational Research Society, 54(4): 325-342.
Ulrich, W. (2006). Critical pragmatism: a new approach to professional and business ethics
Interdisciplinary Yearbook of Business Ethics. L. Zsolnai. Oxford, UK, and Bern,
Switzerland, Peter Lang Academic Publishers. 1: 53-85.
Ulrich, W., and M. Reynolds. 2010. "Critical Systems Heuristics." Pp. 243-292 in Systems
Approaches to Managing Change, edited by Martin Reynolds and Sue Holwell. London:
Springer.
Vickers, G. (1965). The Art of Judgement, Chapman and Hall, London
Vickers, G. (1987). Essays of Sir Geoffrey Vickers. Policymaking, Communication and
Social Learning. G. B. Adams, J. Forester and B. L. Catron. New Brunswick, NJ.,
Transaction Books.
Zhu, Z. (2011). "After Paradigm: why mixing-methodology theorising fails and how to make
it work again." Journal of the Operational Research Society 62 (4): 784-798.
In: Critical Thinking ISBN: 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 69-95 2011 Nova Science Publishers, Inc.

Chapter 3

DEVELOPING CRITICAL THINKING THROUGH

PROBABILITY MODELS

Einav Aizikovitsh-Udi
Harvard University, Boston, Massachusetts, USA

“Learning without thinking is a wasted blessing"

Confucius

ABSTRACT
In light of the importance of developing critical thinking, and given the scarcity of
research on critical thinking in mathematics studies in the broader context of higher-order
thinking skills, we have carried out a research that examined how teaching strategies
oriented towards developing higher-order thinking skills influenced the students’ critical
thinking abilities. The guiding rationale of the work was that such teaching can foster the
students’ skills of and dispositions towards critical thinking. In this research, a primary
attempt has been made to examine the relations between education for critical thinking
and mathematics studies through examining teaching and learning critical thinking
according to the infusion approach, which combines critical thinking and mathematical
content (“Probability in Daily Life” learning unit).The main contribution of this work and
the innovations it is expected to introduce lie in elucidating the connection between
critical thinking and the study of mathematics and creating insights into the mechanisms
of critical thinking development, and its place and importance in the study of
mathematics, in spite of the uncertainty whether critical thinking skills acquired in
studying one field will necessarily be applied by students in other fields, referred to as
“the transfer problem.” In this way it will be possible to strengthen the status of
mathematics studies in imparting higher-order thinking skills in various frameworks, in
parallel with and beyond the formal program of studies. The purpose of this research is to
examine how and to what extent it is possible to develop critical thinking by means of the
learning unit “Probability in Daily Life” using the infusion approach. The research
questions that guided it are: (1) To what extent does the study of “Probability in Daily
Life” in the infusion approach contribute to the development of critical thinking
dispositions? (2) To what extent does the study of “Probability in Daily Life” in the
70 Einav Aizikovitsh-Udi

infusion approach contribute to the development of critical thinking abilities? (3) What
are the processes of construction of critical thinking skills (e.g., identifying variables,
postponing judgment, referring to sources, searching for alternatives) during the study of
the “Probability in Daily Life” learning unit in the infusion approach? The present
research involved nine groups of gifted and high-achieving mathematics students in
eleventh grade from all the social groups and strata of Israeli society. The students
studied the learning unit “Probability in Daily Life” modified by the researchers to
include critical thinking teaching in the infusion approach. The students were then tested
in two critical thinking tests, CCTDI and the Cornell Critical Thinking Test, the results of
which were statistically analyzed, and also selectively interviewed, with subsequent
qualitative analysis of the interviews and lesson transcripts. Thus the research combines
quantitative and qualitative methods.
The research findings can be summed up in the following categories: (i) In all three
iterations of the experimental teaching, a moderate improvement was detected in the
critical thinking dispositions of all experimental groups. (ii) Throughout these iterations,
a moderate improvement was also detected in the students' critical thinking abilities. (iii)
Teaching critical thinking contributed to the construction and use of these skills in the
framework of mathematics. Thus, when teachers consistently emphasize critical thinking
skills, the students are more likely to succeed in the subject of mathematics. (iv) This
research did not detect a clear-cut distinction between the critical thinking abilities and
dispositions of excellent and average mathematics students. That is, no direct correlation
has been found between the development of mathematical knowledge and the
development of critical thinking. On the basis of these findings, the following
recommendations for further research can be made: (1) A more comprehensive
examination of the processes of critical thinking: to what extent could the students
describe, orally and in writing, the processes of thinking, activate them and apply the
thinking skills they studied on the procedural and meta-cognitive level? Did they make an
informed use of terms and strategies of higher-order thinking, including critical thinking?
In other words, it should be examined what use the research participants make of the
“language of thinking,” or, in the words of Costa and Marzano, “do they speak thinking?”
(Costa & Marzano, in Harpaz, 1997). Developing such a language involves, on the part of
the teacher, such skills as using precise vocabulary, presenting critical questions,
presenting data rather than answers, aspiring for exactness, giving directions, and
developing meta-cognition. (2) Examination of the attitudes and perceptions of education
students in colleges for teacher training, practicing teachers and researchers of
mathematical education with regard to teaching that develops critical thinking in
mathematics; evaluation of these students’ and professionals’ critical thinking functions
in teaching, learning, and research. (3) Teaching “Probability in Daily Life” and
conducting the same research among all the strata of the students’ population and not
only among those who study mathematics at the higher level.
It definitely seems that in the last decade, there has been a rapidly growing
awareness of the importance of promoting the development of thinking skills in the
Israeli educational system, and the system has been making considerable progress
towards integrating the curriculum learning materials that contribute to the development
of higher-order thinking skills1. In 1994, the Ministry of Education recognized thinking
skills as a distinct subject of studies. This recognition lead to the establishment of a
Subject Committee for Thinking Skills, which is in charge of consolidating appropriate
didactic materials, as is the case with the rest of the academic subjects in the school
system. The complex and ceaselessly changing contemporary reality, which requires
independent decision-making on a daily basis, makes it extremely important to impart to

1
Critical, deductive, creative, inventive and other types of thinking: on the interconnections between different types
of thinking, see Appendix 10.1
 Developing Critical Thinking through Probability Models 71

students the ability to think critically. Critical thinking is needed in every field of activity,
as it allows the individual to deal with reality in a reasonable, mature and independent
way (Lipmann, 1991). The need for developing critical thinking in different disciplines is
anchored in the ideals of education for democracy, as our freedom to think about and
criticize the reality and society in which we live is a form of expression of our autonomy
as individuals. Today this idea is even more vital, because of the growing need to be
capable of engaging in inquiry and evaluation based on rational considerations regarding
the various messages we are exposed to in different areas of life (Feuerstein, 2002,
Perkins, 1992, Swartz, 1992). In the field of education, mathematics has traditionally
been considered a branch of knowledge particularly suited to the teaching and learning of
higher-order thinking skills, such as critical thinking. Mathematics curricula all over the
world, including Israel, identify the acquisition of these skills as one of their goals. The
idea that mathematics is a discipline suited to teaching critical thinking also appears in
the research literature2. However, in spite of this assumption, very few empirical studies
to date have engaged with the question of whether the study of mathematics indeed
develops or even requires this mode of thinking. The answer to this question is far from
being clear. The present research tackles precisely this basic question, “Is it possible to
develop critical thinking in the framework of mathematics studies?”

RESEARCH QUESTIONS AND METHODOLOGY

The following research questions guided the study:

• To what extent does the infusion-approach study of “Probability in Daily Life”

learning unit contribute to students’ development of critical thinking
dispositions?
• To what extent does the infusion-approach study of “Probability in Daily Life”
contribute to students’ development of critical thinking abilities? What are the
instructional processes (e.g. putting a statement into question, delaying
judgment, referring to sources) by which critical thinking skills are constructed
during the infusion-approach study of the “Probability in Daily Life” unit?

Research population: 180 tenth-grade students, studying in two educational frameworks:

the Kidumatica mathematics program for gifted youth at Ben-Gurion University and the
formal high school framework. About 50 students studied this unit with the researcher in a
general high school in central Israel, and about 40 students studied the same program as part
of the formal high-school curriculum with another teacher. The two groups represent the
multi-cultural composition of the Israeli society: city residents, kibbutz members, the
religious sector and the Arab sector. “Probability in Daily Life, the special learning unit
developed for the purposes of this research, contained 15-16 double lessons (30-32 academic
hours). Research tools: questionnaires3, personal interviews, observation, class records and
analysis of lesson plans.

2
A number of articles on critical thinking in mathematics use the term in other contexts and deal with imparting
technical tools such as making an estimation, comparison or inference, verifying a result, evaluating an
exercise, application and interpretation, solution strategies etc.
3
The following questionnaires were used: the Cornell critical thinking test, version Z (Ennis et al., 1985), checking
the dispositions and abilities for critical thinking (answering research questions 1,2); a content-based critical
72 Einav Aizikovitsh-Udi

UNIQUENESS AND CONTRIBUTION

Educators in Israel, who wonder, like their colleagues worldwide, about the goals of the
education system that could guide the different educational frameworks, may find in this
research an idea that can unify different topics and study programs, in order to prepare
learners for life in a changing society, and develop their ability to think in a systematic and
independent way. More generally, this research is expected to contribute to the public
discourse of the mathematical education community on this issue. It raises the public
awareness of the need to develop critical thinking in the framework of mathematical
education, which may enable future examination and promotion of the development of critical
thinking through mathematics teaching in a fuller and more informed way.
Drawing on infusion-approach study of "Probability in Daily Life," the present research
establishes points of reference to critical thinking dispositions and abilities among students
learning mathematics in different environments (high school and the Kidumatica
mathematics club). This combination has not been examined so far by the literature in the
field. This research has identified and measured differences between dispositions, abilities,
and construction of skills characteristic of critical thinking in mathematics, and completes
other researchers conducted in other environments. The combination of the Cornell test and
the CCTDI test in the evaluation of critical thinking abilities and dispositions is unique to this
research; it has not been performed in previous studies. To conclude, the main contribution of
this research lies in revealing the connection between critical thinking and the teaching of
mathematics. Despite the problem of transfer discussed earlier, the scientific contribution of
this research lies in the new insights it provides into critical thinking, its place and importance
in teaching mathematics. Thus it will be possible to strengthen the status of the study of
mathematics in imparting higher-order thinking skills, both in parallel with and beyond the
formal education program.

IMPLICATIONS FOR THE FORMAL SCHOOL CURRICULUM

Current approaches in the teaching of mathematics according to the new mathematics
curriculum adhere to a conceptual understanding of mathematics and emphasize
investigation, problem solution, mathematical literacy4 and use of mathematical discourse.
The students are supposed to actively construct their knowledge and understanding, while the
teacher functions as a ‘mediator’ by asking questions, posing challenges and assigning
investigation tasks, and helps the students to think in deeper ways about various concepts,
ideas and mathematical contexts. Studying and teaching mathematics in such a way is a very
difficult task, because the ways of teaching and learning are very demanding and creative,
requiring, among the rest, deep knowledge and understanding of mathematics on the teacher’s
part, coping with the unknown on the student’s part, and much intellectual effort on both

thinking questionnaire on “Probability in Daily Life” (answering research question 3); mathematical
knowledge questionnaire (answering research question 4).
4 Mathematical literacy is defined as “an individual’s capacity to identify and understand the role that mathematics
plays in the world, to make well-founded judgements and to use and engage with mathematics in ways that
meet the needs of that individual’s life as a constructive, concerned and reflective citizen." (PISA 2003:
Assessment Framework -Mathematics, Reading, Science and Problem Solving Knowledge and Skills).
 Developing Critical Thinking through Probability Models 73

sides. This research was designed while taking into account the implications of earlier
researches’ findings (Feuerstein, 2002; Zohar & Tamir, 1993; Weinberger, 1998) and of the
pilot study conducted in preparation for this research. these findings point at the importance
of learning experiences that develop critical thinking by means of various specially designed
curricula. The findings of the present research are likely to be useful in composing new study
programs and methods that can be based on the connection between critical thinking and the
study of mathematics, which this research brings to light.

CRITICAL THINKING: AN OVERVIEW

Critical thinking is a topic that has interested humans since ancient times. Development
of critical thinking has been defined as one of the most important goals of education since the
Middle Ages until today. The ancient ‘fathers’ of the idea of critical thinking are considered
to be the sophists5 (740-399 BC) and Socrates (5th c. BC). Socrates, who was and still remains
an extremely influential philosophical figure, dealt mainly with the theory of ethics and the
issues of governing society and the state. Walking the streets of Athens, he approached people
with questions about the nature of the world. In order to understand their opinion on a certain
issue, he first had to clarify their definition of that issue and whether that definition was true.
He was a person who thought independently, and taught others to think for themselves.
Therefore, if one wanted to be a disciple of Socrates, one would have to think independently,
and if necessary, to be able to detach oneself from previously known and generally accepted
ideas and definitions (Bryan, 1987). Socrates is known to have resisted the greatest cultural
innovation of his time – the writing of books. He claimed that writing on parchment does not
allow open argument and contestation, which are crucial for thinking (Regev, 1997). Socrates
used a technique called elenchus (ελενχος), a mixture of questions somewhat like a cross-
examination, which later became known as the “Socratic method” or “Socratic debate,” and
in which Socrates refrained from openly introducing his own opinions. Socrates makes all his
conclusions from the answers of his opponent, which served him later in the debate to defeat
the latter’s opinions; thus in his constant striving for the absolute knowledge he created a
method of critical thinking, and posed an ideal model of critical thinking for his successors.
What made him such a model was that he investigated questions, was the first to raise the
problem of definition, sought after the meaning of things, sought to find self-evident
arguments and proofs, used inductive arguments and did not grant axiomatic validity to
definitions (Bryan, 1987). Socrates was sentenced to death on the charge of treason and
“corruption of youth.” Only later did Plato’s writings, “the Socratic dialogues,” defend the
good name of Socrates and prove that he was wrongfully convicted (Bryan, 1987). The
pedagogy of questioning and thinking, according to Aristotle, begins with wondering, with
the primary question. The ability to ask questions is crucial for a human being, and in Jean
Paul Sartre’s terms, it is essential to be able “to see what is lacking – about facts, reasons,
explanations that we lack – to explain what is present and is experienced as lacking” (Harpaz
& Adam, 2000).

5
The sophists developed the theory of rhetoric, the basis of non-formal logic, which later became an important
component in education for critical thinking.
74 Einav Aizikovitsh-Udi

The Educational and Social Importance of Critical Thinking

As in the distant past, the need for developing critical thinking today is anchored in the
ideals of education for democracy, which postulate our freedom to think about and criticize
reality and society in which we live, as an expression of our being autonomous individuals.
Today this idea becomes even more vital, because of the increasing need to be able to
investigate and evaluate various messages presented to us in different fields, on the basis of
rational considerations. In this sense, to develop a critical approach and attitude towards
various issues means to “be aware” (Feuerstein, 2002). Matthew Lipman in his article “A
Functional Definition of Critical Thinking” points at the traditional distinction between
‘knowledge’ and ‘wisdom’. ‘Knowledge’ refers to the sum of information and ‘truths’ passed
from generation to generation, while ‘wisdom’ refers to the person’s ability to make sensible
decisions in complicated and unclear situations. Wisdom is highly prized, because previous
knowledge is not sufficient in order to know what the best way to act is. According to
Lipman, in periods of transition and change, when the reservoir of traditional knowledge
becomes insufficient to deal with reality, wisdom, which is characterized by intellectual
flexibility and originality, is highly esteemed. In our days, according to Lipman, the term
‘wisdom’ came to be replaced with the term ‘critical thinking’. The principles of critical
thinking, according to Lipman, are the ability to exercise judgment (judgment is activated
when we use our knowledge to arrive at practical decisions), use of criteria in decision-
making (when we have several options that we critically compare to each other in order to
choose the one that appears best), sensitivity to context (when we take into account the
specific conditions of context and choose a suitable way to act, instead of acting as we are
accustomed to, without regard for the specific situation), and finally, self-correcting thinking
(when we encounter a problem that springs from our course of action, we are prepared to
make a re-evaluation and to correct that course). The contemporary reality is complex and
ceaselessly changing. It constantly demands arriving at independent decisions, therefore it is
extremely important to instill in the students at school the ability to think critically, according
to the above principles. Critical thinking is necessary in any field of occupation, since it
allows the individual to deal with reality in a reasonable and independent way.
The Critical Thinking Movement: At the time of unprecedented confusion regarding the
appropriate goals of education, the educational Critical Thinking Movement, with
headquarters in the United States, proposes a profound discussion of one educational goal
rooted far back in the antiquity. This movement has a considerable influence in the U.S. and
is increasingly influential also in other countries, including Israel. The movement’s thinkers –
philosophers, psychologists and educators – claim that critical thinking is a worthy
educational goal that suits the spirit of the present time and answers its challenges. The
movement is a sub-current of a larger and older, internationally renowned movement called
“Education for Thinking,” which began 30 years ago in the United States in response to the
failure of school education to realize its goals. “Education for Thinking” set out to propose an
educational ideal that should guide all the educational institutions. In the framework of the
“Critical Thinking” movement, its title concept was given different and even contradictory
theoretical and didactic definitions. Many educators from different disciplines are trying to
define the field of critical thinking and create a common concept. Yet, in spite of the variety
of definitions and the disagreements on the meaning of the movement’s central idea, the
purpose of the movement is to educate young people for critical thinking and personality,
 Developing Critical Thinking through Probability Models 75

prepared and able to examine the accepted beliefs. David Perkins, one of the movement’s
most notable thinkers, emphasizes the need for fostering critical thinking as a tool for
understanding knowledge, and not as a goal for its own sake (Harpaz, 1996,1997). The
Critical Thinking Movement seeks to encourage students to cast intelligent doubt about what
the authorities – teachers, specialists, textbooks, books, newspapers, television – tell them. It
seeks to bring up critical pupils who ask questions such as, on what grounds does a certain
text or person claim what they claim? From what point of view are they claiming this? Why
prefer their claim over other, contradicting or different claims? The idea of educating for
critical thinking, as well as the idea of educating for creative thinking, has far-reaching
consequences for school education. At present, it is mostly an idea, rather than action, but one
can already see practical attempts to realize this idea in the educational field. One of the ways
to realize the idea of educating for critical thinking is to “translate” it into a range of skills.
Thus, for instance, the devoted promoters of this idea developed a new field called “informal
logic,” which helps to locate, criticize and construct propositions in natural language. Other
supporters of education for critical thinking developed a classification of skills, such as the
skill of examining reliability of information sources, the skill of uncovering basic
assumptions, the skill of identifying biases, etc. Other supporters developed study programs
based on conflicts between different worldviews, standpoints and versions. Still others
composed programs of critical reading, critical watching and critical “consumption” of media
(Harpaz & Adam, 2000).

THEORY OF CRITICAL THINKING

"Critical thinking is that mode of thinking - about any subject, content or problem - in
which the thinker improves the quality of his or her thinking by skillfully taking charge of the
structures inherent in thinking and imposing intellectual standards upon them"

NCECT

Definitions of Critical Thinking

A historical survey over several decades shows that the existing plethora of definitions of
‘critical thinking’, vagueness and lack of true understanding surrounding the term have led to
a structural disagreement about the nature of this phenomenon among researchers,
psychologists, informal logicians, philosophers, educators and theorists (Ennis, 1985,1987;
Lipman, 1991; McPeck, 1981; Passmore,1980; Paul, 1993; Siegel, 1998; Johnson & Blair,
1994). The situation of this term is similar to that of the term ‘environment protection’.
Everyone agrees about the importance of the activity and its goals, but the lack of clarity
about the exact nature of the goals and the means of achieving them prevents necessary action
in many cases. Some see critical thinking simply as “everyday, informal reasoning” (Galotti
1989), whereas others feel differently. Shafersman (1991) proposes that a critical thinker is
one who asks questions, offers alternative answers and questions traditional beliefs. He
believes that such people, who seem to be challenging society, are not welcome, and for this
reason critical thinking is not encouraged. Lipman considers it to be different from ordinary
76 Einav Aizikovitsh-Udi

thinking because it is both more precise and more rigorous, and furthermore, it is also self-
correcting (1991). It has also been described by Halpern (1998) as being “purposeful,
reasoned, and goal-directed. “Since there exist in literature dozens of widely varying
definitions of ‘critical thinking’6, I have no way of posing one definition, as is conventionally
done in dissertations. I will try to dispel this vagueness by presenting different definitions and
the disagreements between different specialists concerning these definitions.
On the basis of extensive reading in the field, it seems that critical thinking is a thinking
that establishes criteria for examining beliefs, opinions and truths, in order to give a rationally
based preference to certain beliefs, opinions and truths over others – and to be prepared to
doubt even these7. Thus, critical thinking is a thinking that criticizes phenomena, ideas and
products on the basis of rational and emotional criteria. Ennis (1987) defines critical thinking
as “reasonable and reflective thinking focused on deciding what to believe or do.” This
definition replaced a narrower definition Ennis proposed in 1962, as “correct evaluation of
statements,” which encountered much criticism and opposition, because it was based on
logical skills alone. Ennis, who is known as one of the most important writers on critical
thinking, presents in his article “A Taxonomy of Critical Thinking Dispositions and Abilities”
(1987) the taxonomy of critical thinking, which includes fourteen dispositions and twelve
abilities, subdivided into sub-abilities. Some of the dispositions and abilities essential for a
critical thinker will be described below: dispositions such as searching for a question, making
an argument, taking care to be well-informed, using reliable sources, searching for
alternatives, taking a stand, and abilities such as clarity, grounding of claims, inference, and
interconnection. Siegel (1988) discusses Ennis’ taxonomy very favorably and refers to the
inclusive set of dispositions, characteristics and abilities proposed by Ennis as a “regulative
ideal”8. He claims that critical thinking guides our judgments and provides us with criteria of
excellence on which evaluation of educational activities can be based – therefore it is called a
regulative ideal. Siegel in his article deals with the question, “Who is a critical thinker?” He
claims that a critical thinker has to be an individual with a certain type of personality,
dispositions, traits of character and thinking habits. The critical thinker has to know how to
evaluate statements, and to be prepared to match judgment and action to a principle, to
demand justification and to question ungrounded claims.
McPeck (1981) defines critical thinking as correct use of reflexive skepticism in a given
field, and sees the essence of critical thinking in the behavioral aspect of doubting, or
“postponement of judgment.” McPeck’s behavioral approach differs from Ennis’ definition
(according to Ennis, the essence of critical thinking is the logical-analytic activity of
analyzing statements). In order to deeper understand both McPeck’s and Ennis’ definitions, I
will review and compare the various approaches again. McPeck sees the essence of critical
thinking in its behavioral aspect, while Ennis completely ignores this aspect. McPeck opposes
those who see critical thinking as primarily evaluation of statements, because evaluation of

6
In fact, each definition relates to a certain area in the field of education and includes several important aspects.
7
The concept of ‘critical thinking’ raises a wealth of associations. Some will imagine critical thinking as doubting
whatever is said, others as a kind of protest, provocation or an inclination to argue. With regard to the term
‘doubt’ I will adhere to the positive meaning of the word: not discarding an old idea and seeking to replace it
with a new idea, but recognizing the value of the old idea while creating and posing a new one alongside with
the old. Without doubting and testing ideas, it would be impossible to develop new and better ideas. Doubt is,
without a doubt, a crucial force in learning and development.
8
Siegel defines regulative criteria for excellence, the ability to choose between methods, kinds of policy, and
educational acts.
 Developing Critical Thinking through Probability Models 77

statements concentrates on questions of validity and not on checking the reliability of

information sources. According to McPeck, we do not routinely analyze conclusions, but
rather evaluate data, information and facts. In order to do this, one needs to be well
acquainted with the field that the evaluated information belongs to. Therefore, according to
McPeck, “acquisition of specific skills is neither a necessary nor a sufficient condition for
critical thinking”; a more detailed definition can be found on the official website of NCECT9:
Critical thinking is that mode of thinking – about any subject, content, or problem – in which
the thinker improves the quality of his or her thinking by skillfully analyzing, assessing, and
reconstructing it. Critical thinking is a self-directed, self-disciplined, self-monitored, and self-
corrective thinking. It presupposes assent to rigorous standards of excellence and mindful
command of their use. It entails effective communication and problem-solving abilities, as
well as a commitment to overcome our native egocentrism and sociocentrism. Watson and
Watson & Glaser (1980) claim that critical thinking is: (1) an investigative approach that
involves an ability to recognize and accept the general need of proving whatever is assumed
to be true; (2) knowledge of the nature of valid conclusions, and of abstractions and
generalizations in which the measure of validity of different kinds of evidence is established
in a logical way; (3) skills of applying the above knowledge and approaches. Critical thinking
is also defined as result-based, rational, logical and reflective evaluative thinking in terms of
what to reject or accept and what to believe, following which thinking a decision is made
what to do (or not to do), and then to act accordingly, taking responsibility for the decisions
that were made and for their implications. Elsewhere, critical thinking is defined as the ability
and readiness to evaluate claims in an objective manner, based on solid arguments (Wade &
Tavris, 1993). From all of the above definitions it can be concluded that critical thinking is
characterized both by behavioral components, such as doubting, postponement of judgment
and inquisitiveness, and by cognitive components, such as the process of investigation and
drawing conclusions. We would like to focus on three specific definitions that deal with both
abilities and dispositions. McPeck defines critical thinking as “skills and dispositions to
appropriately use reflective skepticism” (McPeck, 1981). Lipman claims that critical thinking
is “thinking which enables judgment, is based on criteria, corrects itself, and is context-
sensitive” (Lipman, 1991). The third definition is the one we have based our research. Ennis
(1962) defines critical thinking as “a correct evaluation of statements". Over twenty years
later, Ennis broadened his definition to include a mental element, defining it as “reasonable
reflective thinking focused on deciding what to believe or do” (Ennis, 1985). The abilities
related to critical thinking are divided into two categories: skills, which include the ability to
analyze, evaluate, and draw conclusions, and dispositions, such as the motivation, inclination
and urge of the student to apply critical thinking to discussing issues, making decisions,
and/or solving problems. In addition to critical thinking skills, it is also important to evaluate
the students’ dispositions towards critical thinking, since they may point at the learner’s
inclination to practically apply critical thinking in various contexts.

9
The National Council for Excellence in Critical Thinking, http://www.criticalthinking.org/about/nationalCouncil
78 Einav Aizikovitsh-Udi

Critical Thinking Abilities According to Ennis’ Taxonomy10

Critical thinking depends on a number of skills, such as identifying the source of

information, assessing the source’s reliability, evaluating the extent of the new information’s
consistency with previous knowledge, and making a conclusion on the basis of all these
mental acts. In the literature, critical thinking skills are considered necessary for encouraging
meta-cognitive understanding. According to Ennis (1963,1987,1991,2002) critical thinking is
a reflective activity (in which the person examines his/her own thinking activity) and at the
same time a practical activity, the goal of which is a rational belief or action. There are five
key concepts here: practical, reflective, rational, belief, and action. In light of these, Ennis
upgraded his taxonomy of critical thinking11 and divided it into a system of dispositions and
abilities presented below. The principal areas of the critical thinking ability are clarity,
grounding, inference, and interrelatedness. The critical thinking abilities are: focusing on the
question; analyzing statements; asking questions; evaluating the reliability of the source;
deduction; value-judging; defining terms; identifying assumptions; making decisions about
action; interrelatedness with others. It is important to note that the principal areas presented in
Fig. 2.1.3 have an intuitive dimension: we want to be clear about what is happening; we want
to have an acceptable grounding for our judgments; we want our inferences to be logical; we
want our interrelations with others to be sensitive, and we want that the dispositions and
abilities for critical thinking should be active (Harpaz, 2002).

Critical Thinking Dispositions according to Facione

Critical thinking has been investigated largely in terms of thinking skills that involve the
cognitive domain. For decades, the promotion of students’ thinking has been the focus of
educational studies and programs (Boddy, Watson, & Aubusson, 2003; De Bono, 1976;
Ennis, 1985; Kuhn, 1999). Each of these programs has its own definition of thinking and/or
of skills. Some use the phrase ‘cognitive skills’ (Leou et al., 2006; Zoller et al., 2000) and
others refer to ‘thinking skills’ (Aizikovitsh & Amit 2008, 2009, 2010; Boix-Mansilla &
Gardner, 1998; De Bono, 1990; Egan, 1997; Resnick, 1987; Zohar & Dori, 2003; Zohar,
2004), but they all distinguish between higher- and lower-order skills. Resnick (1987)
maintained that thinking skills resist precise forms of definition; yet, higher order thinking
skills can be recognized when they occur. Our ever-changing and challenging world requires
students, our future citizens, to go beyond the building of their knowledge; they need to
develop their higher-order thinking skills, such as system critical thinking, decision making,
and problem solving (Zohar, 1999; 2000, Zoller, 2002; 2007). There have been significant
changes in the past decades in the field of education. Whereas earlier the teacher was at the

10
. Ennis emphasizes that the dispositions and abilities in his taxonomy relate to general critical thinking. In order to
infuse them into the general curriculum, it will be necessary to teach them several times, at different levels of
difficulty and in the framework of different study subjects (see Fig. 2)
11
In his first article from 1962, Ennis defined critical thinking as “correct evaluation of statements. In 1987, he
replaced this definition with a new one: “Critical thinking is a reflexive rational thinking focusing on the
decision what to believe or to do”. Ennis upgraded his taxonomy, which he published 25 years earlier,
according to his new definition. The first taxonomy only included abilities and skills, while the present one, a
“taxonomy of dispositions and abilities for critical thinking,” also includes dispositions (14 dispositions and 12
abilities).
 Developing Critical Thinking through Probability Models 79

center and the emphasis was put on what to teach, today’s education involves teaching how to
think, and in particular, how to be a critical thinker. Critical thinking is necessary in every
profession, and it allows one to deal with reality in a reasonable and independent manner
(Lipman, 1991; McPeck, 1994; Paul, 1993). There seems to be no clear consensus as to what
exactly critical thinking is. Some see it as simply being “everyday, informal reasoning”
(Johnson & Blair, 1994), whereas others feel differently. Yet, it seems evident at this point
that the ability to think critically is not something that we are born with, and it is widely
accepted that it is in fact a learned ability that we need to teach. There are taxonomies that set
out a list of reasoning skills involved in critical thinking (12 skills according to Ennis’
taxonomy of 1962 or 15, according to Dick, 1991). Many of these approaches assume that
when these skills are taught and used properly, the students will become better thinkers. Other
approaches see dispositions as playing a vital part in the process of critical thinking. Beyer
(1987) describes dispositions for critical thinking as involving "an alertness to the need to
evaluate information, a willingness to test opinions, and a desire to consider all viewpoints."
Halpern (1996) emphasizes the importance of the students’ dispositions, since skills are
useless unless put into practice. In addition to successfully using the appropriate skill in a
given context, critical thinking implies also the disposition to recognize the need for using a
particular skill in a certain situation, and the willingness to make the effort of applying it.
Facione and Facione (1994, 2000) describe dispositions towards critical thinking as
containing elements of intellectual maturity, searching for truth, open-mindedness,
systematicity, self-confidence in critical thinking, analyticity, and inquisitiveness.
They developed the California Critical Thinking Disposition Inventory (CCTDI), which
was originally meant to be used to assess critical thinking dispositions in college students, but
has been successfully adapted also for use in high school. There are seven scales on the
CCTDI. Each describes an aspect of the overall disposition toward using one's critical
thinking to form judgments about what to believe or what to do. People may be positively,
ambivalently, or negatively disposed on each of seven aspects of the overall disposition
toward critical thinking. The CCTDI also provides a total score which gives equal weight to
each of the seven: Truthseeking, Open-mindedness, Analyticity, Systematicity, Critical
Thinking, Self-Confidence, Inquisitiveness, Maturity of Judgment. Truth seeking is the habit
of always desiring the best possible understanding of any given situation; it is following
reasons and evidence where ever they may lead, even if they lead one to question cherished
beliefs. Truth-seekers ask hard, sometimes even frightening questions; they do not ignore
relevant details; they strive not to let bias or preconception color their search for knowledge
and truth. The opposite of Truthseeking is bias which ignores good reasons and relevant
evidence in order not to have to face difficult ideas. Open-mindedness is the tendency to
allow others to voice views with which one may not agree. Open-minded people act with
tolerance toward the opinions of others, knowing that often we all hold beliefs which make
sense only from our own perspectives. Open-mindedness, as used here, is important for
harmony in a pluralistic and complex society where people approach issues from different
religious, political, social, family, cultural, and personal backgrounds. The opposite of open-
mindedness is closed-mindedness and intolerance for the ideas of others. Analyticity is the
tendency to be alert to what happens next. This is the habit of striving to anticipate both the
good and the bad potential consequences or outcomes of situations, choices, proposals, and
plans. The opposite of analyticity is being heedless of consequences, not attending to what
happens next when one makes choices or accepts ideas uncritically. Systematicity is the
80 Einav Aizikovitsh-Udi

tendency or habit of striving to approach problems in a disciplined, orderly, and systematic

way. The habit of being disorganized is the opposite characteristic to systematicity. The
person who is strong in systematicity may or may not actually know or use a given strategy or
any particular pattern in problem solving, but they have the mental desire and tendency to
approach questions and issues in such an organized way. Critical Thinking Self-Confidence:
the tendency to trust the use of reason and reflective thinking to solve problems is reasoning
self-confidence. This habit can apply to individuals or to groups; as can the other
dispositional characteristics measured by the CCTDI. We as a family, team, office,
community, or society can have the habit of being trustful of reasoned judgment as the means
of solving our problems and reaching our goals. The opposite is the tendency to be mistrustful
of reason, to consistently devalue or be hostile to the use of careful reason and reflection as a
means to solving problems or discovering what to do or what to believe. Inquisitiveness is
intellectual curiosity. It is the tendency to want to know things, even if they are not
immediately or obviously useful at the moment. It is being curious and eager to acquire new
knowledge and to learn the explanations for things even when the applications of that new
learning is not immediately apparent. The opposite of inquisitiveness is indifference. Maturity
of Judgment: Cognitive maturity is the tendency to see problems as complex, rather than
black and white. It is the habit of making a judgment in a timely way, not prematurely, and
not with undue delay. It is the tendency of standing firm in one's judgment when there is
reason to do so, but changing one's mind when that is the appropriate thing to do. It is
prudence in making, suspending, or revising judgment. It is being aware that multiple
solutions may be acceptable while appreciating the need to reach closure in certain
circumstances even in the absence of complete knowledge. The opposite, cognitive
immaturity, is characterized by being imprudent, black-and-white thinking, failing to come to
a closure in a timely way, stubbornly refusing to change one's mind when reasons and
evidence indicate one is mistaken, or revising one's opinions without a substantial reason for
doing so. Ennis (1985, 1987, 1989) presents 14 dispositions, the first 13 of which are defined
as necessary for critical thinking, while the last one, “being sensitive,” is not exactly a basic
disposition yet nevertheless has to be present in the totality of dispositions. The dispositions
are: seeking for clarification of a thesis or question; searching for arguments; trying to be
well-informed; using reliable sources; taking into account the general situation; trying to stay
relevant to the central issue; consistently remembering what the original or basic issue is;
searching for alternatives; seriously considering different points of view; postponing
judgment; taking a stand; seeking a high degree of precision; dealing with the components of
the whole in an organized way; sensitivity.

Development and Learning of Critical Thinking

There is an ongoing discussion in the field of education regarding the ways in which
critical thinking skills can be developed. Some researchers believe that there is a need to plan
specific critical thinking courses. Others claim that developing these skills can be
accomplished in the framework of regular courses (Ennis, 1989; McPeck, 1981; Resnick,
1987; Weinberger,1992). There is a debate as to whether these skills are completely general
or specific to subject matter and concepts. Most agree that critical thinking has both general
and specific attributes. Feuerstein's study (2002) showed that after teachers were provided
 Developing Critical Thinking through Probability Models 81

with theoretical and pedagogical knowledge, they were able to foster critical thinking in their
students. Zohar and Tamir (1993) found that critical thinking does not develop on its own.
Based upon this conclusion and upon the small amount of existing research in the field of
critical thinking in mathematics, this study examines the affinity between education for
critical thinking and mathematical studies. Many researchers, beginning with the philosopher
Passmore (1980), hesitate regarding many questions related to critical thinking, such as, what
does ‘being critical’ mean? Is it possible to educate for critical thinking, and what does this
mean? How can we know that we have succeeded in this task? In his article “Teaching to Be
Critical,” Passmore discusses the meaning of education for critical thinking and raises the
question, “Is being critical a matter of habitual behavior acquired through experience, that is,
a habit?”In addition, Passmore relates to the confusion between mere grumbling and critical
thinking. According to Passmore, it should be clear that a critical person is a person who has
imagination. In the same way as imagination should be distinguished from delusion, being
critical has to be distinguished from a grumbling expression of discontent, or from mere
slander. They are as easy to confuse as imagination and delusion. According to Schafersman
(1991), critical thinking is a learned ability that should not be left to develop of its own
accord, nor should it be taught by an untrained teacher. Both training and knowledge are
necessary to promote critical thinking abilities in students. Moreover, Schafersman suggests
that because society does not welcome people who challenge authority, critical thinking is not
often encouraged. Thus, in his opinion, "most people do not think critically." Resnick (1987)
corroborates Schafersman’s point of view, arguing that despite the fact that developing
critical thinking has been one of the most important goals of education for centuries,
problems that demand critical thinking are often dealt with ineffectively. Expertise in any
field can only be achieved with critical thinking (Wagner 1997), and it is therefore necessary
to help students understand how valuable it is and how they can achieve it. In Zohar and
Tamir's (1993) study as well, the researchers concluded that critical thinking does not develop
on its own and efforts are required in order to develop it. As Barak, Ben-Chaim and Zoller
(2002, 2007) summarize, previous research has shown a need for improving critical thinking
skills among students, since most students do not use sophisticated thinking even at the higher
education level. In general, there is a consensus that the ability to think critically is becoming
increasingly important for being successful in contemporary life, because of the ever-
increasing pace of changes and the complexity and interconnectedness of various phenomena
we encounter. People today are not expected to ‘know their place’, but rather to establish and
reinvent their position in the world. As the world is advancing, more and more people are
required to make rational decisions based on evaluative/critical thinking, instead of accepting
others’ authority. Thus, students need to be ready to examine truth values, to raise doubts, to
investigate situations and to search for alternatives in the context of school and everyday life.
In accordance with the above, De Bono (1976) had proposed a long time ago that it is hard to
teach thinking skills by means of a formal logical process, using principles and axioms. He
developed a number of approaches to teaching thinking, and showed that students who
received lessons in thinking produced a greater number of solutions for problems, compared
to students who did not receive such lessons. Our research is based on three key elements: a
critical thinking taxonomy that includes skills and dispositions (Ennis, 1987); the learning
unit "Probability in Daily Life" (Lieberman & Tversky, 1996,2001); and the infusion
approach of integrating subject matter with thinking skills (Swartz, 1992). Ennis claims that
critical thinking is a reflective and practical activity aiming for a moderate action or belief.
82 Einav Aizikovitsh-Udi

There are five key concepts and characteristics defining critical thinking: practical, reflective,
moderate, belief and action. In accordance with the categories this definition employs, Ennis
developed a taxonomy of critical thinking skills that include an intellectual as well as a
behavioral aspect. In addition to skills, Ennis’ taxonomy also includes dispositions and
abilities. In this study, we focus on students’ abilities rather than their dispositions. We have
chosen to use Ennis’ definition and taxonomy of critical thinking because it distinguishes
between abilities and disposition, and because teaching thinking skills according to a
taxonomy suits the hierarchical structure of our learning unit in probability studies.

The Learning Unit "Probability in Daily Life" (Lieberman & Tversky, 2001)

This unit in probability studies is part of the formal high school curriculum of the Israeli
Ministry of Education. It was chosen because its rationale is to make the students to "study
issues relevant to everyday life, which include elements of critical thinking” (Lieberman &
Tversky 2001, Introduction p.3). In this unit, students must analyze problems using statistical
instruments, as well as raising questions and thinking critically about the data, its collection,
and its results. Students learn to examine data qualitatively as well as quantitatively. They
must also use their intuitions to estimate probabilities and examine the logical premises of
these intuitions, along with misjudgments of their application. The unit is unique because it
explores probability in relation to everyday problems. This involves critical thinking elements
such as tangible examples from everyday life, confronting credible information, accepting
and dismissing generalizations, rechecking data, doubting, comparing new knowledge with
the existing knowledge. This unit is characterized by questions such as “Define the term
‘critical thinking’,” “Give examples of a problem while using a controlled experiment,” “Give
examples of failures and misleading commercials,” and “Give examples of a scientific truth
that was dismissed.” While studying the subject, the connection is checked between statistical
judgment and intuitive judgment, and intuitive mechanisms that produce wrong judgments
are explored. While studying the subject, students are expected to acquire the tools for critical
thinking. In the beginning, students learn the mathematical tools necessary for performing
calculations, and later on they use the probability part: causal connection, and mechanisms of
intuitive judgment, which are considered more of a psychological projection (Gilovich,
Griffin & Kahneman, 2002; Kahneman et. al, 1996).

The Infusion Approach (Swartz, 1992)

In light of the evidence that has accumulated in the field of teaching thinking, the
question arises whether thinking skills are general or content-dependent (Perkins & Salomon,
1988,1989; Perkins, 1992). Out of this question there developed four major approaches: the
general approach, the infusion approach, the immersion approach, and the mixed approach.
The general approach teaches thinking skills as a range of general skills detached from other
study subjects, as a separate course in the curriculum. In the infusion approach the skills are
taught in the framework of a specific study subject, and thinking turns into an integral part of
teaching specific materials, while general principles and terminology of thinking are
explicitly emphasized. In the immersion approach, the study material is taught in a thought-
 Developing Critical Thinking through Probability Models 83

provoking way and the students are “immersed” in the topic of study, without explicit
reference to the principles of thinking. The mixed approach combines the general and the
infusion approach. The present research employs the infusion approach, where thinking is
taught and learned in the context of the learning unit “Probability in Daily Life.” It is
important to elaborate here on the distinctions between the general and the infusion
approach. The field of education has recognized for decades the need to concentrate on the
promotion of critical thinking skills. The question is how this can be best accomplished.
Some educators feel that the best path is to design specific courses aimed at teaching critical
thinking, which is called the general skills approach. Integrating the teaching of these skills in
regular courses in the curriculum is a different approach known as the infusion approach. The
question at the heart of the argument is, whether critical thinking skills are general or depend
on content and on the system of concepts specific to that particular content. According to
Swartz and Parks (1994), the infusion approach aims at teaching specific critical thinking
skills along with different study subjects, and instilling critical thinking skills through
teaching the set learning material. According to this approach, such lessons are expected to
improve the students’ thinking and help them to learn the contents in different study subjects.
Swartz also emphasizes that the students should not only employ critical thinking skills in
class, but also be able to activate them in real-life situations and to recognize situations when
these skills should be used. For this, an appropriate motivation should be fostered; otherwise
these skills will remain passive. In this study, conducted according to the latter approach, we
have combined the mathematical content of the "Probability in Daily Life” learning unit with
critical thinking skills according to Ennis' taxonomy, restructured the curriculum, tested
different learning units and evaluated the participants’ critical thinking skills, to examine
whether the learning unit “Probability in Daily Life,” by using the infusion approach, does
indeed develop critical thinking.

Studies Dealing with Critical Thinking in Mathematics

An extensive literature review conducted in this research has shown that a number of
works have been published on the topic of “critical thinking in mathematics,” yet very few of
them proceed from the same context or ‘spirit’ as the present study, namely, that of seeking
for a general definition of “critical thinking” and giving this definition a scientific grounding
(Akbari-Zarin & Gray,1990; Avital & Barbeau, 1991; Battista et al., 1989; Becker, 1984;
Boucher, 1998; Cherkas, 1992; Coon & Birken, 1988; Dion, 1990; Dubinsky, 1989, 1986;
Fridlander, 1997; Garofalo, 1986, 1987; Gray & St. Ours, 1992; Innabi & Sheikh, 2007;
Johnson, 1994; Kaplan, 1992; Kaur & Oon, 1992; Kloosterman & Stage, 1992; LeGere,
1991; McCoy, 1990; Movshovitz-Hadar, 1993; Olson .& Olson, 1997; Lawrenz & Orton,
1989). As pointed out before in the “Theoretical Background” section, critical thinking has
been defined in many different ways, on the basis of various theories. In science, and in
particular in mathematics, none of the classical definitions cited in the “Theoretical
Background” section have been presented. Researchers who do relate to critical thinking in
the field of mathematics use this term in other contexts, and in fact deal with imparting
technical tools such as performing an assessment, checking the correctness of results,
evaluating a certain exercise, comparison, inference, application and interpretation, solution
strategies, etc. Reviewing these articles, we have searched for the term “critical thinking”
84 Einav Aizikovitsh-Udi

used in the sense relevant to the present study. Strategies for critical thinking in learning:
define your purpose, what it is you want to study; clarify questions and answers with your
teachers or other specialists in the subject. The purposes of study can be formulated in simple
phrases: “Plumbing regulations in suburban neighborhoods,” “Structure and terms in the
human skeleton.” Think about what is already known to you on the subject: what do you
already know that may help you in your study? What are your preconceptions on the subject?
What means do you have for carrying out the study, and what is your timetable? Gather
information; keep your thinking open so as not to exclude opportunities, Ask questions; what
are the preconceptions of the sources’ authors? Organize the information you have collected
into structures that make sense to you and ask questions again and again. Within the
framework of mathematics studies, critical thinking does not develop spontaneously but
requires an effort. Critical thinking skills rely on self-regulation of the thinking processes,
construction of meaning, and detection of patterns in supposedly disorganized structures. A
considerable mental work is involved in the processes and judgments it requires. Critical
thinking is not algorithmic, i.e. its patterns of thinking and action are not clear or predefined.
Critical thinking tends to be complex. It often terminates in multiple solutions that have
advantages and disadvantages, rather than a single clear solution. It requires the use of
multiple, sometimes mutually contradictory criteria, and frequently concludes with
uncertainty. The latter conclusion corresponds with Zohar's research (1996, p.21). (i)
Conventional teaching is not appropriate for the changing and challenging world we live in,
which demands critical/evaluative thinking based on rational decisions and dispositions. In
this research we find that combining different instruction strategies (such as asking questions,
independent investigation of phenomena, or experimenting in the framework of open
discussion and drawing conclusions considerably improves the students' critical thinking
abilities and dispositions. These findings correspond with those of earlier researches (Facione,
2002) showing that critical thinking relies on cognitive activity directed at focused,
inquisitive interpretation of relevant information, and constant reference to the student's
dispositions. (ii) (Partial) transfer between disciplines is possible. One of the main goals of
teaching higher-order thinking skills, such as critical thinking, is the transfer of these skills to
all disciplines and fields. However, transfer within and between disciplines is difficult to put
into practice (Bransford et al., 1999). In this research the instruction of higher-order thinking
skills was used within the framework of mathematics studies, but the students' success in
critical thinking tests indicates their ability to transfer their critical thinking skills to other
fields, since these tests are based on generic questions that are not confined to specific
disciplines. I will elaborate on this conclusion further in the “Research Limitations” section.

RECOMMENDATIONS
We live in a period of non-stop dynamic changes in all the areas of life. The amount of
knowledge accumulated by the different research disciplines is immense and ever-growing,
which makes it impossible to endow students with all the information they may need in the
future. Thus, the education system needs to adapt itself to the world of tomorrow. Along with
imparting basic knowledge, education needs to impart skills needed for independent
confrontation with new information and with the challenges of the 21st century.We believe
 Developing Critical Thinking through Probability Models 85

that a graduate of the education system should be capable of critical thinking (which is a
central empowering mental tool necessary to the citizen of the modern democratic society as a
learner, consumer, professional, and more) and adopt critical thinking as a way of life. The
term "critical thinking" refers to the individual's ability to adequately evaluate claims by
means of logical-analytic skills, to use reflective thinking that raises question questions
regarding inclinations, beliefs, perceptions and ways of action (Facion, 2002; Ennis, 1989).
These research findings have major educational implications concerning the training of
teachers for taking part in programs designed to promote critical thinking.
The empirical results convincingly show that conscious, consistent instruction of critical
thinking in mathematics increases the students' chances of success. This conclusion is
extremely important for the process of changing teachers' beliefs and instruction strategies in
the discipline. We propose that the programs of professional development be designed in such
a way as to help teachers better understand what is higher-order thinking and have a more
coherent sense of what is critical thinking. We also propose to encourage teachers to employ a
wider range of teaching strategies, as presented in this and other researches, in order to help
their students fulfill tasks that require higher-order thinking in general and critical thinking in
particular.

RESEARCH UNIQUENESS AND CONTRIBUTION

This research was designed as a continuation of a large-scale pilot study, conducted by
the Ben-Gurion University of the Negev and a general high school (located at the center of
Israel) in 2007. The purpose of the pilot study was to examine the students' critical thinking
abilities in different environments drawing on infusion-approach study of "Probability in
Daily Life". (i)The present research establishes points of reference to critical thinking
dispositions among students learning mathematics in different environments (high school and
a mathematics club). This element has not been examined so far by the literature in the field
(ii) This research has identified and measured differences between dispositions, abilities, and
construction of skills characteristic of critical thinking in mathematics, and completes other
researchers conducted in other environments (iii)The combination of the Cornell test and the
CCTDI test in the evaluation of critical thinking abilities and dispositions is unique to this
research; it has not been performed in previous studies. Educators in Israel, who wonder, like
their colleagues in the West, about the goals of the education system that could guide the
different educational frameworks, may find in this research an idea that can unify different
topics and study programs, in order to prepare the learners for life in a changing society, and
develop their ability to think in a systematic and independent way. In much of the literature,
critical thinking development is referred to as an important goal of the educational system.
This research may contribute to the public discourse of the mathematical education
community on this issue. It also raises the public awareness of the need to develop critical
thinking in the framework of mathematical education, which may enable future examination
and promotion of critical thinking development through mathematics teaching in a fuller and
more informed way. To conclude, the main contribution of this research lies in revealing the
connection between critical thinking and the teaching of mathematics. Despite the problem of
transfer discussed earlier, the scientific contribution of this research lies in the new insights it
86 Einav Aizikovitsh-Udi

provides into critical thinking, its place and importance in teaching mathematics. In this
manner, it will be possible to strengthen the status of the study of mathematics in imparting
higher-order thinking skills, both in parallel with and beyond the formal education program.

Implications for the Formal School Curriculum

Current mathematics teaching approaches espouse the conceptual understanding of

mathematics and stress the significance of problem solution, mathematical literacy and
mathematical discourse. According to this approach, teachers function as mediators between
the students and the information they need to acquire by asking questions, posing challenges,
and research. Thus teachers help students better comprehend mathematical terminology, ideas
and associations. This method of teaching is extremely challenging for both students and
teachers: it necessitates the teacher's profound understanding of mathematics, intellectual
effort and creativity, and the student's confrontation with unfamiliar situations and contents.
The implications of this research for the education curriculum were designed on the basis of
previous studies12. Feuerstein (2002), Zohar and Tamir (1993), as well as Weinberger (1998)
point out the importance for the students to experience learning that develops critical thinking
by means of diverse study programs with special characteristics. The findings of the present
research may assist in developing curricula and instruction methods for different ages and
learning levels in mathematics, on the basis of the connection between critical thinking and
the study of mathematics through the learning unit "Probability in Daily Life." In light of the
above, the implications of this research for the formal school curriculum is in the opportunity
it provides to expand the implementation of programs for critical thinking development and
their infusion into mathematics curricula, according to the requirements specified by the
formal education program 13.

CONCLUDING REMARKS
From this research’s findings and discussion, there arise the following research
recommendations: A more comprehensive examination of the processes of critical thinking: to
what extent could the students describe, orally and in writing, the processes of thinking,
activate them and apply the thinking skills they studied on the procedural and meta-cognitive
level? Did they make an informed use of terms and strategies of higher-order thinking,
including critical thinking? On the basis of the former, it should be examined what use the
research participants make of the “language of thinking,” or, in the words of Costa and
Marzano, “do they speak thought?” (Costa & Marzano, in Harpaz, 1997; Costa, 1991).
Developing such a language involves, on the part of the teacher, such skills as using precise
vocabulary, presenting critical questions, presenting data rather than answers, aspiring for
exactness, giving directions, and developing meta-cognition. Examination of the attitudes and

12
See appendix for the studies on which this research implications are based
13
I.e., "the student knows how to draw conclusions from mathematical models," "the student will develop logical
mathematical thinking skills, such as drawing conclusions, making generalizations, analysis, making and
supporting assumptions, self-criticism.”
 Developing Critical Thinking through Probability Models 87

perceptions of education students in colleges for teacher training, practicing teachers and
researchers of mathematical education with regard to teaching that develops critical thinking
in mathematics; evaluation of these students’ and professionals’ critical thinking functions in
teaching, learning, and research. Teaching “Probability in Daily Life” and conducting the
same research among all the strata of the students’ population and not only among those who
study mathematics at the higher level. Examining the gender, age, and ethnicity aspects of
critical thinking development.
There is neither consensus nor coherence in contemporary approaches to education for
critical thinking. The research reported in this thesis has demonstrated the viability of
integrating the purposeful promotion of critical thinking with the teaching of conventional
mathematics content. It is hoped that the findings of this study will contribute to our
understanding of the nature of critical thinking and to the further development of instructional
approaches relevant to its promotion.

REFERENCES
Aizikovitsh, E., & Amit, M. (2008). Developing critical thinking in probability lesson. In O.
Figueras, J. L. Cortina, S. Alatorre, T. Rojano, & A. Sepúlveda (Eds.), Proceedings of the
Conference of the International Group for the Psychology of Mathematics Education
(Vol.2, pp. 9-13). Morelia, México: Universidad Michoacana de San Nicolás de Hidalgo,
Centro de Investigación y de Estudios Avanzados del IPN.
Aizikovitsh, E., & Amit, M. (2009). Promoting critical thinking abilities via probability
instruction. In M. Tzekaki, M. Kaldrimidou, & H. Sakonidis (Eds.), Proceedings of the
Conference of the International Group for the Psychology of Mathematics Education
(Vol. 2, pp.17-24). Thessaloniki, Greece: Aristotle University.
Aizikovitsh, E., & Amit, M. (2009). An innovative model for developing critical thinking
skills through mathematical education. In L. Paditz & A. Rogerson (Eds.), Proceedings of
the International Conference of the Mathematics Education into the 21st Century
Project: Models in developing mathematics education (pp. 19-22). Dresden, Germany:
University of Applied Sciences.
Aizikovitsh, E., & Amit, M. (2010). Promoting the language of critical thinking through
probability. In V. Durand-Guerrier, S. Soury-Lavergne, & F. Arzarello (Eds.),
Proceedings of the Congress of the European Society for Research in Mathematics
Education. Lyon, France : Institut National de Recherche Pédagogique. Retrieved from
http://www.inrp.fr/publications/edition-electronique/cerme6/wg6-01-aizikovitsh.pdf
Aizikovitsh, E., & Amit, M. (2010). Evaluating an infusion approach to the teaching of
critical thinking skills through mathematics. Procedia – Social and Behavioral Sciences,
2, 3818-3822. doi:10.1016/j.sbspro.2010.03.596
Akbari-Zarin, M., & Gray, M. W. (1990). Computer assisted instruction and critical thinking.
Journal of Computers in Mathematics and Science Teaching, 9, 71-78.
Avital, S., & Barbeau, E. J. (1991). Intuitively misconceived solutions to problems. For the
Learning of Mathematics, 11(3), 2-8.
Ball, D. L. (2000). Working on the inside: Using one's own practice as a site for studying
mathematics teaching and learning. In A. Kelly & R. Lesh (Eds.), Handbook of research
88 Einav Aizikovitsh-Udi

design in mathematics and science education (pp. 365- 402). Dordrecht, The
Netherlands: Kluwer.
Barnett, C. (1991). Building a case-based curriculum to enhance the pedagogical content
knowledge of mathematics teachers. Journal of Teacher Education, 42, 263-272.
Battista, M. T., Wheatley, G. H., & Talsma, G. (1989). Spatial visualization, formal
reasoning, and geometric problem-solving strategies of pre-service elementary teachers.
Focus on Learning Problems in Mathematics, 11(4), 17-30.
Becker, L. C. (1984). Setting the stage with APR problems. Mathematics and Computer
Education, 18(3), 89-95.
Ben-Chaim, D., Ron, S., & Zoller, U. (2000). The disposition of eleventh-grade science
students toward critical thinking. Journal of Science Education and Technology, 9, 149–
159.
Bereiter, C., & Scardamalia, M.(1993). Surpassing ourselves: An inquiry into the nature and
implication of expertise. Chicago: Open Court.
Beyer, B. (1987). Practical strategies for the teaching of thinking. Boston: Allyn & Bacon.
Birenbaum, M. (1993). Who is afraid of a research project? Tel-Aviv: University Publishing
Projects. (In Hebrew)
Boddy, N., Watson, K., & Aubusson, P. (2003). A trial of the five Es: A referent model for
constructivist teaching and learning. Research in Science Education, 33, 27–42.
Boix-Mansilla, V., & Gardner, H. (1998). What are the qualities of understanding? In M.
Wiske (Ed.), Teaching for understanding (pp. 161-196). San Francisco: Jossey-Bass.
Boucher, A. (1998). Critical thinking through estimation. Teaching Children Mathematics, 4,
452-455.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain,
mind, experience, and school. Washington, DC: National Academy Press.
Brown, A. (1987). Metacognition, executive, control, self resolution and other more
mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition,
motivation, and understanding. Hillsdale, NJ: Erlbaum.
Brown, A. L., & Campione, J. C. (1990). Communities of learning and thinking, or a context
by any other name. In D. Kuhn (Ed.), Developmental perspectives on teaching and
learning thinking skills (pp. 108-126). Basel, Switzerland: Karger.
Brown, A. L. (1997). Transforming schools into communities of thinking and learning about
serious matters. American Psychologist, 52, 399-413.
Bryan, M. (1987). The great philosophers. New-York: BBC Books.
Carpenter, T. P., Fennema E., & Franke, M. L. (1996). Cognitively guided instruction: A
knowledge base for reform in primary mathematics instruction. The Elementary School
Journal, 97(1), 3-20.
Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C., & Loef, M. (1989). Using
knowledge of children's mathematics thinking in classroom teaching: An experimental
study. American Educational Research Journal, 26, 499–531.
Coon, A. C., & Birken, M. (1988). The common denominators: A collaborative approach to
teaching reasoning skills through literature and mathematics. Innovative Higher
Education, 12, 91-100.
Costa, A. L. (1991). The school as a home for the mind: A collection of articles. Palatine, IL:
SkyLight.
 Developing Critical Thinking through Probability Models 89

Costa, A. L., & Marzano, R. (1991). Teaching the language of thinking. In A. L. Costa (Ed.),
Developing minds: A resource book for teaching thinking (pp. 251-254). Alexandria,
VA: Association for Supervision and Curriculum Development.
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods
approaches (2nd ed.). Thousand Oaks, CA: Sage.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods
approaches (3rd ed.). Thousand Oaks, CA: Sage.
De Bono, E. (1970). Lateral thinking: Creativity step by step. New York: Harper & Row.
De Bono, E. (1976). Teaching thinking. London: Penguin.
De Bono, E. (1990). Lateral thinking: A textbook of creativity. London: Ward Lock
Educational.
De Bono, E. (1993). Teach your child how to think. Jerusalem: Branco Weiss Institute for
Fostering Critical Thinking. (In Hebrew)
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the
educative process. Boston: Heath.
Dion, G. (1990). The graphics calculator: A tool for critical thinking. Mathematics Teacher,
83, 564-571.
Dirkes, M. A. (1979). Many solutions to get one solution. School Science and Mathematics,
79, 346-348.
Duckworth, E. (1996). The having of wonderful ideas and other essays on teaching and
learning. New York: Teachers College Press.
Egan, K. (1997). The educated mind: How cognitive tools shape our understanding. Chicago:
The University of Chicago Press.
Ennis, R. H. (1962). A concept of critical thinking. Harvard Educational Review, 32, 81-111.
Ennis, R. H. (1963). Needed: Research in critical thinking. Educational Leadership, 21(1),
17-20, 39.
Ennis, R. H. (1985). The logical basis for measuring CT skills. Educational Leadership,
43(2), 44–48.
Ennis, R. H. (1987, Summer). A conception of critical thinking – with some curriculum
suggestions. American Philosophical Association Newsletter on Teaching Philosophy, 1-
5.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron
& R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). New
York: Freeman.
Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed
research. Educational Researcher, 18(3), 4-10.
Ennis, R. H. (1991). Critical thinking: A streamlined conception. Teaching Philosophy, 14(1),
5-25.
Ennis, R. H. (1996). Critical thinking dispositions: Their nature and assessability. Informal
Logic, 18, 165-182.
Ennis, R. H. (1996). Critical thinking. Upper Saddle River, NJ: Prentice-Hall.
Ennis, R. H. (2002). Goals for a critical thinking curriculum and its assessment. In A. L.
Costa (Ed.), Developing minds (3rd ed., pp. 44-46). Alexandria, VA: Association for
Supervision and Curriculum Development.
Ennis, R. H., & Millman, J. (2005). Cornell critical thinking test, level X (5th ed.). Seaside,
CA: The Critical Thinking Company.
90 Einav Aizikovitsh-Udi

Ennis, R. H., & Millman, J. (2005). Cornell critical thinking test, level Z (5th ed.). Seaside,
CA: The Critical Thinking Company.
Ennis, R. H., Millman, J., & Tomko, T. N. (1985). Cornell critical thinking test, level X and
level Z. (3rd ed.). Pacific Grove, CA: Midwest.
Ennis, R. H., Millman, J., & Tomko, T. N. (2005). Cornell critical thinking tests:
Administration manual (5th ed.). Seaside, CA: The Critical Thinking Company.
Ennis, R. H., & Weir, E. (1985). The Ennis-Weir critical thinking essay test. Pacific Grove,
CA: Midwest.
Esterle, J., & Clurman, D. (1993). Conversations with critical thinkers. San Francisco, CA:
Whitman Institute.
Facione, P. A., & Facione, N. C. (1992). The California critical thinking disposition
inventory. Millbrae, CA: California Academic Press.
Facione, P. A., & Facione N. C. (1994). The California Critical Thinking Skills Test: Manual.
Millbrae, CA: California Academic Press.
Facione, P. A., Facione, N. C., & Giancarlo, C. A. (1996). The California critical disposition
inventory: Test manual. Millbrae, CA: California Academic Press.
Facione, P. A., Giancarlo, C. A., Facione, N. C., & Gainen, J. (2000). The disposition toward
critical thinking: Its character, measurement, and relationship to critical thinking skills.
Informal Logic, 20, 61-84.
Feldman, A., & Minstrel, J. (2000). Action research as a research methodology for study of
teaching and learning science. In A. E. Kelly & R. A. Lesh (Eds.), Handbook of research
design in mathematics and science education (pp. 429-455). Mahwah, NJ: Erlbaum.
Feldman, A. (1996). Enhancing the practice of physics teachers: Mechanisms for the
generation and sharing of knowledge and understanding in collaborative action research.
Journal of Research in Science Teaching, 33, 513-540.
Fenstermacher, G., & Soltis, J. (1986). Approaches to teaching. New York: Teachers College
Press.
Feuerstein, M. (2002). Media literacy in support of critical thinking. Unpublished doctoral
dissertation, University of Liverpool, UK.
Fisher, R. (1990). Teaching children to think. Oxford, UK: Blackwell.
Gardner, H. (1991). The unschooled mind: How children think and how schools should teach.
New York: Basic Books.
Garofalo, J. (1986). Metacognitive knowledge and metacognitive process: Important
influences on mathematical performance. Research and Teaching in Developmental
Education, 2(2), 34-39.
Garofalo, J. (1987). Why mathematical problem solving is such a problem: Some general
considerations. Research and Teaching in Developmental Education, 3(1), 8-13.
Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of
intuitive judgment. Cambridge, UK: Cambridge University Press.
Glatthorn, A., & Baron, J., (1991). The good thinker. In A. Costa (Ed.), Developing minds: A
resource book for teaching thinking (pp. 63-67). Alexandria, VA: Association for
Supervision and Curriculum Development.
Goldstein, W., & Hogarth, R. M. (Eds.). (1997). Research on judgment and decision making.
Cambridge, UK: Cambridge University Press.
Gray, S. S., & St. Ours, P. A. (1992). A model for discovering critical thinking skills in
mathematics. In W. Oxman-Michelli & M. Weinstein (Eds.), Critical thinking: Focus on
 Developing Critical Thinking through Probability Models 91

science and technology (Vol. 2, pp. 440-448). Upper Montclair, NJ: Institute for Critical
Thinking.
Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-
student survey of mechanics test data for introductory physics courses. American Journal
of Physics, 66, 64–74.
Halpern, D. F. (1996). Thought and knowledge: Introduction to critical thinking. Mahwah,
NJ: Erlbaum.
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains. American
Psychologist, 53, 449–455.
Harpaz, Y. (Ed.). (1996). Education for critical thinking. Jerusalem: The Ministry of
Education and Culture. (In Hebrew).
Harpaz, Y. (1997). Teaching and learning in thinking communities. Jerusalem: Branco Weiss
Institute for Fostering Critical Thinking. (In Hebrew).
Harpaz, Y., & Adam L. (2000). Communities of thinking. Educational Leadership, 58(3), 54-
57.
Innabi, H., & Sheikh, O. (2007). The Change in Mathematics Teachers' Perceptions of
Critical Thinking after 15 Years of Educational Reform in Jordan. Educational Studies in
Mathematics, 64(1), 45-68. doi: 10.1007/s10649-005-9017-x
Janis, I. L., & Mann, L. (1977). Decision-making. New York: Free Press.
Johnson, R. H., & Blair, A. J. (Eds.). (1994). New essays in informal logic. Windsor, Ontario,
Canada: Informal Logic.
Johnson, R. H., & Blair, A. J. (1994). Logical self-defense. New York: McGraw-Hill.
Johnston, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm
whose time has come. Educational Researcher, 33(7), 14–26.
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1991). The endowment effect, loss aversion,
and status quo bias. Journal of Economic Perspectives, 5, 193-206.
Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and
biases. Cambridge, UK: Cambridge University Press.
Kaplan, G. (1992). Critical thinking and mathematics: Cementing the bond. In W. Oxman-
Michelli & M. Weinstein (Eds.), Critical thinking: Focus on science and technology.
(Vol. 2, pp. 434-439). Upper Montclair, NJ: Institute for Critical Thinking.
Kaspi, M. (1978). Who is thinking? Givatayim: Massada. (In Hebrew)
Kaur, B., & Oon, K. K. (1992). Heuristic? What's that? Australian Mathematics Teacher,
48(2), 16-17.
Kloosterman, P., & Stage, F. K. (1992). Measuring beliefs about mathematical problem
solving. School Science and Mathematics, 92, 109-115.
Kohn, A. (1999). The schools our children deserve: Moving beyond traditional classrooms
and “Tougher Standards”. Boston: Houghton Mifflin.
Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28(1),
16–26.
Lakoff, G., & Mark, J. (1980). Metaphors we live by. Chicago: The University of Chicago
Press.
Lamon, S. (1993). Ratio and proportion: Connecting content and children's thinking. Journal
for Research in Mathematics Education, 24, 41-61.
92 Einav Aizikovitsh-Udi

Lawrenz, F., & Orton, R. E. (1989). A comparison of critical thinking related teaching
practices of seventh and eighth grade science and mathematics teachers. School Science
and Mathematics, 89, 361-372.
LeGere, A., (1991). Collaboration and writing in the mathematics classroom. Mathematics
Teacher, 84, 166-171.
Leou, M., Abder, P., Riordan, M., & Zoller, U. (2006). Using ‘HOCS-centered learning’ as a
pathway to promote science teachers’ metacognitive development. Research in Science
Education, 36, 69–84.
Lieberman, V., & Tversky, A. (1996). Critical thinking: Statistical considerations and
intuitive judgments. Tel Aviv: Open University. (In Hebrew)
Lieberman, V., & Tversky, A. (2001). Probability thinking in daily life. Tel Aviv: Open
University. (In Hebrew)
Linn, M. C. (2000). Designing the knowledge integration environment. International Journal
of Science Education, 22, 781–796.
Lipman, M. (1991). Thinking in education. New York: Cambridge University Press.
McPeck, J. (1994). Critical thinking and the `Trivial Pursuit`: Theory of knowledge. In K.
Walters (Ed.), Re-thinking reason: New perspectives in critical thinking (pp. 101-117).
Albany, NY: State University of New York Press.
McPeck, J. E. (1981). Critica1 thinking and education. New York: St. Martin's Press.
Movshovitz-Hadar, N. (1993). The false coin problem, mathematical induction and
knowledge fragility. Journal of Mathematical Behavior, 12, 253-268.
National Academy of Science. (1995). National Science Education Standards. Retrieved
from: http://books.nap.edu/html/nses/html/index.html
National Research Council. (1996). National science education standards. Washington, DC:
National Academy Press.
Nevo, D. (2005). What is the contribution of art studies in the education system? Retrieved from
http://cms.education.gov.il/EducationCMS/Units/Mazkirut_Pedagogit/Omanut/Maagarey
Meida/Maamarim (In Hebrew)
Nickerson, R. S., Perkins, D. N., &. Smith, E. E. (1985). The teaching of thinking. Hillsdale,
NJ: Erlbaum.
Nisbett, R. E., Fong, G. T., Lehman, D. R., & Cheng, P. W. (1987). Teaching reasoning.
Science, 238, 625-631. doi: 10.1126/science.3672116
Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in
school science. Journal of Research in Science Teaching, 41, 994–1020.
Palinscar, A. S. (1993). Pursuing scientific literacy in the middle grades through collaborative
problem solving. The Elementary School Journal, 93, 643-658.
Passmore, J. (1980). The philosophy of teaching. London: Duckworth.
Paul, R. (1993). Critical thinking: How to prepare students for a rapidly changing world.
Santa Rosa, CA: Sonoma State University, Foundation for Critical Thinking.
Paul, R., Binker, A. J. A., Martin, D., Vetrano, C., & Kreklau, H. (1995). Critical thinking
handbook: 6th-9th grades: A guide for remodelling lesson plans in language arts, social
studies & science. Rohnert Park, CA: Sonoma State University, Centre for Critical
Thinking and Moral Critique.
Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context-bound? Educational
Researcher, 18(1), 16-25.
 Developing Critical Thinking through Probability Models 93

Perkins, D. N. (1992). Smart schools: From training memories to training minds. New York:
Free Press.
Perkins, D. N. (1995). Outsmarting IQ: The emerging science of learnable intelligence. New
York: Free Press.
Perkins, D. N., & Salomon, G. (1988). Teaching for transfer. Educational Leadership, 46(1), 22-
32.
Phillips, C. R., Chesnut, R. J., & Rospond, R. M. (2004). The California critical thinking
instruments for benchmarking, program assessment, and directing curricular change.
American Journal of Pharmaceutical Education, 68(4), article 101. Retrieved from:
http://www.ajpe.org/aj6804/aj6804101/aj6804101.pdf
Piaget, J., & Inhelder, B. (1978). The psychology of the child. New York: Basic Books.
Postman, N. (1979). Teaching as a conserving activity. New York: Delacorte Press.
Quinlan, C. (1989). Inferential thinking for mathematics teachers. Australian Mathematics
Teacher, 45(4), 20-22.
Regev, Y. (1997). Socrates and the internet. Computers in Education, 43, 1-13.
Reid, S. (2002). How to think: Building your mental muscle. London: Prentice Hall.
Resnick, L. B. (1987). Education and learning to think. Washington DC: National Academy
Press.
Ritchhart, R. (2002). Intellectual character: What it is, why it matters, and how to get it. San
Francisco: Jossey-Bass.
Sabar Ben-Yehoshua, N. (2000). Qualitative research. Givatayim: Massada. (In Hebrew)
Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanisms of a
neglected phenomenon. Educational Psychologist, 24, 113-142.
Schank, R. (2001). Coloring outside the lines. New York: HarperCollins.
Scheffler, I. (1989). Reason and teaching. London: Routledge & Kegan Paul.
Shannon, K. M. (1992). Using writing to encourage students to think critically about
mathematics and science. In W. Oxman-Michelli & M. Weinstein (Eds.), Critical
thinking: Focus on science and technology (Vol. 2, pp. 395-407). Upper Montclair, NJ:
Institute for Critical Thinking.
Schraw, G., Crippen, K. J., & Hartley, K. D. (2006). Promoting self-regulation in science
education: Metacognition as part of a broader perspective on learning. Research in
Science Education, 36, 111–139.
Shulman, L. S. (1987). Knowledge and teaching: Foundation of the new reform. Harvard
Educational Review, 57, 1-22.
Siegel, H. (1988). Educating reason: Rationality, critical thinking and education. New York:
Routledge.
Smith, F. (1990). To think. New York: Teachers College Press.
Sternberg, R. (1984). How can we teach intelligence? Educational Leadership, 39(1), 38-48.
Stevenson, F. W. (1992). Exploratory problems in mathematics. Reston, VA: National
Council of Teachers of Mathematics.
Suydam, M. N. (1989). Research on mathematics education reported in 1988. Journal for
Research in Mathematics Education, 20, 379-426.
Swartz, R. J. (1992). Critical thinking, the curriculum, and the problem of transfer. In D. N.
Perkins, J. Lochhead, & J. Bishop (Eds.), Thinking: The second international conference
(pp. 261-284). Hillsdale, NJ: Erlbaum.
94 Einav Aizikovitsh-Udi

Swartz, R. J., & Parks, S. (1994). Infusing the teaching of critical and creative thinking into
content instruction. Pacific Grove, CA: Critical Thinking & Software.
Tashakkori, A., & Teddlie, C. (2003). The past and future of mixed methods research: From
data triangulation to mixed model designs. In A. Tashakkori & C. Teddlie (Eds.),
Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA:
Sage.
Ten Dam, G., & Volman, M. (2004). Critical thinking as a citizenship competence: Teaching
strategies. Learning and Instruction, 14, 359–379.
Tishman, S. (2000). Why teach habits of mind? In A. L. Costa & B. Kallick (Eds.),
Discovering and exploring habits of mind (pp. 41-61). Alexandria, VA: Association for
Supervision and Curriculum Development.
Tobin, K., & Fraser, B. J. (1989). Barriers to higher level cognitive learning in high school
science. Science Education, 73, 659–682.
Tobin, K., Kahle, J., & Fraser, B. (1990). Windows into science classrooms: Problems
associated with higher-level cognitive learning. London: Falmer.
Tobin, K., Tippins, D. J., & Hook, K. S. (1994). Referents for changing a science curriculum:
A case study of one teacher’s change in beliefs. Science Education, 3, 245–264.
Villasenor, A., & Kepner, H. S. (1993). Arithmetic from a problem-solving perspective: An
urban implementation. Journal for Research in Mathematics Education, 24, 62–69.
Wade, C., & Tavris, C. (1993). Critical & creative thinking: The case of love and war. New
York: HarperCollins.
Watson, G., & Glaser, E. M. (1980). Watson-Glaser critical thinking appraisal manual
(Forms A & B). Cleveland, OH: Harcourt Brace Jovanovich.
Watts, M., Jofili, Z., & Bezerra, R. (1997). A case for critical constructivism and critical
thinking in science education. Research in Science Education, 27, 309–322.
Weinberger, Y. (1998). Incorporating the development of higher order thinking in preservice
preparation of science teachers. Unpublished doctoral dissertation, The Hebrew
University, Jerusalem. (In Hebrew)
Weinberger, Y. (1992). Fostering critical thinking in biology instruction. Unpublished
master’s thesis, Tel-Aviv University, Tel-Aviv, Israel. (In Hebrew)
Wiggins, G., & McTighe, J. (1998). Understanding by design. Alexandria, VA: Association
for Supervision and Curriculum Development.
Wilks, S. (1995). Critical and creative thinking: Strategies for classroom inquiry.
Portsmouth, NH: Heinemann.
Wiske, M. (Ed.). (1998). Teaching for understanding. San Francisco: Jossey-Bass.
Zohar, A. (1996). To learn, to think, and to learn to think. Jerusalem: Branco Weiss Institute
for Fostering Critical Thinking. (In Hebrew)
Zohar, A. (1999). Training teachers towards developing the student's thinking. Tel Aviv:
Mofet Institute. (In Hebrew)
Zohar, A. (2000). Meta-cognitive knowledge of teachers teaching higher-order thinking skills.
Dapim, 41-52. (In Hebrew)
Zohar, A., & Dori, Y. J. (2003). Higher order thinking skills and low achieving students: Are
they mutually exclusive? Journal of the Learning Sciences, 12,145-181.
Zohar, A., & Tamir, P. (1993). Incorporating critical thinking into a regular high school
biology curriculum. School Science and Mathematics, 93, 136-140.
 Developing Critical Thinking through Probability Models 95

Zoller, U. (1993). Are lecture and learning compatible? Maybe for LOCS: Unlikely for
HOCS. Journal of Chemical Education, 70, 195–197. doi: 10.1021/ed070p195
Zoller, U. (1999). Teaching tomorrow’s college science courses: Are we getting it right?
Journal of College Science Teaching, 29, 409–414.
Zoller, U. (2001). Alternative assessment as (critical) means of facilitating HOCS-promoting
teaching and learning in chemistry education. Chemical Education: Research and Practice
in Europe, 2(1), 9–17.
Retrieved from http://www.uoi.gr/cerp/2001_February/pdf/04Zoller.pdf
Zoller, U., Ben-Chaim, D., Ron, S., Pentimalli, R., & Borsese, A. (2000). The disposition
toward critical thinking of high school and university science students: An interintra
Israeli-Italian study. International Journal of Science Education, 22, 571–582. doi:
10.1080/095006900289679
Zoller, U., Dori, Y. J., & Lubezky, A. (2002). Algorithmic, LOCS and HOCS (chemistry)
exam questions: Performance and attitudes of college students. International Journal of
Science Education, 24, 185–203. doi: 10.1080/09500690110049060
In: Critical Thinking ISBN: 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 97-121 2011 Nova Science Publishers, Inc.

Chapter 4

THE PROMOTION OF CRITICAL THINKING SKILLS

THROUGH ARGUMENT MAPPING

Christopher Dwyer, Michael Hogan and Ian Stewart

School of Psychology,
National University of Ireland, Galway, Ireland

ABSTRACT
Argument mapping is a method of visually diagramming arguments using a 'box and
arrow' format with the aim of simplifying the reading of an argument structure and
facilitating the assimilation of core statements and relations. The current chapter presents
the findings of a controlled trial in which argument mapping training was compared with
hierarchical outline training as techniques for teaching critical thinking skills. Eighty-one
undergraduate psychology students were allocated to one of three groups: an argument
mapping group, an outlining group, or a control group and were tested on critical thinking
before and after an 8-week intervention period. Results revealed that students in the
argument mapping group scored higher than the control group at post-test on the critical
thinking skills of evaluation and inductive reasoning. Students in the outlining group
scored significantly higher than those in the control group on tests of analysis and
inductive reasoning. There were no significant performance differences at post-test
between those in the argument mapping group and those in the hierarchical summary
group. Results are discussed in light of research and theory on best practice in the
cultivation of critical thinking.

INTRODUCTION
Critical thinking is a metacognitive process which is made up of a collection of sub-skills
(i.e. analysis, evaluation, and inference) that, when used appropriately, increases the chances
of producing a logical solution to a problem or a valid conclusion to an argument. In
education reports around the world, the teaching of critical thinking skills has been identified
as an area of education to be developed and examined, specifically, in higher education
(Association of American Colleges & Universities, 2005; Australian Council for Educational
98 Christopher Dwyer, Michael Hogan and Ian Stewart

Research, 2002; Higher Education Quality Control, 1996), because it endows students with
the capability to reason not only academically, but also in social and interpersonal contexts
where adequate problem-solving and decision making are necessary on a daily basis (Ku,
2009).
Though the benefits of critical thinking are not always obvious to many students in third-
level education, it is well established that good critical thinking ability predicts both academic
and everyday functioning (Quitadamo & Kurtz, 2007). Good critical thinkers are more likely
to get better grades; are better equipped and more likely to use the skills of critical thinking
on an everyday basis (U.S. Department of Education, 1990); and are often more employable
as well (Holmes and Clizbe, 1997; National Academy of Sciences, 2005). The ability to think
critically allows those in the workforce to think independently, analyse data in order to make
inferences, communicate well and make sound decisions. In addition, critical thinking skills
are highly desired by employers for their workforce (El Hassan & Madhum, 2007) and are
also essential for good management (MacPherson, 1999).
However, teaching critical thinking (CT) skills to University students is a major
educational challenge (Kuhn, 1991; Willingham, 2007). There are many reasons for this,
including the broad challenges of embedding CT into an often crowded curriculum, and
designing an effective teaching strategy that targets specific CT skills and offers sufficient
practice so these skills develop in an orderly and cumulative way. Two related problems
discussed in more detail below include: difficulties in defining critical thinking and
constructing thinking frameworks and theories that inform the practice of teachers in the area;
and difficulties associated with the assimilation of text-based argument and the challenge of
teaching students transferable analysis, evaluation, and inference skills using text-based
teaching materials. We will consider both of these issues below and then describe how
theory-driven argument mapping training might serve to resolve these problems.

WHAT IS CRITICAL THINKING?

There are many definitions and measures of critical thinking. This variety can make it
difficult for researchers and teachers to understand or agree on the key components of good
critical thinking and these difficulties may impede their ability to construct an integrated
theoretical account of how best to train critical thinking skills. In the absence of greater clarity
in relation to the components of critical thinking skill and the way these components work
together in the context of solving critical thinking problems, it can be difficult to design
critical thinking training programs.
In the past century, there has been little agreement on how to conceptualise critical
thinking. John Dewey (1933) provided one of the first multi-level models of thinking in his
classic book, How We Think. Each level of thinking in Dewey’s system differs in terms of its
adequacy for the purpose of achieving “active, persistent, and careful consideration of any
belief or supposed form of knowledge in the light of the grounds that support it and the
further conclusions to which it tends” (Dewey, 1933, p. 8).
In the first level of Dewey’s system is the stream of consciousness (e.g. day dreaming),
an “uncontrolled coursing of ideas through our heads” (p. 6). At the next level, Dewey
describes imagination as a more orderly and controlled type of thinking, specifically, where
 The Promotion of Critical Thinking Skills through Argument Mapping 99

“successions of imaginative incidents and episodes that have a certain coherence, hang
together on a continuous thread, and thus lie between kaleidoscope flights of fancy and
considerations deliberately employed to establish a conclusion” (p. 6). The third level of
thinking in Dewey’s system “is practically synonymous with belief” - belief that is accepted
or rejected as a set of conclusions, but “not conclusions reached as the result of personal
mental activity, such as observing, collecting, and examining evidence” (p. 7). Dewey draws
a contrast between this form of thinking and reflective thinking, the highest level of thinking
in his cognitive system:

“…Columbus did not accept unhesitatingly the current traditional theory…Skeptical of

what, from long habit, seemed most certain, and credulous of what seemed impossible, he
went on thinking until he could produce evidence for both his confidence and his disbelief.
Even if his conclusion had finally turned out wrong, it would have been a different sort of
belief from those it antagonized, because it was reached by a different method. Active,
persistent, and careful consideration of any belief or supposed form of knowledge in the light
of the grounds that support it and the further conclusions to which it tends constitutes
reflective thought. Any one of the first three kinds of thought may elicit this type; but once
begun, it includes a conscious and voluntary effort to establish belief upon a firm basis of
evidence and rationality” (Dewey, 1933, p. 8, italics added).

Dewey’s conceptualization of reflective thinking helped to inform more recent

conceptualizations of critical thinking. Similarly, recognition of the importance of critical
thinking in education followed the growth of interest in informal logic, which was initiated in
part by the work of Stephen Toulmin in the late 1950’s (Allen, Feezel & Kauffeld, 1967).
Informal logic is a type of logic that emphasises the justificatory function of argumentation,
namely that a good argument requires sufficient support (e.g. reliable and valid empirical
evidence). Toulmin’s focus on informal reasoning helped to transform argumentation training
initiatives in university, which traditionally had focused on training in formal logic (i.e.,
reasoning using syllogisms). Deliberations in relation to critical thinking skills grew in part
from the notion of informal logic, where the central focus is on the analysis and evaluation of
claims - claims could only be made (and justified) after a sufficient amount of analysis and
evaluation had been conducted on propositions and their logical interdependencies within the
arguments used to support these claims. Since then, dozens of definitions for critical thinking
have been offered (see Table 1).
Though there have been dozens of attempts at defining critical thinking, many of the
definitions are quite vague. Many of the authors present a number of skills necessary for good
critical thinking rather than providing an operational definition that informs measurement and
analysis of the skills listed. The one thing that all the authors seem to agree on is that critical
thinking is in fact a collection of cognitive and metacognitive skills centred on the analysis
and evaluation of beliefs and the ability to draw sound inferences. Though it is reasonable to
suggest that a straightforward, singular description and operational definition of critical
thinking is not possible due to the variety of perspectives on critical thinking, there is a need
for some reasonable group consensus in an educational context; as an agreed upon operational
definition is necessary to conduct educational research in this area and, more importantly, to
compare findings across different groups and intervention studies.
100 Christopher Dwyer, Michael Hogan and Ian Stewart

Table 1. Definitions and Descriptions of Critical Thinking

Author Definition/Description
Glaser (1941) Critical thinking is: an attitude of being disposed
to consider, in a thoughtful way, problems and
subjects that come within the range of one’s
experience; knowledge of the methods of logical
enquiry and reasoning; and some skills in
applying those methods. Critical Thinking calls
for a persistent effort to examine any belief or
supposed form of knowledge in the light of the
evidence that supports it and the further
conclusions to which it tends.
Ennis (1987) Critical thinking is reasonable, reflective
thinking, focused on deciding what to believe or
do.
Paul (1993) A unique kind of purposeful thinking, in which
the thinker systematically and habitually imposes
criteria and intellectual standards upon the
thinking, taking charge of the construction of
thinking, guiding the construction of the thinking
according to the standards, assessing the
effectiveness of the thinking according to the
purpose, the criteria, and the standards.
Allegretti & Frederick (1995) Critical thinking is evaluating the arguments of
others, evaluating one’s own arguments,
resolving conflicts and understanding the source
of conflicts in argumentation; thus coming to a
resolution in complex problems and gaining
confidence in one’s own thinking processes
Wilkinson (1996) Critical thinking is goal-oriented, purposeful
thinking that involves a number of mental skills,
such as determining what data is relevant,
evaluating the credibility of sources and making
inferences.
Bensley (1998) Critical thinking is reflective thinking in which a
person evaluates relevant evidence and works to
draw a sound or good conclusion.
Halpern (2003) Critical thinking is purposeful, reasoned and
goal-directed thinking – the kind of thinking
involved in solving problems, formulating
inferences, calculating likelihoods and making
decisions.

Though debate is ongoing over the definition of critical thinking and the core skills
necessary to think critically, to date, there has been only one definition and list of skills that
stands out as a reasonable consensus conceptualisation of critical thinking. In 1988, a
committee of 46 experts in the field of critical thinking, known as the Delphi Committee,
gathered to discuss a definition of critical thinking. The committee also discussed the skills
 The Promotion of Critical Thinking Skills through Argument Mapping 101

necessary to think critically. The findings taken from this meeting, known as The Delphi
Report, written by Peter Facione (1990), defined critical thinking as:

“…purposeful, self-regulatory judgment which results in interpretation, analysis,

evaluation, and inference, as well as explanation of the evidential, conceptual,
methodological, criteriological, or contextual considerations upon which that judgment is
based.” (p. 3).

Furthermore, the Delphi panel overwhelmingly agreed (i.e. 95% agreement) that analysis,
evaluation and inference were the core skills necessary for critical thinking (Facione, 1990).
These skills (as described by the Delphi Report) are presented in Table 2. The definition of
critical thinking provided by the Delphi Report was adopted by the American Philosophical
Association (APA) and as a result, has become a widely accepted definition for good critical
thinking (Beckie, Lowry & Barnett, 2001). The same definition of critical thinking was also
used by the U.S. Department of Education as a framework for setting its educational goals
(Facione, Facione, Blohm & Giancarlo, 2002). The Delphi definition of critical thinking also
inspired the creation of the California Critical Thinking Skills Test (CCTST; Facione, 1990),
a test that is commonly used as part of the evaluation of critical thinking intervention studies.
At the same time, the challenges of teaching critical thinking skills remain, and a question
remains as to how best to teach critical thinking skills.

Can Critical Thinking Skills Be Taught? A Look at the Previous Research

Critical thinking (CT) courses have been taught at University in varying academic
domains including law, philosophy, psychology, sociology and nursing. Importantly, it is
often argued that critical thinking is a domain-general skill that can be taught alongside any
academic content (Gabbenesch, 2006). At the same time, whether or not CT can be improved
via explicit instruction and how it is best improved are issues that continue to be debated in
the literature. This debate is fuelled in part by difficulties interpreting and comparing research
studies in the area.
CT courses are taught in a variety of different academic domains and are informed by
varying conceptualisations of CT. Different intervention studies also use different measures
of CT performance that are not directly comparable - the California Critical Thinking Skills
Test (Facione, 1990), the Cornell Critical Thinking Test (Ennis, Millman & Tomko, 1985)
and the Watson-Glaser Critical Thinking Assessment (Watson & Glaser, 1980). The reported
reliability and validity of different measures also varies, which has led Abrami and colleagues
to ask: “How will we know if one intervention is more beneficial than another if we are
uncertain about the validity and reliability of the outcome measures?” (Abrami et al., 2008, p.
1104). Abrami and colleagues add that, even when researchers declare that they are assessing
CT using reliable and valid assessment tools, there still remains the major challenge of
ensuring that measured outcomes are related in some meaningful way to the conceptualisation
and operational definition of CT that informs their teaching practice. Often, the relationship
between the concepts of CT that are taught and those that are assessed is unclear and a large
number of studies in this area specify no theory to help elucidate these relationships.
102 Christopher Dwyer, Michael Hogan and Ian Stewart

Table 2. The Core Critical Thinking Skills According to the Delphi Report

Skill Description

Analysis To identify the intended and actual inferential relationships among

statements, questions, concepts, descriptions or other forms of representation
intended to express beliefs, judgments, experiences, reasons, information, or
opinions.

Examining ideas: to determine the role various expressions play or are

intended to play in the context of argument, reasoning or persuasion; to
compare or contrast ideas, concepts, or statements; to identify issues or
problems and determine their component parts, and also to identify the
conceptual relationships of those parts to each other and to the whole.

Detecting arguments given a set of statements, descriptions, questions or

graphic representations, to determine whether or not the set expresses, or is
intended to express, a reason or reasons in support of or contesting some
claim, opinion or point of view.

Analysing arguments: given the expression of a reason or reasons intended

to support or contest some claim, opinion or point of view, to identify and
differentiate: (a) the intended main conclusion, (b) the premises and reasons
advanced in support of the main conclusion, (c) further premises and reasons
advanced as backup or support for those premises and reasons intended as
supporting the main conclusion, (d) additional unexpressed elements of that
reasoning, such as intermediary conclusions, non-stated assumptions or
presuppositions, (e) the overall structure of the argument or intended chain
of reasoning, and (f) any items contained in the body of expressions being
examined which are not intended to be taken as part of the reasoning being
expressed or its intended background.

Evaluation To assess the credibility of statements or other representations which are

accounts or descriptions of a person's perception, experience, situation,
judgment, belief, or opinion; and to assess the logical strength of the actual
or intend inferential relationships among statements, descriptions, questions
or other forms of representation.

Assessing claims: to recognize the factors relevant to assessing the degree of

credibility to ascribe to a source of information or opinion; to assess the
contextual relevance of questions, information, principles, rules or
procedural directions; to assess the acceptability, the level of confidence to
place in the probability or truth of any given representation of an experience,
situation, judgment, belief or opinion.

Assessing arguments: to judge whether the assumed acceptability of the

premises of a given argument justify one's accepting as true (deductively
certain), or very probably true (inductively justified), the expressed
 The Promotion of Critical Thinking Skills through Argument Mapping 103

Table 2. (continued)

Skill Description
conclusion of that argument; to anticipate or to raise questions or objections,
and to assess whether these point to significant weakness in the argument
being evaluated; to determine whether an argument relies on false or
doubtful assumptions or presuppositions and then to determine how crucially
these affect its strength; to judge between reasonable and fallacious
inferences; to judge the probative strength of an argument's premises and
assumptions with a view toward determining the acceptability of the
argument; to determine and judge the probative strength of an argument's
intended or unintended consequences with a view toward judging the
acceptability of the argument; to determine the extent to which possible
additional information might strengthen or weaken an argument.

Inference To identify and secure elements needed to draw reasonable conclusions; to

form conjectures and hypotheses; to consider relevant information and to
educe the consequences flowing from data, statements, principles, evidence,
judgments, beliefs, opinions, concepts, descriptions, questions, or other
forms of representation.

Querying evidence: in particular, to recognize premises which require

support and to formulate a strategy for seeking and gathering information
which might supply that support; in general, to judge that information
relevant to deciding the acceptability, plausibility or relative merits of a
given alternative, question, issue, theory, hypothesis, or statement is
required, and to determine plausible investigatory strategies for acquiring
that information.

Conjecturing alternatives: to formulate multiple alternatives for resolving a

problem, to postulate a series of suppositions regarding a question, to project
alternative hypotheses regarding an event, to develop a variety of different
plans to achieve some goal; to draw out presuppositions and project the
range of possible consequences of decisions, positions, policies, theories, or
beliefs.

Drawing conclusions: to apply appropriate modes of inference in

determining what position, opinion or point of view one should take on a
given matter or issue; given a set of statements, descriptions, questions or
other forms of representation, to educe, with the proper level of logical
strength, their inferential relationships and the consequences or the
presuppositions which they support, warrant, imply or entail; to employ
successfully various sub-species of reasoning, as for example to reason
analogically, arithmetically, dialectically, scientifically, etc; to determine
which of several possible conclusions is most strongly warranted or
supported by the evidence at hand, or which should be rejected or regarded
as less plausible by the information given.
Adapted from Facione, 1990.
104 Christopher Dwyer, Michael Hogan and Ian Stewart

Nevertheless, researchers have attempted to group intervention studies in an effort to

examine whether or not critical thinking can be improved via explicit instruction. For
example, a recent meta-analysis by Alvarez-Ortiz (2007) examined 52 studies which
investigated a wide range of teaching strategies designed to improve CT. The meta-analysis
was specifically conducted in order to answer the questions as to whether or not participation
in philosophy courses improved CT ability. Results of the meta-analysis revealed that
participation in a philosophy course yielded a mean effect size of .26 SD, CI [.12 - .39], with
little evidence to suggest that participation in a philosophy course had any greater effect on
CT performance than any other academic course (mean effect size = .12 SD, CI [.11, .21]).
However, this meta-analysis also suggested that all courses (regardless of academic content)
that directly taught CT (effect size of .40, CI [.08, .71]) or had CT infused into the curriculum
(effect size of .26, CI [.09, .43]) yielded better CT performance than courses that did not teach
CT in some form (effect size of .12 SD, CI [.08, .17]). These findings lend some support to
Gabbenesch’s (2006) claim that CT is domain-general, as the course content was not the key
factor in improving CT, whereas involvement of some form of explicit CT instruction was
fundamental.
Another meta-analysis, conducted by Abrami and colleagues (2008) included 161 CT
intervention studies and examined the efficacy of different types of CT training course. They
used Ennis’ (1989) typology of four CT course types (i.e. general, infusion, immersion and
mixed) to differentiate CT intervention methods. In the general approach to CT training, CT
skills, dispositions and processes “are learning objectives, without specific subject matter
content” (Abrami et al., 2008, p. 1105). Conversely, the infusion method requires specific
course content upon which CT skills are practiced. In the infusion approach, the objective of
teaching CT alongside course content is made explicit. In the immersion method, like the
infusion method, specific course content is required; however, while CT skills are practiced,
CT objectives are not made explicit in the immersion approach. Finally, in the mixed
approach, critical thinking is taught independently of the specific subject matter content of the
course.
Abrami and colleagues (2008) reported a significant effect on CT performance (g+ = .34)
of all CT courses included in the meta-analysis. However, only 91 of the studies assessed
critical thinking ability using standardised tests (i.e. as opposed to using an assessment
devised by a teacher or researcher), and these 91 studies yielded an average effect size of (g+)
.24. Comparing the four CT course types, results of the meta-analysis revealed that courses
using the mixed approach had the largest effect on CT performance (g+ = .94), followed by
the infusion approach (g+ = .54), the general approach (g+ = .38) and the immersion approach
(g+ = .09), respectively. It is important to note that the immersion typology (which had the
smallest effect) is the only approach that does not make CT objectives explicit to students.
Thus, making CT objectives clear to students may be an important part of any course design
aimed at increasing CT ability (Abrami et al., 2008). More generally, the authors concluded
that the enhancement of CT ability is greatly dependent upon how CT is taught and that the
mixed approach to teaching CT worked best as students were required to learn CT skills
separate from other course material and then apply them to the material later on in the course.
Abrami and colleagues’ (2008) meta-analysis was conducted in light of some very broad
distinctions between different types of CT training courses. However, less is known about
how different instructional methods impact overall training benefits. In this chapter, we
propose that teaching strategies that facilitate the assimilation of argument structures (i.e.
 The Promotion of Critical Thinking Skills through Argument Mapping 105

analysis of argument structures), and an assessment of the quality of evidence and the logical
relationships between propositions in moderately complex arguments (i.e. evaluation of
argument structures) may in turn facilitate significant growth in analysis and evaluation skills.
With a guiding theory and suitable experimental controls it is possible to compare different
teaching strategies in this context.

Beyond Text-Based Learning: The Use of Thought Structuring Tools in

CT Education

Central to our theory of CT enhancement is a focus on the problem of working memory

demands associated with the assimilation and simultaneous analysis and evaluation of
arguments. According to various frameworks for thinking (e.g. Dewey, 1933; Bloom, 1956;
Anderson & Krathwohl, 2001; Moseley et al., 2005), there are a number of cognitive and
metacognitive skills that are necessary for good thinking. For example, researcher and
theorists often point to the ability to build understanding through the organisation of ideas
(Moseley at el., 2005); and the ability to recognise, appraise and analyse both a chain of
arguments and the justification of claims through reasoning (Allen, Feezel & Kauffeld, 1967).
While these kinds of organizational, analytical, and evaluative skills may be fundamental
components of good CT, Harrell (2005) notes that students often fail to understand the ‘gist’
(Kintsch & van Dijk, 1978) of text-based information presented to them; and more often,
students cannot adequately ‘follow’ the argument of a text (i.e. the chain of reasoning and the
justification of claims in the chain), as most students do not even acknowledge that
information within a text presents an argument and instead read it as if it were a story.
Conversely, authors who do understand the nature of argumentation often construct verbose
‘maze-like’ arguments that consist of massive amounts of text (Monk, 2001). Students who
are presented with these texts may thus find it very difficult to capture anything more than the
‘gist’ of the argument. For example, because text-based arguments contain many more
sentences than just the propositions that are part of the argument, these sentences may
obscure the intention of the piece and the inferential structure of the argument (Harrell, 2004).
More specifically, as arguments are not sequential in nature, the linear nature of text
sometimes makes it difficult to assimilate the information within a text-based argument (van
Gelder, 2003). For example, when reading text, a person may read a statement on page three
and not read any relevant support (or objection) to this claim until they reach page 16.
Between pages 3 and 16, it could be that a variety of other propositions are presented, which
places cognitive load on the reader. Cognitive load is the burden put upon an individual in
using and distributing working memory resources during cognitive activities such as learning
and problem-solving (Sweller, 1988, 1999). This additional load comes from the need to, for
example, switch attention from one page to another and back and forth, in order to create
some structure for a ‘non-user friendly’ text. While reading text-based materials, students
must figure out the relationship between propositions for themselves, using whatever cues
they can, regardless of the ambiguity of the text. Thus, in attempting to mentally structure
arguments when reading text, the reader faces cognitive load.
Tindall-Ford, Chandler and Sweller (1997) found that learning is impeded when
instructional materials require a high degree of attention switching. They concluded that
encoding environments that increase the cognitive load placed on the reader tend not only to
106 Christopher Dwyer, Michael Hogan and Ian Stewart

slow the learning process, but also reduce overall levels of learning. Presenting information in
a way that reduces the level of attention switching may minimize the cognitive load and
improve learning.
Having available the structure of an argument is crucial for many reasons: it facilitates
logical reasoning, the answering of specific questions about the relation between one
proposition and others, and the ready construction of a ‘mental image’ of the whole argument.
Argument mapping is a learning aid which may facilitate thinking in this regard. For example,
in the argument map, both the propositions and the relationships among them are explicitly
stated and the information is presented in an integrated, organised fashion. Notably, it is
generally the case that integrated, organised representations facilitate learning (Sweller,
1999). In previous research, argument mapping has been identified as a technique that might
circumvent the many obstacles related to reading text and visualizing the argument
simultaneously; and may also enhance overall levels of learning and CT (van Gelder, 2001).
Thus, argument mapping is hypothesised as a tool that may support the cultivation of CT
skills by helping to resolve the problem of working memory demands associated with the
assimilation and simultaneous analysis and evaluation of arguments.

Argument Mapping as a Tool for Critical Thinking Instruction

In an argument map, a text-based argument is visually represented using a ‘box-and-

arrow’ style flow-chart that makes the structure of the argument explicit to the reader by
organising the propositions within the argument and by displaying all the connections
amongst propositions within the argument (van Gelder, 2001). For an example of an
argument map, see Figure 1.
Though computer-based argument mapping is a relatively new technique (van Gelder,
2000), some research has examined the efficacy of teaching CT skills using argument
mapping as a tool of instruction. For instance, in her meta-analysis, Alvarez-Ortiz (2007)
found that students who participated in critical thinking courses that used at least some
argument mapping within the course achieved gains in CT ability with an effect size of .68
SD, CI [.51, .86]. In courses where there was “lots of argument mapping practice” there was
also a significant gain in students’ CT performance, with an effect size of .78 SD, CI [.67,
.89]. These findings compare favourably to the effect sizes observed for participation in
philosophy courses (average effect size = .26 SD, CI [.12 - .39]), any other academic course
(effect size = .12 SD, CI [.11, .21]), and courses that directly taught CT or had CT infused
into the curriculum (average effect size = .49 SD, CI [.39, .59].
Thus, there are a number of studies that have previously used argument mapping as a tool
of CT instruction. For example, Tim van Gelder (2001) and van Gelder and Rizzo (2001)
provided undergraduate philosophy students with a semester-long CT course, in which
students were trained in CT through the use of argument mapping (AM). Students’ CT ability
was tested both before and after one college semester using alternate forms of the California
Critical Thinking Skills Test (CCTST; Facione, 1990). Results revealed an improvement with
an effect size of .84, which implied an impressive gain of almost one standard deviation in CT
ability over the course of the semester. It is important to note that, though the authors credit
much of this gain to AM training, they also admit that this gain could also be due to other
 The Promotion of Critical Thinking Skills through Argument Mapping 107

aspects of the course, such as the practice regime. Notably, this study did not include a
control group or an alternative CT training regime with which to compare AM training.
Similarly, van Gelder, Bissett and Cumming (2004) provided undergraduate philosophy
students with a 12-week CT course taught through the use of AM. Students were pre-tested
using the CCTST. During the course, students were provided with homework exercises and
were free to complete as many practice exercises as they wished. Students also attended one
tutorial per week in which they had access to both argument mapping software and to direct
personal guidance from their tutors. After completion of the course, students were post-tested
using the CCTST. Results revealed that CT scores increased significantly from pre- to post-
testing with a large effect size of .8 SD, CI [.66, .94]. There was also a significant correlation
between performance and AM practice hours (r = .31).
Butchart et al. (2009) compared two groups of students who attended AM-infused CT
modules (i.e. a module with online automated feedback for AM exercises and a module that
contained AM exercises only, with no automated feedback). These modules were compared
in turn with a ‘standard’ CT module (i.e. no AM). CT training was heavily concentrated on
two CT skills: analysis and evaluation, described by the Delphi Report as core critical
thinking skills. Prior to commencement of the course, students completed the CCTST Form A
as a pre-test. During the course, students were provided with eight homework assignments
and 10 sets of exercises. Automated feedback was provided to students in the automated
feedback AM group. After completion of the course, students were post-tested using Form B
of the CCTST.
Butchart and colleagues found that those who received automated feedback for their AM
exercises showed a significant gain in CT ability with a medium effect size of .45. Students
who completed the AM exercises without automated feedback showed a gain with an effect
size .22. Those who participated in a standard CT module showed a gain with an effect size of
.19. Unfortunately, statistical differences among the three groups in this study were not
reported. Furthermore, as admitted by the authors, participants in the automated feedback
group could have been provided with more informative feedback, as opposed to simply
receiving automated notice of a ‘correct’ or ‘incorrect’ response for placement of a
proposition. One could argue that this automatic ‘correction’ of argument mapping exercises
is not an ideal form of feedback, in the sense that an explanation as to why a response is
incorrect would likely have been more informative to the student. Thus, although students
who received automated feedback for their AM exercises showed the largest gain in CT
ability, it is unclear how feedback worked to improve performance in this context.
Therefore, while research suggests that CT courses taught through the use of AM
improve CT ability, there have been a number of problems with the research conducted to
date. For example, two of the three studies described above (i.e. van Gelder & Rizzo, 2001;
van Gelder, Bissett and Cumming, 2004) did not compare AM-infused CT training with a no-
intervention control group or a comparable active intervention control group. In addition,
although Alvarez-Ortiz’s (2007) meta-analysis suggests that semester-long training courses in
AM produce greater gains in CT skills (when compared with standard semester-long courses
in introductory philosophy), AM training has not been directly compared with other methods
of teaching CT skills, apart from one study where a standard CT course was used for
comparison purposes (Butchart et al., 2009).
Figure 1. An example of an Argument Map created through Rationale™.
 The Promotion of Critical Thinking Skills through Argument Mapping 109

Butchart and colleagues reported using the same course structure and teaching the same
content “as far as possible” (Butchart et al., 2009, p. 278), but it is unclear how this worked in
practice. Furthermore, though Butchart and colleagues compared three groups in their study,
these groups were not adequately matched or randomly assigned to experimental conditions.
Participants were assigned to experimental conditions based on the semester in which they
registered for the CT course. Those in the automated feedback AM group participated in the
first semester of the study; those in the standard CT training condition participated in the
second semester; and those in the AM exercises only group participated in the third semester
of the study. Also, the pre-and-post-test scores (and the resultant gains) of the three groups
were not statistically compared, so it difficult to assess whether or not the groups possessed
similar or different CT abilities prior to their participation in the course, and whether or not
gains across conditions are statistically different from one another.
In summary, though evidence suggests that critical thinking can be taught and enhanced,
research studies in this area are difficult to compare because of differences across studies in
the conceptualisations of critical thinking that inform teaching practices and the selection of
measures used to assess performance. Some research studies have examined the efficacy of
AM-infused CT training; however, this research is also difficult to interpret due to the
absence in some studies of a control condition for comparison purposes and the failure to
randomly assign groups to experimental conditions in other studies. Therefore, further
research is needed to provide more conclusive evidence in favour of the claim that argument
mapping is a tool that facilitates critical thinking ability.

Rationale for the Current Research

Argument maps and argument mapping may be a useful pedagogical aid, particularly in
situations where students are working to analyse and evaluate complex arguments. The
current research is part of a larger set of studies designed to examine the effects of argument
mapping on memory for arguments (Dwyer, Hogan, & Stewart, 2010) and growth in critical
thinking skill and reflective judgment. The following study examined the effect of AM
training on CT skill. Critical thinking performance of those who attended an AM-infused CT
seminar series was compared with the performance of those who attended a CT seminar series
using identical content but taught using more traditional, hierarchical outlines (HO; see
below). The performance of students in both of these active critical thinking training courses
was compared to the performance of students who received no explicit CT training. A further
aim of the study was to examine the effect of both AM and HO training on students’
disposition towards thinking.
Based on previous research (Butchart et al., 2009; van Gelder, Bissett and Cumming,
2004; van Gelder & Rizzo, 2001), we hypothesised that AM training would result in larger
gains in CT ability over the course of the semester when compared with both HO training and
the control condition. More specifically, though HO organises information for the reader, the
structure of an argument is represented as a linear flow of text and it does not make use of a
box-and-arrow format, colour cues to represent reasons, objections, and rebuttals, or
relational cues (i.e. but, because and however) that link propositions. However, because
information within an HO is hierarchically organised, we hypothesise that training in HO
110 Christopher Dwyer, Michael Hogan and Ian Stewart

(like AM training) would result in larger gains in CT ability over the course of the semester
when compared with the control condition.

METHOD
Design

A series of six one-way ANCOVAs were used to assess the effect of the three
experimental conditions (AM, HO, & Control) on six ability outcomes: overall CT, analysis,
evaluation, inference, deductive reasoning and inductive reasoning, while controlling for
baseline CT skill ability. Similarly, a series of seven ANOVAs were also used to assess the
effects of the experimental conditions on students’ disposition towards thinking.

Participants

Participants were first year psychology students (N = 81; 57 females, 24 males), aged
between 18 and 25 years, from the National University of Ireland, Galway. In return for their
participation, students were awarded academic course credits. To ensure confidentiality,
participants were identified by ID number only.

Materials and Measures

The CT intervention materials used in this study were the exercise handouts and CT
recordings, and a laptop, a projector and DVDs which were used to present the pre-recorded
seminar series. These materials are available upon request.
The California Critical Thinking Skills Test (CCTST; Forms B) was administered as a
baseline measure at the pre-test session. The CCTST was developed by Peter Facione and
colleagues (1990; 2002). The CCTST consists of 34 multiple choice questions, which
examine overall CT ability as well as five sub-skills: analysis, evaluation, inference, inductive
reasoning, and deductive reasoning. Results are presented as raw scores and are additionally
presented as U.S. national percentile equivalents of approximately 2,000 university students.
Test reliability ranges from 0.78– 0.84 (Facione, 1991).
The CCTST (Form 2000) was administered at the post-testing session. Gain was not
measured from pre- to post-testing (i.e. from Form B to Form 2000). Rather, in accordance
with Jacobs (1995), the CCTST (Form B) was administered as a baseline measure and
analysed as a covariate, whereas the CCTST (Form 2000) was administered as an outcome
measure and analysed as the dependent variable.
The California Critical Thinking Disposition Inventory (CCTDI; Facione & Facione,
1992) was administered at post-testing. Seven subscales of the CCTDI include: truth seeking,
open mindedness, analyticity, systematicity, confidence, inquisitiveness and maturity.
 The Promotion of Critical Thinking Skills through Argument Mapping 111

Finally, a questionnaire was administered at the end of the course which asked students to
rate various facets of the course, such as their ability to understand the course, the quality of
the materials, and the quality of the instruction.

Procedure

The study took place over eight weeks. The two experimental groups attended a 16 hour
CT seminar series over the course of eight weeks, differing only in method of presentation
(i.e. AM-infused or HO -infused CT training). The seminar series was designed to teach CT
according to the framework provided by the Delphi Report and the American Philosophical
Association. The control group did not attend any CT seminars.
In Week 1, the California Critical Thinking Skills Test (Form B) was administered. The
seminar series began in Week 2. Seminars were given to four different groups per week: two
of which were AM groups and two of which were HO groups. Both AM and HO lectures
were identical in content and pre-recorded voice-over, which was dubbed over a
PowerPoint™ slideshow using the Echo360™ system recording. Only the slideshows and in-
class handouts varied (for purposes of presenting either AM or HO strategies for organizing
arguments). The voice-over was performed by the same person (male: research supervisor)
and this person did not facilitate the delivery of these recordings to students in the seminar.
Independent evaluators rated the quality of the voice-over and judged whether or not there
were any substantial differences in the quality of AM and HO delivery. Quality of voice-
overs were rated highly and no differences in quality between AM and HO conditions were
noted.
In the seminars, students were taught skills and then shown how to use them via worked
examples. During the seminars, the recordings were often paused and restarted in order to
allow time for the completion of exercises. Students were given enough time so that they
could actively learn by applying the skills they had just learned. On average, approximately
75% of the time allotted to each class was dedicated to this active learning. The course outline
and what was taught in each class is presented in Table 3.
In Week 8, after completion of the seminar series, the CT ability of all three groups was
again measured using CCTST Form 2000. In Week 8, the CCTDI was administered to all
groups in order to examine students’ disposition toward thinking. Students also completed a
questionnaire which asked them to rate various facets of the course and make suggestions for
improving the course. Students who did not complete the course (e.g. those who simply
dropped out), were also given the questionnaire and were asked for reasons why they did not
complete the course.
112 Christopher Dwyer, Michael Hogan and Ian Stewart

Table 3. Critical Thinking Course Outline

Class Title What Was Taught

No.
1 Pre-Testing • Students completed the CCTST (Form B) pre-test.
2 Session 1: 1. We think in order to decide what to do and what to
“Introduction to believe.
Critical Thinking” 2. We ultimately decide what to believe by adding
supports or rebuttals to our own arguments (i.e.
questioning our own beliefs).
3. Arguments are hierarchical structures. We can continue
to add more levels if we like.
3 Session 2: 1. In order to analyse an argument, we must extract the
“Unpacking structure of the argument from dialogue or prose.
(analysing and 2. Identifying types (sources) of arguments and
evaluating) a considering the strength of each type is another form of
persons’ belief” analysis.
3. The evaluation of the overall strengths and weaknesses
of an argument can be completed after adequate
analysis.
4 Session 3: “Analysis 1. Evaluation includes the recognition of imbalances,
& Evaluation” omissions and bias within an argument.
2. Evaluative techniques can aid recall.
3. Examining whether or not the arguments used are
relevant or logically connected to the central claim is
also an important factor in evaluation.
5 Session 4: We must evaluate:
“Evaluation” 1. Types (sources) of arguments based on credibility
2. The relevance of propositions to the central claim or
intermediate conclusions within the argument
3. The logical strength of an argument structure
4. The balance of evidence within an argument structure
6 Session 5: 1. Evaluation and inference are intimately related.
“Inference” 2. Inference differs from evaluation in that the process of
inference involves generating a conclusion from
previously evaluated propositions.
3. In larger informal argument structures, intermediate
conclusions must be inferred prior to the inference of a
central claim.
7 Session 6: “Making • Review of all the previous 5 sessions
Another’s Argument
Your Own”

8 Post-Testing • Students completed the CCTST (Form 2000) post-test

and the CCTDI
 The Promotion of Critical Thinking Skills through Argument Mapping 113

RESULTS
Means and standard deviations for the three groups are presented in Table 6. A series of
between-subjects ANCOVAs were conducted to examine the effects of experimental
conditions on CT outcomes, while also controlling for baseline CT ability. A preliminary
analysis evaluating the homogeneity-of-slopes assumption revealed that there was no
significant difference amongst groups on pre-test CT or sub-skill performance.
There was a main effect of group on analysis performance, F (2, 77) = 4.74, MSE = .04, p
= .011, partial η² = .11, with those in the HO group scoring significantly higher on the
analysis post- test than those in the control group (p < .05). Post-hoc analysis revealed border-
line difference between the AM and control group, F (1, 77) = 3.59, MSE = .04, p = .06, with
the AM group scoring higher than the control group.
Though there was no main effect of group on evaluation performance, F (2, 77) = 2.35, p
= .103, there were some interesting trends in the data. Specifically, post hoc analyses revealed
that the AM group scored significantly higher than the control group on post testing, F (1, 77)
= 4.29, MSE = .02, p = .042. There was a borderline main effect of group on inductive
reasoning, F (2, 77) = 3.08, p = .052. Post-hoc analyses revealed that the AM group scored
significantly higher than those in the control group, F (1, 77) = 4.44, MSE = .02, p = .038; and
that the HO group also scored significantly higher than those in the control group, F (1,77) =
4.52, MSE = .02, p = .037. No other effects were observed.
A further series of eight ANOVAs were conducted in order to examine the effects of
group on disposition and the sub-scales of the CCTDI. There were no main effects of group
on overall disposition score, or on the sub-scale scores of truth seeking, open mindedness,
analyticity, systematicity, confidence, inquisitiveness or maturity. However, overall
disposition score was significantly correlated with post-test CT performance (r = .45, p =
.001), but not with pre-test CT performance (r = .16, p = .244).

DISCUSSION
We examined the effects of AM-infused and HO-infused CT training on students’ CT
performance. Performance on various sub-skills of CT (i.e. analysis, evaluation, inference,
inductive reasoning and deductive reasoning) was measured both before and after the
intervention, as was students’ disposition towards thinking on post-testing.
Results revealed that students in the HO condition performed better than students in the
control group on analysis and inductive reasoning at post-testing. Students in the AM
condition performed better at post-test than students in the control condition on evaluation
and inductive reasoning. Unlike van Gelder and colleagues (2001, 2003, 2004), we did not
find a significant effect of AM training on overall CT performance. Although we must
evaluate the results of the current study with caution, findings suggest that certain critical
thinking skills (i.e. evaluation and inductive reasoning) can potentially be enhanced by
argument mapping training. These results also suggest that certain critical thinking skills (i.e.
analysis and inductive reasoning) are enhanced by training in hierarchical outlining.
114 Christopher Dwyer, Michael Hogan and Ian Stewart

Table 6. Means and standard deviations (%) and sample size (N) for the three groups at
both pre-test and post-test

Pre-Test Post-Test

N M SD M SD
Overall CT

AM 23 .48 .15 .51 .14

HO 28 .44 .16 .50 .13
Control 30 .41 .15 .43 .13

Analysis

AM 23 .55 .15 .67 .17

HO 28 .49 .18 .72 .16
Control 30 .45 .19 .58 .22

Evaluation

AM 23 .46 .17 .45 .17

HO 28 .41 .14 .41 .17
Control 30 .36 .14 .34 .13

Inference

AM 23 .44 .16 .47 .16

HO 28 .40 .15 .46 .15
Control 30 .41 .16 .43 .16

Inductive Reasoning

AM 23 .49 .16 .59 .13

HO 28 .42 .16 .58 .15
Control 30 .40 .13 .50 .14

Deductive Reasoning

AM 23 .46 .15 .42 .16

HO 28 .45 .16 .42 .16
Control 30 .40 .13 .36 .16

Results also revealed that a positive disposition toward critical thinking was related to
better critical thinking performance at post-testing (Ennis, 1987; Facione, 1990, 1992;
Facione, Facione, Blohm & Giancarlo, 2000; Halpern, 2003, 2004). Notably, the correlation
between pre-test CT performance and dispositions was not significant. This suggests that
 The Promotion of Critical Thinking Skills through Argument Mapping 115

dispositions such as truth seeking, open mindedness, analyticity, systematicity, confidence,

inquisitiveness or maturity may emerge as significant correlates of CT performance only after
students have been exposed to some training in CT skills. However, we must interpret these
findings with caution because we only measured dispositions at one point in time, that is, at
post-test. Furthermore, researchers have identified problems with the measurement of
dispositions, including the problematic nature of measuring CT dispositions using self-reports
(Ku, 2009).
There were a number of limitations in this study. One limitation was the small sample
size, which impacted on the power of our statistical analysis. It was difficult to persuade
students to register for this extra-curricular CT training course, and although we managed to
recruit a relatively large number of students, there was significant attrition from pre-test to
post-test. Reasons for attrition included students having conflicting schedules, being too busy
with other subjects, and other personal reasons. From the 129 students who initially
completed the CCTST pre-test, only 81 completed the post-test. This reduced the power of
our statistical analysis. Some borderline effects may well have been significant if our sample
size was larger.
Another limitation of the current study was the randomization of participants to the
control condition. From a pool of approximately 1,000 eligible students, it was hoped that
roughly 300 would register for the course. Those who did sign up were to be randomly
allocated to the AM group, the HO group, or the control condition. Initially, only 101 students
signed up for the course. As a result, to ensure adequate statistical power in the comparison of
the AM and HO conditions, those 101 students were randomly allocated to one of two
conditions. Therefore, it was necessary to recruit an additional group of students for the
control condition. Those who were assigned to the control condition were students who had
expressed an interest in attending the CT course but who could not attend due to conflicting
schedules. Thus, it may be that those who were recruited for the control condition may have
been ultimately less motivated to take part and perform well on the CT tests, as many students
in the experimental conditions actually rearranged their schedules in order to attend the CT
course. Notably, participants in the control condition performed less well than participants in
the AM and HO groups on certain post-test CT skills and it is possible that these differences
between the experimental groups and the control group may have been a result of differences
in motivation. Although we measured disposition toward critical thinking and found a
correlation between CCTST and CCTDI performance on post-test, the CCTDI does not
provide us with a direct measure of student motivation to perform well on the CCTST. A
measure of students’ motivation would have been useful in the current study, and it could
have been included as a second covariate in the analysis of experimental condition of CT
outcomes.
Another potential limitation of the study is the lack of feedback provided to students
during the course. In a meta-analysis by Marzano (1998), it was found that by providing
feedback to students on the type of strategy they used and how well they were using it to
improve a specific type of cognitive process, students showed a significant gain in
achievement, with an effect size of 1.13. Provision of feedback could potentially have also
motivated students and curbed attrition in our study. Another challenge in the current study
was the selection of a critical thinking test that allowed for adequate measurement of gains
associated with CT training. Though the CCTST measures CT and the CT sub-skills
according to the Delphi definition and framework, the test itself is not necessarily ideal for
116 Christopher Dwyer, Michael Hogan and Ian Stewart

evaluating gain in intervention studies. For example, according to research by Jacobs (1995),
the CCTST Forms A and B are dissimilar in that they possess different levels of difficulty. As
a result, Jacobs has recommended that these tests are not used for purposes of measuring
individual differences or gains from pre- to post-testing; and instead, one form should be used
a covariate measure. In accordance with Jacobs’ research findings, we used Form B as a
covariate and baseline measure of CT and the Form 2000 of the CCTST as the outcome
measure.
Further limitations of the CCTST are apparent when one examines the format of the test.
Though there are 34 items which measure analysis, evaluation, inference, inductive and
deductive reasoning skills, performance is assessed via multiple choice questions (MCQs).
More specifically, the CCTST and many other MCQ tests of critical thinking have been
criticised for being basically tests of verbal and quantitative knowledge (Halpern, 2003), since
the test-takers are not free to determine their own evaluative criteria nor generate their own
solutions to the problem (Ku, 2009). The measurement of critical thinking through MCQs is
also problematic given the potential incompatibility between conceptualisations of critical
thinking and its assessment using MCQs. MCQ tests assess cognitive capacities associated
with identifying single right- and- wrong answers in relation to CT problems, and this
approach to testing is unable to provide a direct measure of test-takers use of metacognitive
processes such as reflective judgment (Halpern, 2003; Ku, 2009).
One solution to this problem would be to use a critical thinking assessment that asks
open-ended style questions, which allow for test-takers to demonstrate whether or not they
spontaneously use a specific critical thinking skill. However, the only commonly used critical
thinking assessment that uses an open-ended format is the Ennis-Weir Critical Thinking
Essay Test (Ennis & Weir, 1985), which has been criticised for its domain-specific nature
(Taube, 1997), the subjective nature in which the tests are scored and potential biases in
favour of test-takers who are more proficient in writing (Adams, Whitlow, Stover & Johnson,
1996). In short, both MCQ and open-ended test formats for assessing critical thinking have
their respective limitations. The current trend is to combine the two response formats into one
test (Ku, 2009).
One final issue to consider in relation to this study is what amounts to a sufficient amount
of argument mapping training. One possible reason why AM did not emerge as a better
training method than HO is that, because AM is a relatively novel method that students must
first master before it becomes useful in promoting CT, more AM training is needed relative to
other CT training methods. Based on van Gelder, Bissett and Cumming’s (2004) finding that
‘deliberate practice’ in argument mapping facilitates growth of CT skills, we have to question
whether or not sufficient practice in AM was provided in the current study. Van Gelder and
colleagues recommend a semester long course in AM-infused CT training. Although the
course in this study was only eight weeks, our study suggests that those who attended CT
training scored higher on some CT skills at post-testing when compared with those in the
control condition. The problem is that these effects were not very large or significant, and we
have to question whether or not this is due to the small sample size or the intensity of the
training provided. One improvement on the current design would be to include more
argument map practice outside the classroom. We initially sought to control for the level of
practice in both AM and HO conditions by restricting work to class-time only. While this was
done with the good intention of controlling for potential confounds in the comparison
between AM and HO training, it may have had a negative impact on the overall efficacy of
 The Promotion of Critical Thinking Skills through Argument Mapping 117

the course. Therefore, extended training in argument mapping through CT, both inside and
outside the classroom, is recommended in future studies.
Further research is needed in order to discover the conditions that most positively affect
CT skill development, including research into the effects of extended training in argument
mapping. Further research should also measure and control for students’ motivation level, as
the lack of motivation of students is one possible cause of the high attrition rate in this study.
In addition, future research should also use measures of CT ability that allow for an
assessment of the meta-cognitive abilities of students. For example, open-ended short answer
questions that allow for both quantitatively and qualitatively scoring rubrics would be ideal,
as they require test-taker to truly consider and evaluate all possible solutions and alternatives
and construct an argument in response to a probe question, rather than simply choosing the
correct answer on an MCQ.
Furthermore, according to the survey taken by students who participated in our study, it
was recommended that future research should also aim to assess student ability throughout
the course and provide students with feedback throughout. Students also suggested that
identification of the student’s initial critical thinking strengths and weaknesses, and how they
are improving over time, may act as an incentive to continued participation and engagement
throughout the course. Though incentives were provided to students (i.e. research
participation credit and two laptops to be awarded to those who showed best effort), it was
also suggested by a handful of students that perhaps too many material incentives and rewards
may not be a good thing, as any sort of performance-based feedback could potentially be
rewarding enough.
It was further suggested that making the course available online would be an ideal
method of increasing participation as it avoids the issue of timetable clashes with other
classes, as students would be able to take the course whenever it suited them. It was also
suggested that argument mapping software be provided to students outside the class setting
(also suggested by van Gelder, Bissett & Cumming [2004]). In the current study, we did not
train students in the use of argument mapping software per se, but rather in the method of
argument mapping using paper and pencil materials and exercises provided in class.
In summary, thinking is an important aspect of the human experience and consciously
thinking about thinking is necessary in academic settings so that students may assess what
they have to learn and what they already know. Students often encounter arguments in
academic text-books that are difficult to analyse and evaluate due to the way in which the
arguments are presented in text format. In order to promote good CT, educators must help
students to lessen the cognitive load associated with reading and assimilating text. Argument
mapping is a learning tool which may help students in this context. Results from this study
suggest that AM training may increase specific CT sub-skills, such as evaluation and
inductive reasoning. However, the observed effects were weak and methodological problems
in the current study prevent us from drawing any strong conclusions in relation to the value of
AM training.
118 Christopher Dwyer, Michael Hogan and Ian Stewart

REFERENCES
Abrami, P.C., Bernard, R.M., Borokhovski, E., Wade, A., Surkes, M.A., Tamim, R. & Zhang,
D., (2008). Instructional interventions affecting critical thinking skills and dispositions: A
stage 1 meta-analysis. Review of Educational Research, 78, 4, 1102-1134.
Adams, M.H., Whitlow, J.F., Stover, L.M. and Johnson, K.W., (1996). Critical thinking as an
educational outcome: an evaluation of current tools of measurement. Nurse Educator, 21,
23–32.
Allegretti, C.L. and Frederick, J.N. (1995). A model for thinking critically about ethical
issues. Teaching of Psychology, Vol. 22, 1, 46-48.
Allen, R., Feezel, J., & Kauffeld, F. (1967). A taxonomy of concepts and critical abilities
related to the evaluation of verbal arguments (Occasional Paper No. 9). Madison:
Wisconsin Research and Development, Center for Cognitive
Learning.
Alvarez-Ortiz, C. (2007). Does Philosophy Improve Critical Thinking Skills? Unpublished
thesis.
Anderson, L.W. & Krathwohl, D.R. (eds.) (2001). A Taxonomy for Learning, Teaching and
Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York:
Longman.
Association of American Colleges & Universities, (2005). Liberal education outcomes: A
preliminary report on student achievement in college. Washington, DC.
Australian Council for Educational Research, (2002). Graduate skills assessment.
Commonwealth of Australia.
Beckie, Theresa, M., Lowry, Lois, W. & Barnett, Scott (2001). Assessing critical thinking in
Baccalaureate nursing students: A longitudinal study. Holistic Nursing Practice, 15(3),
18-26.
Bensley, D.A. (1998). Critical thinking in psychology: A unified skills approach. Pacific
Grove, CA: Brooks & Cole.
Bloom, B.S., (1956). Taxonomy of educational objectives: The classification of educational
goals. Handbook 1: Cognitive domain. New York: McKay.
Butchart, S., Bigelow, J., Oppy, G., Korb, K., & Gold, I. (2009). Improving critical thinking
using web-based argument mapping exercises with automated feedback. Australasian
Journal of Educational Technology, 25, 2, 268-291.
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the
educative process. Lexington, MA: Heath.
Dwyer, C. P., Hogan, M.J. & I. Stewart, (2010). The evaluation of argument mapping as a
learning tool: Comparing the effects of map reading versus text reading on
comprehension and recall of arguments. Thinking Skills and Creativity, 5, 1, 16-22.
El Hassan, K. and Madhum, G., (2007). Validating the Watson Glaser Critical Thinking
Appraisal. Higher Education, 54, 361-383.
Ennis, R.H., (1987). A taxonomoy of critical thinking dispositions and abilities. In Teaching
Thinking Skills: Theory and Practice, eds. J.B. Baron, and R.J. Sternberg. New York:
W.H. Freeman, 9-26.
Ennis, R.H., (1989). Critical thinking and subject specificity: Clarification and needed
research. Educational Researcher, 18, 3, 4-10.
 The Promotion of Critical Thinking Skills through Argument Mapping 119

Ennis, R.H., Millman, J. & T.N. Tomko (1985). Cornell Critical Thinking Tests CA: Critical
Thinking Co.
Ennis, R.H. & Weir, E., (1985). The Ennis-Weir Critical Thinking Essay Test. Pacific Grove,
CA: Midwest Publications.
Facione, N.E. (1990). The Delphi report. Committee on pre-college philosophy. American
Philosophical Association.
Facione, P.A., (1991). The California Critical Thinking Skills Test: College Level Technical
Report #1 - Experimental Validation and Content Validity. Unpublished.
Facione, P.C. & Facione, N.A. (1992). California Critical Thinking Skills Test. California
Academic Press, California.
Facione, P.A., Facione, N. C., Blohm, S. W., & Giancarlo, C. A. (2002). The California
Critical Thinking Skills Test: CCTST. Form A, Form B, and Form 2000. Test manual,
2002 updated edition. Millbrae, CA: Insight Assessment.
Gabennesch, H. (2006). Critical thinking…what is it good for? (In fact, what is it?). Skeptical
Inquirer, Vol. 30, 2, pp. 36-41.
Glaser, E. M. (1941). An experiment on the development of critical thinking. Teachers
College Contributions to Education, No. 843. New York: Bureau of
Publications, Teachers College, Columbia University.
Halpern, D.F. (2003). Thought & Knowledge: An Introduction to Critical Thinking (4th Ed.).
New Jersey: Laurence Erlbaum Associates.
Halpern, D.F. (2004). A cognitive taxonomy for sex differences in cognitive abilities. Current
Directions in Psychological Science, 13, 135–139.
Harrell, M., (2004). The improvement of critical thinking skills. In What Philosophy Is
(Tech. Rep. CMU-PHIL-158). Carnegie Mellon University, Department of Philosophy.
Harrell, M., (2005). Using argument diagramming software in the classroom. Teaching
Philosophy, 28, 2,
Higher Education Quality Council, Quality Enhancement Group, (1996). What are
graduates? Clarifying the attributes of “graduateness”. London: HEQC.
Holmes, J. and Clizbe, E., (1997). Facing the 21st century. Business Education Forum, 52, 1,
33-35.
Jacobs, S. S. (1995). Technical characteristics and some correlates of the California Critical
Thinking Skills Test Forms A and B. Higher Education Research, 36, 89-108.
Kintsch, W. & van Dijk, T.A. (1978). Toward a model of text comprehension and production.
Psychological Review, 85, 363-394.
Ku, K.Y.L., (2009). Assessing students’ critical thinking performance: Urging for
measurements using multi-response format. Thinking Skills and Creativity, 4, 1, 70- 76.
Kuhn, D. (1991). The skills of argument. Cambridge: Cambridge University Press.
Kurfiss, J.G., (1988) Critical thinking: Theory, research, practice and possibilities.
Association for the Study of Higher Education, Report No. 2, Washington D.C.
Marzano, R.J., (1998). A theory-based meta-analysis of research on instruction. Aurora, CO:
Mid-Continent Regional Educational Laboratory.
http://www.mcrel.org/pdf/instruction/5982rr_instructionmeta_analysis.pdf, 26, October,
2007.
MacPherson, K., (1999). The development of critical thinking skills in undergraduate
supervisory management unites: Efficacy of student peer assessment. Assessment &
Evaluation in Higher Education, 24, 3, 273-284.
120 Christopher Dwyer, Michael Hogan and Ian Stewart

Monk, P. (2001). Mapping the future of argument. Australian Financial Review. 16, March,
pp.8-9.
Moseley, D., Baumfield, V., Elliot, J., Gregson, M., Higgins, S., Miller, J. & Newton, D.,
(2005). Frameworks for thinking: A handbook for teaching and learning. Cambridge:
Cambridge University Press.
National Academy of Sciences, National Academy of Engineering, Institute of Medicine
(2005). Rising above the gathering storm: Energising and employing America for a
brighter economic future. Committee on Prospering in the Global Economy for the 21st
Century, Washington, DC.
Paul, R.W. (1993). Critical Thinking – what every person needs to survive in a rapidly
changing world (3rd ed.). Santa Rosa, CA: Foundation for Critical Thinking.
Quitadamo, I.J. and Kurtz, M.J., (2007). Learning to improve: Using writing to increase
critical thinking performance in general education biology. CBE –Life Sciences
Education, 6, 140-154.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive
Science, 12, 257-285.
Sweller, J. (1999). Instructional design in technical areas. Australian Education Review No.
43. Victoria: Acer Press.
Taube, K.T., (1997). Critical thinking ability and disposition as factors of performance on a
written critical thinking test. Journal of General Education, 46, 129-164.
Thomson, A., (2009). Critical Reasoning: A practical introduction (3rd ed.). UK: Routledge.
Tindall-Ford, S; Chandler, P. & Sweller, J., (1997). When two sensory modes are
better than one. Journal of Experimental Psychology: Applied, Vol. 3, No. 4: 257 -287.
U.S. Department of Education, (1990). National goals for education. Washington, DC.
van Gelder, T.J. (2000). Learning to reason: A Reason-Able! approach. In C. Davis, T. J. van
Gelder, & R. Wales (eds.), Cognitive Science in Australia, 2000:Proceedings of the Fifth
Australasian Cognitive Science Society Conference. Adelaide: Causal.
van Gelder, T. J. (2001). How to improve critical thinking using educational
technology. In G. Kennedy, M. Keppell, C. McNaught, & T. Petrovic (Eds.), Meeting at
the Crossroads: Proceedings of the 18th Annual Conference of the Australian Society for
Computers In Learning In Tertiary Education, 539–
548. Melbourne: Biomedical Multimedia Unit, Ùniversity of Melbourne.
van Gelder, T.J., (2003). Enhancing deliberation through computer supported
argument mapping. In Kirschner, P., Buckingham Shum, S., Carr, C. (eds.), Visualizing
Argumentation: Software Tools for Collaborative and Educational Sense-Making, 97–
115, London: Springer-Verlag.
van Gelder, T.J., Bissett, M., & Cumming, G. (2004). Enhancing expertise in informal
reasoning. Canadian Journal of Experimental Psychology 58, 142-52.
van Gelder, T.J. & Rizzo, A. (2001). Reason!Able across curriculum, in Is IT an Odyssey in
Learning? Proceedings of the 2001 Conference of ICT in
Education, Victoria, Australia.
Watson, G. & E.M. Glaser (1980). Critical thinking appraisal. New York:
Psychological Corporation.
Wilkinson J.M., (1996). Nursing process: A critical thinking approach (2nd ed.).
Menlo Park: Addison Wesley Nursing.
 The Promotion of Critical Thinking Skills through Argument Mapping 121

Willingham, D.T., (2007). Critical thinking: Why is it so hard to teach? American Educator,
2007, 3, 8-19.
In: Critical Thinking ISBN: 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 123-140 2011 Nova Science Publishers, Inc.

Chapter 5

BEYOND GDP? TOWARDS A NEW SYSTEM

OF SOCIAL ACCOUNTS

Frédéric Lebaron*
Université de Picardie-Jules Verne, France

By asking in February 2008 Joseph Stiglitz, Amartya Sen, Jean-Paul Fitoussi and a group
of economists and of social scientists to reflect on new measures of "economic performances
and social progress" to make up for the inadequacies of the GDP, the President of the
Republic has contributed to legitimise an already established undertaking whose aim is to
build "alternative" well-being indicators, whose visibility had increased thanks to the Human
Development Indicator (HDI) of the United Nations Development Programme, at the
beginning of the 1990s and accelerated during the recent period with the work conducted in
the OECD, the European Commission, in think tanks and in the academic world, especially
around what is now called "the economics of happiness".
The strong echo of the Stiglitz Commission, whose published report gave rise to an
official and quite mediatic launch campaign, has heightened certain tensions around the
project of new "well-being", "economic performances" or still "sustainability" indicators: is
the purpose for example to extend the principle of "monetisation" to the non-merchant sphere
to integrate the "damages of progress" or to relativise the monetary measures of wealth to the
benefit of other counting units?
This contribution questions some of the issues of the Stiglitz Commission and, more
widely speaking, addresses the various facets of the current movement aiming to promote
"new wealth Indicators". We shall focus more particularly on the divergences running
through that space and on the possible resistances raised by such a movement in various
sectors (public statistics, the social movements, the world of professional economists, etc.).

*
Frédéric Lebaron is professor of sociology at the Université de Picardie-Jules Verne, the director of the Centre
universitaire de recherches sur l’action publique et le politique – épistémologie et sciences sociales and a former
member of the Institut universitaire de France (IUF). He has recently published La crise de la croyance
economics (the Crisis of economic creed) (coll. « Dynamiques socio-économiques » at the Editions du
Croquant), and is preparing a book on social indicators (coll. « Topos plus », Dunod). He supervises the
"Alterindicateurs" column in the Savoir / Agir review, whose publication director he is.
124 Frédéric Lebaron

The report of the "Commission on measuring economic performances and social

progress", so-called "Stiglitz-Sen-Fitoussi commission" or more often "Stiglitz commission",
appointed by President Nicolas Sarkozy in February 2008 [1],underlines the belated
consecration of the quest for better wealth and performance measures which had come to light
as soon as in the 1970s, in France with the "social indicators" promoted by Jacques Delors
[2], and had gained in visibility at the beginning of the 1990s with the launch of the human
development indicator by the PNUD [3]. When the report was published in September 2009
[4], then in the form of two books published by Odile Jacob Pub. in November 2009 [5], high
French and foreign political, administrative officials as well as worldly recognised famous
scientists concluded that in-depth transformation of the statistical recording method in the
socio-economic and environmental fields was necessary. The idea is to overcome the limits of
the current measuring systems, by modifying "performance" measuring modalities which are
considered as basically faulty, legitimately disputed and unsuitable for the contemporary
stakes.
The limits of the Gross Domestic Product (GDP), most of them known for a long time
and sources of an already flourishing [6] scientific literature, are the starting point of this
observation: centred on production, this indicator describes the incomes and their evolution
increasingly poorly, in particular due to the globalisation of economies; "defensive"
expenditures such as the reconstruction of pollution-damaged environment or prison expenses
are then considered as contributions to wealth; households' production, which is non-
monetary to a vast extent, is not taken into account; the production of the non-merchant
sector, assessed by the production costs, is poorly measured; its calculation relies on various
by partial "inputations" (a housing expenses is attributed to owner-households); it is based on
the notion of average and not of that of variance.
According to the Commission, all the national accounting systems set up after the second
world war must hence be revamped thoroughly. The public statistical institutions are bound to
implement, relatively in the short-term, noticeable changes in the intellectual architecture on
which they have been built: their investigation devices, their measuring instruments and other
indicators must thoroughly evolve to correct the just aforementioned defects.
In terms of public policies, this change must enable better assessment of the economic
and social performances at different levels and thus promote the implementation of more
appropriate strategies, on a par with the citizens' expectations. The public "decision makers"
themselves must grab hold of innovations and apply them in their ordinary concrete activity:
they form the explicit target audience of the report.
The particularly ambitious and grandiloquent speech by the President of the Republic on
the launch day contributes to granting an official character to said change:

"A formidable revolution lies ahead of us. Each of us is now fully aware of it. This
revolution will only take place if nurtured in our minds. (...) Statistics, accounting, reflect our
aspirations, the value we ascribe to things. They are indissociable from a Weltanschauung,
from economics, from society, from an idea of man, from his/her relation to others. To
consider them as objective data, external to ourselves, which are unquestionable and
indisputable is undoubtedly reassuring, comfortable, but surely dangerous. (...) For many
years, people whose life was becoming increasingly difficult were told that their standard of
living was increasing. How should they not feel led astray? (...) For a number of years, the
statistics have pointed out an increasingly high economic growth as a victory over shortage,
until said growth appeared to destroy more than it created, by jeopardising the future of the
 Beyond GDP? Towards a New System of Social Accounts 125

planet. (...) Behind the religion of numbers, behind the whole edifice of our statistical and
accounting representations, also lies the religion of market which is always right (...) If the
market had the right answer for everything, it would be common knowledge. If the market
were never wrong, everyone could see it. (...) The market is not meaningful. It does not equate
with responsibility. It does not equate with project. It does not equate with vision. The
financial markets even less. » [7].

Box: A Multifaceted and Long-Term Movement

The movement of the "alternative indicators" (according to one of the labels currently used)
both corresponds to a work completed in certain public bodies, NGOs and by researchers
regrouped in independent centres. It has grown in successive "waves" in scattered sectors,
without coordination nor global coherence, without common denomination and with a rather
small degree of institutionalisation.
The notion of "social indicators" crystallised in the 1960-70s in the United States, primarily
within the academic world, with the publication of several books and then as of 1974, of a
journal entitled Social Indicators Research. Jacques Delors and a group of Ecole nationale
d’administration (ENA) students imported this scientific movement to France in 1967-1968, by
endeavouring to suggest "social indicators" within the framework of the French plan, within
which Jacques Delors was a civil servant. This politico-administrative guarantee confers
mediatic visibility to the movement, even if the latter retracts from public space with the 1973-
1974 crisis. The movement is then again mainly discussed in academic reviews. The human
development indicator no sooner appeared within the United Nations Development Program
(UNDP) than in the beginning of the 1990s, after years of internal discussions about the choice
of the best development indicators and of a necessary synthetic measure, to which partake
economists such as Mahbub ul Haq and Amartya Sen.
At the end of the 1990s, independent companies emerged at the intersection of universities
and certain NGOs. In France, the BIP40 is one of the most known, and also of the most
criticised (especially by the official public statistics). In Canada, the Centre for the Study of
Living Standards (CSLS) enjoys great success with the Index of Economic Well-Being (Andrew
Sharpe and Lars Osberg), often presented as quite "innovating", in particular because it
emphasises the notion of "economic insecurity" among other dimensions. Researchers and
associative actors get together. The Organisation for Economic Cooperation and Development
(OECD), the International Labour Organisation (ILO), to a lesser extent the World Bank, join
the UNDP in a movement which henceforth impregnates certain of the main international
organisations, within which the neo-liberal discourse has got the upper hand. The OECD
published its first social indicators in 2001, but in the form of a dashboard and not of a synthetic
indicator. The national public statistics organisations are now being concerned as well.
Parallel to this movement limited to the most "critical" fringes of international
organisations, the "economics of happiness" develops in the world of academic economists, in
the wake of the work of Easterlin. They are mainly centred on the study of the relations between
wealth (GDP) and well-being or "happiness". The turning point towards the economics of
happiness of the academic research became more obvious in the second half of the 2000 and has
further somehow marginalised the most prominent research work in social sciences in the 1990,
such as the attempts originating from NGOs (in France, the BIP), or independent centres such as
the Institute for innovation in social policy (Miringoff, United States) or the Centre for the
Study of Living Standards (CSLS, Canada) [15].
126 Frédéric Lebaron

For the President of the French Republic, the report is hence the opportunity of a true
"cultural revolution", leading to break away from exclusively merchant measures that the
crisis would have ruled out definitely. He also states that

"France will open everywhere the debate on the conclusions of this report (...) It will
include the study of this report in the syllabus of all the application schools of its public
service [8]."

The decision to mandate Joseph Stiglitz [9], alongside with Amartya Sen [10] and Jean-
Paul Fitoussi [11], to make a collection of proposals in this area, can be considered as the
somehow paradoxical victory of a multifaceted movement, which some years ago still seemed
to be doomed to remain neglected by institutions, before asserting themselves successively, to
a limited extent, in different international organisations (United Nations Organisation,
Organisation for Economic Cooperation and Development, World Bank...). The crisis has
contributed to strengthen the promoters of this statistical "renovation" even more: the
°
introduction to the report [12] explicitly indicates that "certain members of the Commission
believe that the latter makes [their recommendations] all the more urgent [13]". It is in
particular what the President of the Republic expresses in his preface, who states that the
crisis reveals the inadequacy of current statistical tools. The text published by the three
economists to outline the philosophy of the report (entitled "Essay on measuring") goes even
further along those lines, considering that the crisis is even partially the product of deficient
measuring instruments, a thematic which falls into a vaster movement of criticism of the
.
accounting [14] and financial Indicators

THE OUTLINES OF THE REPORT: AMBITION AND TECHNICITY

The text of the report combines two features which may seem contradictory at first view:
ambition and technicity. In more than one way, it comes across as a "manifesto" and contains
a strong programmatic dimension. The recommendations and numerous formulations here
and there vigorously claim for new practices: "a major effort should also be conducted..."; "...
should be taken into account"; "... must be taken into account"; "allocate statistical means to
the fields wherein the available Indicators remain insufficient", etc. If the demands addressed
to the statistical devices remain relatively vague, they are nevertheless numerous, involve
significant public investment and are presented as major inflections which should allow for
noticeable social and political changes.
On the other hand, the report is both long (the French version entailed 324 pages of a
very dense text [16]) and often technical, let alone esoteric, juxtaposing concepts and
indicators borrowed from sometimes contradictory or still quite distinct theoretical
frameworks. Numerous developments, in particular in the first part dedicated to the GDP,
pertain to improving national accounting [17]. As a bureaucratic-scientific project developed
after the war, confronted with many issues (or "anomalies"), national accounting was hence
supposed to go from strength to strength and grow steadily so as to integrate the response to
said issues, but sadly lacking simplicity and coherence. Without giving up totally on using the
GDP, the report recommends for example increased usage of actual household incomes and
consumption, as many items already present in national accounting, but still poorly measured.
 Beyond GDP? Towards a New System of Social Accounts 127

Its purpose is hence to focus the representation of economy on actual consumption and
incomes, including on their unequal distribution (which however comes second, after the
"central tendency" measure), aspects that a representation focused on sole global production
should smooth out. This first point partially challenges a "productivist" vision, which has
hitherto innervated the different streams of political economy and of economic thinking, but
to the benefit of a conception which remains monetary and partially "consumerist" or
"utilitarian".
Insisting repeatedly on the principle of equivalence (an identical service for a household
for example should not be accounted for differently according to the public or private legal
status of its provider), the report appeals for better measure of non merchant services (in
particular healthcare or education) thanks to which households enjoy a well-being which is
for the time being vastly ignored by official statistics. Along the same line of thought, the aim
is to develop patrimony accounts at all levels and to submit the balance sheets thus prepared
to "resistance tests" based upon various exploitation hypotheses, "where there are no market
prices or when said prices are subjected to erratic fluctuations or speculative bubbles [18]."
The difficulties associated with the monetary measure of patrimony, in an extended meaning,
hence open up immense perspectives for national accountants. The vision developed in the
report confers greater significance to non merchant public services and to the wealth they
generate, to the domestic activity and to all the stock variations that affect the activity in the
long run. It hence offers more integrated representation of the whole social activity, less
strictly "economic" or "economicist", a vision which corresponds more to that long-developed
vision in social sciences (sociology, anthropology, political science, etc.).
As regards the "quality of life", which is the subject matter of the second part, the
developments bearing upon the three conceptual frameworks presented as the most "useful"
for analysing this issue (measures of the subjective well-being [19],"capabilities" and "fair
allowances" theories) refer to the corresponding theories or economic work, which are
represented within the Commission by their main promoters. The report attempts to convince
the reader of their "operational" usefulness. The methodological difficulties and the lack of
reliable data are, and the report readily admits it, the main obstacles to scientific knowledge in
the field: quarterly or a fortiori monthly homogenous data bearing upon work conditions or
subjective well-being are not or hardly available. The report thus recommends the
multiplication of investigations for measuring realities hitherto remained little objectivated: a
concern prevailing since the beginning of the movement of social indicators is a similar
instance. Another element to be noted is the report focusing on citizens' subjective perception
of the ordinary activity, as measured by surveys on the timetable which integrate attitude
issues, along the lines of the work of Kahneman on the "U index" ("unpleasant"), a
dissatisfaction indicator constructed on the basis of a detailed subjective assessment of daily
activities.
The indicators presented in the third part dedicated to "sustainability" are abstract
intellectual constructions akin to sophisticated national accounting methods, as is the case of
net adjusted savings (NAS), derived from the work of the World Bank, especially from
research conducted by "Nobel" Kenneth Arrow, a member of the Commission. In a 2004
report, the Bank evaluates the "net internal savings" for the quasi-totality of countries in the
world [20]. The starting point is gross savings measured by national accounting, from which
the capital consumption of generated assets can be deduced, which allows to obtain net
savings. Said value is added the educational expenses, measures of human capital investment.
128 Frédéric Lebaron

The depletion of various natural resources can be assessed, on the basis of the profit derived
from said resources, and the damages resulting from global carbon dioxide pollution can be
estimated. The data obtained from the NAS are, little intuitively at first glance, quite
favourable to China, where gross savings are very high. The variations in NAS do not provide
radically different estimates from gross savings. Especially, the aim of this indicator is to
monetise the notion of sustainability, which proved to be one of the most problematic and
controversial items of the work of the Commission and of its reception. Between the draft and
the final version, the wording has been smoothed out and attenuated, due to quite critical
public reactions from certain members, influenced by “decrease” theories and the ecology
stream.

Box 1. The Recommendations of the Stiglitz-Sen-Fitoussi Report

Recommendation 1: When assessing material well-being, one should refer to incomes

and consumption rather than production.
Recommendation 2: emphasise the household perspective
Recommendation 3: consider income and consumption jointly with wealth
Recommendation 4: give more prominence to the distribution of income, consumption
and wealth
Recommendation 5: broaden incomes indicators to non-merchant activities
Recommendation 6: the quality of life depends on the objective conditions of people and
on their "capabilities" (dynamic capacities). Steps should be taken to improve measures of
people's health, education, personal activities and environmental conditions. In particular,
substantial effort should be devoted to developing and implementing robust, reliable measures
of social connections, political voice and insecurity that can be shown to predict life
satisfaction.
Recommendation 7: quality-of-life indicators in the dimensions covered should assess
inequalities in a comprehensive way.
Recommendation 8: surveys should be designed to assess the links between various
quality-of-life domains for each person, and this information should be used when designing
policies in various fields.
Recommendation 9: statistical offices should provide the information needed to aggregate
across quality-of-life dimensions, allowing the construction of different indexes.
Recommendation 10: measures of objective and subjective well-being provide key
information about people's quality of life. Statistical offices should incorporate questions to
capture people's life evaluation, hedonic experiences and priorities in their own surveys.
Recommendation 11: sustainability assessment requires a well-identified dashboard of
indicators. The distinctive feature of the components of this dashboard should be that they are
interpretable as variations of some underlying "stock". A monetary index of sustainability has
its place in such a dashboard; still, under the current state of the art, it should remain
essentially focused on economic aspects of sustainability.
Recommendation 12: The environmental aspects of sustainability deserve a separate
follow-up based on a well-chosen set of physical indicators. In particular, there is a need for a
clear indicator of our proximity to dangerous levels of environmental damage (such as
associated with climate change or the depletion of fishing stocks).
 Beyond GDP? Towards a New System of Social Accounts 129

Many critical diagnoses are contained in the report: for example, the inability of the GDP,
focused on merchant assessment, to measure not only well-being, but even economic
performance, which net domestic product, or even better, actual household incomes, can
approximate with greater accuracy. The American counterexample used by Stiglitz (an
increasing GDP whereas households cannot see their incomes rise) is the most striking, and it
is also highly symbolical: the "American model" has thus lost its dominating status
explicitly. Similarly, the depiction of the blatant limits of public statistics in terms of
objectivation of the quality of life: lack of surveys, of comparable data, of measure of the
positive or negative time perception as experienced by individuals, etc. Finally, the little
satisfactory, contradictory and poorly unified character of the set of current sustainability
indicators, which cannot only be monetary, but must also be "physical" and which should
measure variations in stocks so as to come closer to the idea of sustainability: the current
consumption should not affect the future consumption possibilities. The report, on this score,
leads to a moderate and pragmatic vision, by advocating limited usage of physical and
monetary indicators.

THE LEGITIMACY OF THE COMMISSION OR THE BOUNDARIES

OF "EXPERT" KNOWLEDGE

Announced at the beginning of 2008, the composition of the Commission proved to be a

major stake, expressed publicly on several occasions by most critical commentators. The
Stiglitz commission indeed includes economists who are admittedly rather "critical" against
mainstream thought and especially neoliberalism in its more radical version, but consecrated
by international academic institutions [21] ; internationally recognised neoclassical
economists who have enriched the well-being theory or to national accounting, and officials
in charge of national and international statistical organisations; finally a few economists,
closer to other social sciences, who have contributed to original empirical work in the field. It
has mobilised members of national (Institut national de la statistique et des études
économiques, INSEE, Observatoire français des conjonctures économiques, OFCE) and
international (OECD) bodies, involved in the reforming movement, as "drafters".
The Commission also illustrates, as this has been far less emphasised, the outstanding
American domination in economics (of happiness), which has become overwhelming in the
contemporary French context. Out of the 24 members of the Commission, 12 hold a position
in an American university, among whom 3 in Princeton, 2 in Chicago and 2 in Columbia. 5
are economics "Nobel" prize winners. The Commission has hence mobilised strong academic
symbolical capital associated with the United States, representatives of statistical
organisations and with recognised expertise: it is in the heart of "international" economic
science that the signal of statistical revolution originates, although prone to multiple internal
tensions can be seen between the lines in the report [22].
The first implicit exclusion relates to human and social sciences. Psychology (apart from
punctual mentions), sociology, demography, political science, let alone history [23],
geography or anthropology, are left aside to the benefit of the national accounting / economic
theory combination, supposedly at the core of the stakes and issues of well-being and socio-
economic performance, which means implicit bias in favour of an utilitarian vision, regardless
130 Frédéric Lebaron

how moderate, of well-being and especially of accounting monetisation, especially in terms of

environment. This contributes to giving more technician and anhistoric vision of the issue,
centred on the way that national accounting and theoretical economics could develop in the
future in new directions [24] and thus fuel the work in this field. It is one of the aspects of the
symbolic "captation" and "closing" due to the creation of the Commission: it restricts a set of
intrinsically pluridisciplinary research issues to the domain of economic science, which
causes favourable bias to both utilitarian and monetary representation of well-being and
sustainability. Constantly reasserted, the measure and the explanation of (economic, social,
environmental) inequalities are thus pushed in the background de facto.
The second exclusion concerns actors of the "civil society" some of whom, quite
variably, have still racked their brains for some years around the issue of alternative
indicators, in particular by constructing indicators [25], databases [26] or by making radical
propositions in this respect, sometimes with strong mediatic and political echo. Still, the
"participative" dimension of the project of "new wealth indicators" has often been put forward
in the numerous social debates on the subject [27]: it is a democratic mobilisation project as
well as a scientific project. The French BIP40, for instance, is not mentioned when the
synthetic poverty and inequality indicators are evoked, in the part dedicated to "quality of
life". The FAIR forum has formed, precisely, around Jean Gadrey, to offer a democratic
approach to the preparation and the choice of indicators. More generally, the constructions of
synthetic indicators are contested for the sake of resorting pragmatically to "dashboards",
which in fine translate in "sets of indicators" (which corresponds to the leadership within
official, national and international statistical organisations [28]). Some authors wish that the
"non-merchant", especially associative, world were more emphasised in the Stiglitz report,
contrary to what authors such as Patrick Viveret or Bernard Perret, associated with the
"second left" (in particular in the Esprit journal) and with the promotion of the "third sector",
used to do a few years ago.
Other authors insist, in the wake of Dominique Méda, on the feminine under-
representation also conspicuous within the Commission, which is the third objective
exclusion, characteristic thereof. Patrick Viveret, answering a question on the
www.terraeco.net site, clearly asserts such a posture: "it is important that a commission with
economists of this calibre should recognise the well-foundedness of the criticisms addressed
to the GDP for several years; but which remained associated with alternative sectors by
public opinion. It tells us that indicators have made us blind and underlines the need to
measure ecological and social sustainability. The point is that this commission has worked
exclusively between economists and statisticians, with few ladies, whereas feminine sensitivity
is needed on the activity-inactivity theme [29]."
Finally, the discussion on sustainability indicators is a far cry from the dominant
environmental specialists, linked with natural sciences, biologists in particular. Still, the
former have already embarked on intense scientific struggle around the construction of
durability Indicators and are involved in quite numerous researches on damage to natural
environment.
The commission finally appears as a relatively confined space, centred on the dominant
academic economic science and on the official statistical devices [30]. This explains that a
portion of the criticisms made thereto precisely pertains to its composition and its intrinsic
limits. such is the case for instance of the criticism made by Dominique Méda, a graduate
 Beyond GDP? Towards a New System of Social Accounts 131

from Normal Sup, a philosophy professor, a member of the Inspection générale des affaires
sociales and senior scientist in the Centre d’études de l’emploi, who explains: "The
commission only included two ladies and a vast majority of economists who worked behind
closed doors, without ever meeting with the civil society. It refused to query the way citizens
might decide together what is really important to them. Furthermore, it left significant room
for very individualistic approaches. I refer to the quality of life measured by individual
satisfaction or to indicators which are vulnerable to criticism, such as net adjusted savings.
On the other hand, the commission has firmly opposed the idea of GDP-alternative or
complementary synthetic indicators. It is a pity. Finally, its recommendations have not been
implemented for the time being, or in homeopathic doses, and apparently they are not on the
agenda any longer today" (an interview to L'Hebdo, by Philippe Le Bé, put on line on 16 June
2010). This social confinement, with its consequences on the generated and diffused
"contents", may however be seen as a condition of the political and social efficiency of the
commission, inasmuch as the political leaders have obviously attempted to mobilise a
maximum of scientific capital, a particular form of symbolic capital.

IN THE MIDST OF RADICAL REJECTION, SCEPTICISM, QUALIFIED

CRITICISM, PRAGMATIC ACCEPTANCE AND PROSELYTISTIC
ENTHUSIASM: A MIXED RECEPTION
If the quite mediatised publication of the report has given rise to numerous comments,
they originate in the first place from the usual actors of the political game, party speakers,
journalists and other "generalists" news analysts. The main newspapers, just like the political
parties, nevertheless cover the event rather obligingly, with limited effort to politicise the
contents (see text box 2 on the communiqué of the PS.) Libération even invites Joseph
Stiglitz, whose democrat orientations and criticisms of neoliberal policies are well-known, to
comment upon current affairs in its pages on 15 September 2009.
The range of attitudes, as can be described sketchily for want of a (underway)
quantitative study, reveals vast interpretive amplitude, from radical rejection to proselytistic
enthusiasm via all nuances and "viewpoints", all possible tones, on the report: as in numerous
cases, the announcement also functions like a "projective test" which sometimes tells as much
about commentators as about the study object. Certain analysts thus interlace their
presentation of the report with their own ideological conceptions: Le Figaro, who was privy
to reading the report, outlined on 11 September a legitimisation of "individualistic"
approaches, which triggered a harsh reaction from Jean Gadrey, who embodies the "critical"
wing of the commission [31]. Christine Boutin, on the right of the political spectrum, seizes
the issue of the criticism of the GDP. In an official commentary, "Christine Boutin, the
president of the Christian-Democrat Party (PCD), in the wake of the Copenhagen climate
summit, considers the reformation of the GDP as a "matter of urgency" so as to incorporate
"the environmental and human capital", as suggested by the Stiglitz report. "Among the
various causes of failure" in Copenhagen she notes, in a communiqué, the "unfounded fear of
the signatory countries to have to curb their growth and hence to become poorer" (...), "the
blatant inability of the people in power" to measure their increasing wealth "other than
through the GDP" which is but a "productive activity indicator". If France adopts a "double
132 Frédéric Lebaron

measure of its growth", one in compliance with international classifications, the other taking
into account the social and environmental dimensions of the economic activity, it will be "at
the forefront of the struggle to protect the planet and to eradicate poverty", Ms Boutin
concluded". Some appeal for the use of happiness indicators, after the "model" practiced by
the State of Bhutan since the 1970s with "gross national happiness", incidentally mentioned
quite often in magazine commentaries as illustrating these "alternative indicators": this rather
"conservative", possibly "traditionalist" monarchy, defends a hostile position to the
development of tourist industry and uncontrolled globalisation. On the whole, the reactions
nevertheless seem to be more numerous and more favourable on the left than on the right of
the politico-mediatic spectrum.

Box 2. The Socialist Party And The Stiglitz Report

Committed to an undertaking of doctrinal renovation around the notion of "care", the

Socialist Party harshly reacts to the publication of the Stiglitz report, by supporting its goals
while condemning current public policies and by mobilising an indicator, the "gross domestic
happiness", calculated by the Centre for the study of living standards, which is clearly
favourable to the "plural left" government policy. "The conclusions of the Stiglitz report reflect
the rather wide consensus today on the necessity to have economic indicators which are not
limited to measuring the production of goods and services. For enlightened citizens'
information, democratic debate, experts work, political choices, items such quality of life,
environment, health, culture, quality of public services or still inequalities in plain and simple
synthetic indicators must be considered. The Socialist Party demands their implementation and
eagerly anticipates their revelations. Indeed since 2002, the right-wing policy has accumulated
regressions, which is clearly shown by existing well-being indicators. The "gross domestic
happiness" index calculated by the Centre for the study of living standards (Canada) reached its
peak in France in 2002, after an exceptional growth period between 1997 and 2002, and has
decreased continuously since then. Is it any wonder? (...). as usual, the words of Nicolas
Sarkozy totally contradict his actions. His whole policy defeats the logic associated with well-
being indicators in place. And it is particularly shocking to hear him suggest to put an end to
"the religion of numbers", whereas he was the one to impose that "religion", in particular for the
security policy, with many perverse effects. Considering well-being in all its dimensions and
the necessity to be done with outrageous consumer society lies at the heart of the socialists
project and of the civilisation offensive launched at the Summer University in La Rochelle. ".
Communiqué of the Socialist Party, September 2009.

Reactions, in particular, accumulate on newspapers Websites and on "blogs": 54

"reactions" for instance on the Figaro site, where, alongside with asserted supports, more than
one internaut is extremely critical against the report, against Joseph Stiglitz, sometimes
against Nicolas Sarkozy, and in some cases point out a contradiction between the critical tone
of the report and the orientations of the government in power, also emphasised on left-wing
newspapers websites. Some scepticism can be felt as regards the spirit of the report. For
Fanch12,

"the temperature is either too high to too low, who cares, change thermometer! As
feelings today have superseded reality, we shall feel less warm or less cold all of a sudden.
Fantastic, is it not? (Figaro site, 11/09/2009). For Sacripant, "able to transcend his ideological
 Beyond GDP? Towards a New System of Social Accounts 133

preferences and to call upon Stiglitz who makes no bones about his political preferences(at the
antipodes of Nicolas Sarkozy's) and then capable of ignoring the recommendations he might
have advocated (see Attali), and finally to carry on with sudden insights unseating his own
camp. Nicolas Sarkozy, a potential great reformer, but constantly held on a tight lead by the
daemons of demagoguery, electoralism and communication. What a pity!" (Figaro site,
11/09/2009).

The 94 reactions to the article announcing the Stiglitz report on the Libération site are
also often negative, since they often first and foremost address the President of the Republic
and "the right", again accused of "breaking the thermometer", of pursuing an "ideological
scrambling" strategy", etc. Scepticism about the spirit of the report is just as assertive: "Well,
one may wonder how they're going to measure such a relative value as well-being! A first-
class hoax! We shall rise by 10 degrees at least on the Sucker scale" (professeurMaboul -
NuttyProfessor, Libération site, 14/09/2009). The spontaneous politicised way in which the
report was received outlines the ambiguity of its significance: it is perceived, and often
denounced, as a political "feat" pulled off by the President of the Republic, a simple
communicational "soliciting" operation. "I would like to add to my previous comment that Mr
Sarkozy has a gift for misinterpreting the reality of things.

On the brink of the biggest economic and financial crisis of the century, Mr Sarkozy was
preaching for full employment, full blast economics, etc. etc. (al of us were full blast
economics). Not once could he forecast that crisis. This time again, with an oxygen mask on
his nose, he offers a "gross national happiness" and ideas worthy of an exalted. We're going to
relapse, Mr President, and you still don't catch the tide of the way history is going. Once
again, you will of course save the world and the whole universe. But you might be well-
inspired to take the appropriate measures before the situation really stinks. "We the people do
beg you" ("onvouslavaitdit - wehadtoldyouso", Libération site, 14/09/2009).

The propositions of the Commission sometimes tie in with alterglobalist criticism.

"It would pay to listen to alterglobalists. They have been saying for a long time that GDP-
based calculation of wealth is an aberration. GDP measures partly false things. For example,
a storm destroys and conditions reconstructions which raise the GDP, whereas everything was
destroyed. It does not take the weight of education into account, since knowledge-induced
wealth is difficult to assess, because it takes a while to realise. It is contrary to neoliberal
short-term conception. » (Robert, Libération site, Monday 14 September).

The press, sites and blogs [32] offer numerous supporting attitudes, which may be
unconditional, marked or more nuanced [33]. The "movement" is launched officially, which
strengthens the actors mobilised. At the same time, the national and international statistical
offices will play a decisive role for the future of the propositions put forward by the
commission.
The level of consensus on the stakes and the significance of the report appears rather low,
which confirms the assumption of notable and generalised cognitive disturbance, as well in
political actors (diagrammatically, the left supports the "spirit" of the report by blaming the
government, whereas the right is divided between those that follow the government and the
most critical, who abruptly reject what appears to them as a "severe ideological blow") as in
journalists and internauts, as many unconstrained interpreters who deliver a portion of actual
political and symbolical usages which can be derived therefrom.
134 Frédéric Lebaron

Box 3: "Eco is Facts" or Head on Clash Against Profane Neoliberalism

On the Figaro site, the reactions to the slightly early announcement of the publication of
the report are numerous and sometimes hefty. The reaction of that Internaut expresses a
"neoliberal" ideological posture which can be found on numerous occasions and which
crystallises a number of discursive features, as invoking economic discipline as a source of
scientific truths omitted in the report, or still the "concrete" character of monetary measure.
"[HDI] is not an economic performance measuring instrument. To what extent is life
expectancy characteristic of eco-performances? Still, it is included in the HDI... HDI
represents a subjective vision of what Good looks like for a nation whose main objective it to
have well-fed citizens regardless of what they do. (...). Economics is not poppycock, it's not
simplistic, it's not so complicated either. Eco is FACTS. When you're loaded with it, there are
two possibilities. Either you've stolen it or you've got someone who found it judicious to give
you money in exchange for something he deemed worthy of that price. That's it. Anything
else is crap. Either you earn money without any constraints or you extort it. This is concrete.
The rest is Byzantine considerations on "right, but what shall we do with this money is not
ours (yet)?" Enough of governments' mass extortion led insidiously by "higher aspirations"
which are always Trojan horses for the cupidity of the men in power who capitalise on poor
people's jealousy. (...) Who creates wealth in France? The CAC 40 or Healthcare? Small and
mid-sized companies or the ministry of economy? Growth IS an imperfect indicator, but it is
the best we have because it is concrete: How much dough have we generated? Obviously,
currency does not represent all the value, but it still may represent the whole merchant value.
And merchant value is what we are willing to exchange voluntarily. The rest, taxes, etc. is
akin to foreclosures (with or without cooperation). It is wealth too, but wealth acquired by
force, whereas capitalists' wealth you have in mind is a created wealth. (…). As Stiglitz is
knowingly a NEW KEYNESIAN, he is diametrically opposed to neoliberal theses".

THE SPACE OF "EXPERT" POSITIONS

In the world of economists, the legitimacy of the Commission, with its five "Nobel" prize
winners, imposes relative silence on the French academic elite. The Paris School of
Economics however organised in March 2010 (within the framework of the Master of the
Public Policies and Development Ecole Normale Supérieure) a conference around the results
of the commission, with François Bourguignon, Marc Fleurbaey (members of the
commission), Denis Cogneau. No economist in the Toulouse School of Economics,
traditionally with predominantly neoliberal orientation, has weighed in visibly in the debate.
Most "moderate" neoliberal economists hardly step in, but the council of economic analysis,
predominantly neoliberal for several years, was referred to in 2010 by the President of the
Republic and German Chancellor Angela Merkel about the request for creating a set of 20
well-being and sustainability indicators. Christian de Boissieu, president of the Council of
economic analysis (official advisers of the prime minister), defends this project publicly,
which is in line with the Stiglitz report. The ultraliberals are fairly isolated in view of an
apparently overwhelming position, since it encompasses a spectrum of "streams" ranging
from critical sociologists to official economists.
 Beyond GDP? Towards a New System of Social Accounts 135

It is among heterodoxical economists and socio-economists that stances, which can be

characterised generally as a support embellished with criticisms, are the most visible. Jean
Gadrey, quoted enormously and repeated on a multitude of websites of associations, political
parties, newspapers (in the first place of course the monthly newspaper Alternatives
économiques which hosts him) expresses this critical support. He is significant of a current
stance in the promoters of synthetic well-being indicators, among whom Dominique Méda,
who participates with him in the FAIR forum, Florence Jany-Catrice (senior lecturer at the
university of Lille) or still Patrick Viveret can be mentioned, followed by many other authors
and organisations abroad. This "internal criticism" has materialised rather rapidly during the
work of the commission, after a fit of temper against its composition, which led Dominique
Méda to contest it publicly. An Emeritus economics professor in Lille 1, Jean Gadrey has
been since the beginning of the 2000s one of the main promoters in France of the new wealth
indicators movement. He has recently developed his critical and radical analyses, increasingly
inspired by the thematic of "decrease", via the FAIR forum (and in Alternatives
économiques). In a text published on the FAIR forum with Dominique Méda, Jean Gadrey
developed in June 2009 [34] a nuanced criticism of the Stiglitz report, which will operate as a
reference stance in a "non-official" sphere of influence: he maintains that fair diagnoses can
be found therein, as well as interesting but timid propositions, and at the same time the
expression of the stranglehold of economists and of a dangerous "monetisation" volition of
indicators, especially in terms of environmental sustainability [35]. His criticisms focus
around this third aspect incidentally, which reflects the growing intensity of the stakes around
public policies in terms of environment and their economic consequences. These stances are
widely shared by the organisations and increasingly numerous initiatives around these stakes.
As regards sociologists, reference, although significant, to social inequalities (including
inequalities between socio-professional categories) does not raise overflowing enthusiasm,
since it is again "captured" by economists and relegated to a secondary and corrective role.
The report admittedly remains mostly "economic", whether in the first or third part, with a
focus on "performance" measures, and the "social indicators" issue as that of economic
performances or sustainability, has not mainly been treated through the prism of social
inequalities, as often put in the foreground by social sciences. In the same manner, the
authors' blind faith in "subjective" indicators, as measured quite roughly by answering
"satisfaction" questions can only raise cautious reactions in specialists of questionnaire-based
investigation methods, who know that answers to questions not always have a single
meaning. Finally, monetisation of environmental damage measuring comes across as a
dangerous technical sophistication since it may strengthen the "artificial" market construction
processes.
Rejection stances nevertheless are voiced among economists, including those called
"heterodoxical". Such is the case first of all of the Marxist and Keynesian economists most
bent on "value-added sharing" (hence GDP sharing). Christophe Ramaux, a Keynesian
economist, a senior lecturer in Paris and a member of the Left Party, defends the GDP as a
productive activity measuring instrument in an article in Politis published in July 2009,
entitled "Long live the GDP! But not on its own": although ascribing an "anti-capitalist"
character to the GDP(by officially integrating non-merchant services) and reminding that we
are dealing here with a convention based on an irreplaceable monetary measure for
macroeconomic analysis, Christophe Ramaux still acknowledges the relevance of other
indicators. He criticises nevertheless the limits of the "ecological footprint" as an
136 Frédéric Lebaron

environmental indicator which is advocated in the report and more widely speaking in the
public space. Michel Husson, a former economist of the Revolutionist Communist League
(LCR in French) and close to the "critical left" (he is a chronicle writer in the Regards review
supervised by Clémentine Autain), an administrator of the INSEE and an economics
professor shows, in an ironic note published on his personal web site, that the subjective
happiness indicators advocated vigorously by the Commission are highly correlated with the
logarithm of the GDP, which leads him to relativise the scope of the "alternative" developed
by the commission and to repeat a "productivist" stance often encountered in the field of
economics of happiness. Many criticisms address the environmental indicators suggested. As
noted by Yannick Rumpala, they have multiplied impressively over the last years, in
connection with the recompositions of the public action [36], within the framework of a new
mode of governance, analysed incidentally as one of the aspects of "new public management
[37]." Their signification and their usages may appear ambiguous and ambivalent.
On the economists' and ultra-liberal thinkers' front finally, quite violently hostile stances
at times target Joseph Stiglitz primarily, who is coined as one of the main enemies of the "true
liberals". But they have little access to the mediatic space, apart from the blogs and of
"reactions" on the sites. Slightly isolated in his fiercely reactionary stances in Le Figaro,
Alain-Gérard Slama frontally contests the usage of indicators such as the HDI:

"The adoption of these new [HDI] criteria has seen the emergence of another
configuration of the planet (...) We're surely much the better for it: history-laden tumultuous
nations, bursting with invention, noise and fury, cynically overestimated by the GDP index
have been superseded by the fabulous destiny of the Northern countries with their civilising
ideal, whose unfathomable boredom and stifling conformism has been depicted by Bergman's
unforgiving look [38]".

On his blog [39], the ultraliberal essayist Guy Sorman goes even further:

"Jean-Philippe Cotis has presented the Stiglitz report to his UNO colleagues where it is
suggested to replace the conventional measure of national wealth (GDP) with a qualitative
evaluation of the Gross national happiness (sic). The communication has raised the interest of
the poorest countries (Morocco in particular) who thus hope to realise that they are no so poor
after all! The Americans have looked askance at that project(Stiglitz is rather gibed at in the
United States) and have reminded that the already existing qualitative measures (as adopted
by the UNO) surprisingly reveal that the richest nations are also those where people live the
best and longest lives".

The financier Georges Ugeux, whose blog is often quoted, tends to agree and well
illustrates the discrepancy between the narrow-minded world of economic elites and the
problematics raised by the Commission: "I was in a restaurant in New-York and one of my
table neighbours asked me what President Sarkozy had in mind when suggesting to
incorporate life enjoyment in the Gross National Product.

I had not paid much heed to these presidential words which had apparently interested the
my American interlocutors, who wondered whether France had a monopoly on life enjoyment.
And they went on to mention the attempt to include French gastronomy in Unesco's cultural
heritage of humanity. The conversation evolved into various directions: I felt ill at ease at
 Beyond GDP? Towards a New System of Social Accounts 137

first. I was rather upset by such a superficial statement in the midst of widespread
unemployment. This reminds of the "brioches" (little buns) that Marie-Antoinette wanted to
distribute to the people who had no more bread in 1789. How can a recently jobless man's or
woman's life enjoyment be measured? Is this statement sarcastic or thoughtless? (...) The
French Commission on this subject confirms the flimsiness of its proposer, Nobel prize-
winner John Stiglitz who, when he was the Chief Economist of the World Bank, raised many
eyebrows."

THE "SEQUELS" TO THE REPORT: THE REVOLUTION UNDERWAY?

Several critical comments insist on the cumbersome changes implied by the
implementation of the report's propositions. The GDP remains quite dominant in the ordinary
economic and social analysis, inspite of the increasing and dormant questioning of this
indicator, which can only be exacerbated by the publication of the report. The most radical
propositions concerning general patrimony accounts have met with scepticism, expressed for
instance by French economist André Babeau in the Commentaire review [40].
In a press conference on 17 November 2009, followed by articles (especially in the daily
newspaper Le Monde), Jean-Philippe Cotis, the director general of the INSEE, announces that
the INSEE will take into account the impulse given by the Stiglitz report and puts forward a
far-reaching work programme, as well as a bundle of early results in its extension, in
particular in terms of international comparison. Nevertheless, he maintains the idea of
"completing" the GDP with other indicators and insists on the multidimensionality of the real
world. In the light of this omnipresent GDP and of the still limited legitimacy of "alternative"
indicators, changes in perspective stand little chance of translating in short-term turmoils in
the official representation of economic and social performances. The director of the INSEE
besides insisted on a continuity rather than a break:
"this report, which marks a milestone, still does not break away from current statisticians'
work. It rather calls for accelerating engaged mutations; the better to meet with social demand.
Innovate the better to meet with social demand, such is the mandate anyway we have been
given by the highest authorities of the State. As the Anglo-Saxons like to say,
"mainstreaming" work remains to be done: i.e. to switch from emerging work in prototype
stage to regular production, accessible to a very wide audience (...) At international level, the
INSEE will work in close cooperation with the OECD and Eurostat to implement the
orientations of the Stiglitz report".

Jean Gadrey, commenting the recent evolutions of the INSEE, expresses moderate
optimism, symmetrical to the scepticism with which he had welcome the launch of the
Stiglitz operation.
The controversies around the Stiglitz report hence refer to a very specific configuration: a
Commission formed fairly closely around dominant economists, which proposes multifaceted
renovation of the statistical device with the explicit support of official institutes, but which
both "delineates" and limits the magnitude and the possible practical consequences thereof,
while leaving the main operational choices "open" (and to the responsibility of the statistical
administrations); a Head of state quite involved in the promotion of a discourse which seems
to question some of his previous ideological stances and triggers significant cognitive
disorder in his electoral "clientele"; "profanes" torn between "politicking" interpretations,
138 Frédéric Lebaron

radical rejection, enthusiasm and all possible attitude nuances concerning a projected turmoil
of the official representations of economics; political actors who endeavour to integrate the
"new discourse" while politicising their analysis of the content of the report; critical experts
who are pleased about the legitimisation of their investments while deploring that a few
dominant economists have closed the case on a problematic they have kept close to their
chests, which economists have set a monopoly on the thematic of alternative indicators, while
undermining a number of potential effects: the report may thus paradoxically close the space
of the debate around academic economists and simultaneously render the most prominent
promoters of alternative indicators less visible and less legitimate to some extent.

REFERENCES
[1] A first version of this abstract of our text has been published in Savoir / Agir, in the
"Alterindicateurs" column. F.Lebaron, "The Stiglitz report: towards a statistical
revolution", Savoir / Agir, 10 December 2009.
[2] J.Gadrey, F.Jany-Catrice, Les nouveaux indicateurs de richesse (The new wealth
indicators), Paris, La Découverte, 2nd ed., 2008.
[3] On the birth of the HDI, E.Stanton, « The Human Development Indicator : A History »,
Working Paper Series, 127, PERI, University of Massachussets, February 2007.
[4] See the official web site of the Commission: http://www.stiglitz-sen-
fitoussi.fr/fr/index.htm
[5] J.Stiglitz, A.Sen, J.-P.Fitoussi, Performances économiques et progrès social. Richesse
des nations et bien-être des individus, (economic performances and social progress.
nations' wealth and individuals' well-being), Paris, Odile Jacob, 2009 and J.Stiglitz,
A.Sen, J.-P.Fitoussi, Performances économiques et progrès social.Vers de nouveaux
systèmes de mesure (Economic performances and social progress. Towards new
measuring systems), Paris, Odile Jacob, 2009.
[6] See for example D.Méda, Au-delà du PIB. Pour une autre mesure de la richesse
(Beyond the GDP. For another wealth measure), Paris, Flammarion, 2008.
[7] Nicolas Sarkozy, Great amphitheater of the Sorbonne, Monday 14 September 2009,
repeated in J. Stiglitz, A.Sen, J.-P.Fitoussi, op. cit.
[8] The contrast between the communication offensive around the Stiglitz report and the
small echo of the reports of the Economic, social and environmental council, as that
dedicated to sustainable development Indicators (Philippe Le Clézio, June 2009), is
significant.
[9] Joseph Stiglitz was Chief economist at the World Bank up to 2000, before he was
awarded the prize of economic sciences in memory of Alfred Nobel in 2001 (alongside
with M. Spence and G.Akerlof). He now teaches in the Graduate School of Business of
Columbia. He came to world fame by his critical stances on the management of the
Asian crisis by the IMF and by his neo-keynesian orientations, in particular during the
subprime crisis. He has thus come closer to the "alterglobalists".
[10] Amartya Sen, "Nobel prize-winner" in 1998, teaches currently in Harvard. A theorician
of development, well-being and of economics as a "moral science", he is an honorary
president of Oxfam, an international NGO and a founder of the International ethical,
 Beyond GDP? Towards a New System of Social Accounts 139

scientific and political collegium, an association based in France, founded in 2002 by

Milan Kucan, president of Slovenia, Michel Rocard, a forme French Prime Minister,
co-presidents, Stéphane Hessel, vice-president, and Sacha Goldman, general secretary.
[11] Jean-Paul Fitoussi is the prime mover of the Stiglitz commission and undoubtedly the
author or at least the partial inspirer of some of the most committed texts derived
therefrom (including that of the President of the Republic). A professor in Sciences Po
Paris, where he is a member of the scientific council, Jean-Paul Fitoussi was Chairman
of the French monitoring centre for economic conjunctures. He is a member of the
Economic analysis council. Known for his neo-keynesian orientations, Jean-Paul
Fitoussi is bent on the independence of economic experts.
[12] It is entitled Synthesis and recommendations.
[13] p.9.
[14] For a presentation of the issues of environmental (private) accounting: Jacques Richard,
« Pour une révolution comptable environnementale » (For an environmental accounting
revolution), Le Monde de l'économie, 5 February 2008.
[15] The publication by Odile Jacob pub. consists of two books, whereof the first, prepared
by the three supervisors of the Commission, includes an essay on measurement which
goes beyond collective, more moderate conclusions. For want of direct testimonies, it
is difficult to known far more on the nature of disagreements and resistances raised by
the analyses of the three supervisors.
[16] About the history of the national accounting in France, see F. Fourquet, Les comptes de
la puissance. Aux origines de la comptabilité nationale et du plan (The accounts of
power. At the origins of national accounting and of the plan), Paris, Encres, 1980.
[17] p. 15.
[18] Here appears a recent breakthrough of the problematics for measuring the "subjective"
happiness, derived in particular from the New Economics Foundation, at the origin of
the "Happy Planet Indicator" and which have set the trend in the wake of international
conferences on the issue of "happiness" measure.
[19] See Vers de nouveaux systèmes de mesure, (Towards new measuring systems) op. cit.,
p. 325 et sq.
[20] After the model of the three people responsible, who can be classed diagrammatically
in the category of "consecrated heretics", as coined by Pierre Bourdieu: P.Bourdieu,
Homo academicus, Paris, Minuit, 1984.
[21] By mobilising reputed academic economists known for work in some concurring cases,
the people responsible have run the risk of adopting ecumenical compromise positions,
which is indeed the case on numerous occasions, where conceptual frameworks or
partially contradictory indicator are juxtaposed, without the report leaning in favour of a
particular item. Cf. the remarks by J.Gadrey and D.Méda on this score, art. cit.
[22] Any reflexive perspective on the social usages of quantification and of "indicators" is
for instance largely and strangely missing from the report. See A.Desrosières, Pour une
sociologie de la quantification et Gouverner par les nombres (for a sociology of
quantification and governing through numbers), Paris, Ecole des Mines, 2008.
[23] The next step of the international consecration of the economists working on new well-
being measures will undoubtedly be the attribution of a "Nobel" prize.
[24] One of the most quoted and discussed is the « Happy Planet Index » of the New
Economics Foundation : http://www.happyplanetindex.org/. The Institute for
140 Frédéric Lebaron

Innovation in Social Policy has developed the Social Health Indicator in the United
States, with a strong impact. In France, the BIP40 has initiated a construction
movement of local Indicators: www.bip40.org
[25] In particular that of Ruut Veenhoven of the University of Rotterdam :
http://worlddatabaseofhappiness.eur.nl/
[26] See for example: http://nopib.fr/
[27] The main cleavage in the debates on indicators indeed concerns the opposition between
synthetic indicators, defended by the most radical, in particular in associations and
think tanks, in the wake of the UNDP and the "dashboards" defenders
[28] http://www.terra-economica.info/Sortir-de-la-demesure-et-accepter,7598.html
[29] This is the case here of a device for quasi-undermining the controversy, avoiding
confrontation of adverse stances - whereas the heterodoxical and not institutional voices
are relegated to the edge or outside the device thus set up.
[30] http://alternatives-economiques.fr/blogs/gadrey/2009/09/13/le-figaro-et-le-%C2%AB-
rapport-stiglitz-%C2%BB/
[31] A systematic corpus of stances has been constructed. See a few appended illustrations.
[32] Such as for instance on the site of "feminist" orientation, the news which consecrate its
first column to wealth indicators: http://www.lesnouvellesnews.fr/index.php/
component/content/article/39-editorial/55-editorial
[33] As soon as in the report draft put on line in June 2009, they put forward a first analysis:
J.Gadrey, D.Méda, « Commission Stiglitz : un diagnostic juste, des propositions
(encore) timides » (Stiglitz commission: a fair diagnosis, (still) timid propositions),
IDIES, 9 June 2009.
[34] We have repeated this stance in the Alterindicateurs column of the Savoir / Agir
review: F.Lebaron, art. cit.
[35] Yannick Rumpala, « Mesurer le « développement durable » pour aider à le réaliser ? »
(Measuring "sustainable development' for assisting its realisation?), Histoire & mesure,
vol. XXIV – n 1, 2009, p. 211-246.
[36] See also the research of Jean-Pierre Le Bourhis in the CURAPP.
[37] A.-G.Slama, « Le bonheur devient un droit » (Happiness becomes a right), Le Figaro,
16/09/2009.
[38] Blog http://gsorman.typepad.com/guy_sorman/2010/02/keynes-combien-de-divisions-
.html
[39] A.Babeau, « Mesurer le bien-être : retour sur le rapport Stiglitz » (Measuring well-
being: a look back on the Stiglitz report), Commentaire, Hiver 2009-2010, Vol. 32, 128,
p.1054 et sq.
In: Critical Thinking ISBN 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 141-157 2011 Nova Science Publishers, Inc.

Chapter 6

A FOUR-COMPONENT INSTRUCTIONAL MODEL FOR

TEACHER TRAINING IN CRITICAL-THINKING
INSTRUCTION: ITS EFFECTIVENESS AND
INFLUENTIAL FACTORS

Yu-chu Yeh
Institute of Teacher Education
Research Center for Mind, Brain & Learning
Center for Creativity and Innovation Studies
National Chengchi University
64, Chih-nan Rd, Sec. 2, Taipei 116, Taiwan

ABSTRACT
This study investigated the effectiveness of a training course in critical-thinking
instruction with an emphasis on four components that are most likely to bring about
teachers’ improvement in personal teaching efficacy and teacher behaviors during the
training. Eighty-two preservice teachers participated in a 16-week training session in this
study. Based on both qualitative and quantitative analyses, the findings suggest that
providing guided practice and generating reflective teaching are crucial to the
successfulness of a teacher training program and that a training course in critical-thinking
instruction does, in fact, produce more lasting effects if it simultaneously imparts
professional knowledge, raises personal teaching efficacy as well as heightens reflective
teaching. At the same time, it is found that professional knowledge and field practices are
indeed decisive in teachers’ overall improvement in personal teaching efficacy and
teacher behaviors during teacher training.

Keywords: critical thinking, instructional model, personal teaching efficacy, teacher

behavior, teacher training, professional knowledge.
142 Yu-chu Yeh

It has long been recognized that critical thinking is an effective learning strategy
(Halpern, 1998; Jawarneh, Iyadat, Al-Shudaifat, & Khasawneh, 2008; Klein, Olson, &
Stanovich, 1997; Lawson, 1999; Roberts-Cady, 2008), the key to emotional intelligence
(Elder, 1997) and a prerequisite for leaders in business (Harris & Eleser, 1997;
Dilenschneider, 2000). Since students’ critical-thinking ability can be significantly improved
through effective teaching (e.g. Alshraideh, 2008; Browne & Meuti, 1999; Carmen &
Kurubacak, 2002; Ellis, 2001; Hittner, 1999; Jawarneh et al., , 2008; Mackinnon, 2006;
McCarthy-Tucker, 2000; Nelson & Oliver, 2004; Semerci, 2006; ; Yanchar, Slife, &Warne,
2009; Yeh, 2008a, 2008b; Yang & Chung, 2009), teacher educators should be held
accountable for turning out effective cultivators of critical-thinking skills. Previous studies
have suggested that, aside from dispositions, teachers’ professional knowledge, personal
teaching efficacy, and teacher behaviors are fundamental to the effective instruction of critical
thinking (Yeh, 1997, 2007, 2008b). With this in mind, a course which emphasized four
components was designed in this study to help teachers to become confident cultivators of
critical-thinking skills. Apart from this, this study was conducted to identify the relationships
among the fundamental components of effective critical-thinking instruction and to determine
the most crucial factors vis-à-vis teachers’ professional growth in this area.

FUNDAMENTALS OF EFFECTIVE CRITICAL-THINKING INSTRUCTION

The first of the four fundamental components of effective critical-thinking instruction on
the part of a teacher is having a sufficient body of professional knowledge about critical
thinking. Shulman (1987) identified three distinct types of professional knowledge: content
knowledge, pedagogical knowledge, and pedagogical content knowledge. Content knowledge
and pedagogical content knowledge are especially important for teaching such complex-
thinking skills as critical thinking, creative thinking and problem-solving. By integrating the
concepts of professional knowledge (Grossman & Richert, 1988) and critical thinking
(Alshraideh, 2008; Bailin, Coombs, Browne & Meuti, 1999; Facione, Sanchez, Facione, &
Gainen, 1995; Gadzella & Masten, 1998; Giancarlo & Facione, 2001; Halpern, 1998, 2003;
Harris & Eleser, 1997; McCarthy-Tucker, 2000; Paul & Elder, 2001; Yeh, 1997, 2008b,
2009), this study redefined content knowledge and pedagogical content knowledge so that
“content knowledge” includes a teacher’s understanding of critical thinking and related
concepts while “pedagogical content knowledge” takes into account a teacher’s knowledge as
to how best to employ appropriate teaching strategies to accommodate students’ needs during
the teaching of critical thinking.
As for the second fundamental component of critical-thinking instruction—teacher
efficacy, it has been reported that teacher efficacy is closely related to teachers’ adoption of
innovation strategies, their commitment to teaching, and their employment of effective
classroom strategies (Aguirre & Speer, 2000; Albion, 2001; Guesky & Passaro, 1994;
Kulinna & Silverman, 2000). Teacher efficacy is commonly broken down into two factors:
teaching efficacy and personal teaching efficacy (Gibson & Dembo, 1984). Since research
findings have demonstrated that personal teaching efficacy is a stronger predictor of teacher
effectiveness and is more conducive to effective teaching than teaching efficacy (Dembo &
Gibson, 1985; Gusky, 1988), this study focuses exclusively on personal teaching efficacy.
 A Four-Component Instructional Model for Teacher Training … 143

Accordingly, in teaching critical thinking, a teacher’s personal teaching efficacy refers to his
or her confidence with respect to building upon students’ prior knowledge, dispositions and
skills related to critical thinking (Yeh, 2006).
Besides professional knowledge and personal teaching efficacy, teacher behaviors, the
third fundamental component in effective critical-thinking instruction, play an important role
in the teaching of critical thinking. Teacher behaviors that contribute to students’ capacity to
learn critical thinking skills can be divided into four categories: behaviors that relate to
improving students’ prior knowledge; critical-thinking dispositions; critical-thinking skills;
and overall critical-thinking ability. Such teacher behaviors include: increasing students’
prerequisite knowledge; teaching critical-thinking skills, schema-driven strategies, and ways
to build a frame of mind which is conducive to thinking; offering a healthy mix of facilitating
methodologies by employing an infused-instruction approach as well as anchored instruction;
keeping students focused on undertaking tasks; building a constructive learning environment;
giving students cues, prompts and positive feedback; supplying examples and practice
opportunities; asking higher-order questions and extended questions; allowing a variety of
answers; providing successful learning experiences; rewarding student interactions;
encouraging students’ engagement in group discussions along with cooperative learning; and
monitoring the entire learning process (Alshraideh, 2008; Browne & Meuti, 1999; Chen,
2001; Larson, 2000; Halpern, 1998, 2003; Paul & Elder, 2001; Yeh, 1997; 2007; 2008b).
The fourth and final fundamental component in effective critical-thinking instruction is
teachers’ own critical-thinking dispositions because they may influence teaching outcomes in
a subtle way. In the course of teaching critical thinking, teachers must be self-confident,
open-minded, creative in following curricula, sensitive to students’ feelings, analytical about
students’ learning problems and systematic in problem-solving. It need not be explained that
such personal teaching traits characterize teachers’ critical-thinking dispositions (Giancarlo
& Facione, 2001; Halpern, 2003, 2008; Paul & Elder, 2001; Yanchar et al., 2009).
As for the relationships among the four fundamental components discussed above,
namely a full body of professional knowledge, a high degree of teacher efficacy, positive
teacher behaviors and teachers’ own wealth of critical-thinking dispositions, Robinson (1995)
noted that teachers’ existing knowledge and experience had a strong impact on both their
development and use of effective teaching strategies. In a computer simulation, Yeh (1997)
found that personal teaching efficacy functioned as a mediator between knowledge and
teacher behaviors and that, at the same time, the relationship between personal teaching
efficacy and teacher behaviors was bi-directional. Hence, in terms of critical thinking, the
improvement of teachers’ professional knowledge should increase both their personal
teaching efficacy and teacher behaviors. Another point is that critical-thinking dispositions
are significantly related to individuals’ ego-resiliency, which refers to a person’s ability to
change his or her model of perceptual and behavioral functioning in order to adapt to
situational constraints (Facione et al., 1995); such dispositions can be influential on a
teacher’s learning of professional knowledge and formation of personal teaching efficacy, not
to mention his or her adaptation of positive teacher behaviors.
144 Yu-chu Yeh

INTERVENTIONS FOR TEACHER TRAINING IN

CRITICAL-THINKING INSTRUCTION
For a training course in critical-thinking instruction to be effective, it must be able to
improve teachers’ professional knowledge, personal teaching efficacy, and teacher behaviors.
Providing sufficient guided practice and stimulating reflective teaching are two effective
interventions to achieve this. Guided practice sessions based on an analytical understanding
of teaching events are often more constructive than those in natural settings (Yeh, 2006);
furthermore, belief change often comes about as a result of the interaction of practice and
reflection in teaching (Tillema, 2000). Guided practice can also lead to mastery with regard to
experiences, which has been put forth as a powerful way to enhance teacher efficacy
(Bandura, 1995). In support of this, Yeh (2006) concluded that guided practices contribute to
preservice teachers’ improvements in personal teaching efficacy as it pertains to their
teaching of critical thinking. Also relevant in this regard, Tillema (2000) determines that a
change in beliefs can be attributed to the interactive effects of practice and reflection in
teaching.
The other intervention—reflective teaching—results in instructors’ reconstruction of
professional knowledge and teacher beliefs as well as improvements in teaching practice
(Rodriguez & Sjostrom, 1998; Tillema, 2000; Yeh, 2004). Two mechanisms have been found
essential for nurturing reflective teaching: self-awareness and mindfulness. Collier (1998) has
recently pointed out that building a high level of self-awareness before the student teaching
experience is critical to pre-service teachers’ learning of reflective teaching and their
becoming thoughtful practitioners. Titone, Sherman, and Palmer (1998) also attest to the
theory that giving feedback to increase self-awareness and that encouraging “mindfulness” in
the teacher-learner setting are effective ways to foster reflective teaching.
Accordingly, while self-awareness and mindful learning contribute to reflective teaching
and then encourage the improvements in professional knowledge, personal teaching efficacy,
and teacher behaviors, guided practices may carry effects on the employment of reflective
teaching as well as the reconstruction of professional knowledge, personal teaching efficacy,
and teacher behaviors. It is apparent that four components are crucial to bringing about the
improvement of personal teaching efficacy and teacher behaviors in critical-thinking
instruction; these components are: (a) increasing self-awareness and mindful learning, (b)
reconstructing knowledge and personal teaching efficacy, (c) employing reflective teaching,
and (d) conducting guided practices. This study therefore incorporates these four components
into the training course in critical-thinking instruction.

HYPOTHESES
This study examined two hypotheses. The first supposition is that a four-component
instructional design which especially emphasizes guided practice and reflective teaching
should have a positive impact on a teacher’s improvement in the areas of professional
knowledge, personal teaching efficacy, and teacher behaviors vis-à-vis critical thinking. The
second presupposes that a teacher’s critical-thinking dispositions, professional knowledge and
field practices correlate with his or her personal teaching efficacy and teacher behaviors
 A Four-Component Instructional Model for Teacher Training … 145

during the teaching of critical thinking. The integrated model of the proposed hypotheses in
this study is displayed in Figure 1. More details about the four-component instructional
design are described in the section of experimental design.

• Giving test results of critical-thinking, • A strong sense of critical-

professional knowledge, and personal thinking dispositions
teaching efficacy • Abundant Professional
• Requesting self-evaluations of teacher knowledge
behaviors • Frequently use of teacher
• Giving lectures and conducting class behaviors
discussions

Improvements in
Increasing self- Reconstructing • personal teaching
awareness and professional knowledge efficacy
mindful learning and personal teaching • teacher behaviors
efficacy

Improvements in
Employing Provoking reflective
• professional
guided practices teaching
knowledge

• Developing critical-thinking tests followed

by class discussions
• Field practices of positive teacher
behaviors

Figure 1. The integrated model of tested hypotheses.

METHOD
Participants

Eighty-two preservice teachers enrolled in a post-bachelor teacher education program

participated in this study. They had an average age of 29.79 years (SD = 3.41) and an average
teaching experience of 2.81 years (SD = 1.42). Among the participants, 38 were males, 44
were females, and of these, 49 were teaching in junior high schools, and 33 in senior high
schools.

Instruments

All instruments employed in this study were The Questionnaire of Dispositions toward
Critical Thinking (QDCT), The Questionnaire of Professional Knowledge for Critical-
thinking Instruction (QPK-CTI), The Questionnaire of Personal Teaching Efficacy in
146 Yu-chu Yeh

Teaching Thinking (QPTE-TT) and The Checklist for Teacher Behaviors in Critical-thinking
Instruction (CTB-CTI). The first three of these instruments are 6-point Likert-type
questionnaires while the final one is a checklist. QDCT scores indicate the degree of
dispositions toward critical thinking. The QDCT, with a Cronbach’s α coefficient of .88 (20
items), comprises four factors: systematicity and analyticity, open-mindedness, intellectual
curiosity, and reflective thinking. The item response options are “never” to “always,” scored
from 1 to 6 points (Yeh, 1999).
QPK-CTI scores reveal participants’ self-evaluation of their own professional knowledge
in teaching critical thinking. With a Cronbach’s α coefficient of .95 (9 items), the QPK-CTI
measures two factors: content knowledge about critical thinking and pedagogical content
knowledge about critical thinking. The item response options are “totally disagree” to “totally
agree,” scored from 1 to 6 points (Yeh, 1999).
QPTE-TT scores reflect the trainees’ level of confidence in their ability to teach learners
to think critically. The QPTE-TT, with a Cronbach’s α coefficient of .86 (12 items), assesses
three factors: improvement in thinking dispositions, sharpening of overall thinking ability,
and broadening of prior knowledge and thinking skills (Chen, 2001). The scoring system of
the QPTE-TT is the same as that of the QPK-CTI.
Finally, CTB-CTI scores evaluate the frequency of using positive behaviors in teaching
critical thinking. The CTB-CTI has a Cronbach’s α coefficient of .95 (21 items), and it covers
four factors: expanding students’ prior knowledge, upgrading their critical-thinking
dispositions, cultivating their critical-thinking skills, and ameliorating their overall critical-
thinking ability (Yeh, 1999).
Besides these four tests, five reflection items related to instructional design were included
to evaluate the participants’ attitudes toward the training course. The scoring system for these
items is the same as that of the QPK-CTI. The statements which participants evaluated are:
(1) The self-evaluation of my personal teacher behaviors enhanced my self-awareness in
employing favorable teacher behaviors and, therefore, increased my use of positive teaching
behavior for critical thinking; (2) Developing critical-thinking tests contributed to improving
my understanding of critical thinking; (3) Developing critical-thinking tests contributed to
improving my own critical-thinking ability; (4) Class discussions of critical-thinking test
items contributed to improving my critical-thinking ability; and (5) I have often tried to
employ positive teacher behaviors for critical thinking this semester.

Experimental Design

This study employed a pretest-posttest design. Besides completing the pretests and
posttests which were identical, participants attended a weekly two-hour experimental
instruction session, the aim of which was to improve the participants’ professional
knowledge, personal teaching efficacy, and teacher behaviors pertaining to critical-thinking.
The instruction lasted for sixteen weeks. The course design emphasized the following four
components: (a) increasing self-awareness and mindful learning, (b) reconstructing
knowledge and personal teaching efficacy, (c) employing reflective teaching, and (d)
conducting guided practices.
 A Four-Component Instructional Model for Teacher Training … 147

More specifically, the course design was based on three basic assumptions. First,
returning the graded pretest results of all tests to the participants and also telling them the
mean averages within the group would increase self-awareness about professional knowledge,
personal teaching efficacy and teacher behaviors, which would, in turn, provoke mindful
learning during the classes which followed. Secondly, it was assumed that developing critical-
thinking tests as out-of-class assignments followed by in-class discussions would improve the
participants’ understanding of critical-thinking, and this would then further enhance their
professional knowledge and personal teaching efficacy. The third assumption was that the
employment of guided practice and reflective teaching would result in a more frequent use of
positive teacher behaviors; it follows that this would lead to improvements in professional
knowledge, personal teaching efficacy, and teacher behaviors.

Procedures

The step-by-step timing and procedures of the teaching activities are depicted in Figure 2.
Two types of test were administered during the semester: Type A tests included three
questionnaires for measuring critical-thinking dispositions, professional knowledge, and
personal teaching efficacy (QDCT, QPK-CTI, and QPTE-TT). Type B test refers to the
checklist for teacher behaviors (CTB-CTI). All participants were administered pretests A in
the first class and were subsequently asked to do pretest B based on two periods of classroom
teaching (90 minutes in total) which they recorded between the end of the latter part of the
first and the early part of the second weeks.

Week 1 Administer Pretest A: QDCT,

Do Pretest B QPK-CTI, and QPTE-TT
(Record class teaching)

Week 2 Give results of Pretest A

Lecture and discuss
Week 3 Assign groups and request the
related topics of critical-
development of test items
thinking instruction
Week 5 Lecture on teacher behaviors

Week 6 Analyze Pretest B based on

Take turns Develop
CTB-CTI
presenting test and
items and revise Week 7
participate in test Give results of Pretest B
class items
discussions
Week 14
Do Posttest B and analyze
Posttest B
Week 15
What the teacher does
Give Posttest B results; administer
What the students do Week 16 Posttest A and reflection items

Figure 2. Flowchart of instructional procedures.

148 Yu-chu Yeh

The participants were shown their results of Type A pretests in the second week and were
given pretest B results in the seventh week. Pretest B results were not immediately returned
because, as a self-evaluation checklist, it was believed that a full understanding of each
checking item was required to enable participants to analyze the self-evaluation items. A
thorough lecture and a discussion of the self-evaluation checking items had been held in the
sixth week. Toward the end of the semester, the participants were required to do posttest B
(again by recording their teaching and doing follow-up self-evaluations). The posttest B
results were given in the sixteenth week before posttests A were administered. The five
reflection items were added to posttests A.
As for teaching activities, all participants were randomly assigned to groups of four or
five people during the third week. Each participant was asked to develop one multiple-choice
test item geared toward other teachers for each of the following critical-thinking skills:
assumption identification, induction, deduction, explanation, and the evaluation of arguments.
These concepts had been explained in the preceding week. From the fifth week on, the groups
took turns presenting their test items to the class; 40- to 50- minute discussions on the test
items followed each week. Meanwhile, lectures and discussions on topics related to critical-
thinking instruction continued from the second week to the fifteenth week. The main topics
for the lectures included:

a) definitions and components of critical thinking,

b) factors that influence the learning of critical thinking;
c) teaching approaches;
d) instructional models;
e) curriculum design;
f) effective teacher behaviors,
g) effective student behaviors, and
h) teaching strategies.

RESULTS
Effectiveness of the Training Course

Three repeated measures analyses of variance were employed to examine the changes
among the participants in terms of the three indices for effective critical-thinking instruction:
professional knowledge, personal teaching efficacy, and teacher behaviors. It should be noted
that the employed data of teacher behaviors in this study, as determined by Test B, were
analyzed by the researcher. The correlation coefficient of the CTB-CTI scores between the
researcher and the participants was .93. The means and standard deviations of the QPK-CTI,
QPTE-TT and the CTB-CTI are displayed in Table 1. The first and the third analyses yielded
significant effects (Wilks’ Λs = .18 and .70, respectively, ps < .001), whereas the second
analysis did not (Wilks’ Λ = .99), thus demonstrating that the training course significantly
improved the teachers’ perceived professional knowledge and their use of positive teacher
behaviors but not their personal teaching efficacy. Subsequent paired t-tests and comparisons
of the means revealed that the teachers’ perceived knowledge in both content knowledge and
 A Four-Component Instructional Model for Teacher Training … 149

pedagogical content knowledge pertaining to teaching critical thinking had significantly

increased in the posttest than in the pretest, ts(62) = 14.88 and 19.53, ps < .001. Moreover, the
teachers reported that they were employing more favorable teacher behaviors to improve
students’ prior knowledge, critical-thinking dispositions, critical-thinking skills, and overall
critical-thinking ability at the time of the posttest than they were at the time of the pretest,
ts(71) = 4.45, 4.34, 5.01, and 5.78, ps < .01, respectively.

Table 1. Means and Standard Deviations of the QPK-CTI, QPTE-TT, and CTB-CTI

Pretest Posttest
Inventory M SD M SD
QPK-CTI (n = 65)
Content knowledge 2.30 .93 4.26 .72
Pedagogical content knowledge 1.99 .84 4.20 .82
QPTE-TT (n = 71)
Enhancing thinking dispositions 4.12 .62 4.23 .68
Improving overall thinking ability 4.31 .58 4.39 .68
Increasing prior knowledge and thinking skills 4.43 .54 4.42 .56
CTB-CTI (n = 74)
Increasing prior knowledge 2.38 1.37 3.42 1.84
Enhancing critical-thinking dispositions 14.95 9.25 24.05 16.67
Improving critical-thinking skills 8.69 8.76 16.95 14.02
Improving critical-thinking ability 3.07 3.08 6.69 5.69

To further confirm the effectiveness of the employed interventions, the relationships

between the three indices for effective critical-thinking instruction and the five reflection
items related to the interventions were analyzed. On a six-point scale, the mean scores for the
five items were 5.08 (SD = .89), 5.03 (SD = .90), 4.95 (SD = .92), 5.15 (SD= .90), and 4.69
(SD = .93), respectively. The results of Pearson correlation analyses clearly showed that each
of the five items and the three indices of effective critical-thinking instruction had a positive
correlation (see Table 2). Of particular interest is that all five items had the highest degree of
correlation with professional knowledge, rs(76) = .58 to .78, ps < .001.

Table 2. Correlations among the Scores of Professional Knowledge, Personal Teaching

Efficacy, and Teacher Behaviors in the Posttest and Five Reflection Questions

Q1 Q2 Q3 Q4 Q5
Posttest professional knowledge .58*** .78*** .70*** .75*** .67***
Posttest personal teaching efficacy .54*** .57*** .51*** .56*** .61***
Posttest teacher behaviors .21* .32** .35*** .22* .32**
* p < .05. ** p < .01. *** p < .001.

Noteworthy too is that the second highest correlation was with personal teaching efficacy,
rs(76) = .51 to 61, ps < .001. Comparatively, each of the five items had a significantly lower
degree of correlation with teacher behaviors, rs(76) = .21 to .35, ps < .05. As a result of the
150 Yu-chu Yeh

findings, it can reasonably be stated that the teachers had a positive attitude toward the
training course. Furthermore, the correlation coefficients suggest that there was a certain
degree of training effect as shown in the teachers’ improvement in terms of personal teaching
efficacy though the effect was not significant in the repeated measures analyses of variance.

The Relationships Among the Four Fundamentals of Effective Critical-

Thinking Instruction

Two canonical correlation analyses were used to investigate the relationships among the
four fundamental components of effective critical-thinking instruction. Both analyses
involved two independent variables (IV, critical-thinking dispositions and professional
knowledge for teaching critical thinking) and two dependent variables (DV, personal teaching
efficacy for critical thinking and teacher behaviors for critical thinking). However, the first
analysis employed the pretest scores, whereas the second one used the posttest scores.The
first analysis yielded a significant canonical correlation (Wilks’ Λ = .73, p < .001). The
canonical correlation was .51, representing 26% overlapping variance for the pair of
canonical variates. The pair of canonical variates had a high factor loading on critical-
thinking dispositions (.85) on the IV side and a high factor loading on personal teaching
efficacy (.94) on the DV side, which indicates that critical-thinking dispositions and personal
teaching efficacy seem to have had a strong positive correlation in the pretest (see Table 3).

Table 3. Canonical Correlation Analysis of Pretest IVs and Pretest DVs

Canonical variate
Correlation Coefficient
Pretest IV set
Critical-thinking dispositions .85 .92
Professional knowledge .42 .54
Percent of variance Total = .45
Redundancy Total = .11
Pretest DV set
Personal teaching efficacy .94 .83
Teacher behaviors .61 .36
Percent of variance Total = .63
Redundancy Total = .16
Canonical correlation (ρ) .51
Squared canonical correlation (ρ2) .26
Wilks’ Λ .73 ***
*** p < .001.

The second analysis also yielded a significant canonical correlation (Wilks’ Λ = .46, p <
.001). The canonical correlation was .73, representing 54% overlapping variance for the pair
of canonical variates. The pair of canonical variates had a high factor loading on professional
knowledge (.99) on the IV side and a high factor loading on personal teaching efficacy (.99)
 A Four-Component Instructional Model for Teacher Training … 151

on the DV side. Therefore, as shown in the posttests, abundant professional knowledge is

most assuredly related to a strong sense of personal teaching efficacy (see Table 4).

Table 4. Canonical Correlation Analysis of Posttest IVs and Posttest DVs

Canonical variate
Correlation Coefficient
Posttest IV set
Critical-thinking dispositions .30 .12
Professional knowledge .99 .97
Percent of variance Total = .54
Redundancy Total = .29
Posttest DV set
Personal teaching efficacy .99 .96
Teacher behaviors .40 .12
Percent of variance Total = .57
Redundancy Total = .31
Canonical correlation (ρ) .73
Squared canonical correlation (ρ2) .54
Wilks’ Λ .46 ***
*** p < .001.

The Most Crucial Factors for Improving Personal Teaching Efficacy and
Teacher Behaviors

To further understand the most crucial factors related to teachers’ improvements in

personal teaching efficacy and teacher behaviors, a third independent variable, “frequency of
practicing teacher behaviors,” was added to the foundation of the second canonical
correlation and then reanalyzed.

Table 5. Canonical Correlation Analysis between CT-Dispositions, Professional

Knowledge, Practicing Frequencies and Posttest DVs
Canonical variate
Correlation Coefficient
Posttest IV set
Critical-thinking dispositions .28 .14
Professional knowledge .96 .68
Frequency of practicing teaching behavior .84 .37
Percent of variance Total = .56
Redundancy Total = .33
Posttest DV set
Personal teaching efficacy .99 .96
Teacher behaviors .39 .11
Percent of variance Total = .57
Redundancy Total = .33
Canonical correlation (ρ) .76
Squared canonical correlation (ρ2) .58
Wilks’ Λ .42 ***
*** p < .001.
152 Yu-chu Yeh

The analysis yielded a significant canonical correlation (Wilks’ Λ = .42, p < .001) of .76,
representing 58% overlapping variance for the pair of canonical variates. The pair of
canonical variates had a factor loading on professional knowledge (.96) and frequency of
practicing teacher behaviors (.86) on the IV side and a high factor loading on personal
teaching efficacy (.99) on the DV side. Thus, on the basis of the results of the posttests, there
is strong evidence to support the theory that abundant professional knowledge and frequent
practice of teacher behaviors are related to having a strong sense of personal teaching efficacy
(see Table 5).

DISCUSSION AND CONCLUSIONS

Effectiveness of theTraining Course

This study employed a four-component design to improve teacher effectiveness in

teaching critical-thinking by: (a) increasing self-awareness in terms of professional
knowledge, personal teaching efficacy and teacher behaviors when participants were given
their test results; (b) helping to reconstruct professional knowledge and personal teaching
efficacy by means of lectures and class discussions; (c) encouraging reflective teaching by
increasing self-awareness and mindful learning, and (d) improving professional knowledge,
personal teaching efficacy and teacher behaviors through guided practices. By and large, the
participants’ positive attitudes towards the instructional design and the significant results
from the related analyses suggest that the instructional design that was developed here was
effective in achieving the presumed instructional goals although its effects on personal
teaching efficacy were not as strong as the impacts on professional knowledge and teacher
behaviors. Nevertheless, the findings in this study not only support the notion that teacher
effectiveness in teaching critical thinking can be significantly improved by means of a well-
designed course (Yeh, 1997, 2008a, 2009) but also demonstrate that guided practice and
reflective teaching are powerful tools for teachers’ professional growth (Carter, 1990; Collier,
1998; Tillema, 2000, Yeh, 2004). Evidence that led to these conclusions is observed, first, in
the teachers’ significant improvements in professional knowledge and teacher behaviors and,
secondly, in the strong correlations between the first and the fifth reflection items and the
posttest scores of professional knowledge, personal teaching efficacy, and teacher behaviors.
Moreover, this study confirms that those with a greater wealth of professional knowledge at
the completion of the training tended to highly value and benefit from opportunities to
evaluate their own teacher behaviors, which agrees with the claim by Rodriguez and Sjostrom
(1998) that reflective teaching is related to professional knowledge. As a result, it can be said
with confidence that guided practice and self reflection in teaching are crucial to teachers’
acquisition of professional knowledge, development of personal teaching efficacy, and
employment of effective teacher behaviors during the teaching of critical thinking.
Another important intervention found in this study is the inclusion of class discussions. It
had been earlier hypothesized that devising critical-thinking tests followed by class
discussions would contribute to teachers’ improvements in both professional knowledge and
personal teaching efficacy. The following findings support this: (a) there was a significant
change in knowledge at the completion of the training course; and (b) there was a high
 A Four-Component Instructional Model for Teacher Training … 153

correlation between the second, third, and fourth reflection items and the posttest scores of
professional knowledge along with personal teaching efficacy. These findings help validate
the idea that discussions generally contribute to learners’ understanding of a topic and the
development of critical thinking ( Yeh, 2008b, 2009). In brief, class discussions are most
probably one of the most effective ways to increase professional knowledge and personal
teaching efficacy in a specific domain.

Crucial FactorsfFor Improving Personal Teaching Efficacy and

Teacher Behaviors

This study investigated the predictive powers of professional knowledge and critical-
thinking dispositions on teachers’ personal teaching efficacy and teacher behaviors in both the
pretests and the posttests. Though there were significant canonical correlations for both
analyses, the correlation patterns were different. In the pretests, critical-thinking dispositions
were a better predictor of personal teaching efficacy and teacher behaviors than was
professional knowledge. By contrast, in the posttests, professional knowledge became a better
predictor than critical-thinking dispositions. The low predictive power of professional
knowledge in the pretests may have been due to the low variance—almost all of the teachers
had very limited professional knowledge about teaching critical thinking (The mean for
content knowledge was 2.30 and for pedagogical content knowledge 1.99). However, upon
completion of the training course, the teachers’ professional knowledge had significantly
improved (The mean for content knowledge was 4.26 and for pedagogical content knowledge
4.20). These results strongly imply that professional knowledge is a more influential factor in
teachers’ enhancement of personal teaching efficacy and their use of teacher behaviors than
critical-thinking dispositions. It is therefore important to help teachers-in-training build up
professional knowledge, especially that of pedagogical content knowledge.
On the basis of the results of this study, it appears that in addition to professional
knowledge, a large number of practice sessions is a crucial factor in teachers’ improvement in
personal teaching efficacy and teacher behaviors vis-à-vis critical thinking. According to
Tillema (2000), reflection after practice is a more fruitful way of affecting student teachers’
belief change than preparing them to be reflective before they enter their practice teaching.
This study, however, encouraged reflection both before and after practice by means of giving
test results and requesting self-evaluations of teacher behaviors twice during the training. The
results here suggest that such a reflection approach may be a more effective way than a pure
reflection-after-practice approach in terms of raising the motivational force of teaching
practice sessions in improving teacher effectiveness.

CONCLUSION
Preparing confident cultivators of critical thinking and nurturing reflective teachers can
not be over emphasized in teacher education. To achieve this educational ideal, as a teacher
educator, I designed a training course which featured a four-component process with a special
focus on guided practice and reflective teaching. The findings in this study indicate that the
154 Yu-chu Yeh

instructional design and interventions incorporated in the training course were effective. In
essence, the study findings show that teacher effectiveness in teaching critical thinking can be
improved as long as the training course is meticulously designed.
Worthy of special note, however, is that a training course in critical-thinking instruction
can indeed produce more lasting effects if it imparts professional knowledge, raises personal
teaching efficacy as well as enhances reflective teaching all at the same time. This study has
also developed instruments in the form of questionnaires for measuring four fundamental
components of a successful critical-thinking instruction; these tools will be valuable in further
research. Finally, the teacher training in this study was focused more on the use of the “direct
approach” for teaching critical thinking, in which critical-thinking skills are regarded as
general skills. Further studies may apply this instructional design with the “infusion
approach”, thus focusing more on how to teach critical-thinking skills in a specific domain.

REFERENCES
Aguirre, J., & Speer, N. M. (2000). Examining the relationship between beliefs and goals in
teacher practice. Journal of Mathematical Behavior, 18(3), 327-356.
Albion, P. (2001). Some factors in the development of self-efficacy beliefs for computer use
among teacher education students. Journal of Technology and Teacher Education 9(3),
321-347.
Alshraideh, M. (2008). The effect of Suchmans’ inquiry model on developing critical
thinking skills among university students. International Journal of Applied Educational
Studies, 4(1), 58-69.
Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Common misconceptions of
critical thinking. Journal of Curriculum Studies, 31(3), 269-283.
Bandura, A. (1995). Exercise of personal and collective efficacy in changing societies. In A.
Bandura (Ed.), Self-efficacy in changing societies (pp. 1-45). NY: Cambridge University
Press.
Browne, M. N., & Meuti, M. D. (1999). Teaching how to teach critical thinking. College
Student Journal, 33(2), 162-170.
Carmen, C., & Kurubacak, G. (2002). The use of the internet to teach critical thinking.
Society for Information Technology and Teacher Education International Conference,
2002(1), 2504-2505.
Carter, K. (1990). Teachers’ knowledge and learning to teach. In W. R. Houston (Ed.),
Handbook of Research on Teacher Education (pp. 291-310). New York: Macmillan.
Chen, Y. M. (2001). Teacher training for the employment of teaching strategies in critical-
thinking via a computer simulation: An example in gender equality issues. Unpublished
thesis, National Sun Yat-sen University, Kaohsiung, Taiwan.
Collier, S. T. (1999). Characteristics of reflective thought during student teaching experience.
Journal of Teacher Education, 50(3), 173-181.
Dembo, M. H., & Gibson, S. (1985). Teachers’ sense of efficacy: An important factor in
school improvement. The Elementary School Journal, 86(2), 173-184.
Dilenschneider, R. L. (2000). Critical thinking. Executive Excellence, 18(5), 5-7.
 A Four-Component Instructional Model for Teacher Training … 155

Elder, L. (1997). Critical thinking: The key to emotional intelligence. Journal of

Developmental Education, 21(1), 40-41.
Ellis, T. J. (2001). Multimedia enhanced educational products as a tool to promote critical
thinking in adult students. Journal of Educational Multimedia and Hypermedia, 10(2),
107-124.
Facione, P. A., Sanchez, C. A., Facione, N. C., & Gainen, J. (1995). The dispositions toward
critical thinking. The Journal of General Education, 44(1), 1-25.
Gadzella, B. M., & Masten, W. G. (1998). Critical thinking and learning processes for
students in two major Fields. Journal of Instructional Psychology, 25(4), 256-261.
Giancarlo, C. A., & Facione, P. A (2001). A look across four years at the disposition Journal
of General Education, 50(1), 29-55.
Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation. Journal of
Educational Psychology, 76, 569-582.
Grossman, P. L., & Richert, A. E. (1988). Unacknowledged knowledge growth: A re-
examination of the effects of teacher education. Teaching and Teacher Education, 4(1),
53-62.
Guesky, T. R., & Passaro, P. D. (1994). Teacher efficacy: A study of construct dimensions.
American Educational Research Journal, 31(3), 627-643.
Guesky, T. R. (1988) Teacher efficacy, self-concept, and attitude toward the implementation
of instructional innovation. Teaching and Teacher Education, 4(1), 63-69.
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains. American
Psychologist, 53(4), 449-455.
Halpern, D. F. (2003). Knowledge and thought: An introduction to critical thinking (4th ed.).
Hillsdale, NJ: Lawrence Erlbaum.
Harris, J. C., & Eleser, C. (1997). Developmental Critical Thinking: Melding Two
Imperatives. Journal of Developmental Education, 21(1), 12-19.
Hittner, James B. (1999). Fostering critical thinking in personality psychology: The trait
paper assignment. Journal of Instructional Psychology, 26(2), 92-97
Jawarneh, M., Iyadat, W., Al-Shudaifat, S., & Khasawneh, L. (2008). Developing critical
thinking skills of secondary students in jordan utilizing monro and slater strategy, and
mcfarland strategy. International Journal of Applied Educational Studies, 4(1), 58-69.
Klein, P., Olson, D. R., & Stanovich, K. (1997). Structuring reflection: Teaching argument
concepts and strategies enhances critical thinking. Canadian Journal of School
Psychology, 13(1), 38-47.
Kulinna, P. H., & Silverman, S. (2000). Relationship between teachers’ belief systems and
actions toward teaching physical activity and fitness. Journal of Teaching in Physical
Education, 19, 206-221.
Larson, B. E. (2000). Classroom discussion: a method of instruction and a curriculum
outcome. Teaching and Teacher Education, 16(5-6), 661-677.
Lawson, T. J. (1999). Assessing psychological critical thinking as a learning outcome for
psychology majors. Teaching of Psychology, 26(3), 207-209.
Mackinnon, G. R. (2006). Contentious issues in science education: Building critical-thinking
patterns through two-dimensional concept mapping. Journal of Educational Multimedia
and Hypermedia, 15(4), 433-445.
156 Yu-chu Yeh

McCarthy-Tucker, S. N. (2000). Teaching style, philosophical orientation and the

transmission of critical thinking skills in the U.S. public schools. Korean Journal of
Thinking & Problem Solving, 10(1), 69-77.
Nelson, T., & Oliver, W. (2004). Maximizing critical thinking skills with technology. In P.
Kommers and G. Richards (Eds.), Proceedings of World Conference on Educational
Multimedia, Hypermedia and Telecommunications, 2004, 3982-3986.
Paul, R., & Elder, L. (2001). Critical thinking: Tools for taking charge of your learning and
your life. Upper Saddle River, NJ: Prentice Hall.
Roberts-Cady, S. (2008).The role of critical thinking in academic dishonesty policies.
International Journal for Educational Integrity, 4, 2, 60-66.
Rodriguez, Y. E., Sjostrom, B. R. (1998). Critical reflection for professional development: A
comparative study of nontraditional adult and traditional student teachers. Journal of
Teacher Education, 49(3), 177-187.
Robinson, B. (1995). Teaching teachers to change: The place of change theory in the
technology education of teachers. Journal of Technology and Teacher Education, 3(2/3),
107-117.
Semerci, N. (2006). The effect of problem-based learning on the critical thinking of students
in the intellectual and ethical development unit. Social Behavior and Personality, 34,
1127-1136.Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new
reform. Harvard Educational Review, 57(1), 1-21.
Titone, C., Sherman, S., & Palmer, R. (1998). Cultivating student teachers’ dispositions and
ability to construct knowledge. Action in Teacher Education, 19 (4), 76-87.
Tillema, H. H. (2000). Belief change towards self-directed learning in student teachers:
Immersion in practice or reflection on action. Teaching and Teacher Education, 16(5-6),
575-591.
Yanchar, S. C., Slife, B. D., &Warne, R. (2009). Advancing disciplinary practice through
critical thinking: A rejoinder to bensley. Review of General Psychology, 2009, 13(3),
278–280.
Yang, S. C., & Chun, T. (2009). Experimental study of teaching critical thinking in civic
education in Taiwanese junior high school. British Journal of Educational Psychology ,
79, 29–55.
Yeh, Y. (1997). Teacher training for critical-thinking instruction via a computer simulation.
Unpublished dissertation, University of Virginia, VA, US.
Yeh, Y. (1999). A study of substitute teachers’ professional knowledge, personal teaching
efficacy, and teaching behavior in critical-thinking instruction. Journal of Chengchi
University, 78, 55-84
Yeh, Y. (2004). Nurturing reflective teaching during critical-thinking instruction in a
computer simulation program. Computers & Education, 42(2), 181-194.Yeh, Y. (2006).
The interactive effects of personal traits and guided practices on preservice teachers’
changes in personal teaching efficacy. British Journal of Educational Technology, 37(4),
513-526.
Yeh, Y. (2007). Aptitude-treatment interactions in preservice teachers’ behavior change
during computer-simulated teaching. Computers & Education, 48(3), 495-507.
Yeh, Y. (2008a). Collaborative PBL meets e-learning: How does it improve the professional
development of critical-thinking instruction? In T. B. Scott and J. I. Livingston (Eds.),
 A Four-Component Instructional Model for Teacher Training … 157

Leading-edge educational technology (pp. 133-158). Hauppauge, NY: Nova Science

Publishers, Inc.
Yeh, Y. (2008b). Integrating collaborative PBL with blended learning to improve preservice
teachers’ critical-thinking skills. In A.R. Lipshitz and S. P. Parsons (Eds.), E-learning:
21st century issues and challenges (pp. 155-175). Nova Science Publishers, Inc.
Yeh, Y. (2009). Integrating e-learning into the direct-instruction model to enhance the
effectiveness of critical-thinking instruction, Instructional Science, 37, 185–203.
In: Critical Thinking ISBN 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 159-173 2011 Nova Science Publishers, Inc.

Chapter 7

CRUCIAL CONNECTIONS: AN EXPLORATION OF

CRITICAL THINKING AND SCHOLARLY WRITING

Roisin Donnelly and Marian Fitzmaurice

Learning, Teaching and Technology Centre
Dublin Institute of Technology, Ireland

ABSTRACT
Academic writing in the context of producing quality research articles is something
which all academics engage in and there is evidence of increased attention to supporting
the development of the writing and subsequent output of academics and research
students. However, while scholarly writing is learnt in complex ways, critical thinking is
an intrinsic part of such writing, and is highly valued across all the academic disciplines
and indeed is a high priority on both employability and citizenship agendas. However, in
practice the teaching of critical thinking is difficult and there is a lack of discussion about
what it means within the context of the writing process. This study describes a pedagogic
intervention with a group of academic staff to support the participants not only to explore
critical thinking in their own writing, but also to consider in depth how they would apply
this learning to their work with students in higher education. Within the context of an
academic writing module on a postgraduate programme for academic staff in higher
education, an action research approach was used with participants to improve their
understanding of the role of critical thinking in the academic writing process. The data
suggests that the pedagogic intervention resulted in greater confidence in terms of
participants’ critical writing skills and also supported them to help their own students in
the academic writing process. An exploratory model is proposed for critical academic
writing encompassing a series of scaffolded in-class activities, virtual peer learning, and
tutor feedback – culminating in the publication and dissemination of individual practice-
based educational research.

Keywords: Advanced academic literacy; Academic writing; Collaborative dialogue; Critical

thinking; Peer review.
160 Roisin Donnelly and Marian Fitzmaurice

INTRODUCTION
Over the last few decades, critical thinking has been defined in a number of different
ways. Elder & Paul (1994) suggest that critical thinking is best understood as the ability of
thinkers to take charge of their own thinking. More recently, Duron, Limbach, & Waugh
(2006) define critical thinking as the ability to analyse and evaluate information and conclude
that “critical thinkers are considered to be able to raise vital questions and problems,
formulate them clearly and gather and assess relevant information, use abstract ideas, think
open-mindedly, and communicate effectively with others” (p. 160).
Teachers in all disciplines agree that critical thinking is an important educational outcome
for their students, and indeed there is general consensus that ccritical thinking concepts and
tools are the essential core of all well-conceived instruction. However, although teachers are
able to articulate the critical thinking skills that they would like their students to exhibit, the
cognitive steps between actual student performance and desirable student performance often
remain unarticulated and vague. Taken further, there is an implicit assumption that academics
have an agreed understanding of the concept of critical thinking but this tacit understanding is
seldom articulated or discussed. As academics working with postgraduate students we were
interested in ways of supporting students to be critical in their academic writing. Academic
writing can be seen as a continuum of increasing complexity developing from undergraduate
to postgraduate writing and beyond (Stacey & Granville, 2009). The term advanced academic
literacy (AAL) has been used to refer to the writing expected of participants in higher levels
of a discipline and this is a cumulative process of which enculturation into the disciplinary
norms is central. Badley (2009) reminds us that good academic writing should always be a
problematic and tentative exercise in critical reflective thinking. Clearly, there are key
elements of academic writing of which critical thinking is paramount that need to be
developed and the pedagogic challenge is to devise relevant supports for postgraduate
students so that they can develop as academic writers.
While the social and ideological underpinnings of academic writing have been
investigated (Canagarajah, 2002; Casanave, 2002; Johns, 1997), the relationship between
critical thinking and academic writing is an under researched area.

CONTEXT
There is increasing pressure on academics to undertake research and to publish in higher
education and indeed their practice offers rich and interesting fields for investigation but there
are few opportunities provided within an increasingly busy and pressurised academic
environment for developing their academic writing. In recognition of this, a 10 ECTS module
entitled ‘Writing and Disseminating Research’ was developed and had shared delivery as part
of two masters programme in Applied eLearning and Higher Education, which are accredited
professional development programmes for academic staff. Ultimately there was dual purpose
to the module, to enhance academics own writing as well as supporting their own students in
the same endeavour.
Each year, participants are drawn from a variety of Higher Education Institutions in
Ireland and from a range of disciplines and course participants range from newly appointed
 Crucial Connections 161

staff to their institutions, to those that have been teaching for anywhere between 5-25 years.
Our experience of working with the participants is that this multi disciplinary setting provides
for interesting and critical discourse about academic writing. In terms of their subject
disciplines, there is an eclectic mix, with many subject disciplines being represented ranging
across apprentice education, undergraduate and postgraduate education. The study had a
number of objectives:

1) To focus on how critical thinking informs the practice of academic writing; this
involved exploring the definitional debates surrounding critical thinking and arriving
at a definition that could be adapted within different disciplines.
2) To support academics in improving their own writing through an increased
awareness of the concept and practice of critical thinking.

SETTING THE DISCUSSION IN THE LITERATURE

For many years, academic writing has been a distinct teaching and research subject in US
higher education and is strongly emerging as a subject in UK higher education, but in Ireland
there are few initiatives in this area. Findings from research carried out in the UK with
academic and support staff shows that 90% of respondents believe it is necessary to support
students in their writing (Ganobscik, 2004). Initially, there were two models in the UK for
teaching academic writing. The first is a skills approach, which seeks to teach writing as a set
of discrete techniques without relation to a discipline and the second is an academic
socialization model, which views academic writing proficiency as something that students
absorb through immersion in disciplinary practices. However, Lea & Street (1998) argue for
an academic literacies approach, which challenges the assumption implicit in the skills and
academic socialization approach that it is the students who are in deficit and need to learn to
adapt to the university. Academic literacies theorists make the case that ‘writing is not a
student problem only, but a challenge for all members of the university as they attempt to
adjust to new forms and technologies of writing and studying, as well as a variety of student
backgrounds and experiences’ (Ganobscik-Williams, 2006, p. 4). In working with academic
staff we became aware of a real challenge because as subject specialists they often do not feel
that they can work effectively with students on their writing. The authors began to consider
how this new theoretical framework might inform the practice of academic staff and decided
to put into practice an initiative to support lectures in terms of their writing skills, whilst
examining the role of critical thinking in this process and supporting the academic staff to
work more effectively with their students on their writing. The work seeks to contribute to the
discussion about the role of writing in the university and draw lessons for readers from our
experience of implementing an initiative with academic staff drawn from a variety of higher
education institutions in Ireland and from a diverse range of disciplines.
162 Roisin Donnelly and Marian Fitzmaurice

TEACHING CRITICAL ACADEMIC WRITING

The recognition that academic writing needs to be taught is growing and the call for
teaching writing has come from outside and from inside the university (Bergstrom, 2004;
Ganobcsik-Williams, 2006). Prior to the 1990’s, there was very limited provision of writing
support in the UK tertiary education sector and this can be contrasted with the situation in US
universities, where dedicated writing support has been a feature of first year programmes
since the late nineteenth century (Ivanic & Lea, 2006). However in the UK there is now an
emerging body of work on student writing and three models are evident, study skill, academic
socialization and academic literacies. Indeed, there is a now a growing body of research in the
field of academic literacies and despite the variety of contexts, the findings in regard to
students’ struggle with writing and the gaps between tutors’ and students’ expectations and
understanding are remarkably consistent (Ivanic & Lea, 2006). Research findings from very
different institutional contexts and student groups all indicate ‘a complex relationship
between the acquisition and development of subject-based knowledge and writing in higher
education’ (Lea, 2004. p. 740). She argues that as subject specialists, academics often
overlook the ways in which writing and textual practices are central to the process of learning
(Lea, 2004).
It is very frequent in higher education for the teaching or support for writing to be
separated from mainstream study but a successful programme developed in the USA ‘Writing
in the Disciplines’ approaches the development of academic writing through the disciplines.
However, it does not take account of the increasing number of interdisciplinary or
multidisciplinary contexts where students are undertaking courses, which are becoming more
common at undergraduate and postgraduate level. Jacobs (2005) reports on an initiative at a
tertiary institution in South Africa on integrating academic literacies into the disciplines of
study and argues for the importance of creating discursive spaces for the collaboration of
academic literacies practitioners and disciplinary specialists. Attention to student writing must
be integrated in mainstream contexts and it seemed to us that working with academic staff on
their writing could offer the potential for opening up such discursive spaces.
Stierer (1997) writes about removing some of the guesswork from the process of meeting
the writing requirements of courses and calls for a more systematic and explicit approach for
helping students to identify and critique the kinds of expectations they are expected to fulfill
in relation to written assignments. Students are always expected to be critical in their writing
but while lecturers are able to articulate the critical thinking skills that they would like their
students to exhibit in terms of their writing, the cognitive steps between actual student
performance and desirable student performance often remain unarticulated and vague. In the
module there was an attempt to implement strategies to support students to develop their
critical thinking skills in order to bring a criticality to their writing. Some of these students are
expert writers in their own disciplines, while others may not have considerable experience
and find academic writing really difficult. However, each member of the group is
encountering a new area of study with its own particular discourse and supporting the writing
process gave them the opportunity to develop new skills.
 Crucial Connections 163

RESEARCH DESIGN
Cohen, Manion & Morrison (2000) argue that action research is a powerful tool for
change and improvement at the local level and this approach was used in this study because it
offered the potential to bridge the gap between research and practice by potentially delivering
useable solutions to real-world problems. Different action researchers have described the
process in different ways, some as cycles of reflective action, some as flow diagrams, some as
spirals of action (Mc Niff et al., 1996). Also, in action research there are different types of
studies ranging from small-scale evaluative case studies which have a defined start and end
point to studies which are cyclical in nature and more long term (Tight, 2003). This research
study had a defined start and end point and an approach outlined by Coughlan & Brannick
(2001), comprising of a series of steps, diagnosing, planning action, taking action and
evaluating was followed, as it was the most suited to the context. Figure 1 illustrates the data
collection cycle undertaken in the study.

Figure 1. Action Research Cycle for Critical Academic Writing.

The action research cycle began with a review of the situation, drawing on informal
conversations with participants, colleagues, a review of the literature and a process of values
clarification for ourselves with regards to the relationship between critical thinking and
academic writing. Following on from this, the ‘Critical Academic Writing Module’ was
164 Roisin Donnelly and Marian Fitzmaurice

designed to develop academic writing through the application of critical thinking skills. It is
arguable that using the traditional lecture format may not adequately foster active learning or
critical thinking skills in these participants, as it is based on a teacher-centred approach. As a
result, it was important to adjust the structure of class delivery to promote such skills. Not
only would this make the module work more enjoyable for both participants and tutors, it
could equip participants with the skills necessary in their future practice. During a class
session, the tutor needed to consider the kinds of active learning that could encourage critical
thinking. To enhance the overall learning experience, it was necessary have a broad
understanding of what active learning constituted for these participants. In-class strategies
included requesting participants to be involved in the learning experience by, for example,
giving information and ideas, sharing experiences, and offering opinions. Table 1 shows a
variety of the short activities used to help progress the participants through the process of
critical academic writing.

Table 1. List of In-class activities

Students asked first to read an article and:

Individually
Produce a drawing/visual summary of the text One group highlights key points, another
Everyone selects one sentence from the text
that they have found meaningful (a main
point or an idea with which to argue)
Produce one question that you would ask the
author
Produce a ‘bare bones’ summary (25 words)
Write a Critical Synopsis of the Text (See
Appendix 1)
In pairs/small groups
‘blacks out’ everything that is NOT necessary
In pairs, analyse a passage in the article for
‘voice’
In groups, discuss the textual features of the
discipline that are evident
In pairs, examine the main argument/s that the
author/s are making
In groups of 4, discuss how convincing the
arguments made by the author/s are and
present to entire group

Towards the end of the first semester, once the participants had experienced the range of
in-class activities, a virtual peer learning set entitled ‘online journal club’ was established in
the virtual learning environment, webcourses. This was envisaged as a way to share insights
and conversation themes with the participants. In the first instance, articles on the process of
critical academic writing were distributed by the tutors to the group, but thereafter, they were
encouraged to select relevant articles themselves. In the asynchronous discussion board in the
VLE, directed commentary on each article was posted by participants who were divided into
small groups of four; contribution is regular and periodic (weekly). Discussion questions
posed by the tutors are used to stimulate reflection and conversation. In particular, these
questions are designed to help the online journal club participants:

• identify key points addressed by the article, and put them in context;
• discuss the validity of the findings, and;
• consider how the findings apply to practice with regard to critical thinking and
academic writing.
 Crucial Connections 165

Participants are encouraged to take a RADICAL approach to evaluating these articles

online: Read, Ask, Discuss, Inquire, Collaborate, Act and Learn. To support the
participants further in this, a set of guidelines are used, shown in Table 2:

Table 2. RADICAL Approach to Critical Academic Writing

Read the article critically

Ask the key questions for yourself
Discuss the meaning and shared interpretation
Inquire into other sources of knowledge and insight
Collaborate with others who know or care about the issues
Act by sharing their postings and working to change practice
Learn from what others share online and from your actions and collaborations and re-start the cycle

Finally, the tutors highlight commonalities or uniquely important ideas from the online
collaborative dialogue. The peer learning sets have the potential to improve communication
and mutual support between participants and also to encourage them to make links between
taught sessions on critical acadmic writing and foster and advance their understanding of
classroom writing practices. This also leads into a crucial aspect of the module – the role of
formative feedback to students by tutors. By providing formative feedback that seeks to
discover and clarify intended meanings, the tutors tap into the developing writers' basic desire
to communicate their ideas. The process-oriented writing instructional approach favoured by
the module tutors ensured that deep-level revision was most productive in terms of writing
skills development. The tutor formative feedback underpinned with an inquiring stance
engaged the participants in negotiation over the emerging meaning of their texts. Sample draft
compositions were used to explore the assumptions and implications of this instructional
stance to the participant writing.
At the mid way point and at the end of the module, participants were invited to complete
questionnaires with eight open-ended questions designed to evaluate the module, but with a
specific focus on the experience of the critical writing process; the data from these
questionnaires were then analyzed. In looking for patterns and trends in the responses, the
following steps were undertaken: reading through the responses and developing categories for
the different themes emerging; as we read through the comments, we assigned at least one
category to each response. Once the data had been studied and categories determined, the next
step was to pinpoint categories that were related and identify the main themes. Four themes
emerged from the data: becoming critical; perception of the peer review process; and
challenges of critical academic writing; perceived impact of critical academic writing on
practice.

DISCUSSION OF FINDINGS
Becoming Critical

To engage critically with a written text, the reader ideally needs to have an understanding
of what the authors are doing and reasonable knowledge of the field of enquiry and this can
only be achieved by critical reading of relevant texts.
166 Roisin Donnelly and Marian Fitzmaurice

I will be getting my students to read more critically, through approaching their reading
with a clear sense of the importance of focusing on the evidence provided in the account and
whether the reasoning follows logically to the conclusion that has been drawn.
Design Lecturer

Attention is drawn here to the importance of approaching a text as something that

demands a response from you as a reader and to see your reading as an active process that
requires you to be critical. The critical reading of a text is mostly about assessing the quality
of the case that has been made and so the critical reader is interested in whether there is
sufficient evidence to support a claim and whether other possible interpretations have been
considered. Thus, the participants recognized the importance of working with students to
develop the skill of critical reading as an important step to becoming critical in writing. Most
of the participants reported that the strategies used in class and outlined in Table 1 were
useful in terms of the development of their own academic writing and would be used by them
with their own students.

Teaching my students how to construct an argument and focus their work is the most
challenging but I have now some strategies that I can employ with them.
Business Lecturer

One of the most important things I will be sharing with my students is the importance of
developing a logical line of reasoning
Social Work Lecturer

Allied to this, the importance of being able to critically select the texts that are most
central and relevant to your study purposes was also seen as important.

Engaging my students with the process of researching the literature in my discipline and
accessing relevant journals is key.
Architecture Lecturer

There is a clear need to select what to read and what not to read and it could be argues
that making critical choices about what to read is in fact the first step and so the importance of
focusing with students on techniques for deciding what to read. In terms of the writing
process, the issue of attention to planning also emerged as important.

I will be emphasizing to them the need for planning to ensure steady progress.
Ecology Lecturer

In addition, it was felt by a number of the lecturers that there was a real need to support
students in terms of the development of their academic writing through facilitated sessions
focusing on the key elements of academic writing, and the module had been enabling for
them in that regard.
I will now be facilitating sessions on academic writing with my own students and
enabling structured peer feedback to occur in these. I would also like to have optional ‘dip-in’
sessions for those with different levels of experience.
Law Lecturer
 Crucial Connections 167

It has become clear to me that supported, dedicated academic writing sessions must be
included at least at MA level. The process should be considered a journey towards excellence.
Business Lecturer

Clearly, the importance of structured sessions on academic writing and the confidence to
support students in terms of becoming critical in the academic writing process grew through
participation in the module and through the range of different activities that were developed
and the peer review process emerged as of particular significance.

PERCEPTION OF THE PEER REVIEW PROCESS

In the module, peer review was the process of making judgments about the quality of
critical academic writing which involved a peer reading and examining the draft journal
papers in various stages of completion and providing feedback. This feedback led to
reflection and discussion, with the ultimate aim of improving participant learning. The
greatest value of peer review here was the influence that different disciplinary peers had on
improving individual’s critical academic writing practice:

I got the most interesting and constructive comments back from the peers who were from
various different backgrounds and disciplines to myself.
I now realize the benefits of a critical friend.
Law Lecturer

A study by Mürau (1993) considered the effects of the peer review process on writing
anxiety. By working together, although perhaps having an initial sense of apprehension about
the process and what it entails, participants come to realize the similar problems and
difficulties that their peers share and feel less isolated. There can be a fear of exposure of
one’s work to peers and also a sense of unease at having to give criticism:

It is lonely at times writing for publication whether in my own discipline or more broadly
in education, so it was a positive thing to see how someone else feels about my writing.
Business Lecturer

I was a bit intimidated by this as I am now to it; however it was a good learning
experience.
Social Work Lecturer
Initially I had a fear of being wrong in offering my opinion but once I was honest about
comments it felt good to have a judgment to offer.
Instrumentation Lecturer
By providing the participants with experience in writing collaboratively and critiquing
one another's writing, it is argued here that collaborative writing promotes active learning and
provides them with experience working as part of a team. Peer review gave the participants
experience in critical thinking and promoted their editorial skills. These classroom techniques
raised participants’ comfort level at having their work evaluated by others in a professional
setting. The module evaluation feedback confirmed that participants who completed the
module were more likely to write collaboratively in future, and participants reported that they
would seek collaborative writing opportunities in their workplace:
168 Roisin Donnelly and Marian Fitzmaurice

As well as the face-to-face reviews we conducted, I got sent texts and emails throughout
that kept me going when I was going to give up. The group peer review sessions were very
useful also and are something that I intend to use in my own practice.
Social Care Lecturer

The collaborative peer review sessions facilitated the participants in learning how to read
carefully, with attention to the details of a piece of writing (whether their own or another
writer's). Academic writers have very little opportunity and few spaces to share their writing-
in-progress (Antoniou & Moriarty, 2008) but the module provided a space for this and all
agreed that their writing was strengthened by taking into account the responses of both the
actual and anticipated readers. As a result, they were making the transition from writing
primarily for themselves or for the tutors, to writing for a broader audience, which was a key
transition for the participants as they developed their post-graduate work and learnt to write
papers of publishable quality. Wrapped around these benefits were the development of
participant skills in formulating and communicating constructive feedback on a peer's work,
as well as knowing how to gather and respond to feedback on their own work. However, it is
worth highlighting that there can be a downside to collaborative peer review, particularly if
one of the peers is a stronger writer than the other in the first instance, as one participant
noted:

This has the potential to place a burden on the more experienced writers in the process.
Law Lecturer

CHALLENGES OF CRITICAL ACADEMIC WRITING

The two key challenges identified by the participants revolved around time management
of the writing process and grappling with scholarship, particularly knowing when to stop
reading and start writing; this symbiotic relationship between reading and writing is important
to acknowledge:
It was difficult to transfer the chaos of my concepts and all the reading I was doing into a
structured piece of work.
Getting to grips with a new type of literature was the biggest issue for me.
Law Lecturer

The editing of my paper took much, much longer than I had allocated time for.
Architecture Lecturer

The synthesis of literature is complex and requires time and space to think.
Law Lecturer
Whilst the complexity of writing is not to be underestimated, discovering efficient
writing skills does take time; it consists of lengthy procedures of conducting thorough
research and the ability to write skillfully. Improving the efficiency of one’s academic output
is a valuable skill to acquire. The participants had to grapple with challenging discipline-
specific subject matter and exacting logic; all whilst contending with the educational research
field’s rhetoric, accepted language and writing style, required format, and of course, critical
thinking. There is a very clear sense form the data that building a repertoire of critical
thinking and writing skills that enable the participants to enter the academic debates in their
 Crucial Connections 169

subject, and even to challenge accepted thinking, is worth the time investment as it can result
in can result in changes to practice.

Perceived Impact of Critical Academic Writing on Practice

From the 20 participants on the module, only five had previously published in
disciplinary journals, and none in educational research outlets. However, participation in the
module had some important results and all lecturers reported that there had been real learning
in terms of writing in an academic context and more specifically, writing for a journal.

One of the most useful aspects is that this was an opportunity to spend time learning and
improving my writing abilities in an academic context.
Social Work Lecturer

I have a much better sense of how to approach the whole area of writing for a journal.
Business Lecturer

They had also become more confident about their writing:

I now appreciate that no research idea, no matter how small could be of real interest to
someone out there.
This module has given me the confidence to pursue publishing my own work
Apprentice Lecturer

The module required each participant to target a specific journal and write an article and
as outlined earlier a framework was provided for individuals to engage in the writing process.
The support, guidance and practice led to improvements in their writing over the timescale of
the module and this resulted in an increase in confidence and an improvement in their writing
process.

Towards a Model of Critical Academic Writing

The academic writing process demands that the participants are thinking critically and
over the course of the module a model emerged which can/has provided for our participants
an approach to the teaching academic writing in the context of their own disciplinary field.
Figure 2 shows the six components of the model and how they interrelate: in-class activities,
virtual peer learning sets, the support of tutors’ feedback, the role of cross programme
dissemination which took the form of a participant-led conference and an educational
research forum, the online and in-class resources distributed and the impact on participants’
practice of critical thinking skills and critical academic writing.
Figure 2. Model of Critical Academic Writing.
 Crucial Connections 171

CONCLUSION
This work contributes to the discussion about the role of writing in the university and
draws lessons for readers from our experience of implementing an initiative with academic
staff drawn from a variety of higher education institutions in Ireland and from a diverse range
of disciplines. Antoniou and Moriarty (2008, p. 164) contend that ‘successful academic
writing does not depend on innate talent and ability but, like all writing, develops with
dedication and practice’. There is much to be gained by adopting the practice of explicitly
teaching academic writing skills with a particular focus on developing the skill of being
critical. During the module, the lecturers through structured exercises and activities had on-
going practice and gained confidence in academic writing in an educational discipline and
also gained confidence in articulating what it means to be critical as a writer. Academic
writing is a developmental process as is critical thinking and the pedagogic intervention with
academic staff on two postgraduate programmes detailed in this chapter has highlighted the
value of focusing on the role of critical thinking in the writing process.
Since writing and publishing are increasingly important in a successful academic career it
is imperative that there is support for lecturers to develop their writing. Morss & Murray
(2001) argue that despite the emphasis on publishing to enhance individual and institutional
profiles there is not sufficient research or support for academics aiming to improve quality
and productivity in writing. Moore (2003) suggests that to help academics write, we need to
initiate discussions and undertake research and in this chapter we have sought to contribute by
sharing the work we undertook with a group of lecturers. The comments of the lecturers
indicate that the project has been successful and many have presented their work at
conferences and some have published for the first time in peer review higher education
journals. Also, a model has been developed which we hope can be used by lecturers to
support their students in their academic writing endeavors so that the tacit understanding
which lecturers have of the concept of critical thinking in the writing process will be
articulated, discussed and become a focus of their educational practices.

REFERENCES
Antoniou, M., & Moriarty, J. (2008). What can academic writers learn from creative writers?
Developing guidance and support for lecturers in Higher Education. Teaching in Higher
Education, 13(2), 157-167.
Badley, G. (2009). Academic Writing as shaping and re-shaping. Teaching in Higher
Education, 14(2), 209-219.
Canagarajah, A.S. (2002). Critical academic writing and multilingual students. Ann Arbor:
University of Michigan Press.
Casanave, C.P. (2002). Writing games: multicultural case studies of academic literacy
practices in higher education. Mahwah: NJ: Laurence Erlbaum.
Duron, R., Limbach, B., & Waugh, W. (2006). Critical Thinking Framework For any
Discipline. International Journal of Teaching and Learning in Higher Education, 17(2),
160-166.
172 Roisin Donnelly and Marian Fitzmaurice

Elder, L., & Paul, R. (1994). Critical thinking: Why we must transform our teaching. Journal
of Developmental Education, 18(1), 34-35.
Ivanic, R., & Lea, M.R. (2006). New Contexts, New Challenges: the Teaching of Writing in
UK Higher Education. In Teaching Academic Writing in U K Higher Education (pp.7-
15). Hampshire: Palgrave Macmillan.
Jacobs, C. (2005). On being an insider on the outside: new spaces for integrating academic
literacies. Teaching in Higher Education, 10(4), 475-487.
Johns, A.M. (1997). Text, role and context: Developing academic literacies. Cambridge:
Cambridge University Press.
Lea, M.R. (2004). Academic literacies: a pedagogy for course design. Studies in Higher
Education, 29(6), 739-756.
Leki, L. (1998). Academic Writing: Exploring Processes and Strategies. 2nd ed. New York:
Cambridge University Press.
Ganobcsik-Williams, L. (ed.) (2006). Introduction: Starting Point Theory and Pedagogy in
Teaching Academic Writing in U K Higher Education (pp.3-6). Hampshire: Palgrave
Macmillan.
Moore, S. (2003). Writer’s retreat for academics: Exploring and increasing the motivation to
write. Journal of Further and Higher Education 27(3), 333-342.
Morss, K., & Murray, R. (2001). Researching Academic Writing within a Structured
Programme: insights and outcomes. Studies in Higher Education, 26(1), 35-52.
Mürau, A.M. (1993). Shared writing: Students’ perceptions and attitudes of peer review.
Working Papers in Educational Linguistics, 9(2), 71-79.
Schatzman, L. & Strauss, A. (1973). Field Research: Strategies for a Natural Sociology.
Englewood Cliffs: Prentice Hall.
Stacey, J.D., & Granville, S. (2009). Entering the conversation: reaction papers in advanced
academic literacy. Teaching in Higher Education, 14(3), 327-339.
Srierer, B. (2006). Schoolteachers as Students of Academic Literacy and the Construction of
Professional Knowledge within Master’s Courses in Education. In Teaching Academic
Writing in U K Higher Education. Hampshire: Palgrave Macmillan.
Sword, H. (2007). The Writer’s Diet. New Zealand: Pearson Education.

APPENDIX 1:
A CRITICAL SYNOPSIS OF A TEXT
Notes on Questions

Critical reading is part of academic study and requires you not just to passively read a
text but also to assess the texts of other scholars. In order to do this well, a structured
approach can be helpful. The 5 questions outlined below provide a framework for a critical
reading of a text.

Question 1: Sketch a Simple Outline of the Key Arguments or Ideas

This requires you to read the entire article and summarise the main arguments and ideas.
 Crucial Connections 173

Question 2: What Are the Authors Seeking to Do in Writing this Article?

The abstract, introduction or conclusion should make clear what the purpose of the
authors is. Authors may be seeking to do any of the following:

• Contribute to theory
• Report their own research
• Criticise what is currently being done
• Review the work of others
• Express opinions
• Give advice on future policy directions.

Question 3: What Are the Authors Saying that Has Relevance to My Work?

This question requires you to consider the links if any to your own project or research.

Question 4: How Convincing is What the Authors Are Saying?

This question requires you to evaluate the arguments put forward by the authors.

• Are the arguments supported with strong evidence?

• What claims are made?
• Are there unsubstantiated claims?
• What data set is drawn on and are the claims clearly related to this?
• Are the claims consistent with other articles you have read?
• Do the claims resonate in terms of your own research or professional
experience?

Question 5: What Use Can I Make of This?

This question requires you to think about the following:

• Do you agree or disagree with the claims made by the author?

• In your own writing, is this a key text that you will use and discuss in depth or
will you only refer to it briefly?

(Adapted from Wallace & Wray, 2006)

In: Critical Thinking ISBN: 978-1-61324-419-7
Editors: Ch. P. Horvath and J. M. Forte, pp. 175-188 2011 Nova Science Publishers, Inc.

Chapter 8

HOW CAN CRITICAL THINKING BE RECOGNIZED

IN THE CLASSROOM?

Patrícia Albergaria-Almeida1, José Joaquim Cristino Teixeira-Dias2

and Mariana Martinho3
1
Research Center for Didactics and Technology in Teacher Education
Department of Education
University of Aveiro, Portugal
2
Department of Chemistry
University of Aveiro, Portugal
3
Department of Education
University of Aveiro, Portugal

ABSTRACT
A crucial goal of Higher Education is to support students in developing their ability
to think critically. However, “critical thinking” can be an ambiguous expression. In this
chapter, we intend to clarify its meaning, as well as to characterize the most significant
indicator of critical thinking: higher-order questioning, as well as the relationship
between critical thinking and active learning. We also propose to shed light on the
teaching approaches which seem to inhibit students’ critical skills and to present teaching
and learning strategies to enhance critical thinking. Finally, we will describe the
curriculum of a first year chemistry course at a Portuguese university aimed at fostering
students’ critical thinking through the encouragement of quality questioning. We will
describe, in detail, the teaching, learning and assessment strategies designed and
implemented in this course.

1. QUESTIONING AS THE MAIN INDICATOR OF CRITICAL THINKING

The Commission of European Communities (2006) considers critical thinking as a key
competence for lifelong learning in terms of an “attitude of critical appreciation and
curiosity, an interest in ethical issues and respect for both safety and sustainability - in
176 P. Albergaria-Almeida, J. J. C. Teixeira-Dias and M. Martinho

particular, as regards scientific and technological progress in relation to oneself, family,

community and global issues” (p. 15). The Boyer Commission Report (1998) reinforces that
courses should be “structured in such a way as to create student-centered learning
environments where inquiry is the norm, problem solving becomes the focus, and thinking
critically is part of the process” (p. 4). Therefore, critical skills are identified as important
generic competences to be developed by students. However, it is well-known that frequent
university classes do not promote students’ criticality (Braxton & Nordvall, 1985).
According to Sternberg (1987) and to Browne & Freeman (2000), critical thinking is a
participant activity that implies a partnership between the teachers and their students. Even if
critical thinking can be accelerated through the implementation of specific teaching, learning
and assessment strategies, these will only be effective if the students are willing to become
fully engaged in effective learning. Actually, critical thinkers are those who are predisposed
to think critically (Halpern, 1999). Critical thinkers evaluate the outcomes of though
processes (Halpern, 1998). Thus, critical thinking can be taught as: decision making (Dawes,
1988), cognitive process (Rabinowitz, 1993), problem solving (Mayer, 1992), or argument
analysis (Kahane, 1997).
Several authors (Almeida, Pedrosa de Jesus & Watts, 2011; Browne & Freeman, 2000;
Cuccio-Shirripa & Steiner, 2000; Meyer, 1994; Shaw, 1996) also conceptualize critical
thinking as directly related to higher-level questioning. According to Cuccio-Shirripa and
Steiner (2000, p. 21):

“questioning is one of the thinking processing skills which is structurally embedded in

the thinking operations of critical thinking, creative thinking and problem solving. It consists
of the smaller micro-thinking skills of recall, comprehension, application analysis, synthesis
and evaluation (…) Questions guide knowledge construction in the formation and changing of
the cognitive networks or schemata”.

Facione, Facione & Giancarlo (1996, p. 71) also see a straight link between critical
thinking and questioning:

“Critical questioners, critical learners, are curious, challenge authority, internalize,

practice scientific and critical thinking and questioning. Critical questioners have the
motivation, inclination to drive and involve themselves in meaningful critical thinking, while
making decisions and/or solving problems.”

Actually, Browne & Freeman (2000, p. 302) consider questioning as “the primary
behavioral characteristic of critical thinking classrooms”. Almeida, Pedrosa de Jesus and
Watts (2011) also conceptualize student questioning as the most important indicator of critical
thinking.
In 1966, Raths, Wasserman, Jonas and Rothstein considered the interaction between the
students and the teacher as a privileged site to enhance criticality. More recently, Meyer
(1994) saw each declarative statement made by the teacher, or by the students, as potential
stimuli to ask questions. Furthermore, this author, as many others (e.g., Braxton & Nordvall,
1985; Browne & Freeman, 2000; Browne & Keeley, 1998; Ruggerio, 1996), consider a
separate category of critical questions. These questions are seen as especially significant for
the critical thinking classroom. Shaw (1996) underlines that, in a first moment, critical
 How Can Critical Thinking Be Recognized in the Classroom? 177

thinking requires comprehension. Thus, questions that disclose information are important
starting points for activating critical thinking. Browne and Freeman (2000, p. 302) suggest a
list of questions to promote critical thinking:

• What words or phrases are being used in an ambiguous fashion?

• What descriptive and value assumptions provide the foundation for the
reasoning?
• What evidence was provided for the claims in the reasoning?
• What is the quality of the proffered evidence?
• Are the analogical components of the argument persuasive?
• What important omitted information is omitted from the reasoning?
• What rival causes might explain the conclusion?
• What alternative inferences can reasonably be drawn from the evidence?”

On the other hand, Almeida, Pedrosa de Jesus and Watts (2011) do not defend the
existence of a distinct category of critical questions. Instead, these authors suggest that
encouraging “critical questioners” is more important than promoting critical questions
(Almeida, Pedrosa de Jesus and Watts, 2011, p. 117).
According to Almeida, Pedrosa de Jesus & Watts (2011), being critical depends on three
key aspects: context, competency and disposition. Mayer (1986) emphasizes the role of
context, stating that is mainly the learning context that promotes or shrinks students’
criticality. As stated by Mayer (1986, p. 253):

“The key to developing critical thinking lies in creating conditions for participation rather
than passivity, and in providing opportunities for emotional engagement with the materials.”

2. THE RELATIONSHIP BETWEEN ACTIVE LEARNING AND

CRITICAL THINKING
Several studies (Almeida, Pedrosa de Jesus & Watts, 2008; Almeida, Teixeira-Dias &
Medina, 2010; Pedrosa de Jesus, Teixeira-Dias & Watts, 2003; Teixeira-Dias, Pedrosa de
Jesus, Neri de Souza & Watts, 2005) found that encouraging student questioning is a key
strategy to promote active learning. Browne and Freeman (2000) also point out the
encouragement of active learning as a fertilizer of classroom criticality. The strong point of
active learning is that it eases personal interest within the subject matter, this means
provoking students into discussion, questioning and evaluation. Critical thinking arises from a
state of doubt that can only be accomplished through cognitive dissonance (Festinger, 1957),
cognitive disequilibrium (Graesser & Olde, 2003) or controversy (Browne & Freeman, 2000).
So, in order to enhance critical thinking, teachers must stimulate controversy intentionally
(Brod, 1986). However, as stated by Browne and Freeman (2000, p. 302):

“the special nature of critical thinking as an educational objective results in the necessity
to design classrooms to match the demands of a mental activity that is less than welcome for
many students.”
178 P. Albergaria-Almeida, J. J. C. Teixeira-Dias and M. Martinho

Frequently, both teachers and students feel uncomfortable with controversy. However, it
is controversy that promotes discussion and reflection, and these are both essential when
fostering critical thinking. But it is not solely discomfort that inhibits critical thinking.
In 1987, Sternberg presented several myths inhibiting the teaching and learning of critical
thinking:

1) the teacher has nothing to learn from students

2) critical thinking is exclusively the teacher’s job
3) there is a correct way to teach critical thinking
4) the more important is the “right” answer
5) discussion is a means to an end
6) the role of a course in critical thinking is to teach critical thinking

When the students and the teacher step out of their comfort zone and take risks by
experiencing controversy, “cognitive magic is possible” (Browne & Freeman, 2000, p. 308).
Pithers and Soden (2000, p. 244) underline that the teacher should aim at challenging student
ideas:

“For example, by facilitating the generation of hypotheses, the interpretation of

information or data, specification of criteria, or helping students to understand the judgmental
processes for applying principles to new situations or for making predictions”.

In the next section, we will describe the curriculum of a first year chemistry course,
aiming to promote students’ critical thinking through fostering students’ high-level
questioning.

3. HOW TO DESIGN A CHEMISTRY UNIVERSITY COURSE AIMED AT

PROMOTING STUDENTS’ CRITICAL THINKING
Having in mind the encouragement of students’ critical thinking, the curriculum of a 1st
year chemistry course, at the University of Aveiro in Portugal, aims at promoting active
learning, fostering student higher-level questioning, and consequently enhance student
criticality and promote deep approaches to learning.
Johnston (2010) suggests some activities to develop first year students’ intellectual
competences. It is important to emphasize that questioning is linked to several intellectual
skills, as shown in Table 1.
Following the suggestions of Johnston (2010) and Mbajiorgu and Reid (2006), the
teaching, learning and assessment strategies at this chemistry course for 1st year students were
conceived, designed and implemented in order to foster deep learning (Entwistle, McCune, &
Walker 2001). Almeida, Pedrosa de Jesus and Watts (2011) found that university students
adopting deep approaches to learning are frequently critical questioners. The strategies
implemented are described in the following sections.
 How Can Critical Thinking Be Recognized in the Classroom? 179

Table 1. Student development in practice

Intellectual competences Examples of student activity from practice

Curiosity, openness to new ideas and
willingness to listen to other views
Scepticism and critical awareness of
the need to gather data, information
and present evidence
Communication which is concise,
accurate and persuasive
Adapted from Johnston 2010, 54; bold added by the authors of this chapter.
Brainstorming, formulating questions,
investigating relevant information, carrying out
sustained inquiry work, discussing ideas with
peers and lecturers. Acknowledging diversity and
including all points of view
Asking focused questions, checking sources,
keeping detailed notes of practical work.
Identifying mistakes and improving performance
Devising models, making analogies and
comparisons, categorizing information, focusing
on key questions, recognizing audience needs and
academic conventions. Utilizing a variety of
communication technologies. Resisting the
temptation to plagiarize

A) The Basic Structure of the General Chemistry Course: Lectures,

Laboratory Classes and Tutorials

The General Chemistry course comprises three kinds of classes: lectures, laboratory
classes and tutorials.
Students should attend a two-hour lecture per week (not compulsory). Lectures should
provide the students with the understanding of the contents approached. These classes are of
crucial importance to students’ academic success in chemistry. Lectures should be seen as
hours of active study. Before each class, students are expected (i) to read the material
provided by the teacher; (ii) to identify topics that could represent obstacles to learning; and
(iii) to identify topics that could raise doubts or questions. During lectures students are
stimulated to ask questions either orally or in a written format, as we will explain later.
The main aim of the laboratory classes is to initiate the student in the research process in
chemistry. Simultaneously, it is expected that students will learn to appreciate chemistry as an
experimental science. To accomplish this aim, all laboratory tasks are planned and performed
individually. During laboratory classes, the teacher acts as a facilitator, assisting students to
overcome their difficulties. Each student must attend a two-hour laboratory class per week.
Practical classes are compulsory.
Practical laboratory sessions were conceived in order to promote the development of
concepts and understanding and not merely as a handmaid of lectures. In laboratory classes,
the students have opportunities to: (a) identify the main objectives of the work; (b) identify
and overcome any conceptual and practical difficulties encountered; (c) plan and execute the
work; (d) record and discuss the results and observations in their lab book; (e) answer the
questions raised in their laboratory manual
180 P. Albergaria-Almeida, J. J. C. Teixeira-Dias and M. Martinho

Tutorials provide the student with guidance about learning methods. Students should use
these classes to clarify their doubts and/or ask questions about the contents taught in lectures
and also about the practical tasks. Each student is expected to attend a two-hour tutorial per
week, but these are not compulsory classes.

B) Innovative Teaching and Learning Strategies

Besides the formal classes described earlier, the following strategies were also
implemented:

1) small pauses during lectures to encourage students’ oral questions. In the middle of
the lesson, the teacher stopped lecturing for two or three minutes, and invited the
students to think about or to discuss the class topics with their colleagues. At the end
of the break, students had the opportunity to raise oral questions. If the students felt
more comfortable, they could write their questions instead, and the teacher would
answer orally at the beginning of the next lesson. Some instances of students’
questions asked after these pauses are:

Teacher, could you explain better what is anthropomorphism?

Are all dry cell batteries not rechargeable?

2) teacher’s written questions during lectures to facilitate the organization of teaching

and learning and to serve as a role model to students. For instance, throughout the
‘Water and aqueous solutions’ topic, the teacher presented seventeen written
questions. These had diverse degrees of difficulty and served different functions.
Some instances of the written questions drawn from our project are:

How can you describe the polarity of a water molecule?

Why are there substances that are gaseous, others are liquids and others are solids?
In the absence of gravity, why are water drops exactly spherical?

3) ‘Questions and answers in Chemistry’ online forum to encourage and facilitate

students’ questioning. Students could use this tool to ask written questions related to
the topics taught during lectures and/or practical laboratory sessions. Questions
related to everyday phenomena with a chemical background were also welcomed.
The teacher answered all questions within two days, also on the online forum. All
questions and answers are available online to all chemistry students. An instance of a
written question asked through the online forum is:

I think the teacher said that in the reaction NH3(g) + HCl(g) -> NH4+Cl-(s), the
reagents have a lower energy than the products. However, it was also said that the energy
of the Cl- and NH4+ pair is higher than the energy of the pair Cl-H and NH3. Could you
confirm this information and explain why this happens?
 How Can Crritical Thinking Be Recognized in the Claassroom? 181

4) ‘Probllem-based casses’ online foorum to encouurage studentts to ask queestions and

suggesst possible expplanations for the phenomenna proposed by b the teacherr. This kind
of actiivity also aimmed to enhannce the discuussion betweeen students. All A of the
problemm-based cases proposed too students werre based on reeal-life situatiions with a
social, ecological or
o technologiccal impact onn society. Sttudents were invited to
analyze these situatiions through the
t eyes of a scientifically
s i
informed citizzen. One of
oblem-based cases
the pro c proposedd to the studennts is shown inn Figure 1.

Problem-b based case “Acid rain”

Having in mind your knnowledge aboutt acids and bassis, ask questioons or proposee possible
e
explanations fo
or the followingg phenomenonn:

Marble an nd limestone are a the most usedu materials in the construuction of builddings and
monuments. Bo
m oth marble andd limestone aree composed of calcium carboonate (CaCO3), differing
o
only in the crysstalline structuure. Even if booth are known by
b their high durability,
d builddings and
m
monuments bu
uilt with marblle and limestonne are graduallly eroded throough the action of acid
r
rain. How can we explain thiss phenomenon?

More inforrmation about acid rain availlable in: Atkinss, P. and Loretta, J. (2005). Chemical
Principles – Thee Quest for Insiight (3rd ed.). New York: W. H.
P H Freeman and Company. p. 3996.

Fiigure 1. Problem
m-based case abbout acid rain.

Fiigure 2. Example of an interacttion moment beetween studentss, in one of the online

o forums (in
Poortuguese).
1882 P. Albergariaa-Almeida, J. J.
J C. Teixeira--Dias and M. Martinho
M

Figure 2 presents an a example of o an interactiion moment between

b studeents in the
online forum aboutt acid rain. It I was interessting to find that besides proposing
explannations for thee situation prooposed, studennts also raisedd questions rellated to the
acid raain topic. For instance: We talk about accid rain, but is i there basic rain? The
situatio
ons proposed in these forrums aimed at a triggering perplexity orr cognitive
disequilibrium (Graeesser & Olde, 2003), leadingg to the discusssion among students.
s

5) chemisstry mini-projects where thhe students aree given the oppportunity to creatively
develoop a small-group project on a chemistry topic.
t Besidess scientific knoowledge, it
is also expected thatt students develop other kinnds of compettences, such ass the social
compeetence. Accordding to Daviees (2009), grooup work is a learning strrategy that
compriises a social dilemma,
d in which
w learnerss face contradictory demandds between
self-intterest and alttruism. Mini-pprojects compprise nine phhases, as summ marized in
Figure 3.

Phase 1 – Tea
P am Organisati tion and Regiistration
The organization and registration
r off the teams was
w facilitatedd through thee use of a
sooftware speciffically conceivved for this puurpose.

P
Phase 2 – Top
pics Randomly
ly Assigned within
w Teams

1
• Team orrganization and registration

2
• Topics raandomly aassigned w
within teaams

3
• Readingg the papeer and asking questiions

4
• Meetingg with thee teacher

5
• Proposing up to two researrch questiions

6
• Writing,, producin
ng and delivering th
he poster

7
• Oral preesentation
n of the po
osters

8
• Evaluation of the mini‐reseearch projects

9
• Exhibitio
on of the posters

Fiigure 3. Processs of developmennt of the mini reesearch projectss.

 How Can Critical Thinking Be Recognized in the Classroom? 183

Phase 3 – Reading The Paper and Asking Questions

Each team should read the paper related to their topic. All of the papers were chosen from
‘Scientific American’, as shown in Table 2. Moreover, after reading the paper, the students
should ask questions about scientific contents and post them in the online forum created to
support the development of mini-projects. This forum was only accessible to the students that
subscribed to the chemistry mini-projects.
Some instances of questions posted in the online forum are:

• According to the Scientific American paper “The secrets of stardust”, the

particles of dust originated in comets contain organic materials and also contain
structures that can promote chemical evolution if these structures are underwater.
What exactly are these “structures”? And how do these structures promote
chemical evolution when underwater?
• The temperature needed to transform liquid hydrogen into solid hydrogen is
about 14 Kelvin. However, if the pressure rises, then the temperature will also
rise. Thus, to perform this transformation, an extremely low temperature is
needed as well as a high pressure. Thus, at which temperature does liquid
hydrogen becomes solid hydrogen?
• This paper explains the importance of rocks in the origin of life on our planet.
However, we are curious to know why carbon was the most important atom in
the origin of life. Was it possible to substitute carbon by a different element?
Would it be possible for another element to promote the same kind of bonds
between atoms?

Table 2. Mini-projects topics and corresponding Scientific American paper

Topic - Title of the Scientific American Paper Year Month

Melting below zero 2000 February
The secrets of stardust 2000 December
Intelligent gels 1993 May
The evolution of the periodic system 1998 September
Combinatorial chemistry and new drugs 1997 April
New chemical tools to create plastics 1997 May
Catalysis on surfaces 1993 April
Metal clusters and magic numbers 1997 December
Nanotechnology and the double helix 2004 June
Little green molecules 2006 March
Zinc fingers 1993 February
Simulating water and the molecules of life 1998 November
Making new elements 1998 September
How should chemists think? 1993 February
The ice of life 2001 August
Drugs by design 1993 December
Making metallic hydrogen 2000 May
The complexity of coffee 2002 June
Life’s rocky start 2001 April
Creating nanophase materials 1996 December
184 P. Albergaria-Almeida, J. J. C. Teixeira-Dias and M. Martinho

Phase 4 – Meeting with the Teacher

At this stage, students should have the initiative to schedule a meeting with the teacher.
The teacher met with each group separately. Each meeting lasted 30 minutes. In these
meetings, the students must raise questions about their topic and the teacher will only provide
appropriate feedback, orientation and guidance for students and answer their questions.

Phase 5 – Proposing Up to Two Research Questions

Students were asked to raise up to two research questions that should guide their research
projects and their posters. Some instances of these questions are:

• Is it useful to spend millions of Euros in the research and discovery of new

compounds that currently do not have any utility?
• To what extent is the study of the stardust important for the humankind?
• What is the environmental impact of the superficial fusion?
• How can melting below zero be explained?

Phase 6 – Writing, Producing and Delivering the Posters

During this phase, each group selected and organized the information according to their
research questions and produced the poster. Figure 4 shows a poster produced by the team
researching “Drugs by design.”

Figure 4. Poster about the topic “Drugs by design.”

Phase 7 – Oral Presentation of the Posters

Each ‘project team’ presents its project to the other students and to teachers. Each
presentation is subjected to questions from both peers and teachers. Some instances of these
questions are:
 How Can Critical Thinking Be Recognized in the Classroom? 185

• Why are there some medicines (such as the H1N1 vaccine) that take a short time
for being produced while other drugs take so many years for being discovered
and produced? Why is there this difference?
• You said the flavor of the coffee had its origins in the amino acids. How does
this happens? Are the amino acids bonded to the caffeine?

Phase 8 – Evaluation of the Mini-Research Projects

After the presentation of the posters, the students were evaluated based on: (i) the
performance in each of the following tasks: asking questions about the Scientific American
paper, meeting with the teacher, asking the guiding questions; (ii) the scientific quality of the
poster; (iii) the aesthetic quality of the poster; (iv) the presentation of the poster; (v) the
answers to the questions raised by both peers and teachers; (vi) the questions asked during the
presentation session. The score for each student’s participation in the mini-research projects
was added to the final grade of the student.

Phase 9 – Exhibition of the Posters

After the presentation session, the posters were exhibited for a month in the building
dedicated to the teaching of first-year students of Science and Technology.

C) Assessment Strategies

In order to promote the alignment between teaching, learning and assessment (Biggs &
Tang 2007; Pedrosa de Jesus & Moreira, 2009), the following assessment strategies were
considered:

1) a multiple-choice test due to the large number of students in the chemistry course.
This test also included two open questions: a question-posing case (students should
raise questions) and a problem-based case (students should propose explanations) as
shown in Figure 5.
2) participation in the two online forums considering both the number and the quality of
the participation;
3) performance in practical laboratory work considering both students’ performance in
practical classes and the content of the lab book;
4) participation in the mini-projects considering the development of the project, the
quality of the poster, the quality of the poster presentation, and the questions asked
during the mini-projects presentation session.
1886 P. Albergariaa-Almeida, J. J.
J C. Teixeira--Dias and M. Martinho
M

In the lastt decades, as thhe harmful connsequences of the acidic atm

mospheric polluutants has
bbecome more evident
e in the 1960s, the Norrth American and
a the Europeean Union govvernments
s
started regulatiing the emissioons of the acid atmospheric pollutants
p (mainnly sulphur diooxide and
n
nitrogen oxidess).

in forests in lakees in ecosystemss in public mon

numents

However,

- in some parts of Euroope and North America, the rain was stilll acidic, in spiite of the
ssevere control of pollution;
- in some ecosystems
e – mainly
m forests – the negativee impacts of acid rain were suuperior to
t
those foreseen by the governmments. Discusss, explain, com
mment on these observations ini detail.

Fiigure 5. Problem
m-based case abbout acid rain.

CONCLUSION
N

The curricuulum of this first year chemistry coursee shows that it i is possible to create a
quuestion-based learning envvironment com mbining tradiitional classess, such as leectures and
laaboratory pracctical classes, and innovativve strategies such
s as mini-rresearch projeects, online
foorums and problem-based cases. Actuallly, even the strategies typpically teacheer-centered,
suuch as lectures, included soome innovative features in ordero to prom
mote students’ motivation
annd, consequen ntly, engage thhem in criticall learning. It is the teacher’s responsibilitty to create
a learning atm mosphere that fosters studennts’ active leaarning. But it is the studennts’ role to
acctively engagee in this learning environmeent, developingg their criticality.
As stated by
b Halpern (19999, p. 70),

“criticaal thinking skillls can be taughht and learned, and when studdents learn thesse skills
and apply thhem appropriateely, they becomme better thinkerrs.”

REFFERENCESS
Almeida, P., Peedrosa de Jesuus, H., & Wattts, M. (2008).. Developing a mini-projectt: students’
A
questions and
a learning sttyles. The Psychology of Edducation Revieew, 32(1), 6-177.
A
Almeida, P., Pedrosa
P de Jesus,
J H. & Watts, M. (2011).
( Kolb’’s Learning Styles and
Approachees to Learning through the Use
U of Studennts’ Critical Quuestions. In S. Rayner &
 How Can Critical Thinking Be Recognized in the Classroom? 187

E. Cools (Eds.), International Perspectives On Style Differences In Human Performance:

Leading Edge Research, Theory And Practice (pp.115-128). New York: Routledge.
Almeida, P., Teixeira-Dias, J. J., & Medina, J. (2010). Enhancing the Scholarship of Teaching
and Learning: the interplay between teaching and research. International Journal of
Teaching and Case Studies, 2(3/4), 262-275.
Biggs, J., & Tang, C. (2007). Teaching for Quality Learning at University (3rd Ed). England
& NY: Society for Research into Higher Education & Open University Press.
Boyer Commission on Educating Undergraduates in the Research University (1998).
Reinventing Undergraduate Education: A Blueprint for America’s Research Universities.
Retrieved October 2010 from http://naples.cc.sunysb.edu/Pres/boyer.nsf
Braxton, J. M. & Nordvall, R. C. (1985). Selective liberal arts colleges: higher quality as well
as higher prestige? Journal of Higher Education, 56, 538-553.
Brod, H. (1986). Philosophy teaching as intellectual affirmative action. Teaching Philosophy,
9, 1-10.
Browne, M., & Freeman, K. (2000). Distinguish Features of Critical Thinking Classrooms.
Teaching in Higher Education, 5(3), 301-309.
Browne, M. & Keekey, S. M: (1998). Asking the right questions: a guide to critical thinking.
New Jersey: Prentice Hall.
Chin, C., & Osborne, J. (2008) Students’ Questions: a potential resource for teaching and
learning science. Studies in Science Education, 44 (1), 1-39.
Commission of the European Communities (2006). Recommendation of the European
Parliament and of the Council on key competences for lifelong learning. Official Journal
of the European Union, 10-18.
Cuccio-Schirripa, S., & Steiner, H. E. (2000). Enhancement and analysis of science question
level for middle school students. Journal of Research in Science Teaching, 37, 210-224.
Davies, W. M. (2009). Groupwork as a form of assessment: common problems and
recommended solutions. Higher Education, 58, 563-584.
Dawes, R. M. (1988). Rational Choice in an Uncertain World. Fla.: Harcourt Brace.
Entwistle, N. J., McCune, V., & Walker, P. (2001). Conceptions, styles and approaches
within higher education: analytic abstractions and everyday experience. In R. J.
Sternberg, & L. F. Zhang (Eds.), Perspectives on Cognitive, Learning and Thinking styles
(pp. 103-136). New Jersey: Erlbaum.
Faccione, P., Facione, N. & Giancarlo, C. (1996). The motivation to think in working and
learning. In E. Jones (ED.), Preparing Competent College Students: Setting New and
Higher Expectations for Student Learning (pp. 67-79). San Francisco: Jossey-Bass.
Festinger, L. (1957). A theory of cognitive dissonance. Stanford, CA: Stanford University
Press.
Graesser, A. C. & Olde, B. A. (2003) How does one know whether a person understands a
device? The quality of the questions the person asks when the device breaks down.
Journal of Educational Psychology, 95 (3), 524-536.
Halpern, D. F. (1999). Teaching for Critical Thinking: helping college students develop the
skills and dispositions of a critical thinker. New Directions for Teaching and Learning,
80, 69-74.
Halpern, D. F. (1998). Teaching Critical Thinking for Transfer Across Domains: disposition,
skills, structure training, and metacognitive monitoring. American Psychologist, 53, 449-
455.
188 P. Albergaria-Almeida, J. J. C. Teixeira-Dias and M. Martinho

Johnston, B. (2010). The First Year at University: Teaching Students in Transition. Berkshire:
The Society for Research into Higher Education and Open University Press.
Kahane, H. (1997). Logic and Contemporary Rhetoric. Belmont: Wadsworth.
Mayer, J. (1986). Teaching critical awareness in an introductory course. Teaching Sociology,
14, 249-256.
Mayer, R. E. (1992). Thinking, Problem Solving, Cognition. New York: Freeman.
Mbajiorgu, N., & Reid, N. (2006). Factors Influencing Curriculum Development in
Chemistry. Hull: Higher Education Academy Physical Sciences Center.
Meyer, M. (1994). Rhetoric, Language, and Reason. Pennsylvania: Pennsylvania State
University Press.
Pedrosa de Jesus, H., & Moreira, A. (2009). The role of students' questions in aligning
teaching, learning and assessment: a case study from undergraduate sciences. Assessment
and Evaluation in Higher Education, 34(2), 193-208.
Pedrosa de Jesus, H., Teixeira-Dias, J. J. C., & Watts, M. (2003) Questions of Chemistry.
International Journal of Science Education, 25 (8), 1015-1034.
Pithers, R. T. & Soden, R. (2000). Critical thinking in education: a review. Educational
Research, 42(3), 237-249.
Rabinowitz, M. (1993). Cognitive Science Foundations of Instruction. Hillsdale, N. J.:
Erlbaum.
Raths, L. E., Wasserman, S., Jonas, A. & Rothstein, A. (1966). Teaching for Critical
Thinking: Theory and Application. Columbus, Ohio: Charles-Merrill.
Ruggerio, V. R. (1996). Becoming a Critical Thinker. Boston: Houghton Mifflin.
Shaw, V. F. (1996). The cognitive processes in informal reasoning. Thinking and Reasoning,
2, 51-80.
Sternberg, R. J. (1987). Teaching critical thinking: eight ways to fail before you begin. Phi
Delta Kappan, 68, 456-459.
Teixeira-Dias; J. J. C., Pedrosa de Jesus, H., Neri de Souza, F., & Watts, M. (2005).
Teaching for quality learning in chemistry. International Journal of Science Education, 27(9),
1123-1137.
 INDEX

attribution, 139
# authorities, 5, 19, 27, 75, 137
authority, 24, 32, 43, 44, 47, 52, 54, 67, 81, 176
21st century, 84, 119, 157
autonomy, ix, 4, 71
aversion, 91
A avoidance, 26
awareness, ix, 5, 8, 42, 51, 62, 70, 144, 152, 161, 179
abstraction, 37
academic settings, 117
B
academic success, 179
access, 18, 32, 107, 136
balance sheet, 127
accounting, 124, 126, 127, 129, 139
base, 88
acid, 181, 182
batteries, 180
action research, x, 90, 159, 163
behaviors, x, 24, 141, 142, 143, 144, 148, 149, 150,
adaptability, 56
151, 152, 153
adaptation, 143
belief systems, 155
adults, 7, 33
benchmarking, 93
age, 87
benefits, 98, 104, 167, 168
agencies, 52, 58
Bhutan, 132
alertness, 79
bias, 26, 44, 79, 91, 112, 129
allocated time, 168
blogs, 132, 133, 136, 140
amplitude, 131
bones, 133, 164
anchoring, 7
boredom, 136
anthropology, 127, 129
Botswana, 66
anxiety, 63, 167
brain, 21, 24, 25, 33
APA, 5, 6, 8, 101
brain activity, 25
aptitude, 6
brain damage, 21
aqueous solutions, 180
Britain, 4
argument mapping, ix, 97, 106, 117
buns, 137
Aristotle, 3, 73, 87
business ethics, 67
articulation, 46
Asian crisis, 138
assessment, vii, xi, 1, 2, 3, 6, 7, 10, 12, 16, 17, 19, C
20, 21, 22, 27, 28, 29, 31, 35, 83, 89, 93, 95, 101,
104, 105, 116, 117, 118, 124, 127, 128, 129, 175, cabinet, 40
176, 178 calcium, 181
assessment tools, 101 calcium carbonate, 181
assets, 127 calibration, 11
assimilation, ix, 32, 97, 98, 104, 105, 106 capital consumption, 127
atoms, 40 capital punishment, 9
190 Index

carbon, 128 commercials, 82

carbon dioxide, 128 common sense, 40
case studies, 21, 44, 163, 171 communication, 14, 39, 47, 52, 59, 77, 133, 136,
case study, 23, 94 138, 165, 179
causal inference, 21 communication technologies, 179
causal reasoning, 23 communities, 21, 88, 91
causation, 26 community, 4, 21, 39, 42, 58, 61, 72, 80, 85, 176
CCTDI, viii, 31, 70, 72, 79, 85, 110, 111, 112, 113, compilation, 55
115 complement, 55, 57
central executive, 33 complexity, 12, 21, 28, 40, 42, 43, 55, 58, 59, 81,
challenges, 42, 51, 72, 74, 84, 86, 98, 101, 127, 157, 160, 168
161, 165, 168 compliance, 132
chaos, 58, 59, 168 composition, 71, 129, 130, 135
chemical, 180 comprehension, 59, 118, 119, 176, 177
Chicago, 88, 89, 91, 129 computer, 42, 106, 120, 143, 154, 156
children, 33, 88, 90, 91 computer use, 154
China, 128 concept map, 155
citizens, 51, 54, 131, 134 conception, 7, 27, 89, 127, 133
citizenship, x, 3, 94, 159 conceptual model, 51, 61
civil society, 130, 131 conceptualization, 2, 27, 99
clarity, vii, 1, 6, 19, 27, 75, 76, 78, 98 conference, 93, 134, 137, 169
classes, 14, 16, 117, 176, 179, 180 confidentiality, 110
classification, 55, 75, 118 configuration, 136, 137
classroom, 4, 14, 18, 30, 32, 88, 92, 94, 116, 119, confinement, 131
142, 165, 167, 176, 177 conflict, 43, 49, 55
cleavage, 140 conflict resolution, 55
climate, 128, 131 confrontation, 84, 86, 140
climate change, 128 Confucius, 69
closure, 80 congress, 23, 87
coercion, 52 consciousness, 47, 49, 52, 53
cognition, ix, 6, 10, 21, 29, 30, 31, 32, 70, 86 consensus, 30, 42, 79, 81, 87, 99, 100, 132, 133, 160
cognitive abilities, 117, 119 constituents, 44, 47
cognitive ability, 7, 9, 34 construct validity, 7
cognitive activity, 10, 84 construction, viii, 27, 70, 72, 84, 85, 100, 106, 128,
cognitive capacities, 116 130, 135, 140, 176, 181
cognitive dissonance, 177 constructivism, 39, 94
cognitive level, ix, 26, 70, 86 consumption, 75, 126, 128, 129
cognitive load, 105, 117 contradiction, 132
cognitive map, 39 control condition, 109, 113, 115, 116
cognitive process, 6, 10, 11, 20, 60, 115, 176 control group, x, 11, 13, 97, 107, 111, 113, 115
cognitive psychology, 5, 6 controlled research, 23
cognitive representations, 21 controversial, 24, 32, 128
cognitive skills, 6, 15, 27, 33, 78, 92 controversies, 137
cognitive style, 8 convention, 135
cognitive system, 99 conversations, 41, 163
cognitive tool, 89 cooperation, 134, 137
cognitive variables, 26 cooperative learning, 143
coherence, 87, 99, 125, 126 coordination, 125
collaboration, 162 Cornell Critical Thinking Test, viii, 70, 101, 119
college students, 3, 8, 11, 18, 26, 30, 79, 95 correlation, viii, 70, 107, 114, 115, 148, 149, 150,
colleges, ix, 70, 87 151, 153
colonisation, 48 correlation coefficient, 148, 150
color, 79 correlations, 152, 153
 Index 191

corruption, 73 disequilibrium, 177, 182

course content, 12, 13, 14, 34, 104 disorder, 137
course work, 14, 15, 16 disposition, 1, 9, 10, 16, 19, 34, 79, 82, 88, 90, 95,
covering, 15 109, 110, 111, 113, 114, 115, 120, 155, 177
creative thinking, 5, 6, 75, 94, 142, 176 dissatisfaction, 127
creativity, 5, 86, 89 distribution, 127, 128
crises, 60 distribution of income, 128
critical academic writing, xi, 159, 164, 165, 167, 169 diversity, 179
criticism, 3, 76, 86, 126, 130, 131, 133, 135, 167 dough, 134
crystalline, 181 draft, 128, 140, 165, 167
CST, vii, 38, 55, 66 drawing, 23, 55, 77, 84, 85, 86, 117, 163, 164
cues, 18, 105, 109, 143 dreaming, 98
cultivation, x, 97, 106 drugs, 13
cultural heritage, 136 durability, 130, 181
culture, 132
currency, 42, 63, 134
curricula, ix, 60, 71, 73, 86, 143 E
curriculum, ix, xi, 13, 70, 71, 72, 78, 82, 86, 88, 89,
ecology, 128
93, 94, 98, 104, 106, 120, 155, 175, 178
economic activity, 132
cycles, 61, 163
economic consequences, 135
economic growth, 124
D economic indicator, 132
economic performance, 123, 124, 129, 134, 135, 138
damages, 123, 128 economic theory, 129
data collection, 163 economics, 123, 124, 125, 129, 130, 133, 135, 136,
data set, 173 138
decision makers, 124 education, viii, ix, x, 2, 4, 17, 28, 32, 33, 34, 43, 58,
deduction, 78 61, 69, 70, 71, 72, 73, 74, 76, 78, 80, 83, 84, 85,
deductive reasoning, 11, 30, 110, 113, 116 86, 87, 88, 90, 92, 93, 94, 95, 97, 98, 99, 118,
deep learning, 178 120, 127, 128, 133, 153, 154, 155, 156, 159, 161,
defects, 124 167
defence, 66 educational assessment, 30
deficiencies, vii, 1 educational institutions, 74
deficit, 161 educational objective, 2, 28, 32, 118, 177
Delta, 29 educational practices, 171
delusion, 81 educational psychology, 3
democracy, ix, 71, 74 educational research, xi, 7, 99, 159, 168, 169
Democrat, 131 educational system, ix, 70, 85
democratisation, 53 educators, 2, 4, 5, 6, 10, 17, 74, 75, 83, 117, 142
demography, 129 egocentrism, 77
Department of Education, 98, 101, 120, 175 e-learning, 156, 157
dependent variable, 21, 110, 150 elementary teachers, 88
depth, x, 124, 159, 173 emotion, 30
designers, 55, 62 emotional intelligence, 142, 155
destiny, 136 emotional state, 26
detection, 84 empirical methods, 6
development of thinking skills, ix, 70 empirical studies, ix, 10, 71
developmental process, 171 employability, x, 159
dialogues, 12, 73 employees, 58
dichotomy, 51 employers, 98
direct measure, 115, 116 employment, 54, 142, 144, 152, 154
directives, 47 encoding, 105
discomfort, 178 encouragement, xi, 175, 177, 178
192 Index

enculturation, 160 formation, vii, 1, 9, 22, 46, 143, 176

enemies, 49, 50, 136 foundations, 34
energy, 180 fragility, 92
engineering, 39, 42, 56, 60, 63 framing, 37, 61, 63
England, 38 France, 87, 123, 124, 125, 126, 131, 132, 134, 135,
environment, x, 25, 48, 49, 50, 75, 92, 124, 130, 132, 136, 139, 140
135, 160 freedom, ix, 39, 52, 71, 74
environmental aspects, 128 Freud, 23
environmental conditions, 128 full employment, 133
environmental management, 59 functional approach, 27, 30, 35
environmental sustainability, 135 functionalism, 44, 56
epistemology, 34, 43, 53
equilibrium, 44
ethical issues, 40, 118, 175 G
ethics, 50, 73
game theory, 43
ethnicity, 87
gender equality, 154
Europe, 64, 95
general education, 120
European Commission, 123
generalizability, 8, 17, 30, 32, 33, 34
everyday life, 81, 82
geography, 129
evidence, vii, x, 3, 8, 9, 10, 12, 14, 15, 16, 17, 18, 20,
Germany, 66, 87
21, 23, 24, 25, 26, 32, 77, 79, 82, 99, 100, 103,
gifted, viii, 70, 71
104, 105, 109, 112, 152, 159, 166, 173, 177, 179
goal-setting, 20
evolution, x, 9, 124
God, 61
exclusion, 129, 130
goods and services, 132
execution, 7
google, 37
exercise, 51, 71, 74, 83, 110, 160
governance, 136
expenditures, x, 124
government policy, 132
experimental condition, 109, 110, 113, 115
governments, 64, 134
experimental design, 145
grades, 92, 98
expertise, 24, 34, 48, 49, 50, 51, 54, 88, 120, 129
gravity, 180
exploitation, 127
Great Britain, 3
exploratory model, xi, 159
Greece, 87
exposure, 12, 54, 167
Gross Domestic Product (GDP), v, x, 123, 124, 125,
extraneous variable, 25
126, 129, 130, 131, 133, 135, 136, 137, 138
grounding, 13, 37, 76, 78, 83
F grouping, 6
growth, 99, 105, 109, 116, 124, 131, 132, 155
factor analysis, 8 guidance, 6, 18, 61, 107, 169, 171, 180
faith, 60, 135 guidelines, 51, 165
fear, 26, 57, 61, 63, 131, 167 guiding principles, 55
feelings, 132, 143
financial, 125, 126, 133
financial crisis, 133 H
financial markets, 125
happiness, 123, 125, 129, 132, 133, 136, 139
fishing, 128
harmony, 79
fitness, 155
health, 43, 58, 128, 132
flexibility, 8, 9
helplessness, 50
flights, 99
hermeneutics, 46
fluctuations, 127
high school, 71, 72, 79, 82, 85, 94, 95
force, 43, 50, 54, 76, 134, 153
higher education, vii, x, 29, 81, 97, 159, 160, 161,
Ford, 105, 120
162, 171
formal education, 72, 86
formal reasoning, 21, 88
 Index 193

higher-order thinking, viii, ix, 69, 70, 72, 78, 84, 85, insecurity, 125, 128
86, 94 institutional change, 44, 46
history, 32, 129, 133, 136, 139 institutionalisation, 125
holism, 37, 57, 61, 64 institutions, 124, 126, 129, 161, 171
homework, 107 instructional design, 144, 152, 154
homogeneity, 113 instructional materials, 105
horses, 134 instructional methods, 104
house, 59 instructional practice, 12, 28
household income, 126, 129 integration, 2, 92
housing, x, 58, 124 intellectual flexibility, 74
human, 8, 34, 39, 42, 43, 44, 47, 48, 49, 52, 59, 62, intelligence, 9, 31, 93
73, 84, 117, 124, 125, 127, 129, 131 intentionality, 50, 52
human activity, 52 interest groups, 46
human capital, 127, 131 International Labour Organisation, 125
human development, 124, 125 interrelatedness, 42, 55, 78
human experience, 117 interrelations, 78
human intentionality, 48, 52 intervention, x, 39, 55, 59, 97, 99, 101, 104, 107,
human perception, 43 110, 113, 116, 144, 152, 159, 171
humanism, 44, 53 investment, 127, 169
hypothesis, 5, 19, 103, 106, 109 investments, 138
hypothesis test, 5 Ireland, 97, 110, 160, 161, 171
Israel, ix, 71, 72, 74, 85, 94
Israeli educational system, ix, 70
I issues, viii, 14, 29, 32, 38, 40, 42, 43, 47, 48, 53, 58,
60, 63, 73, 74, 77, 79, 82, 98, 101, 102, 123, 126,
ideal, 30, 32, 33, 34, 50, 51, 54, 73, 74, 76, 107, 115,
127, 129, 139, 154, 155, 157, 165, 176
117, 136, 153
idealism, 39
ideals, ix, 71, 74 J
identification, 15, 19, 117
identity, 28 jobless, 137
ideology, 47, 53, 66 Jordan, 91
image, 44, 58 journalists, 131, 133
imagery, 50 junior high school, 156
imagination, 81, 98 justification, 30, 76, 105
imbalances, 112
IMF, 138
Immanuel Kant, 4, 37, 38, 48 K
immersion, vii, 1, 12, 13, 16, 27, 82, 104, 161
Keynes, 37
improvements, 40, 144, 151, 152, 169
Keynesian, 135
in transition, 2
kill, 35, 109
incidence, 47
knowledge acquisition, 16
income, 128
incompatibility, 116
independence, 139 L
independent variable, 150, 151
individual differences, 8, 10, 13, 34, 116 laptop, 110
individuals, ix, 10, 46, 71, 74, 80, 129, 138, 143, 169 laws, 23, 38
induction, 23, 31, 92 lead, ix, 7, 20, 22, 32, 43, 70, 79, 133, 144
industry, 58, 132 leadership, 130
inequality, 130 learners, 4, 72, 85, 153, 176
inferences, 26, 32, 78, 98, 99, 100, 103, 177 learning environment, 143, 164, 176
infusion approach, viii, 13, 18, 28, 29, 69, 81, 82, 87, learning process, 106, 143, 155
104, 154 lesson plan, 71, 92
194 Index

liberalisation, 47 meta-analysis, 13, 14, 28, 104, 106, 107, 115, 118,
liberalism, 54 119
liberation, 49 metacognition, 1, 10, 12, 19, 28, 32, 34
life expectancy, 134 metacognitive skills, 16, 20, 28, 31, 99, 105
life satisfaction, 128 metaphor, 55, 56, 60
lifelong learning, 175 methodological procedures, 44
light, viii, x, xi, 41, 42, 69, 73, 78, 82, 86, 97, 98, 99, methodology, 23, 39, 44, 46, 47, 48, 53, 55, 56, 57,
100, 104, 124, 137, 175 61, 67, 90
limestone, 181 military, 42
liquids, 180 Ministry of Education, ix, 70, 82, 91
literacy, vii, 38, 40, 43, 57, 58, 59, 62, 63, 72, 86, 90, misconceptions, 29, 154
92, 159, 160, 171, 172 mission, 3, 43
localization, 21 mixing, 67
logical reasoning, 11, 30, 106 modelling, 43
logistics, 42 models, vii, 5, 7, 15, 26, 27, 32, 33, 44, 51, 60, 61,
longitudinal study, 32, 118 62, 86, 98, 161, 162, 179
love, 94 modernism, 65
modules, 107
molecules, 40
M monopoly, 136, 138
Moon, 2, 33
magnetic resonance, 24
Morocco, 136
magnetic resonance imaging (MRI), 24
motivation, 26, 29, 30, 48, 49, 50, 51, 77, 83, 88,
magnitude, 137
115, 117, 172, 176
majority, 131
multiplication, 127
man, 43, 59, 60, 124, 137
mutations, 137
management, 39, 43, 44, 57, 59, 61, 66, 98, 119, 136,
138, 168
manipulation, 22, 25, 27 N
mapping, vii, ix, 27, 49, 51, 97, 98, 106, 107, 109,
113, 116, 117, 118, 120 naming, 38, 39
Maryland, 1 National Health Service, 58, 62
mass, 134 National Research Council, 92
materials, ix, 14, 17, 21, 30, 70, 82, 98, 105, 110, natural resources, 128
111, 117, 177, 181 natural science, 130
mathematical knowledge, viii, 70, 72 natural selection, 8
mathematics, viii, ix, 69, 70, 71, 72, 81, 83, 85, 86, negative mood, 26
87, 88, 90, 91, 92, 93 neglect, 20, 46, 47, 53
mathematics education, 87, 93 neoliberalism, 129
mathematics studies, viii, ix, 69, 71, 84 Netherlands, 88
matrix, 55, 58, 66 neuroscience, 21, 24, 25
matter, vii, 1, 2, 12, 15, 16, 17, 27, 28, 43, 49, 80, neutral, 26, 30, 42, 44
103, 104, 127, 131, 168, 169, 177 New Zealand, 172
measurement, 7, 32, 40, 90, 99, 115, 116, 118, 139 NGOs, 125
measurements, 21, 119 nursing, 101, 118
media, 13, 75
mediation, 29
memory, 5, 16, 35, 109, 138 O
mental activity, 99, 177
objectivity, 54
mental image, 106
observable behavior, 7, 20
mental representation, 22
obstacles, 106, 127, 179
mental state, 25
officials, 124, 129
mentor, 48
open-mindedness, 8, 9, 10, 79
messages, ix, 71, 74
 Index 195

openness, 9, 179 positivism, 39, 47

operations, 39, 42, 43, 48, 176 postmodernism, 67
operations research, 39, 42, 43, 48 poverty, 130, 132
opportunities, 60, 84, 143, 152, 160, 167, 177, 179 power relations, 40, 44, 52
oppression, 52 practical activity, 78, 81
optimism, 63, 137 pragmatism, 57, 67
organism, 55 preparation, 11, 73, 94, 130
organize, 14, 23 preservice teachers, x, 141, 144, 156, 157
originality, 74 president, 123, 124, 126, 131, 133, 134, 136, 138,
overlap, 22 139
oxygen, 133 prima facie, 7
primacy, 51
principles, vii, 1, 2, 5, 12, 13, 14, 15, 16, 17, 18, 19,
P 23, 27, 43, 47, 74, 81, 82, 102, 103, 178
prior knowledge, 143, 149
Pacific, 29, 35, 90, 94, 118, 119
probability, vii, 3, 82, 87, 102
parallel, viii, 44, 55, 69, 72, 86
probe, 46, 117
participants, ix, x, 9, 10, 11, 25, 26, 70, 83, 86, 107,
problem solving, vii, 5, 6, 15, 21, 31, 33, 43, 57, 62,
110, 115, 148, 152, 159, 160, 163, 164, 165, 166,
77, 78, 80, 88, 90, 91, 92, 94, 98, 105, 120, 142,
167, 168, 169
143, 176
pattern recognition, 18
problem-based learning, 156
pedagogy, 73, 172
problem-solving strategies, 88
peer assessment, 119
production costs, x, 124
peer review, 165, 167, 168, 171, 172
professional development, 85, 156, 160
percentile, 110
professional growth, 142, 152
performance appraisal, 60
professionals, ix, 70, 87
performance indicator, 61
profit, 30, 58, 128
performance related pay, 61
project, 53, 88, 103, 123, 125, 126, 130, 132, 134,
performers, 11
136, 171, 173, 180
permit, 14, 23
projective test, 131
personality, 74, 76, 155
proposition, 106, 107
persuasion, 102
protection, 75
Philadelphia, 32
prototype, 137
physical activity, 155
psychological variables, 26
physics, 90, 91
psychologist, 5
pilot study, 73, 85
psychology, x, 2, 5, 14, 15, 16, 18, 23, 24, 25, 28, 29,
PISA, 72
30, 31, 32, 33, 34, 35, 90, 93, 97, 101, 110, 118,
Plato, 3, 73
155
plausibility, 103
psychometric approach, 8
playing, 79
public awareness, 72, 85
pluralism, 37, 42, 55, 57, 61, 64
public investment, 126
polarity, 180
public opinion, 130
polarization, 32
public schools, 156
policy, 60, 61, 64, 76, 132, 173
public service, 126, 127, 132
political leaders, 131
publishing, 169, 171
political parties, 131, 135
politics, 51
pollution, x, 124, 128 Q
population, ix, 70, 71, 87
Portugal, 175, 178 quality of life, 127, 128, 129, 130, 131, 132
Portuguese university, xi, 175 quantification, 139
positive attitudes, 152 quarks, 40
positive correlation, 149, 150 quartile, 11
positive feedback, 143 query, 131
196 Index

questioning, xi, 13, 44, 73, 112, 137, 175, 176, 177, risks, 41, 178
178, 180 rubrics, 14, 117
questionnaire, 25, 72, 111, 135 rule-based approach, vii, 1, 21, 23, 26, 27, 28
rules, vii, 1, 2, 3, 5, 7, 12, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 34, 62, 102
R

random assignment, 25 S
rationalisation, 51
rationality, 35, 50, 52, 54, 99 safety, 175
reactions, 128, 132, 133, 134, 135, 136 savings, 127, 131
reading, ix, 10, 14, 16, 28, 75, 76, 97, 105, 106, 117, scaffolded in-class activities, xi, 159
118, 131, 165, 166, 167, 168, 172 scarcity, viii, 69
reading skills, 14 scarcity of research, viii, 69
reagents, 180 schema, 31, 143
realism, 39, 46 schemata, 176
reality, ix, 37, 41, 49, 51, 70, 74, 79, 132, 133 scholarship, 168
reasoning, vii, x, 1, 3, 4, 5, 6, 7, 8, 9, 11, 15, 17, 18, school, ix, 4, 60, 70, 71, 74, 81, 85, 86, 88, 90, 91,
19, 20, 21, 22, 24, 26, 27, 30, 31, 32, 33, 34, 35, 92, 93, 126, 154
75, 79, 80, 88, 92, 97, 99, 100, 102, 103, 105, school improvement, 154
110, 113, 117, 120, 166, 177 science, 3, 28, 29, 38, 42, 43, 49, 59, 83, 88, 90, 91,
reasoning skills, 4, 5, 7, 8, 18, 19, 21, 79, 88 92, 93, 94, 95, 127, 129, 130, 138, 155, 179
recall, 112, 118, 176 scientific knowledge, 40, 127
reception, 128 scientific method, 25, 39
recognition, ix, 70, 99, 112, 160, 162 scientific observation, 3
recommendations, ix, 19, 30, 70, 86, 126, 131, 133, scientific progress, 27
139 scientific study of CT, vii, 1, 2, 6, 20, 27, 28
reconstruction, x, 124, 144 scope, 14, 136
reductionism, 57, 58, 64 Second World, 42
reference system, 41 secondary students, 155
reflective practice, 55, 56, 62 security, 132
reform, 4, 60, 88, 91, 93, 156 selectivity, 41
regression, 9 self-assessment, 32
regulations, 84 self-awareness, 144, 152
rejection, 131, 138 self-concept, 155
relevance, 3, 43, 53, 102, 112, 135 self-confidence, 8, 79, 80
reliability, 75, 77, 78, 101, 110 self-discipline, 77
religion, 125, 132 self-efficacy, 154
remodelling, 92 self-evaluations, 153
replication, 23, 28 self-monitoring, 11, 12
reproduction, 67 self-reflection, 49
requirements, 86, 162 self-regulation, vii, 1, 10, 19, 20, 84, 93
researchers, viii, ix, 12, 19, 70, 72, 75, 80, 85, 87, 98, self-reports, 24, 115
101, 104, 115, 125, 163 seminars, 111
resistance, 60, 127 sensitivity, 74, 80, 130
resolution, 18, 24, 40, 58, 62, 88, 100 sequencing, 14
resources, 17, 43, 46, 105, 128, 169 services, 127, 135
response, 10, 17, 42, 74, 107, 116, 117, 119, 126, sex, 119
165, 166 sex differences, 119
response format, 116, 119 shape, 61, 62, 89
restructuring, 60 shortage, 124
rewards, 117 showing, 18, 22, 84
rhetoric, 3, 4, 5, 73, 168 signalling, 44
risk, 60, 62, 139 signals, 22, 61
 Index 197

simulation, 143, 154, 156 supervisors, 139

skeleton, 84 support staff, 161
skill acquisition, 6, 16, 26, 27, 28 Supreme Court, 23
social consequences, 44 sustainability, 123, 127, 128, 129, 130, 134, 135, 175
social construct, 47 sustainable development, 138, 140
social control, 47, 53 Switzerland, 67, 88
social group, viii, 70 syllogisms, 99
social indicator, 123, 124, 125, 127, 135 synthesis, 6, 7, 168, 176
social inequalities, 135 systemic change, 58, 62, 63
social influence, 24
social learning, 41, 60
social movements, 123 T
social order, 47
Taiwan, 141, 154
social policy, 125
talent, 171
social relations, 47
tanks, 123, 140
social sciences, 63, 125, 127, 129, 135
target, 61, 124, 136, 169
social theory, 51, 52, 56
taxes, 134
socialization, 161, 162
taxonomy, 5, 6, 8, 28, 29, 30, 32, 34, 76, 78, 79, 81,
society, viii, ix, 13, 33, 47, 54, 70, 71, 72, 73, 74, 75,
83, 89, 118, 119
79, 81, 85, 124, 132, 181
teacher effectiveness, 142, 152, 153, 154
sociology, 101, 123, 127, 129, 139
teacher training, vii, ix, x, 70, 87, 141, 154
Socrates, 3, 73, 93
teachers, viii, x, 6, 29, 70, 75, 80, 84, 85, 86, 87, 88,
software, 107, 117, 119
90, 92, 93, 94, 98, 141, 142, 143, 144, 148, 150,
solution, 5, 50, 58, 71, 72, 83, 86, 89, 97, 116
151, 152, 153, 155, 156, 160, 176, 177, 178
South Africa, 162
teaching experience, 144, 154
special creation, 9
teaching strategies, viii, 69, 85, 104, 142, 143, 154
specialists, 75, 76, 84, 130, 135, 161, 162
technician, 130
species, 8, 103
techniques, vii, x, 42, 43, 44, 97, 112, 161, 166, 167
speech, 22, 124
technological progress, 176
stability, 44, 47, 53
technologies, 161
stakeholder groups, 41
technology, 43, 91, 93, 120, 156, 157
stakeholders, 40, 46, 48, 50, 55, 56, 59, 60, 61
temperature, 132
standard deviation, 106, 113, 114, 148
tension, 58, 59, 60
standard of living, 124
tensions, 123, 129
state, 73, 128, 137, 177
territory, 58
states, 20, 24, 50, 126
tertiary education, 162
statistics, 123, 124, 125, 127, 129
test data, 91
strategic management, 65
test scores, 15, 109
strategy use, 6, 19
testing, 13, 17, 39, 56, 76, 107, 110, 113, 114, 116
stream of consciousness, 98
textbook, 3, 14, 15, 16, 89
stress, 86
textbooks, 3, 12, 23, 75
structuralism, 44, 52
therapist, 23
structure, ix, 14, 31, 47, 60, 61, 82, 97, 102, 105,
think critically, ix, xi, 10, 29, 71, 74, 79, 81, 93, 98,
106, 109, 112, 164, 181
100, 101, 175, 176
student achievement, 118
thoughts, 22, 30
student motivation, 115
tones, 131
student teacher, 153, 156
traditions, viii, 38, 41, 43, 47, 56, 60, 64
style, 59, 106, 116, 156, 168
training, x, 3, 11, 13, 15, 18, 30, 31, 32, 54, 81, 85,
subjective well-being, 127, 128
93, 97, 98, 99, 104, 106, 107, 109, 111, 113, 115,
subskills, 6
116, 117, 141, 144, 148, 150, 152, 153, 154, 156
suburban neighborhoods, 84
training programs, 98
Sun, 154
traits, 8, 30, 76, 143, 156
supervisor, 111
transcripts, viii, 70
198 Index

transformation, 42, 47, 50, 53, 124 vulcanization, 35

translation, 46, 52
transmission, 156
treatment, 23, 25, 156 W
trial, x, 23, 30, 88, 97
Wales, 120
triangulation, 94
war, 42, 94, 124, 126
triggers, 137
Washington, 29, 32, 88, 92, 93, 118, 119, 120
tutor feedback, xi, 159
waste, 22
water, 64, 180
U weakness, 103
wealth, x, 76, 123, 124, 125, 127, 128, 130, 131,
united, 74, 123, 125, 126, 129, 136, 140 133, 134, 135, 136, 138, 140, 143, 152
United Nations, 123, 125, 126 weapons, 60
United Nations Development Programme, 123 web, 58, 62, 118, 136, 138
United States, 74, 125, 129, 136, 140 websites, 132, 135
universe, 37, 61, 133 well-being, 123, 125, 127, 128, 129, 132, 133, 134,
universities, 125, 162 135, 138, 139, 140
urban, 94 William James, 57
USA, 1, 66, 69, 162 Wisconsin, 118
witnesses, 50, 54
workers, 13
V workforce, 98
working memory, 105, 106
validation, 155
workplace, 167
variables, viii, 5, 12, 25, 26, 27, 28, 42, 58, 70
World Bank, 125, 126, 127, 137, 138
variations, 127, 128, 129
worldview, 50
vested interests, 54
worldwide, 72
virtual peer learning, xi, 159, 164, 169
writing process, x, 159, 162, 165, 166, 167, 168, 169,
vision, 40, 125, 127, 129, 134
171
visions, 49
visualization, 88
visually diagramming arguments, ix, 97 Y
vocabulary, ix, 9, 62, 70, 86
voice-overs, 111 young people, 74