Systems Research and Behavioral Science

Syst. Res. 17, 5163 (2000)

&

Research Paper

Generic Research Designs in the Study

of Education: A Systemic Typology
James Steve Counelis*
School of Education, University of San Francisco, California, USA

A typology is a classification system by type, that is, by the formal structure of a study.
In this typology a distinction is drawn between a generic research design and a
methodology. The former the generic research design refers to the over-arching
formal plan for achieving specified disciplinary goals. The latter the methodology refers integratively to: (a) the data-generating processes; and (b) the datal
reduction procedures for discerning datal patterns.
In this typology, generic research designs are cross-classified by two dimensions: (a) the
researcher's assumptions about time and space; (b) the researcher's topics of study (1)
the idea; and (2) the object of the idea: the unit set and the class or set.
There are 10 generic research designs: (1) the review of research; (2) the conceptual
book review; (3) the philosophical study; (4) the simulation; (5) history; (6) the case study;
(7) the post hoc analysis; (8) the survey; (9) the experiment; (10) the meta-analysis.
Copyright # 2000 John Wiley & Sons, Ltd.
Keywords epistemic perspective; general systems theory; generic research design; interdiscipline; metatheory; methodology (a) data-generating processes and (b) data reduction
procedures for discerning datal patterns; taxonomy; time/space manifold; typology

Those who are enamoured in practice without

science are like a pilot who goes into a ship
without rudder or compass and never has any
certainty where he is going.
Leonardo da Vinci (14521519),
Ms. G, 8 (The Library, Institut de France)
INTRODUCTION
Education is one of the oldest subjects of study in
the history of Western culture. The clerisy in
* Correspondence to: James Steve Counelis, School of Education,
University of San Francisco, CA 94117-1080, USA.

CCC 10927026/2000/01005113$17.50
Copyright # 2000 John Wiley & Sons, Ltd.

every culture has written on education, be they

religious and civic leaders, writers and artists,
philosophers, scientists and teachers. Further, the
surviving folklore in oral traditions offers substantial evidence that educational matters were
concerns of protoliterate societies. In twentiethcentury America, education is still debated and
speculated upon by almost everyone. Education
as human capital investment is one index of the
significance of education in our contemporary
political debates (Sweetland, 1996).
Educational study has broadened and, hopefully, deepened. The broadening range of educational inquiry attracted this writer to do the
present study on its apparent diversity. This
Received 7 November 1997
Accepted 28 May 1998

RESEARCH PAPER
study's purpose is to bring one form of conceptual order to what appears as an infinite variety
of research studies on education. The diversity of
educational research has generally been classified
in methodological terms. And its substantive
emphases range across the disciplines. To accomplish this task of conceptual unity, the metatheory underpinning the diversity in educational
research requires description (Blauberg et al.,
1977; Fiske and Shweder, 1986).
METATHEORY
From the viewpoint of general systems theory, all
professions are interdisciplines, namely, a set of
disciplines that are centered at the intersect of
theory and praxis with regard to some human
need. By example, medicine and dentistry are
interdisciplines at the intersect of the natural and
behavioral sciences with regard to human health.
Education is an interdiscipline too. It is centered
at the intersect of the social/behavioral sciences
with regard to administration, teaching and
learning in all societal contexts (Counelis, 1979;
Klein, 1990).
Interdisciplines deal with self-organizing open
systems in which cybernetic processes control the
exchange of information with a system's environment. Open systems are anisotropic and have
non-reversible systemic processes. Biological,
cultural, economic, educational, political, social
and all other human behavioral systems, therefore, are anisotropic and not time/space reversible. Systems processes are evolutionary,
descriptively recognizing many activity levels
which chaos and catastrophe theories further
suggest. Studies of open systems like those in
education are always about discrete societal
agents in larger encompassing systemic cultural
environments (Kroeber and Kluckhohn, 1952/
1963). The intercourse of societal agents and
systemic cultural environments is anisotropic
and evolutionary.
In the study of open systems, two time/space
manifolds are assumed by researchers. These are:
(a) the kairotic time/space manifold; and (b) the
chronotic time/space manifold. The kairotic
time/space manifold refers to the researcher's
c 2000 John Wiley & Sons, Ltd.
Copyright *

52

Syst. Res.
assumption in which a specific systemic object
derives meaning and significance from its uniqueness in place p at time t (Gottschalk, 1950/
1969; Tosh, 1984/1991; Sanjek, 1990). The adjective `kairotic' is derived from the Greek noun
kairoB, which refers to the unique moments in a
temporal process that are characterized by the
qualitative, the experiential, and the singular.
Review the following list: the Cenozoic era, Jesus'
life in Roman Judaea, the Industrial Revolution,
Whitehead and Russell's Principia Mathematica,
Schutz's phenomenological vision of the social
world, the development of microcomputers,
along with Andrew Lloyd Webber's `pop opera'
The Phantom of the Opera. These are examples of
systemic objects and events deriving meaning
and significance from their uniqueness in time
and place. These objects and events are within
the kairotic time/space manifold.
The chronotic time/space manifold refers to
the researcher's assumption that a specific
systemic object derives its meaning and significance through its existential generality and
continuity across time and space. The adjective
`chronotic' is derived from a second Greek
noun wrooB. It refers to the qualitative,
calculable but repetitively patterned elements of
a temporal process. Both time and space are
primitively assumed to be homogeneous and
infinite. Their measurement is conceptually and
pragmatically possible. Examples of such
systemic objects and phenomena are the heliocentric system, Skinner's operant conditioning,
electrical currents, Mendeleev's periodic table of
elements, DNA and RNA molecules, and Einstein's theory of relativity.
In the sixth book of Plato's Republic, two pairs
of metatheoretical notions are presented. These
dyadic notions are: (a) the idea vs. the object of
the idea; (b) the one vs. the many (Plato
Respublica 6. 507, 510). The first dyadic notion is
the epistemic distinction between the concept
about a systemic object vs. the systemic object
itself. An example of this first dyad is the concept
of a cell (i.e., a written description and/or an
image of a cell) vs. an actual specimen of a cell
(e.g., a paramecium) being examined under a
microscope. The importance of this first dyadic
distinction can be illustrated by the rather
Syst. Res. 17, 5163 (2000)

James Steve Counelis

Syst. Res.
common cavalier use of the plural term `data'.
When implicitly reified to mean epistemic
experiences or perceived empirics (i.e., the object
of the idea), correctly construed the term `data'
refers to the conceptual content of the written
records reflecting the experiences or perceived
empirics (i.e., the idea).
The second dyadic notion is the epistemic idea
of a unique, single systemic object or unit set (i.e.,
the one) vs. a class or set of unique systemic
objects (i.e., the many). An excellent example is
found in Breuer and Freud's Studies in Hysteria
(1895/1937). The case of the hysteric Anna O. is a
unique object or a unit set. The class or set of
`hysterics' is composed of all five unique cases of
hysteria, namely, Anna O., Elizabeth von R.,
Emmy von N., Katherina, and Lucie R. The
significance of this distinction rests in circumventing the careless designation of a unique
systemic object as representative of a defined set
when the similarities and differences of apparently similar objects are not known. Breuer and
Freud defined `hysteria' by the common characteristics of the five hysterics (Breuer and Freud,
1895/1937).
These dyadic sets of epistemological notions
are assumptions which all research scholars use
whether or not they are defined. These concepts
will be used to construct a systemic typology for
generic research designs of the social/behavioral
sciences in the interdiscipline of education.
This systemic typology will contain only those
educational studies that are theoretical, to
use Aristotle's notion of theoretical knowledge
(Aristotle, Metaphysica 1. 993b). These educational
studies are grounded in the formal and empirical epistemologies of contemporary social/
behavioral sciences. When appropriate, the
natural sciences and the humanities are used.
A further defining notion is required. In this
systemic typology, a distinction is drawn between a generic research design and a methodology. The former the generic research
design refers to the over-arching formal plan
for achieving specified disciplinary goals. The
latter the methodology refers integratively
to: (a) the data-generating processes; and (b) the
datal reduction procedures for discerning
datal patterns (Keeves, 1988, 1997). And most

RESEARCH PAPER
importantly, methodology is always a necessary
and integral part of every generic research
design.
There is a third dimension not depicted in this
typology of generic research designs. This
dimension denotes the perspective, framework,
viewpoint or assumption which all inquirers
carry into every research project. Some epistemic
perspective always influences the research
results. Several representative epistemic viewpoints are: historicism or a hermeneutic stance,
psychodynamic or cognitive theory, determinism, reductionism or empirically defined science.
Epistemic perspectives, frameworks, or assumptions certainly `nudge' the inquirer toward
particular research questions with their implied
methods for the data sought and therefore the
kinds of datal patterns produced. Epistemology
models ontology (Polkinghorne, 1995/1997,
p. 148).
GENERIC DESIGNS OF EDUCATIONAL
RESEARCH: A SYSTEMATIC TYPOLOGY
A typology is a classification system by type, that
is, by the formal structure of a study. A typology
is not used to classify topics either by process, or by
substance or even by purpose. Hence the term
`generic research design' is aimed to classify the
common formal structures of research designs
used in the social/behavioral sciences. Example:
the process called `empirical correlational
methods' can be used in experiments, metaanalyses, and the case study, depending upon
the researcher's intention and the formal characteristics of the generic research design fulfilling
those intentions. Likewise, a hermeneutic exegetical process can be used in a case study, a
philosophical study, a conceptual book review
and in a conceptual review of research. Of
course, a particular generic research design
provides the formal structure for the researcher's
intentions.
This systemic typology of generic designs used
in educational research is built upon two dimensions. The horizontal dimension is defined by
the researcher's sense about time and place when
conducting a specific research study, namely,

c 2000 John Wiley & Sons, Ltd.

Copyright *

Generic Research Designs in the Study of Education

Syst. Res. 17, 5163 (2000)

53

RESEARCH PAPER
(a) the kairotic time/space manifold; (b) the
chronotic time/space manifold. The vertical
dimension is defined by two classes of the
systemic objects being studied by the researcher.
These two classes are: (a) the idea the study of
concepts about systemic objects; (b) the object of
the idea the study of the systemic objects
themselves. In the latter category, there is a
division of these systemic objects into the unit set
and the class. The attached chart presents this
systemic typology (Counelis, 1984). The researcher's epistemic perspective inherently is found
within the topic under study. Nobel physicist
and Anglican theologian John Polkinghorne
asserts that ontology controls epistemology
(Polkinghorne, 1988, p. 335). So it is with this
study and every other study.
The First Systemic Typological Subset:
The Idea
In educational research textbooks since the 1950s,
it is common to find history, the case study, the
survey and the experiment to be the basic
research designs. There are, however, four other
generic kinds of social/behavioral sciences scholarship which significantly extend and guide
these sciences and the interdiscipline of education. These are: (a) the review of research; (b)
the conceptual book review; (c) the philosophical
study; (d) the simulation model. In this systemic
typology, these types of educational inquiry are
classified under the vertical epistemic category of
the idea a particular concept being studied.
The Research Review
Generically, the review of research cannot be
other than a time-bound kairotic study. The
writer of the research review is a prisoner of his/
her biography, culture and the research status of
the subject at the time when the researcher is
reviewing a specific topic. The reviewer writes a
coherent conceptual study on the research findings (i.e., ideas) for an area of interest. With the
research review being a kairotic study, it is useful
to note that the review of research is not history.
The reviewer's intent is not an interpretive
narrative on some aspect of a discipline's history
c 2000 John Wiley & Sons, Ltd.
Copyright *

54

Syst. Res.
(Cooper and Hedges, 1994; Dunkin, 1996;
Murray and Raths, 1996).
Regardless of intended use, the burden of the
reviewer of research is to bring a coherent
conceptual understanding to a significant body
of scholarship on a given topic. Smith and
several colleagues suggested five criteria for
judging the adequacy and fairness of research
reviews. They are: (a) how well the sample of
studies reviewed represents the population of all
possible studies on the topic; (b) the clarity of
assumptions and methods; (c) the absence of an
ex post facto exclusion of studies based solely on
the judgment of the reviewer; (d) the extent to
which the reviewer's conclusions follow the
findings; (e) the unprejudiced attitude of the
reviewer through each stage in the procedure
(Smith, 1980). Only the last criterion is incorrect.
No inquirer is ever without prejudice. This writer
would therefore seek the explicit statement of the
reviewer's epistemological perspective and require the consistency of its use throughout the
research review.
The Conceptual Book Review
In company with the research review, there is the
conceptual book review. The spare serious literature on academic book reviewing contains little
beyond a few clues, complaints about reviewers'
ethics, and other odds and ends. This writer
believes that the conceptual book review is
significantly beyond the 500-word book note.
Without question, the conceptual book review is
an important form of scholarly critique in making
useful contributions to the professional research
literature in all disciplines.
Like the reviewer of research, the conceptual
book reviewer is time-bound a prisoner of
his/her biography and culture. The reviewer is
tied to the research status of the discipline at the
time when the book is published. The conceptual
book reviewer has these contributions to make:
(a) placing the book's subject matter in its
appropriate disciplinary framework; (b) assessing
the achievement of the author's stated purposes
or implied goals; (c) noting the volume's subject
matter contributions to the discipline; (d) writing
a critical commentary judiciously scaled by stated
criteria; and (e) writing the conceptual book
Syst. Res. 17, 5163 (2000)

James Steve Counelis

Syst. Res.

RESEARCH PAPER

Figure 1. The typology of generic research designs in the study of education

c 2000 John Wiley & Sons, Ltd.
Copyright *

Generic Research Designs in the Study of Education

Syst. Res. 17, 5163 (2000)

55

RESEARCH PAPER
review with good humor (Kamerman, 1978;
Sarton, 1950; Wolfe, 1998). Like the reviewers of
research, conceptual book reviewers must state
explicitly their epistemic perspective and consistently apply it in the conceptual book review's
text.
The Philosophical Study
The philosophical study is clearly a chronotically
oriented generic design of research. With the
exception of the historian of philosophy, the
educational philosopher writes on topics which
are not intended to be time-bound, though the
popularity of philosophical styles and topical
fashions is a function of a specific time and place.
Philosophers write as if their ideas are general
enough to be culturally universal. Hence beauty,
logic, empirical interpretations of statistics,
meaning as understanding or explanation,
justice, truth and error, the good and the moral,
the commonweal, being, essence and mercy are
ideas exhibiting timelessness.
Since the ancient Greeks, logic and mathematics have been within the ambit of philosophy.
But their philosophical status is not universally
held today by field professionals. There is little
doubt that formal studies are indeed chronotic.
Mathematics, logic and statistics are applied to
the study of systemic objects in the real world as
in historical cohort case studies (e.g., Andelman,
1998), surveys and experiments. In themselves,
formal studies do not lose their structural
chronotic character. But social/behavioral scientists using surveys or experiments place logic,
mathematics, and statistics in studies about the
`object of the idea'.
The Philosophy of Education Society published
in 1954 their policy study, titled `The Distinctive
Nature of the Discipline of the Philosophy of
Education'. This study is a model of brevity and
incisive thought. The committee's study briefly
delineated the content and cognitive processes of
educational philosophy. Further, the PES committee described three interconnected phases of
philosophical thinking: the descriptive/analytical
task, the critical/evaluative task, and the speculative task. In this statement, educational philosophers provide guidance for both reading and
writing philosophically about education as an
c 2000 John Wiley & Sons, Ltd.
Copyright *

56

Syst. Res.
interdiscipline (Philosophy of Education Society,
1954).
The Simulation
The simulation is an axiomatically driven conceptual system. The intent is to generalize
abstracted elements of an empirical system. A
simulation works repetitively and therefore is
chronotically a trans-temporal system. Explicitly
this system is an abstraction of high order. With
formal concepts, simulations are built through
`hard-wire' engineering and software design.
Some formal ideas are: (a) axiomatic methods;
(b) calculus for modeling dynamic systems; (c)
dynamical systems theory: modeling chaos,
catastrophe and complexity processes; (d) fuzzy
set theory; (e) game theory; (f) graph theory; and
(g) structural equation models. Simulations are
achieved through algorithms and three-dimensional constructs. Singular in modeling interest is
artificial intelligence, e.g., human/machine learning, robotal motion, natural language programming,
ethical
decision-making,
medical
diagnostics, engineering/architectural design,
and `virtual reality'. Computer models for the
social/behavioral sciences are rapidly developing their specific relevance to administrative,
teaching and learning processes (Franklin, 1995;
Reimann and Spada, 1995).
Summary Notes on the First Systemic Typological
Subset: The Idea
Generic forms of conceptual inquiry used in the
interdisciplinary study of education have art,
rigor and integrity. The results from these conceptual inquiries are always knowledge-oriented
propositions. The review of research is to present
an estimate of the status and treatment of a topic
within its discipline. The conceptual book review
is to bring the results of the book into a critical
holistic context of the discipline. Philosophy
brings formal and conceptual clarity to educational research and axiological understanding
to theoretical, practical and productive aspects of
education. Simulations order a system's processual elements by formal rigor and parsimoniously expressed axiomatics. Epistemic criteria
evaluate cogency in conceptual studies.
Syst. Res. 17, 5163 (2000)

James Steve Counelis

Syst. Res.
The Second Systemic Typological Subset:
The Object of the Idea
The second part of this systemic typology is set
by disciplined studies on the object of the
idea that is, people and societal phenomena.
Four cells constitute this section of the systemic
typology. By a cross-hatch, the two time/space
manifolds the horizontal dimension and the
two classes of denumerable objects the vertical dimension, create a four-celled typology.
These cells contain six generic research designs:
(a) history; (b) the case study and post hoc
analysis; (c) the survey; (d) the experiment and
meta-analysis.
History
History, biography and autobiography are about
the one and the singular the unique systemic
object or event within a time/space framework
(Gottschalk, 1950/1969; Tosh, 1984/1991). That
has been the historian's perspective since
Herodotus and Thucydides.
A number of principles are used to categorize
history. Mandelbaum uses narrative, explanation, and interpretation as categories (Mandelbaum, 1977). White's poetic categories of
romance, tragedy, comedy and satire applied to
the emplotment the way in which the story's
sequenced events occur become classifications
(White, 1973). Histories are also classified by
various contents, such as, cities, demography,
economics, families, ideas, and technology, as
well as by such historical intellectual frameworks
as the Enlightenment, Romanticism, the Age of
Analysis, or the Global Village (Tosh, 1984/1991;
Leatherman, 1998). Further, epistemic debates
over historical generalization and explanation
provide another entrance into conceptualizing
history as art or science. Gottschalk details six
groups of historians by their attitudes and
practice in using generalizations (Gottschalk,
1963, ch. 8). Also, Nichols draws a `genealogy
of historical generalizations' through the example
of the American Civil War. He uses a genetic
principle to account for the cumulative intellectual process of the historians' guild (Gottschalk,
1963, ch. 9). A kairotic discipline, history cannot
generate law-like propositions, such as tested

RESEARCH PAPER
hypotheses achieved by experiment or predictive
equations based on ribbons of chronological
data.
The historian's armamentarium contains sophisticated scientific tools. The following classes of
methods for discerning the authenticity of
historical materials are listed synoptically in
order to present a comprehensive but selective
overview of their range and character: (a)
anthropology, economics, linguistics, political
science, psychology and sociology used in the
interpretation of autobiography, historical texts
and events; (b) mathematics and statistics used
on chronological and measurable empirical data;
(c) the contemporary voice and/or film recording
of news events and rare historical moments; (d)
computer use: content analysis, textual criticism,
authorship studies, the assembly of ancient
Greek pottery from coded shards, stylistic
analyses of texts, paintings, music and textiles;
(e) natural sciences used in dating, historical
validation and preservation of artifacts: (1) the
chemistries of inks, paints, papers, cloth, metals,
soils, and coprolites; (2) X-ray; (3) mass spectrometry; (4) neutron activation analysis; (5)
thermoluminescence; (6) tree ring dating; (7)
radio-carbon and potassium/argon dating; and
(8) geology: stratigraphic and petrological studies. Also one should add the following scientific
techniques: (a) new archeological excavation
techniques; (b) aerial and documentary photography; (c) historical linguistics; and (d) forensic sciences. Collectively these scientific methods
improve the evidentiary quality of historical
research (Study of History, 1985; Hermann and
Hummel, 1993).
The Survey
The survey is characterized by the kairotic time/
space manifold and defined classes of enumerated units. William the Conqueror's Domesday
Book (A.D. 10811086), the first census of the
United States in 1790, election polls, marketing
studies and annual higher education questionnaires such as HEGIS are examples of surveys.
Survey results are patterned summaries of the
system's unit observations for a given time and
place. Surveys are built upon previously determined frequency classes.

c 2000 John Wiley & Sons, Ltd.

Copyright *

Generic Research Designs in the Study of Education

Syst. Res. 17, 5163 (2000)

57

RESEARCH PAPER
A generic research design, the survey's purposes are explicitly to describe certain aspects of
a social system at a given time and place. A social
system's characteristics that are denumerable
become the observers' sources of information.
The data are procured by a researcher from direct
observers using a systematic recording device.
Paper-and-pencil questionnaires, in-person and
telephone interviews, together with focus
groups, are among the most frequently used
survey instruments for gathering data. The
reduction of data into patterns produces the
survey study's results. These results are formed
into propositions. The response patterns for the
original research questions describe the social
system being studied. Survey results carry the
intellectual imprint of the inquirer.
The survey's art rests upon two elements. The
first element is the design of the research
questions for study. The second element is
found in the creative and insightful writing of
questionnaire items and interview queries. The
craft of the survey is built upon the following: (a)
selecting the appropriate population sample or a
specifically designated set of persons; (b) discerning and selecting the useful survey processes
that are suggested by the original research
questions; (c) designing and executing the
pertinent data-generating processes; (d) designing and executing the appropriate data-reduction
procedures to yield datal patterns; (e) writing
direct responses to the original research questions. Whether the survey results are used for a
TV or radio script, a newspaper column, or a
book, the text becomes a descriptive/explanatory
essay through which the writer opines as he or
she pleases.
The Case Study
Using qualitative and/or quantitative data, the
case study is a partial image of some reality, built
to achieve a particular end the researcher's
valued goal. Indeed, case studies are analogical
constructs. The analogue is the actual observed
person, situation or idea. The analogy is the
written descriptive/explanatory summary of
conceptualized observations about the analogue.
The comparison and contrast of analogue to
analogy yields the degree of congruity between
c 2000 John Wiley & Sons, Ltd.
Copyright *

58

Syst. Res.
the two. Using the theories of truth by correspondence (i.e., correlation) or truth by coherence, high degrees of agreement in form,
substance and/or concept between analogue
and analogy are interpreted as evidentiary truth
(Counelis, 1990).
The case study is always the study of the
singular, be it a person, a social group (e.g., the
ethnographic study of a first grade class), an
institution, a specifically selected set of persons
(e.g., a psychometric study of gifted/talented
high school students), or even some product of a
human system, such as a text (e.g., a postmodernist's quotidien), institutional behavior (e.g.,
critical systems analysis Tsivacou, 1997), a
musical work, technical inventions, a new hybrid
plant or cloned animal. Thus the unit category
for the case study in this systemic typology.
The reason for the chronotic classification of
the case study is not quite so apparent. The
researcher's intention provides the reason for
placing the case study within the chronotic time/
space manifold. The case researcher's intention is
finding a generic pattern of elements within a set
of similar cases. That generic pattern of elements
is the Hegelian concrete-general (Wallace, 1892,
pp. 287310). To find this generic pattern, the
common elements of the set of cases are read at
one level of abstraction above the observable
details. These patterned elements are not timebound to the idiosyncratic detail. This patterned
set of common elements the Hegelian concrete-generals collectively is the schema. This
schema is chronotic in nature. This schema is
used in post hoc analysis, one of the generic
research designs to be described later.
In case study research, data-generating
methods stem from the social/behavioral
sciences, the natural sciences and the humanities.
Given the disciplinary source of the data-generating processes, the appropriate data-reduction
procedures then are employed. Both quantitative
and qualitative data are used in case studies,
depending upon the specific research questions
being posed (Counelis, 1993; Yin, 1994). In Habits
of the Heart, Bellah and his colleagues well
illustrate a case study on `individualism and
commitment in American life' (Bellah et al.,
1985). Another example is the Seymour Martin
Syst. Res. 17, 5163 (2000)

James Steve Counelis

Syst. Res.

RESEARCH PAPER

Lipset case study through which he describes

America's contemporary political organizing
principles in American Exceptionalism: A DoubleEdged Sword (Lipset, 1996). In this case study,
Lipset remembers Alexis de Tocqueville's insightful 1840 study Democracy in America. Lipset
notes the United States to be qualitatively
different from other Western nations in its political organizing principles, these being liberty,
egalitarianism, individualism, populism, laissez
faire, sectariancongregationalism, and derivatively, voluntarism. Lipset therefore defines
America as a political `outlier' (Lipset, 1996).

Organization of Work reflect the post hoc study

of the case. For these studies, each scientist used
systemic cases from different social levels the
person, the national revolution, and the societal
economy. But each writer's research purpose was
the same the inductive improvement of the
validity and reliability registered by a nomothetic
proposition the datal schema. The pattern of
the datal schemas exhibited in a defined set of
cases is the substantive basis for all post hoc
analyses. Decidedly, the epistemic perspective of
the researcher codes the meaning attached to
such studies.

Post Hoc Analysis

In the social/behavioral sciences, the purpose of
post hoc analysis is to study the empirical
schemas the Hegelian concrete-generals of
a societal structure that was initially established
in a specific case study. The post hoc analysis of a
defined set of cases requires that their patterned
elements the datal schemas be structurally
similar to the original case study. A propositional
principle describes the whole set of cases (i.e., the
one). This rule-like proposition is theoretical,
trans-case and trans-temporal (i.e., chronotic).
Note the two following examples of post hoc
analysis. In his well-known 1938 volume The
Anatomy of Revolution, historian Crane Brinton
used the American, English, French and Russian
revolutions as systemic cases to build his `fever'
model of the revolutionary process. Brinton's
nomothetic propositions on the revolutionary
process are trans-case, trans-cultural, and transtemporal (i.e., chronotic) (Brinton, 1938/1952).
Stanley H. Udy Jr wrote an excellent book,
Organization of Work: A Comparative Analysis of
Production among Nonindustrial Peoples (1959). He
completed a study on work organization in 150
non-industrial societies across the world. He
used 426 case studies from the Human Relations
Area Files. Systematically collecting 11 variables,
he was able to develop 64 empirical propositions
about the organization of productive work.
Udy's socio-economic propositions are transsocietal, trans-cultural, and trans-temporal (i.e.,
chronotic) (Udy, 1959).
Breuer and Freud's Studies in Hysteria, Brinton's The Anatomy of Revolution, and Udy's

The Experiment
In the social/behavioral sciences, the experiment
is concerned with the many the person and
groups in society. Its design is to set the
conditions such that a rule-like pattern is
discerned for sets of variables. The rule in
experimental design is to control explicitly the
possible sources of variance and uniqueness.
Indeed, the experimenter has three devices for
controlling these sources for invalidity, namely,
randomization, the design of intervention content and the sequencing of interventions. Formal
design procedures (i.e., Graeco-Latin squares,
randomized sampling of variables, the placebo
ruse, and the partition of variances, among
others) all point to the rule-making results of
the experiment. The Solomon four-group design
optimally typifies experimental design. Correlation, curve-fitting, multivariate analytical
methods, spectral analysis along with hypothesis
testing by parametric, non-parametric and distribution-free statistics are among a number of
cognitive processes used for discerning patterns
of variables. The variables are expressed in lawlike propositions. These propositions are epistemically coded by the ideas of objectivity and
empiricism held by the experimenter.
In 1963 Campbell and Stanley, along with
Tatsuoka and Tiedman, wrote two classic essays
in which they conceptually outlined the foundational character of experimental design for the
interdiscipline of education and the social/
behavioral sciences (Campbell and Stanley,
1963; Tatsuoka and Tiedman, 1963). For the
current status of measurement and modeling for

c 2000 John Wiley & Sons, Ltd.

Copyright *

Generic Research Designs in the Study of Education

Syst. Res. 17, 5163 (2000)

59

RESEARCH PAPER
the social/behavioral sciences, see Keeves'
encyclopedic sources (Keeves, 1988, 1997).
Meta-analysis
Meta-analysis is the collective statistical re-study
of the summary findings (i.e., empirical concepts) from many controlled experiments on the
same topic. The burden of meta-analysis is to
raise significantly the number of empirical cases
(i.e., the many) for an improved inductively
derived empirical concept. Analogically, this is
the identical goal of the post hoc analytic process
for cases. Review this well-known example of
meta-analysis.
The 1979 Glass and Smith meta-analysis on
achievement and class size provides an excellent
example. Their pool of studies on achievement
and class size consisted of 77 empirical studies,
that spanned 70 years, and involved nearly
900 000 cases (i.e., pupils) from 12 countries.
Collectively recalculating the common statistical
data from the 70 studies, the Glass and Smith
results generated the inferences that a clear and
strong relationship exists between achievement
and class size; and that more is learned in small
classes than in large classes. In fact, there are
more than 30 percentile ranks of achievement in
classes ranging from one to 40 pupils. These
empirical results are trans-case, trans-cultural,
and trans-temporal (i.e., chronotic) (Glass and
Smith, 1979).
Summary Notes on the Second Systemic Typological
Subset: The Object of the Idea
The interdiscipline of education is studied
through six generic research designs. The inductive inferences derived from these generic
research designs are made about live actors in
real-world situations. The discursive forms for
these inferences are: (a) the narrative and
explanatory discourse for history; (b) the descriptive/explanatory statistical essay for the survey;
(c) the descriptive/explanatory exposition for the
case study and post hoc analysis; (d) the
nomothetic or rule-like proposition (i.e., mathematical or statistical equation) or a tested
empirical hypothesis for the experiment and
meta-analysis; (e) Post hoc analysis and metaanalysis are respectively analogical equivalents
c 2000 John Wiley & Sons, Ltd.
Copyright *

60

Syst. Res.
for the collective re-study of cases and experiments. Several forms of discourse are extended
and complex (i.e., the historical and explanatory
narrative, the descriptive/explanatory exposition
of the case and post hoc analysis). Other forms of
discourse are more parsimonious in text. These
make for denser texts to read (i.e., the descriptive/explanatory essay for the statistics of surveys, the inferential statistics of experiments and
meta-analyses). These studies are fashioned by
the epistemic perspective held by the specific
researcher.

GENERAL IDEAS
Not the only approach toward understanding
today's diversity in educational research, the
simplicity and reasonableness of this typological
technique suggested its viability. Indeed, this
systemic typology of 10 generic research designs
used in the social and behavioral sciences' study
of education provides some important general
notions about the set.
First: This systemic typology provides for a
broadening of the notion of a generic research
design beyond those given in current textbooks
on educational research. There are two additions,
namely, the research review and the conceptual
book review. These two significant forms of
scholarly study are generally practiced but not
usually understood as generic research designs
in their own right.
Second: Plato's notion of `the idea' in relation to
the two time/space manifolds provides a significant conceptual link for two pairs of generic
research designs, namely, (a) the research review
and the conceptual book review in relation to (b)
the philosophical study and the simulation.
Recognition of scholarly research on `concepts'
as opposed to scholarly research on `objects and
events' is not explicitly noted in current educational research textbooks.
Third: The temporally biased or kairotic descriptive character of the generic research designs'
results should be noted in the following: (a) the
review of research; (b) the conceptual book
review; (c) history; (d) the survey.
Syst. Res. 17, 5163 (2000)

James Steve Counelis

Syst. Res.
Fourth: The chronotic and nomothetic character
of the following six generic research designs'
results should be noted: (a) the philosophical
study; (b) the simulation model; (c) the case
study and post hoc analysis; (d) the experiment
and meta-analysis.
Fifth: These 10 generic forms of inquiry contain
methodologies: (a) data-generating processes and
(b) data reduction procedures through which
higher-level datal patterns, abstractions, hypotheses or concepts are expressed as inductively
implied or deductively inferred propositions.
Hence the over-arching character of the generic
research design encompasses all required
methods related to data creation, data reduction,
datal analysis and the discernment of datal
patterns.
Sixth: These 10 modes of inquiry are generic in
that they are parsimoniously construed as overall
formal plans for disciplinary and interdisciplinary inquiry. Metatheoretical concepts epistemically format this systemic typology. These general
statements about all 10 generic research designs
provide the structural and cognitive limits for
these generic research designs. Ultimately, the
propositions generated through these generic
research designs collectively change our world
view over time.
Seventh: This systemic typological understanding of generic research designs does not confuse
it with a larger project design, wherein several of
these generic research designs could be used for
appropriate segments within a larger project.
Planning a research project for the resolution of a
real-world problem is complex. Certainly the
study of a discipline-centered problem through
the social/behavioral sciences is complex as well.
No doubt, these generic research designs can be
used scientifically to build the complex of social
and behavioral knowledge within which the
interdiscipline of education resides. Of course,
the humanities and natural sciences are used
when the need arises.
Eighth: The qualities and kinds of knowledge
derived with these generic research designs are
decisively but inherently determined by the
researcher's choice of epistemological perspective, framework, viewpoint or assumptions. Epistemology models ontology.

RESEARCH PAPER
PERSONAL PERSPECTIVE
By persuasion, I am a general systems theorist. I
believe that general systems theory is useful in
building interdisciplines such as education. I
believe this longer-range goal is important
because interdisciplines are like halfway houses
in expanding a truth-yielding science about
human behavior in society. I further believe this
systemic typology of generic research designs is a
helpful comprehensive tool for teaching educational research at the graduate level. In fact, I
have worked on this typology since 1972,
refining it through instructional use and research
application (Counelis, 1972). With advanced
training, research practice and field application
of research results, the interdiscipline of education becomes a joint contributor to the longterm scholarly enterprise of building the interdisciplinary character of the social/behavioral
sciences, remembering always the required consistency of an epistemic perspective. Whether
interdisciplines will contribute to Wilson's consilience the interlocking of causal explanations
across disciplines we will see (Wilson, 1998)!
As a general systems theorist, I remember
Polkinghorne's observation that epistemology
models ontology. If I read Wilson's consilience
aright, it will be a very attractive result!
If we are to understand what is going on in
any area of cultural and intellectual work, we
must understand its social context.
C. Wright Mills,
The Sociological Imagination (1959).

REFERENCES
Andelman, C. (1998). Women and Men of the Engineering Path: A Model for Analyses of Undergraduate
Careers, US Government Printing Office, Washington, DC.
Aristotle, Metaphysica 1. 993b.
Bellah, R. N., Madsen, R., Sullivan, W. M., Swidler, A.,
and Tipton, S. M. (1985). Habits of the Heart:
Individualism and Commitment in American Life,
University of California Press, Berkeley, CA.
Blauberg, I. V., Sadovsky, V. N., and Yudin, E. G.
(1977). Systems Theory: Philosophical and Methodologi-

c 2000 John Wiley & Sons, Ltd.

Copyright *

Generic Research Designs in the Study of Education

Syst. Res. 17, 5163 (2000)

61

RESEARCH PAPER
cal Problems (Syrovatkin, S., and Germogenova, O.,
trans.), Progress Publishers, Moscow.
Breuer, J., and Freud, S. (1895/1937). Studies in Hysteria
(Brill, A., trans.), Beacon Press, Boston, MA.
Brinton, C. (1938/1952). The Anatomy of Revolution,
Vantage Books, New York.
Campbell, D. T., and Stanley, J. C. (1963). Experimental
and quasi-experimental designs for research in
teaching. In Gage, N. L. (ed.), Handbook of Research
in Teaching, Rand McNally, Company, Chicago,
Ch. 5.
Cooper, H., and Hedges, L. V. (eds) (1994). The Handbook of Research Synthesis, Russell Sage, New York.
Counelis, J. S. (1972). A typology of generic research
designs. The Generator of Division GAERA 2, 2931,
36.
Counelis, J. S. (1979). Education about education.
Educational Studies 9, 407424.
Counelis, J. S. (1984). The metatheory of taxonomy. In
Majumder, D. D. (ed.), Advances in Information
Sciences and Technology: Pattern Recognition and
Digital Technique (Vol. 1), Indian Statistical Institute,
Calcutta, pp. 121136.
Counelis, J. S. (1990). Analogy and expert systems.
Systems Research 7(3), 193195.
Counelis, J. S. (1993). The case study: Its systemic
metatheory and the social/behavioral sciences.
Kybernetes: International Journal of Systems and Cybernetics 22(5), 2234.
Dunkin, M. J. (1996). Types of errors in synthesizing
research in education. Review of Educational Research
66, 8797.
Fiske, D. W., and Shweder, R. A. (eds) (1986).
Metatheory in Social Science: Pluralisms and Subjectivities, University of Chicago Press, Chicago.
Franklin, S. P. (1995). Artificial Minds, MIT Press,
Cambridge, MA.
Glass, G. V., and Smith, M. L. (1979). Meta-analysis of
research and the relationship of class size and
achievement. Evaluation and Policy Analysis 1, 216.
Gottschalk, L. (1950/1969). Understanding History: A
Primer of Historical Method, Knopf, New York.
Gottschalk, L. (1963). Generalization in Writing History,
University of Chicago Press, Chicago.
Hermann, B., and Hummel, S. (eds) (1993). Ancient
DNA: Recovery and Analysis of Genetic Material from
Paleontological, Archeological, Museum, Medical, and
Forensic Specimens, Springer-Verlag, New York.
Kamerman, S. E. (1978). Book Reviewing, The Writer,
Boston, MA.
Keeves, J. P. (ed.) (1988). Educational Research, Methodology, and Measurement: An International Handbook,
Pergamon Press, Oxford.
Keeves, J. P. (ed.) (1997). Educational Research, Methodology, and Measurement: An International Handbook
(2nd edn), PergamonElsevier, Oxford.
Klein, J. T. (1990). Interdisciplinarity: History, Theory and
Practice, Wayne State University Press, Detroit, MI.
c 2000 John Wiley & Sons, Ltd.
Copyright *

62

Syst. Res.
Kroeber, A. L., and Kluckhohn, C. (1952/1963).
Culture: A Critical Review of Concepts and Definitions,
Random House/Vintage Books, New York.
Leatherman, C. (1998). The faculty saying their field is
in `disarray', historians set up a new society. The
Chronicle of Higher Education 44(35), A12A13.
Lipset, S. M. (1996). American Exceptionalism: A DoubleEdged Sword, Norton, New York.
Mandelbaum, M. (1977). The Anatomy of Historical
Knowledge, Johns Hopkins University Press, Baltimore, MD.
Murray, F. B., and Raths, J. (1996). Factors in the peer
review of reviews. Review of Educational Research 66,
657658.
Philosophy of Education Society (1954). The distinctive
nature of the discipline of philosophy of education.
Educational Theory 4, 13.
Plato, Respublica 6. 507, 510.
Polkinghorne, J. [C.] (1988). The quantum world. In
Russell, R. J., et al. (eds), Physics, Philosophy, and
Theology: A Common Quest for Understanding,
Vatican Observatory, Vatican City State, pp. 333
342.
Polkinghorne, J. [C.] (1995/1997). The metaphysics of
divine action. In Russell, R. J., et al. (eds), Chaos and
Complexity: Scientific Perspective on Divine Action
(2nd edn), Vatican Observatory/Center for Theology and the Natural Sciences, Vatican City State/
Berkeley, CA, pp. 147156.
Reimann, P., and Spada, H. (eds) (1995). Learning in
Humans and Machines: Towards an Interdisciplinary
Learning Science, PergamonElsevier, Oxford.
Sanjek, R. (ed.) (1990). Fieldnotes: The Makings of
Anthropology, Cornell University Press, Ithaca, NY.
Sarton, G. (1950). Second Preface to Volume 41: Notes
on reviewing learned books. Isis 41, 149158.
Smith, M. L. (1980). The Benefits of Psychotherapy, Johns
Hopkins University Press, Baltimore, MD.
Study of History (1985). Encyclopaedia Britannica (15th
edn) (Vol. 20), pp. 223314.
Sweetland, J. W. (1996). Human capital theory: foundations of a field of inquiry. Review of Educational
Research 66, 341359.
Tatsuoka, M. M., and Tiedman, D. V. (1963). Statistics as an aspect of scientific method in research on
teaching. In Gage, N. L. (ed.), Handbook of Research
on Teaching, Rand McNally, Chicago, Ch. 4.
Tosh, J. (1984/1991). The Pursuit of History: Aims,
Methods, and New Directions in the Study of Modern
History, Longman, London.
Tsivacou, I. (1997). The rationality of distinctions and
the emergence of power: a critical systems perspective of power in organizations. Systems Research and
Behavioral Science 14, 2134.
Udy, S. H., Jr (1959). Organization of Work: A Comparative Analysis of Production among Nonindustrial
Peoples, Human Relations Area Files Press, New
Haven, CN.
Syst. Res. 17, 5163 (2000)

James Steve Counelis

Syst. Res.
Wallace, W. (1892). The Logic of Hegel: Translated from
The Encyclopaedia of the Philosophical Sciences (2nd
edn rev., augmented), Oxford University Press,
Oxford (reprint edn: Geoffrey Cumberlege, London,
1931/1950 2nd edn).
White, H. (1973). Metahistory: The Historical Imagination
in Nineteenth Century Europe, Johns Hopkins University Press, Baltimore, MD.

RESEARCH PAPER
Wilson, E. O. (1998). Consilience: The Unity of Knowledge, Knopf, New York.
Wolfe, A. (1998). The solid responsibilities of book
reviewing. The Chronicle of Higher Education 44(33),
B45.
Yin, R. K. (1994). Case Study Research: Design and
Methods, 2d edn. Sage, Thousand Oaks, CA.

c 2000 John Wiley & Sons, Ltd.

Copyright *

Generic Research Designs in the Study of Education

Syst. Res. 17, 5163 (2000)

63