The Case for Case-Based Research

Steven Gordon
Journal ofInformation Technology Case andApplication Research; 2008; 10, 1; ABVINFORM Global
Pg. 1

The Case for Case-Based Research

Editorial Preface
The Case for Case-Based Research
Steven Gordon
Editor-in-Chief
Babson College, Babson Park, MA 02457
pordon @ babson.edu

SETTING THE SCENE

JITCAR was founded by Shailendra Palvia in 1999 in response to an impression among
researchers that publishing high-quality case-based research in the major journals in our field was
difficult and becoming increasingly so (Palvia, 1999). Dub6 and Par6 (2003) document that
between 1990 and 1999, case articles accounted for only 15 percent of manuscripts published in
the seven journals that they identified as the top outlets for information systems (IS) research.
Additionally, case research was the primary research technique in only 80 percent of these
articles, or 12 percent of the total.
Fast forward nine years to the present. The top IS journals other than JITCAR continue to
publish a low percentage of case-based research. Furthermore, in the on-going discussion about
relevance versus rigor in our discipline (see, for example, King and Lyytinen, 2006; Desouza et
al., 2006; Klein and Hirschheim, 2003), case research is typically viewed as low in rigor.
Skeptics ask, "How can lessons be generalized from a sample of one?'and, consequently, "What
value can be extracted from case-based research?"

RIGOR IN CASE RESEARCH

Before answering these questions, it is useful to define what we mean by "rigor" in the context of
academic research. In the physical sciences before the early 1900s, rigor typically implied two
things: 1) that an experiment showed what it was intended to show, i.e., that there were no
unidentified factors that could have caused the observed effects; and 2) that the experiment was
described sufficiently well as to be repeatable by other scientists. Because of the latter
requirement, the concept of "rigor" became tightly entwined with accuracy of measurement. The
advent of quantum mechanics forced a redefinition of rigor because quantum events could not be
replicated with certainty. A particle could be found in one place or another, but its location could
be specified only by a probability distribution. Similarly, in the biological sciences,
reproducibility exists only at a statistical level. A drug might be effective in 99.9% of the patients
who take it, but its degree of effectiveness and the ideal dosage varies among individual patients.
For some patients, the drug might not work at all and might even have negative or dangerous side
effects. Eliminating alternative causes for observed effects is also more difficult in the biological
sciences than in the physical sciences. As a result, rigor in the biological sciences requires

JITCAR, Volume 10, Number 1, 2008

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

The Case for Case-Based Research

methodologies to control for variability in the biological systems and their environments and to
address the statistical validity of observed results.

In the social sciences, including management disciplines such as management information

systems, research is hampered by the inherent complexity of the systems being investigated.
Social systems are orders of magnitude more complex than biological systems. In most real
social systems, hundreds of variables, most beyond the control of the investigator, change
simultaneously. In addition, the subjects of experiments cannot be directly manipulated as they
often are in physical or biological experiments. Furthermore, many of the underlying variables of
interest, such as personality traits, motivation, and fear of change cannot be directly observed.
Instead, they are implied by observable indicators or measurable surrogate factors that
presumably correlate with the variable or variables of interest. As a result, deficiencies exist even
in research that is as rigorous as possible in the traditional sense. Specifically, it is impossible to
control for or assess the impact of all of the factors that could affect what is ultimately observed.
And, the impact of an independent variable on the dependent variable can be predicted only with
a low degree of statistical certainty. There may be a high degree of certainty that the independent
variable has some impact (that is, the coefficient of the variable in an equation is statistically
significant), but, because social systems are so complex, rarely is the magnitude of the impact
known with any degree of certainty.
It is interesting to explore how case-based research compares to statistical research according to
our original definition of rigor: 1) that the research shows what it intends to show; and 2) that the
experiment is described in sufficient detail as to be repeatable. I will argue that both research
methodologies can be rigorous, but that they take different paths to achieving rigor and, by
necessity, define rigor differently.

In assessing the rigor of a particular research effort, it is important to ask if the research shows
what it intends to show. For both statistical and case-based research, the research question must
be stated clearly and unambiguously. But, the nature of research questions in quantitative,
statistical studies differs implicitly from those of case-based studies. Statistical research asks
whether hypothesized relationships exist in a statistical sense. If such relationships are shown to
exist, it cannot be inferred that they will exist for a particular organization or in a particular
context. Case-based research, in contrast, asks whether hypothesized relationships exist in a
given context. If such relationships are shown to exist, it cannot be inferred that they will exist in
any other context.

In assessing rigor, we must also ask if the experiment is sufficiently well described as to be
repeatable. In statistical studies, this requires describing how the subjects were selected, what
instruments were used to measure the variables of interest, and what statistical methods were used
to test the hypothesized relationships. For case-based research, this requires describing how the
subject or subjects were selected, what observational techniques andlor instruments were used to
measure the important variables, and how the data were analyzed to demonstrate the hypothesized
relationships. In both methodologies, different researchers analyzing the same data should be
able to reach the same conclusions.

JlTCAR, Volume 10, Number 1,2008

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

The Case for Case-Based Research

It is worth noting that not all case-based research follows a positivistic philosophy, which holds
that laws of causation exist and can be deduced by following scientific principles. Information
systems case researchers have also followed other philosophies, sometimes labeled "critical" or
"interpretive" (see Klein and Myers, 1999). Critical research seeks to focus the reader's attention
on social injustice or evil so as to improve social or working conditions, enhance productivity, or
increase the value of information systems and technology. Interpretive research operates on the
philosophy that reality is in the mind of the beholder. Phenomena are understood differently by
different people, and reality is an interpretation of what has been seen or heard. The value of
such research lies more in understanding complex social and organizational phenomena from
multiple perspectives than in drawing or proving hypotheses. JITCAR will accept interpretive
cases if interpretive principles, such as those advocated by Klein and Myers (1999). are
appropriately followed.

RELEVANCE IN CASE RESEARCH

Case-based and statistical research studies are different but complementary in the nature of their
relevance. The relevance of statistical research depends in large part on how much of the
variance in the dependent variable can be explained by the independent variables. In information
systems studies, this percentage is often low. Top-ranked journals, such as MIS Quarterly and
Information Systems Research, regularly present models in which less than 15% of the variance is
explained or in which the authors extol the significance of an added variable that changes the
percent of variance explained by less than 10%. Very few studies present models that explain as
much as 50% of the variance in the dependent variable. Therefore, practitioners have a hard time
assessing whether statistical research is applicable to their individual contexts. While an
executive might expect to obtain a desired change by manipulating the independent variable in
his or her organization, he or she cannot be confident that the change will occur at all, and it is
quite possible that change will occur in a direction counter to what is desired.
The relevance of case-based research depends in large part on how well the context of its
intended use matches the context of the research case. If the case is adequately descriptive,
executives and other managers should be able assess the quality of this match. For some, the case
will have no value. Others might say, "This case could have been written about my organization
and my situation." They might justifiably feel confident that they can obtain the same impact by
following the same actions as those described in the case. Most others will find themselves
somewhere between these two extremes. Differences in organizational structure, culture,
location, size, business practices and other factors will require them to examine how their
outcomes will be affected by differences between their environment and that described by the
case.
Because of these differences in how statistical and case-based studies are applied, case study
research increases the relevance of statistical research in the same domain and vice versa.
Someone who would like to emulate the protagonist of a case to produce a similar outcome will
feel more confident if statistical studies support that outcome, even if his or her situation is
somewhat different from that described in the case. Similarly, someone attempting to manipulate
independent variables corresponding to a statistical study will gain confidence if a case study
supporting the statistical research provides a parallel to his or her situation. Case studies that

JITCAR, Volume 10, Number 1, 2008

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

The Case for Case-Based Research

provide counter-evidence to the results of statistical studies are equally valuable because they
help identify situations for which theory based on statistical observation does not or may not
apply. Researchers often fail to consider the case study method as a means for testing theory.
Jans and Dittrich (2008) found only 23 theory-testing cases in a sample of 689 cases in five fields
of business research published between 2000 and 2005 in top journals. Yet, the value of a case
study in confirming or disconfirming theory cannot be denied.
Case research is most commonly conceived, and perhaps most appropriately applied, in theory
exploration and development (Benbasat et al., 1987). This is good news for case researchers
because business theory is, for the most part, highly nuanced and in constant development.
Statistical research for theory development is hampered by the "curse of dimensionality." It is
impossible to account for or control all the variables that interact in a normal business
environment without analyzing an impossibly large number of cases or observations. In casebased research, however, any factor can be observed and its impact traced, in time and in state, to
document its effect and to bring nuance to existing theory. It might be impossible to "prove" that
A affects B, but a case can document what happens as A occurs and can demonstrate, following a
chain of events, how it might ultimately affect B.

SETTING THE BAR

In view of the arguments presented above, how should the editors of JITCAR judge the quality of
a case-based research manuscript? I believe that the following requirements should be met:
1. The research question or questions must be clearly stated;
2. The research questions must be well motivated;
3. Prior research relevant to the questions of interest must be addressed;
4. The context of the case must described sufficiently well as to provide confidence in
repeatability; and
5. The evidence in the case and the processes of data collection and analysis must provide
answers to the research questions with appropriate caveats.
Below we examine each of these requirements in turn. A more formal and complete explication
of appropriate research methods and presentation can be found in Yin (2003).
Stating the research question clearly is the first step in providing rigor. It allows the readers and
reviewers to determine whether the case demonstrates what it intends to demonstrate. In
exploratory and theory-building research, the questions should be framed very broadly, as in
"How does A affect B?', "Why does A affect B?', or even "What affects B ? ' I n theory-testing
cases, the research question should be framed much more narrowly, as in "Does Theory X Apply
in Context Y."
Two components are necessary to motivate the research. The first is plausibility. If you intend to
ask how A affects B, you must first show that it is plausible for A to affect B. Plausibility may
not be of great concern in the earliest, exploratory stages of theory building, when you are simply
asking "What affects B?', but where theory already exists to explain the causes of B, the case
researcher is obliged to address why the research question even merits the reader's attention. The
second component is value. Why should the reader care? The researcher needs to make the case
that the answer is important in a variety of contexts, the more the better.

JITCAR, Volume 10, Number 1, 2008

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

The Case for Case-Based Research

A review of relevant research establishes whether the case builds or tests theory. If prior research
provides answers to the research question, then the case should be framed as theory-testing
research. If prior research is incomplete in the sense that it cannot provide answers to the
research question, the case researcher should demonstrate that multiple, different answers to the
case question conform to existing theory. Such a demonstration establishes value by proving that
the case supplements existing theory.
Case research is not strictly repeatable. So, there must be enough detail in the case to allow the
reader to predict with some certainty how likely it would be for the same effects to be observed if
the same actions were taken in similar situations. This requires the case to provide depth in
describing the context. It also requires the case to show, to the extent possible, a chain of
causality between the independent variables, or actions taken by the case protagonists, and the
dependent variable, which is the outcome observed. While statistical research can imply
causality with some repeatability in similar populations, case research can more clearly and
believably demonstrate causality and, implicitly, repeatability. Description of the data collection
and analysis processes is also important to provide evidence that the conclusions are unbiased.
Finally, the evidence in the case should provide answers to the research questions with
appropriate caveats. This closes the loop and demonstrates the value of the case. This
requirement is closely linked to demonstrating a chain of causality. To the extent that the reader
can follow the causal chain and relate cause and effect to the research question, he or she will
believe that the research question has been answered. Nevertheless, as questions in the social
sciences rarely have a black and white answer, caveats are important. The researcher must
address what factors besides those addressed by the research question could have caused the
observed effects.

CONCLUSION
In concluding, the questions posed at the start of this essay can now be answered. "How can
lessons be generalized from a sample of one?' I hope to have argued persuasively that research
on a sample of one can, if done properly, explore questions of interest, help develop new theory,
lend weight to existing theory, and disconfii existing theory or strongly held hypotheses.
"What value can be extracted from case-based research?'Its value derives from its ability to
build and test theory plus its power to provide practitioners with the capacity to assess whether
existing or newly-developed theory is applicable to their particular needs.
As I take the reins of JITCAR's leadership from Dr. Palvia, I want to thank him for his insight
and tireless work in creating a highly recognized publication outlet for serious case researchers. I
am mindful of JITCAR's mission to promote case-based research in IS and hope, during my term
as Editor-in-Chief, to continuously improve the rigor and relevance of the manuscripts we
publish. The time for case-based research to flourish is now!

REFERENCES
1.

Benbasat, I., Goldstein, D., and Mead, M. (1987), The case research strategy in studies of information
systems, MIS Quarterly, Vol. 11, No. 3, 369-386.

JITCAR, Volume 10, Number 1, 2008

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

The Case for Case-Based Research

Desouza, K., El Sawy, O., Galliers, R., Loebbecke, C., and Watson, R. (2006), Beyond rigor and
relevance towards responsibility and reverberation: Information systems research that really matters,
Communications of AIS, Vol. 17, No. 16,2-26.
Dubt, L. and Part!, G. (2003), Rigor in information systems positivist case research: Current practices,
trends, and recommendations, MIS Quarterly, Vol. 27, No. 4, 597-635.
Jans, R. and Dittrich, K. (2008), A review of case studies in business research, in Case Study
Methodology In Business Research, Eds: Jan Dul and Tony Hak, Oxford (England): ButterworthHeinemann.
King, J. and Lyytinen, K. (2006), The market of ideas as the center of the IS field, Communications of
AIS, Vol. 7 , NO. 38, 2-19.
Klein, H. and Hirschheim, R. (2003), Crisis in the IS field? A critical reflection on the state of the
discipline, Journal of AIS, Vol. 4, No. 10, 237-293.
Klein, H. and Myers, M. (1999), A set of principles for conducting and evaluating interpretive field
studies in information systems, MIS Quarterly, Vol. 23, No. 1, 67-97.
Palvia, S. (1999), Research based on cases and applications studies, Journal of Information
Technology Cases and Applications, Vol. 1, No. 1.
Yin, R. (2003), Case Study Research: Design and Methods, 3d Ed., Applied Social Research Methods
Series, Vol5, Thousand Oaks, CA: Sage Publications.

Steven R. Gordon is Editor-in-Chief of JITCAR and Professor of Information Technology

Management at Babson College, Wellesley, MA. Dr. Gordon's research and consulting interests
focus on three areas: how IT affects corporate innovation, governance of information technology
services, and ecomrnerce in the financial service industry. His research appears in the
Communications of the ACM, Information & Management, Information Systems Management,
the International Journal of Service Industry Management, Journal of Strategic Information
Systems, and other academic journals. He is the editor of Computing Information Technology:
The Human Side (IRM Press) and Information Technology and E-Business in the Financial
Services (Ivy League Publishing, 2004). He is a co-author of the textbooks Information Systems:
A Management Approach (Wiley, 3rd edition) and Essentials of Accounting Information Systems
( U . of Phoenix).

JITCAR, Volume 10, Number 1, 2008

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.