CHAPTER 1

The Science in Social Science

1.1 INTRoDucTION
TRIS BOOK is 1bout research in the social sciences. Our goal is practical:
designing research that will produce valid inferences about social and
politicallife. We focus on political science, but OUT argurnent applies to
other disciplines such as sociology, anthropology; history, econornics,
and psychology and to nondisciplinary areas of study such as legal
evidence, education research, and clinical reasoning.
This is neither a work in the philosophy of the social sciences nor a
guide to specific research tasks such as the design of sUTveys, conduct
of fie1d work, or analysis of statistical data. Rather, this is a book about
research design: how to pose questions and fashion scholarly research
to make valid descriptive and causal inferences. As such, it occupies a
middle ground between abstract philosophical debates and the hands-
on techniques of the researcher and focuses on the essential logic un-
derlying all social scientific research.
1.1.1 Two Styles of Research, One Logic of Inference
Our main goal is to connect the traditions of what are conventionally
denoted "quantitative" and "qualitative" research by applying a uni-
fied logic of inference to both. The two traditions appear quite differ-
ent; indeed they sometimes seem to be at war. Our view is that these
differences are mainly ones of style and specific technique. The same
underlying logic provides the framework for each research approach.
This logic tends to be explicated and formalized clearly in discussions
of quantitative research methods. But the same logic of inference un-
derlies the best qualitative research, and all qualitative and quantita-
tive researchers would benefit by more explicit attention to this logic
in the course of designing research.
The styles of quantitative and qualitative research are very different.
Quantitative research uses numbers and statistical methods. It tends to
be based on numerical measurements of specific aspects of phenom-
ena; it abstracts from particular instances to seek general description
or to test causal hypotheses; it seeks measurements and analyses that
are easily replicable by other researchers.
4 . The Science in Social Science
h' ontrast covers a wide range of approaches,
Qualitative researc ,m c , hes relies on numerical mea-
but by definition, nonehof on one or a small number
surements. Such work as en. depth analysis of historical ma-
of cases, to use intensive to be concerned with a rounded
terials, to be discursive in met o ,an t or unit Even though they
h' ount of sorne even .
or compre enSIve acc f qualitative researchers generally un-
have a small number o t' from their studies. Sometimes
ounts of I orma lOn d
earth enormous am .. linked with area or case stu _
this kind of work in .the social decision, institution, loca-
ies where the focus lS on pa. A' also the case with quantitative
tion, issue, or piece of in its own right: a major
research, the 15 :! or decision, or a world crisis. Why
change in a nahon, an electlO , J dd ly I'n 1989
7
More gener-
. ollapse so su en .
did the East German regIme c . t gimes of Eastern Europe col-
ally, why did almost the always, the event be
lapse in 1989? SometImes, bu . 1 r of event such as a politIcal
chosen as an exemplar of a partlcu to reject a waste
revolution or the decision a I:artlcuf ar k's linked to area studies
. c: t' es thlS kmd o wor 1 f h
disposal sIte. _o:n
e
1m. d culture of a particular part o t e
where the focus 15 on the hlstory an t' analyzed dosely and in full
world. The particular place or even 15
detail. .. 1 cientists have debated the merits of
For several decades, s. es area studies versus comparative
case studies using quantitative methods
studies, and "sclenhlc PI' rich textual and contex-
versus "historical" believe that sys-
tual understanding. Son:
e
lId to truth in the social sci-
tematic statistical analysls. lS .t e on y TOha hemently disagree. This
t f quahtatlVe researc ve . 1
ences. Advoca . o livel debate; but unfortunately, lt a so
difference of to . to Ya quantitative-systematic-general-
bifurcates the sOCIal m
h
'stic-discursive branch. As the
. . h d qualItatlve- umam . f f
lzmg branc an a o histicated in the analysls o sta 15-
former becomes m?re comprehensible to those who
tical data (and. the1r wo . es) the latter becomes more and more
have not studled the techmqu
f
'h lyses to the seemingly non-
d f th . relevance o suc ana
convince o e Ir '. t . which its practitioners are
replicable and nongenerahzable even s m
interested. . b k' t show that the differences between
A major purpose of th1S . lS are only stylistic and are
the quantitative and l unimportant. AH good research
methodologicaHy understood-to derive fro:n
can be understood---:mde: ' B th quantitative and quahtatIve
"""n'lP llooerlving- 10glC of mference. o
Introduction . 5
research can be systematic and scientific. Historical research can be an-
alytical, seeking to evaluate alternative explanations through a process
of valid causal inference. History, or historical sociology, is not incom-
patible with social science (Skocpol 1984: 374-86).
Breaking down these barriers requires that we begin by questioning
the very concept of "qualitative" research. We have used the term in
Our title to signal our subject matter, not to imply that "qualitative"
research isfundamentally different from "quantitative" research, ex-
cept in
Most does not fit clearly into one category or the other. The
best often combines features of each. In the same research project,
some data may be collected that is amenable to statistical analysis,
while other equally significant informabon is not. Patterns and trends
in social, political, or econornic behavior are more readily subjected to
quantitative analysis than is the flow of ideas among people or the
difference made by exceptional individualleadership. If we are to un-
derstand the rapidly changing sodal world, we will need to indude
information that cannot be easi1y quantified as well as that which can.
Furthermore, alI social sdence requires comparison, which entails
judgments of which phenomena are "more" or "less" alike in degree
(Le., quantitative differences) or in kind (i.e., qualitative differences).
Two excellent recent studies exemplify this point. In Coercive Cooper-
atian (1992), Lisa L. Martn sought to explain the degree of interna-
tional cooperation on econornic sanctions by quantitatively analyzing
ninety-nine cases of attempted economic sanctions from the post-
World War TI era. Although this quantitative analysis yielded mueh
valuable information, certain causal inferences suggested by the data
were ambiguous; henee, Martn carried out six detailed case studies of
sanctions episodes in an attempt to gather more evidence relevant to
her causal inference. For Making Demacracy Work (1993), Robert D. Put-
nam and his colleagues interviewed 112 ltalian regional councillors in
1970,194 in 1976, and 234 in 1981-1982, and 115 community leaders in
1976 and 118 in 1981-1982. They also sent a mail guestionnaire to over
500 Community leaders throughout the country in 1983. Four nation-
wide mass surveys were undertaken especially!p
r
this study. Never-
the1ess, between 1976 and 1989 Putnam and hiscolleagues condueted
detailed case studies of the politics of six to satisfy the
"interocular trauma tic test," the investigators "gained an intima te
knowledge of the internal political maneuvering and personalites that
have animated regional politics Over the last two decades" (Putnam
1993:190).
The lessons of these efforts should be c1ear: neither guantitative nor
qualitative research is superior to the other, regardless of the research
address-ed. Since mdnv nI sntefC::1t 50GJl SC1-
t;:ntists canno! be meaningfuHy fonnul<1ted in th"t permil statstl
cal of witn quantitdtive dnta,. ,Vi.> do flor wish to
encouragt' the exdusive use of quantitative ri'Clmiques, VVE' are not tr."-
t aU social sdentists out oI thE' library and into the
center, 01" to replace idiosyncratie eonversations v;ith struc!ured nter-
vrevo/5. \v!:' ,1rgue that nonstatistieal researen h'il! prQduee more
reliable result.'i if researehers pay attenton to the rules of st'enU{k in-
fercnce--rules thu are MJmenmes more dearly stated in the style or
Vr."r1<o;,"IV defined stntisticili melhods that under-
,;bstract fomlai modeIs applkable
to aH knds nf rt:seardl, even that ror whkh vari"bles carU10t bt> meil-
sured quanhtativdy, The very i'lnd even unrcil1istic, nah,lfe o
statistical modeLs 15 vI/har lTUIkes the rules ni inierenc,,' shine through so
de"rly,
The nlles o inference that \<\,'12' di::;cuss are not reievant to all ssues
that are of lo social sdentisi5L Many nI' he most irnn<,rL"1
pohtical life-abour 5uch as agency, ob-
dtizenship. and the proper reiation-
betvveen national socicties an intemational po!itics--are phi1-
mthcr than But the rules are rdevant to alI n.:seardt
lhe goal is to learn fads about the real worM. thc dis-
tinctive chara<:tenstic that sets soda! setenee apart from casual ooser-
vation is tIMt social saenee seeks to arnve ay vald i21ferenc('S bv toe
of weU-estaolisheu pmcedures o inqury. Our foens
researd\ means that we sidestep lssues in the
sdence as ''.'en as controversies the role of
the nature and existence oi tmth, ilHU fl'.'-
!ated We asstmw tnat ir is possible to have sorne knc)\vledge
oi toe external world but thar 5uen 5 uneertan.
nothing in our set of ruk:s tnat we mU3t run
Uf sueh a or coHect <1.11 relevant
w!? can maK vali sdentifk inferences. An impor-
tant topic is worth studying even H very !ittl.e infoffi1aton s avaHable.
The o uny research design in his siruation wiH be
ul1crtain condusons. out so long as we honestly report our
chis kind of study can oe very usefuL Umited intt'1rmation
15 often a feature of sodal inquiry. Because lhe social worl.d
tllat us understand those
to understan them
even ,,{!ten "bout cm cnndusions s high.
gene}' of a problem mar be so tilat data gathered by the
userul sdentific metnods mig.ht oe ohsolete befre it can be
lated. If a distraught person s nmning al HS swnging an ax, admi
i
f lntroducton
a.n
tm;ke
inforrn,1t1011 about the
Dhen,,,, .. ,,,,,, afe ,,{ten
S TI.e Scmce in Social Scence
able scientific but thc oi hH1t-' is not
('tnL Facts. C.JD be f.'c1Ht?cted \lf quantit;t\'l?
more ot" .. 'SS systematcaliy. ano lhe fnm.c:r is heiter than th,,
aHeL hu OUT particular defniton (>f selKe rtC(lUlres the addtonal ster
llf iluempting tn infer the immedatc d,11a lo :;omething brodder
\hat 1:; not directiy ubservoo, Thar somethjng may involve j
f,:rcnce-.using ob!iervtions frmn the world to learn abour other unob-
servoo facts. Or that smething .may nvolve (IlUsa! il!faCl1(c-leamng
abnut causal effects from tue data observed. 'Hle domain of can
be restricted in spacte and behilvor in American election,;
sinee 1%0, so<.ial me)'vemt:nts n F;stem Europe since 198t:)-or t can be
extensve--h1.llnan beha\'ior sim:e the nventon of In either
case,lhe mark of sdentifk research Js (he oi rnak-
ing infereno2s that go the ol;!sen',lus coHectoo.
:t The pmcedu:res a.re public. Scientific resean:h uses co(hfie,< and
methods lo and data whooe reliablHty can there-
fore be assessed, Much social research in he qualtdtn:, follows
fe\Ver ruk'S oE research proCedufl' or f nf.:rence. As Robert K
Merton l94911%8:71-(2) pui il, "'1'he of qualilatve
,lata fren resides in a unfathomlble
and inlfabIe ' . is pub-
le, no! prvate." /'vterton' s statemeni is not [roi' o al!
still tme o sorne but
-mmy had no method-snmetimes as f the use uf ex-
methds wouki dminsh ther Nevertheless they cannot
but U$e some methnd. St ..,mehow ask ques-
tions, nfer informatiun alx>u! 'rhe w()rld fmm thes observatons, "ud
m,,!ke nfererKi::s "DOU! cause nd eH!;'cL H the method and
sea rcher' s observa tons "nd nerenee" are dt
the
o what tvas dne. \Ve cannot
ev,tluate lhe (lf seit:ctivn Iha! wen.' used to n:cord bservatons,
the ways ll whCl observathm" \vere and tbe \vhch
(ondusions Viere drawll, '.Ve (annat eara from th",ir mcthods l'r n:u"","<:
their TL"$ults. Sucl research is l10t ,1 "el. V\'hether or l10t it makes
":",,un,,"-,' t s no! a contril::ll.!tOH to soda! sen..:""
explidr or n.oi-hiwt' Hrnitatons. Tht: dvan-
tage oE b har those limitatlons can b(' understood M\d, if 1'0&-
sDle, addressed. in addition, the methLx1s can be and shared. This
protess aHows n:::;:;ean::h fl.>sults to br
ers ;md research :;tudes in b,' F""I;,-",,.,'
,1, The (cmdusions are uncertain.
prneess, tts
the world tha!
l'search-
and scholars lo learn,
an
d3 tu learn abtYllt
I
f
In!roduef1on 9
oE ,d the ret1I \\'odd or 011
nferenel! abou! <1 c;nb;d d't'l in l'he f1'\11 world le; unintcrpretabl, A re-
\vho tn Lhvt! issuc (d un(ertainiY i...; Fner
ng tlmt he' O[ she knows everything perfeetl\' 'tl[ [ha! h "l!' ';1:1
idea how certain or UH,'rt1n the results "re. Etther wav, inferences wlh-
ou uncertai.nty eshHlales re not science ,);. we define 'jI.
4. Toe content s the method. Fina!!;', scientfic research "oh,,'l'('s to " $el ('jf
mIes (lf inference on v",hch ts vaJiditv Explicating the mOSl im-
portan! nJles is a task of dlis oool<.': The n'l1!eni' of ",\"1C!1<:\"< s
the meihods and rules. lwt the m,lttef, "nce \A;P can use
these methuc]:. ro This W,1S
over a C,'nftlrv agu when Kar! Pcarson Utl
Q
1: lb} lh,ll "lw fl,'ld
01 scienc" b unl.mted; its mah"ria " endless; en:!\' gn>uP of natura!
!10mena, l'verv of 5r'ci,! lite .. (very s!age nf nr '
me'nt ig material f(\1' science. Th{ of ,l sdence consists "kme in It;:
merhod, flot in ts materaL"
Thes" (our eatures 01' science have il further imnlc't('l' "\'J' ' ..""'"
L,., r -' ,J A' "u.\,..J:,. u
lis v'st s il S(lcil1i . rL:searcher 01' teitm oi reseilrchers
lilbors undel' hmitiltions uf aod a.nd mistakes are
umnroidablc, yet such errors .vin be ponted out others. LTn -
the social ch,:r.Kter 01' scence can be sinee it
means that out" "york need no! to be an im-
portnnt contributinn-whether tu [he
lo
s"
to redirect) the concems uf
scholars and uses methods tI.> arrive al nfc'f'
ences lhat are ccmsis!ent w1th rules o sdenee and the tnformation ,1t
OUf it is t) ,'" A 1 1 1. ,,<1 "i:: a .ll1lnDutlOl1. t,c contrfbutun
u even a minO!' ;rtide is than !taL P( he wnrk" hat
fOft'ver in 2 desk Jrnver or v\'thin tht' cnnfnes f a compu!er.
1,1.3 Sdl'l1ce ami
Sodal sdence constitutt:.'s an aUcmpt to make sense s{}Clal stth!u.ons
that we as more 'Ir "ss ''''L>. J h) h . , vv,. ".C U " Oh'"
ever, tlai wha! \Ve perct:\"'" as complexlty is not entirelv inllerent in
phenomen: tIJe vvorld is l10t naturallv divid'" I'n"" ' J -' aH cnrn-
rrldr>. uf thi'
lnd{vtL rrUJst in'
s mpossible, even in prindplt'.

1 {} , The s,.-imct' in Social Sdence
pIe, or events. On the cntr.lr\. the p,-,rceved complexty of a.si.tu-
aln depends in part on lit)\"; weil we Hi simplir).. ,m..:l
capudty tu simplify depends on whether we can spearv outeomes ano
explanatory in a coherent \-vay. Ha\'1.r:
g
. obs:rvatlOns
mav iissist us in lhis but is usually lt1Sl.oenL 1 hus ccmpicx-
ity'; is :mrtiy nmefitmal an he state (lf our theory., . .
Scentific methuds can be as valtl<lbie fOT mtnnslCally eomplex
events as for simpler onCS. Complexity is Hkely lo make OUT inferences
less certain but should nof make them any 1e5..'> scientifk. Um:'ertainty
and limited data shou1d not cauSE' us ro abandon scientific reSE'arch.
On he contrarv: the biggest payoff for using the rules of scientifk n-
ference CKcun; 'predsely \vhen data are limited, observation tools
aTe undear, and rclatonships are uncertatn.
vVith dear and data" method may be lest?
mportant, since c'l/en partial1y mies of inference may producE'
ansvlers that are rol.lghly corred,
sorne complex, and in some scnsc tmiqucf evens vvth
enormous ramficatons, The eollapsc of the Roman Empire, the
Freneh Revolut1.on., the American Civil War, World Wal' 1, the 11010-
caust and the of Germanl' In 199(\ (.re al! examples of
such events.. TIlese events seem to be the resul.t of complex interachons
o many forces whose conjuneture crudal lo the event having
taken plan:, That is, ndependently caused sequences of events
forces at a glven plaee and time, their interaction appeanng
to brinO' about the events fFfirschman 1970). Ftlrther-
b
more. it IS oiten diJ{kult to evens ''''ere inevHable
produds of l;r,ye<;e;ale
on f,",,,,,vn
of our
5eems to have played a outside the
providt>d crocial links in thc of events.
chance oHen
01' the theorv
Oue \vav to understand sueh events i5 by """,'k:'j",,,,
com:cptuaing eaeh case as a member of a das:; events ahout which
meaningful can be made, This method often
\Vdl ter ordinarv v\'ars or revolutions, but some wars and revolutlOns,
being much extreme than others, are "outHers" in the stanstkal
di.stribution. Furthermore, notable early wars or l'eVO!utlOns may exert
rnpact on subsetluent events of the same dass-we
think of tite Freneh Re\'olubon-that cauhon is necessarl' in
comparing them \vith their successors, which rnay be to sume
he product of mit.'ltion the dass o E'vens can be usetu1,
but ir is not appropriate.
Another wa\' scientifkaHv \vith rare' large-scale events is
to engage in anall's!s:' "the mental construction o a
lntroduction . 11
course of \?yents ,'.'hich s altered through modifications in one or more
'conditiolls'" \'ebt'f [190Sj 19'19:173), The applicatitm o t11is idea in
a sy5tematic, scientific wa)' is illustrated in a particularl)' ex-
ample of arare event fmm geology and evolutionary biolngy, both
historically oriented natural sciences. Stephen J. Goulcl has suggestt>d
that oue way ro distingush systematic features oi evolution from sto-
chastic, chance events mal' be to imagine what the wodd \VouId be
like f aIl conditions up to a spedfic p.)int were fixe-i and then the rest
of history vvere remu. He coutends that if it were possible to "replay
the tape off lite," to le evolution ocrur again fmm the beginnng, the
world's mgansms toda)' would be a completely different Gould
1989a),
A unique event on vv'lch students of evolution have recentlv [0-
cused is [he sudden extinct1.on of tile dinosaurs 65 million vcars' agt1,
GouIel 0989a:318} "\ve mus! assume thar
not have evolved on our planet jf a cosmic catastrophe had not
daimed the dino&'1urs as vktims," If this statement is truc, the extinc-
bon oi the dinosaurs was as imp.,rtant as any histor(al event rOl'
human beings; however, dinosaur extinction dtx>s nor fal! neatlv into
a dass oi events fhat eould be studied in a systematc,
fasnion througn the applk"lton of laws in a straightforward
way.
Nevcrtheless, dinosaur cxt1netion can be studied after-
native hypotheses can developed and tesied with respect ro their
observable mplicdtions, Orw hypothesis to account for dinosaur ex-
tinctiol1, developed by Luis Alvarez and eol1aborators ai Berkelev in
the late 1970s (VV, Alvarez and Asaro, a cosmk coHiion:
a meteorite crashed rnto the earth lt aboul 72JX10 kilorneters an nouL
creating a blast greater than that from a fuH-scale nuclear \Var. H this
hypothesis is corred,. t would have the observable that
iriditlm element ('omrnon inmeteorites but .are on should
be found in the o the eartn's crust tha! corresponds to
st.>diment lnid clown mllion years indeed, ihe discoverv
01' iridium at predkted in the earth been as
confirming evidence fer (he Although this is un unambgu-
ously unique event., there are rmmy other observable implicatons. For
one example" it shouJd be possible lo find the metorite's crater some-
where 01'1 Earth (and severai candidates haye been foundl.
The issue of the cause{s) o dinosaur extinchon rernains unresoh' .. 'C1,
although the has genel'ated mueh valuable research. For
l-k)t\-'e\d&f .. an that t:"v \:',)kanic crup ..
tklrLSf !;>; 31S<J Cf)t'tsHent. t..,,'ith the ot tridiun1, dnd l,)A)!1Sb!ent
!he meteuri!e hypoth(>$ls wth the findil1g that aH the specic'5 extim1imb did !lO! (KCUr
simultarwously.
12 . The Scicl1C'e in Sodal St.ience
our pnrposes, the pnint nf example t..;; that sClentific gc)neraliza-
tkms dre usehJl in . CYeH highly t1!1u:;udl evento; th;l! do no( tal]
ntp i1 large ditss nf evenls. The AlvilR'J: hypothesis cannot be test!:\:!
wth reference to a ser uf common evento" bu ii does ha\'e ohsen:abte
imp!catiol1s for orher phenomena th"t c;m be cv"luilred. We should
nle .. hmvever, tha! a hypothesis is no! considered a reasonahly
explanation until it has been evaluated empiricaHy and passed a num-
ber or demanding tests. At a minimum, iis implications mus be con-
sistent \vim OUT knowledge of (he externa! wodd; at l:x:st, ir shouid
preclct what 1mre Lakatns 097m refers lo as "new facts," th"t those
fornlerlv unobserved.
TI1e pout is !tal even apparently such as dinosaur
extioction can he sfudiL-a sdentiicalIy 'Ve p<ly attention ro impnw-
theory, dala, and om use ni the data. lmprovng OUT theory
through conceptual darification and specification of var"lbles can
generare more observable implications and e'len test causal thcurtes
uf unque events sueh as dinosaur extindion. lmpnwing our data al-
JO\\'s us to observe more of these observable implications, ami impr"-
ing our use o da.) permits more oi these impleatinns ro be extrac'ied
from existing data. That a set of events to Di' studed is highly complex
dnes not render careful resean:::h irrc!evant \\fhether ,ve study
milny phenomena or fe\v-or even one-the study \vil! be improved f
we eoned data on ilS manv observable oE our thenry as
L2
""P""'" research at lis best is a creative process of and
within a weH-estab1isned struttUTe o scientfk
ftlr a metharucal process of U"""''''",,,,
the seholar mus nave tue ""AH"'''"'
and evalua-
o! mind ro
oi looking al the \vorkt ro ask new questions, ro
revse research desms appropriately, fu,d then ro coHect more data of
a diiferent l:ype than inlel1ded. However, if !he researcher's
findings are to be vald and aceepted by scholars in this field, aH these
revisions ana recol1siderations rrlUs:t take accordi.ng to expicit
consistent vvith the rules of A dyl1amic process
nquiry necu"TS \'vithin a stable structun" oi rules.
&xial sdentisls often research \vith a considered dt,sign, co1-
lect some data, and drav."' condusions. But mis process 1S rarel}' a
smooth une and s not !x"'St done in this ordcr: conclusions
rareiy foHm'\' easiiy from a resean:::h design afld data coHected in ateor-
:-'1aor Components of Research Design 13
dance \\,ith lt. L)r1Ce an hL15 dat.;) rfOYided
ti researcb he or shc o.ften find an in1perh:ct tit aUl,ong U1C
main research quesbons, the tbcory dnd tIle data at hand. j\t thlS o,tage.
researchers oiten discouraged. They mistakenty hdie"e that
other scientists find dose, immedi"ie fi!s between data ,md re-
Ths perception is fiue to the faet that nvestgators often take
do"\,vn t:he scaffolding after puttng up their buildings, lea\'-
ing iittlc trace o the agony ana uneerta.inty of con5tructiO!:, Thus tn.e
pro<:ess of inquir\' seems more ;md cut-and-dned than Ir
acroaUy 15.
Some of OUT advice is direded toward researchers who an' tr:ring to
make cmlnections l:x:tween th0)\/ ;.md data, At times, they can design
more data-collectinn proc01ures in order tn evaluare a
theory al other times, the)' can tlSe the data tht'v h;:rve and rt,:'Cast
a tlteretical questin (or cven pose an t.'ntrely diff('renr question that
was not originany foresl'Cn) to produce a more importan! reseaTeh
project. The f it adheres to rules of 'wiH stiH be
sdentific and produce relable inferences aaour the vvorld.
VVherever possibJe, researchers should also improve their research
d
m.;i<'llS bdore conductin<> anv field research. I'lO\vever, data has a
'-.. 17 o h .
disciplining thoughL lt is exrremdy wmmon lo find t at me
rescarch dl'sign fans apart ,vh('n the very first obscrvat1on5 are
co!lel::ted--lt is not tnat the throry i5 \vrong bui that the data are no!
suited to answeling the qUi .. '5tiOns originally posed. Understanding
from the outset vvnat can and \vhat cannot be done ar this later stnge
can he:!p the rescan:::her al least 50me of the problems \vhen
first designing the research.
For malytical purpose5, "ve divide al! rese<1rch nto four
the research thl:' IheoTY, the da!ll, the use [he
dfll. These components are nor usual1y separately ud
scholars do not aUend to them in an)' order. In fact., for
qualitative researehers who begin their neld work bdore choosing a
precise research question, data comes follO\VL-a by the
Hmvever, this particular breakdow'n, \\'hkh we explain in sect:ons
1.2.1--1,2.4, s particularly usel:ul for understanding the nature rt.'"
sean:::h designs. In urder to darify precisely wnat cvuld he done t re-
sources were redireded
t
our ad,dce in the remainder of this sedion
as..sumes that researchers have unlimited time and resources. OE
course, in any actual research sltuahon, OT\t? must always make eom-
promises. believe that understandng the advice in the four
gories that fol1ow will help n."Sean:hers mi1ke these compmmlses :n
5uch a way as ro improvE' their research most, even \vhen 0"1
fad thdr researeh 15 subject to external constraints.
The SOalce in SoCal Sdence
121
Ihroughout ths book, \Ve con.sder what to do once v;e dt:'nt\' bt'
object of researen, Givcl1 a rescaren questi(lT), what art: tlw ro
conciue! tbat rescaTen 5(1 that We can obtilin "dUd explanatons nE
and politic,11 plwnmcn" 7 ()m Jiscussion begins with d research qU("S-
tlOn and then proceeds in the stages o desgnng ilod eonductin'" the
L. t"'f
But ''lhere do rescaTen yuestiol1s originate? How dOes a
scholar choose tn/:' tapie for There is no simple ans\ver to ths
question. Like others, K"d Popper (1968:32) has argued that "tltere s
no suen thing as a logieal methud oi hi1ving !lew ideas ... ,
eontalns 'an irrahona! element' m " 'creat1ve intuition.' u fhe nlies ni
at (he earliest of tne I'eseafch process are les"
th"n are the rules ror oteI' rL'Se;lrch adivines. filere are texts on Jc-
signing laboratory expe-r!t1I::'nts un social statistical criteria 1m
drawing a {or " sun'cy oi attitudes 011 pubhc policy, ana man-
ua!s on eonducting participanl observaol1 of a buream::ratk offke, But
there is no mJe foI' 'whch researeh pwject tu conduct. nor if
we should decide to w()rk, are theI'e rules governing
\vhere ,ve shuld conduct it
"Ve can pro pose ways to select a sample oi comrmmities in order ro
the impact Di alternative educational polkies, or ways to coneep"
ethnic conflict in a t11anner conduclve to the forrnulation and
oE as to its inddence. Hur there are no rules tnat tell
educatonal poliey or ethnic conflict. n terrns off
sodal sdenee tnere are bet!er ana \VOTSe '.vavs ro the
of he Easl German govemment in 1989iust as -thert' are hetter
and worse ways ro study the be!\'\'een i\ C;1n .. ",.-."
tion on laxes the lik.::lihCK)( ot electoral suecess, Sur there 15 no
\vay to dettlrrnine whether ir ls better lo the of the East
reginw 01" the role ni taxes in U.5. electoral politcs,
Ihe specific topk that a social "cienns! studes may nave } personal
and idiosyncratic lt is no acddent that reserch on particular
1S likely to be by people oi that group: 'vmnen have
oHlm led the ,vay in the history of women, blacks in the historv oi
blaeks, immigrants in the o immigration. Topcs ma)' be
mfiuenel.i by and Vah.1'S. The student ()f hird-
world polines S likeJy to hdve a dL'Sire for trave! and a
tolerance for difficult living condtions tnan the sl:udent o' mngres-
"ioIlal polky mlking; the annlyst of intemational eoot't'ration
nave a particular distaste for violtmt conflcL '
These personal ma ,'alm,'5 "iten provide lile motivation
Majen Components of Rescarch Design 15
ro beemnc J sZlCial scientist ;:lnd, ah'!', ro cho()se a particular
questiOlL As such, thc\' 111.1)' ((\nstltuk he ",hll" [zlr
in: } particular rescarch pmject-and approprt'ltdy so, But, ll(' matter
ho\\' persona! or idiosyncratic the re"son;; tor choosing ; topic, the
methods of "enc(' and rules of infere!1cc in ths book wll
help schlars ..h;'vise more poweriul rescareh dtesgns, Fmm ti,,"
spective o a potential mntribution to social personal rei1-
sons are neilher necessary nor sufficient justficattll1s for the choice
oi a tpc. In most cases, they should not appt.>ar in om senolady
writings. Toput t most direcHy but quitt;> no 011'::
what we think-the schnlady communitv caH:S H'hat \Ve can
aemonstrilte.
rules fUi choosng ,) c!G not exis, th<c'tc are
individual detern1ining thc
of a research 'nterprise io he scholarly community. IdeillI:v,
aH rt.'Semch in the sodal sdences should [\'.'0 critt'ria,
FirsC a rcsearch 5!n)uld pr)5C a tha! in the rC;J!
The topk should be conset.uential inr poltica!, sociaL or eco-
l1mC ior understandng something that affects
rnany x'oplc's Uve's, or for understanding and predctng eVl:'nts that
might be harnlful or beneficaJ Shively 1990:15). Second, a j-""''',,,.,,,,,
project should makc a mnfriufiul1 tu un
flm: out c0llecfin'
ftilfans af the word. Tnis lHer crlterion does out imply
!hat al! n>seareh that mntribl.ltes lo our stock of soda! sdenee
natioos in tact ims at makng Gms;l inferences, Snmetimes
the statc o knO\vledge in ; field is sl1ch lhM tnl1ch ilnd
description ls nt"f'\led bdore \Ve can take on the ehIlenge of
nation. Often he mtributon of a pl\)t:'ct wil! be descrptivc
nierence. Sumetimcs the may no! c"cn be descriptl\?e
bui rathel' will t'X' the dos!:' oDser\'ation of partculilf evcnts <JI' he 5um-
mar)' of his!orkal detaL meet OUf second crterion
because are prerequisites to
Our firsi criterion directs our attention to tl10 rteal worle! of poHtics
,1nd social pbmomeoa and to rile current and historical re('ord (lf tht:,
('vents and problems Ihat Jives. VVhether el rescaTen
question meets this eritt:'rion is a sodetal judgment. The "ec
ond criterion direets our attention <) the literature uf
sderKe, to he inteHeetual puzzlcs m)t psed, lo th;,t re-
man to be ,1l1d to rhe scientific theorics md melhods 2\'ajlablc
lo sIn' them.
16 . The Sci'7ce in Social Sdeu<."'e
OUT first criteron. Ten major wars during tlw last fOUT hundred
llave killed almost thir:y mi1lion peop1l,' tLevy 19S5:372j; snme
"limltt.>d \'\'iUS," such as those between the Unted States and North
Vietnam and between lran nnd fraq, nave eaeh dainK>t1 over a mllion
and nuclear war, were it io oeeur,. cOUld km billions of hum,m
being5. Polit.ical mismanngement, both domestk and international, has
led t economic prvation on a global bass-as in the 19305-as well
as to regional and local depressl0n, as evidenced by the t:ragic expe-
riences of much of Africa and Latin Amerka during the 19805. In
generaL emss-nation11 variation in political institutions lB associated
vdth great variation in the conditlons of ordinarv human Iife, whieh
are reHectcd in differences in lite and infant mortal1ty be-
tween countrt.>s with similar le veIs of economic deveJopment (Rus5ett
1978:913-28), Within the lJntL't.1 programs designed to aHeviate
po v ert Y or social disorganization seem to have varied greatly in their
efficacy; It camwt be doubted that research \vhich contributes even
marginaHy to an understanding f these ls5ue5 is mportant.
\Vhile SOt-:iaJ sclentists have an abundanee of sgnific.cmt questions
that can be im/estigated, the r001S fnr understanding them are &Caree
aOO ra!:her crude. Much has been written about vvar or msery
that adds 1mIe lO the underst.md.ing uf these issues because it rals
either to describe tht'Se phenomena systematically or to valid
causal O descriptive inferences. BriHiilnt insights can contribute to un-
derstanding by yielding new bUI brilliance is
not a method of .1\11 hypotheses need h) be eva1u-
at'd empirically befor(' Gm
This book offers no advice on """'"'.V.<HH
ever, is to the importanee
eonstitutes a contribunon to
a contribution ro knowledge.
brillanL What it can do, how-
cnnducting researcn so lha tt
OuT secand l"riterion lor a research question, "'making ti
contribution/' means expicitly a research desgn \vithin the
framework o the existing sooalSclentific literature, This ensures that
the investigator understand the "sta te of the art" and minimizes !:he
d'lnce or duplicating what has aIready done. lt also
that the work done \,;'ilJ be to thu5
success oi' {he of scholars h,ken as a v\'1101e.
contribution ro the lterature can be done in mi1ny diiferent \"'<'a1's.
We lst a few o the here:
1. Choose a
contribution.
ivlajor Cornpncnts of Research Desgn 17
C:-h<)t)st.'> dn in the that
\{;f one h.::'ievt' has nt l:wen
whether it s inJeed fais,e or vfhetlwf s"me otht:'f theor\' l.'i n'frl'CL
3. Atternpt ro resolve or pren.id .. further evdenct' ni one sde ni a contnr
verSy in the rUeratun.::.it-perhaps that the (pntro\'crsy i/'/3S
unfounded fmm the sta.rt.
4. Design research to iHurninate nr evall1ate unquestioned assumptions in
he ltemrure.
5. A.rb"U that an importallt topie has heen overlooked in the literature and
then to cpnITibut:e a systemate studv 1'0 the area.
6. Show tha! Iheorit'S 01' evidence for sorne purpose ll un\'
ture could he applt>d in another literJture tu solve "n but appar-
cntfy unrebtL4:1 problern.
Focusing too mueh on making a r:ontrbution to a litera-
ture without some attention to topics that have real-world importance
nms the fisk of desn.mding ln politicaUy Con-
versely, attention to the curren! poltica! to i;:,;-
sues of the amenabiity of a subject to syslem'ltlc study within tht.?
framework a body of social sCeI1ce knowledge leads to careless
.. vork that adds litile to our deeper understanding.
Our two erireda for choosing research questions are not
in opposition to one another. In nm, understanding
'''TOrId phenomena is enhanced by tile generatiol1 and evauation
explanatory hypotheses through the use of the method. But
in the shorf term, t11ere mav be a contradiction betvveen use-
furness and kmg-term VeliLle. For instanee, \1ankiw (990)
pont's out that maaoeconomic theory 2nd applit'd macroeconomics
divergeti sharply during the 1970s ,md 19805: modds that had been
ShtH\'Il to be lheoretical1v incoherenr still llSt'd ro forecast the
direction o the ES. ,vhile the rte,,, tht:oreticaI de-
to correc tuese flaws remained speculative and vI/ere not suffi-
dentIy refined to make acrurate pretHetons.
The criteda of practica! applicabiIity to the real world and
tion ro to one another when a
,dI begin with " rea!-
ht:> hrear uf
men and 'VOlTH"I1.. he lransition lO de-
Others may stalt \vlh ,.n
th soda! science mcrature: .1 coniradiction between
studif's of under llr
an nconsistency between thcories \oting and recen!
election outcmt>s. The distinct10n hehveen rhe criteria o (ourse,
18 The Setene!' in Social ':Xiencl:'
not hard ilDd bsL Snme rese,ych questim15 Siltisty both criter;a tn)!1)
the but in ft..:searchers oJtcn
nearer (me than the other.
4
Whercver it begins, the m)ccss of dt?sgnng rescareh ro mswer a
spedfic questlon 5hou1d mc>ve h)\\'ard the satlsfaction of OUT two c.ri-
teri,L Ana obviously our drt'ction of movernent wll depend on wlwre
,ve start. Lf we are motivated by a sodal sdentific puzzle, we mus! ask
ho\v io make thaL research topie more relevant lo reaJ-\Norld topics of
signficance-for instanee, how might aboratory experiments beUer
illuminate real-world strategic choices by poIitica decsion-makers nr,
what behavoraJ cousequences might tbe theory have. lf ,ve vvith
a rea!-world \ve should ask huw that problem can be studied
\vith modem scientifc methods so that ir contribues to the stock o
sodal sdencE' Ir be i.hilt VI'e w decide tha!
too far from (me criteion or he jo; not the mn5r truitfu! approach.
Laboratory eX'?erimenters may argue tnat the &earen fnr externa! refer
en!s 1S premature and rhat mure progres" \vHi be made by refining the-
urj' ana nlethoJ in the more controlled environment oi the laboratory.
i\nd i.n terms of a long-h:,m1 rescareh tney may be righL
the schoLar mot1va!E'd a real-,vorJd pmblem rnay argue
that aecurate description 15 needL,(l before moving to explanatiOl'L And
sueh a researrher mav also be right Ac"Curate description s an impor-
tant step in researrh pmgrams.
in either case, a research ana f possibJe a specifk researcn
pmjecr, should aim lo our two aitera: it snould deal witn d
rt'al-world to contribute. or
to i1 iterarure. Si.no: OUf main concem in
this b(,ok is research more scientiflc. we wHl
who sta.rts wilh the "real .. v,'orld" pt'Y-
of
rathel" han wilh
literature. it 15 ,,;ssentlal to devise ti workable plan fOi
fhal (anno! bt into t1 re5carch
Ji Ciwsal 3hould be
ha! will make nO contri-
19
Eter;):url' ",houkl f'L1\'-
ing cho:sen a \\' ente!' il "n/ith tht:: litcrl.:U[\.:',
What questnns o interest to U5 have aln,;\ldy been aH5wert.
M
j? Hw
can we imd refine out' question su hal lt seeu1S capable o)
.. witn he iun!s ;lYdildb1t<> \-Vl: ma\' "tart v;ith a lS511t',
but we wH have to curne te h"th ,,th he literature uf
science and he problems ni inference.
the
such an oxymown sh,mld no\ be caBed ,1
1991:4; see a150 \Vmx..15 ,md Wallnn 1982).
'file development ot a as tlw fir:'it 01
It 50m.etimes comes in but it ne.d nor. in fac:t.,
we cannot a theof)' ,vitlwut of prior work 011 tlle
and the 'lince even tIle re"earel!
don \vould \VhieVer d!110unt o'
data has bL"CU there are sorne
ate ilnd the usefulncss oE a theorv. VV
of these here but SilYe more detailed discussio)l ior
choose theores thdt could be wmnf!;. indL't'tL more 1"
learned from theories thar are wrong rhan frO!'n theories th"t ,re
$(1 lhar could not bto' \vrnng in : \'Ve need ro
a din.:'Ct answer ro ti",.' VVhdt ('vidence
COl1\'lm::e us iha! Wt' are
H lhere 1S no anf-'wer ro
then v;e du no! have a theon,r
;::hnse one that is capil-
as Thjs
choice vll allenA' more tests nI' the
\vth more d"ra nd a
of will the
al risk f falsifi;:d nwre tunes,
"nd wiIi make it to cullect data 50 ro tmld
r'nr the
20 . TIw Sdcna: n Socia! Sdence
Tllin.1, in cnnCft.:te )S
st<lled theor!e,; ,md hypthesl':-' SelY,.' ll\l purpnst' bUi hh)biuscah?:. The-
Orlt'S th,1t are static't.i precise!y ilnd make specific prtxiictions can be
shown niore ro be \VTong ana dre therefore beUer.
Sorne researchers recommend follmving the principIe of "parsi-
rnonv." LJnfortunatelvI the word has bcen used in so manv ways in
caslll conversatinn schlariy \vritings that the principie be-
come obscured (see Sober [lQ881 for i\ complete discllssion). The dear-
est definition ol parsimony V'<1S gh'en by (]961:-t7): "Simple
theories nave highef prior probabililies."7 Parsimony is thcrefore a
iudgment, or even assurnption, about he nature of the ,vorld: ! s as-
sumed 1(1 be simple, The of theori",s thar a
simple \yorld is a rule thar applies in situations there is
" high degree oi certanty hat the workl is indet'Xi simple, Scholnrs in
physics seern to find parsmony appropriate, but hose in biolog.\r often
think oi ir as In tlw soal some forcefully ddend par-
simony in ther subfields Zdincr 1984J, but we believe t s
occasionaJ!r appropriate. Gi\'en the precise ddinition ni parsimony as
an assumption about lhe .,'rld, ,>,ce should never insist on parsirnony
as n uf but it i5 usdul in hose
situations wherc \\'e ha,,' sorne o tIte smpHcity of the
'levorld 1-ve an'
Our pont is that we do nor advise researchers to parsilTlony as
an essential good, snce there seems HUle reason lo adopi iI: unless ,ve
alrl'adv know ajo! about a sUbiecL We do no! even need tn
dvoid since t is directl.v implied by
[he maxim as aH our
evidence relative to the
eS:lgi:Hell can ead ro what \ve call
(see SL'Ctlon 4,1), out [hese are prob
lems nf research and not assumptions abour the world. .
/\.11 our advice hus far applies ir we han: no! coHected our data
and begun imy Hmvever, ifwe have gathered the
\ve can certain!y use these mies to modify Ollr and gather
He\V data, and thus new observabie of the new
Of CTlUTse, this process s time consurning, and
\'\'ilsteful nf the da collected. VVhat then about the
situaton where our IS in nbvinus net't o but
\ve cannot afford in collect additonal data? Ths sltualion-l1 \vhch
. This has comt' (o l:w KnUWl1 dS the
cnncept s similar t CXcan razor.
Maior Components (lf Researdt Design . 21,
/\11\ schobr CHl Hl\C up \\'th ,1 "plau:.iblc" 111,,>
for <"In\' (li data alter the "eL tti di) so 1k)(h-
about lhe verad!)' uf the theorv, Th0 thcory wil! fi [he data
still Inav x:' wildh,' wrong-ndeed, demonstrahh' h'mng ,,th
other d:lta. llurrul1 al'(' verv a pattdl):'
t very at recognizing nonpattems. (Ms! oi' us c':en se\:
... ', .... ," in 'random ink Ad hoc adjustmen!s in a theory th<"lt
unt fit existing data IYtUst be usted rarely imd \,,:ith considerable
discipline.
8
. .
rilere is stiH the problem of Wh,lt lo do when ,ve hilVe fnushed Otlr
and and wish tn work on improving theory,
In this sittwtion. ",,'e fpl10vdng !wn rules: if our pre-
",',,-.>,,,,,, 1S condtional on variables and we ilre willing to drop
oue oi the condit)ns, ,ve may d so, For example, if vve hypothesizt"C1
oricrinallv tha! demoCTatic countres with social vvettare
te;;'s do' not fght ench other, ir \mld be 1:0 extend that
hvporhesis lo aU mndcm dcrnonades and thus !:'valuatc our theor)'
against more cases and ncrease its ehances ot being .. '
pint is thlt <lfter the we mav modi!y our 111 a
'way that make$ it apply to a of phenomena. SinoS' such "n
aJteration in our thess it mon, fuUv to modifca-
oon in l:his direction should not to ad hili: that
merely to "sdve" dr1
ro phenomena that llave
The oppost{' practke, napproprate, AHer ob-
the data, we should add a restridi\'t: conditon and
as j our th,,'ory, with ihat qualifkatiol1,. has been shown
to be corred, lf our orginal" \vas that modern do
not fight \vars wlh one al10ther due to their conslitutional t
\vould be less found lo ,mr
restrict the pm:X1sition to dernocracies with advanced
once f has bccu he data tlmi such ti
!o make PUl' corred, 01' suppose that
\vas that ren>lutions only occur under
oi severe econornk bU we find that this s not true in one
o our case studies, In
stualioll wnuJd flor be
such as, rcvolutlons never uelllr
\vhen he miltarv s the
ro dd general
uds (lf , ... rn"npntv
ershp is "P1''''''''''''''
the f.'Conomv is bast>d" OH J smiill
- H \\"(1- thn'(' <Ji n\11\\'orki
(ontribution ro ii scholadv the natuH::: 4,)1 dliH..leU1hl VI- HI corn::\ct th1 snu-
tioa: $Olnt,'tme wil! our ,,'lh Mw!her ;e 01 ti! ,md uelno!l"t'r;!e UM!
we wefe wrong,
i5 \\-:1DT'L Sucb ,1 Iil(lfC!Y
_misleading) Vi/ay of'my b correct, e:xcept in (()Un-
Since we nave discovered that our theory is lncorrect
f"cmmtrv x, ji dQs nor help to fum ths ialsfication into a spurous
generaiization. WiLhemt forts to cnl1ect Dew we wil! han? no
admis$ble evidence to support rhe new verson (lf tht, theory.
So OUT bask rule "vith resped to altering our theorv after observing
fue data is: me am ma/(' the theof!! les;; rcstridivc (50 tf1l1t ii nruers a broadcr
nmge pltenomeml ud s expDst'd lo mure Dpporhmities {or fl115ificatimrJ.
[mI me s!u:ruld flof maite rt more rc::;tricfi,w WitilOut CO!!ecfing new data fo test
the neu' Vi?fSiOft the tl/f;ory. lf >ve cannot collect additonal data, then
we are stuek; aud .ve do not pmpose any magica!way of getting un-
stuck. At some point, dedding thnt \Ve are wrong lS indeed, nega-
uve tindings can be valuable for a scholariy literature. Who
vvould not prefer (mE' finding o',[er any number nE flimsy
positive findings based on ad. boe theories?
!',4oreover. if \Ve are wrnng, we !'leed not stop \<vriting after admitting
defeat, V'v'e muy acid a section to our artlde or a chapter lO our book
about tuture empinen! resean::h and curren! rncclH'ticaI SpeCUL"tion. n
this contexto we have consderab!y more fn.'Cdom. We ma)' 5uggest ad-
ditional conditions that be attnchea t our theorv, ii
we beileve they mighr solve the probIem propuse a modification uf
p",id",.,n thcry nr pmpose a range of entirely different theo-
lve cannot conduae anything ,vHh a great deal of
o?rtainty perhaps iha! lhe ,ve stated nt the outset is
out \\'e do have Ihe of !lev\" researen
:r data-cnllection that could be used to dc'cide v\'!lcther our
speculations iire COlTi'ct These can be very in sug-
oP;:.ti'"v areas vvhere futun: researchers can !oo!c
as \Ve discllssed sodal sdence aoes not
to rules: the need tor sometimes mandates
that lhe textbook be discarded
f
And dar" can discipline thought.
llenee rescarchers wm somebmes, after data, have i.nsp-
ratons abour loen-v they should have consfmcted the theorv in the first
Such a moditkation, even if restrichve, may be \vorthwhHe iE we
can convinee oUrSt'fVes nd nthers l:hat mcldfying the theory in the
\vay that We' propuse is h'C could have done beiore vve co1-
ecred the data f 1ge had it, Bu unhl tt.'sted v'(h 11(,,1' Jata.
the status uf such a \ViH remain ver\' tmcertain. and ir should be
labelcd 35 such.
Onc COlbcqucnce off hes"! mies " !ht
jiten \tT\ useiuL "S[!l:"i. m 't's'rch dat must be
or other l1h?ms. Preliminary datil-
questons or modHy thc
"Data" are svstematicaHy collected dements of nformation the
world, They 'can be qualiative or quantitative in style. d,lta
are colleded to evaluate a ver}' specfic theory, but nor sO nt:-equently,
scholars col1ect data behm.' knowing predsely \.vhat they are mterested
in finding out Moreov.:'r. cven if dJtil ,me co!1edea ro evaluate a
'f' - ma\' ultimatelv be interf'sted in questlms
oH: 'r-' ", .. ::>,
that nor ccurred to them prevnus!y.
In either case-,when data afe gathered for a purpose or
when data are used for some purpose not dearly in mina \vhen the)'
were gathereLl-certain rules \'\tll impmve the quallty 01 those data. ln
princtple, Wt' can think aDOU! these mpm\n.
g
ebta,
[mm the rules in section 1.2.2 fUf llnpmvmg theory. ln . am
data-colh."Cton efforl requires some (lE thcory, just as formulat-
lng any sorne data (see . . 19(4): ' .
Our first nd most important tor unprovmg data LJtIahty
i5: record and fi'lmrt the ;mxess whid he data are \Nithoul
this information we cannot determine lvhether standard pnxe-
dures in analyzing the data \viH produce biasea On1y by
knovdng the process by whch the data were generated W1Jl ,ve be able
t - valid nr causal inferences. In a
opinion polI. recording the data-generation pmcess t!,at ,ve
kmnv the E'x.ct method v\'hich the v;as dr:nvn and ti).:
dile questiofls that were asked, In a qualtative {',lse
studji, reporting the mies ""hiel: v;e choose the smaH num-
ber nf cases for IS critica!. We addlt10nilJ in
d1apter 6 for case in qualitative but even more im-
portant than choosing a good method is being lo record dnd
report ,\-'hatt'ver method was lIsed and aH tl1(> inrormatlOn neces'k1fy
for someone else lO PpJy it'l
in section 1.2,2 \ve for theories th, are df
24 The S.:it'lVf in Soda! 'Sci('!Kt'
()Uf
data qunHty is in un!{"T .f.{, t; di.1'fa eH a,o.: lJhIU'i!
of its (ll'St'TUablt, . This me3rtS cullectng as
aata in as nMn\' diverse wnh',ts as possible. EdCh addit1c;nal mpli-
cat1cm of our th00ry which wc "bsern! pw\-kh:s aHotlwr cnntcxt in
which to 1..'\'all.1ilte (s The more ObSi:'fVilblt, implicatkms
which ilr hmnd to be COI1SLstent \-virh the theory, the more powerful
the explanatol1 ami tbe more ccrialn the results.
When adding ddta on ne\\' obseJ'yable irnplcations oi a theorv, vI/t'
can la) coHect more obscrvations on the same dependent variabe, or
lb} record addithm<ll dependent vari"bles, VVe can, for inst.mce, dis-
to shorter time perinds nr ,1reas, We can
collect nt'orrnatm on dependen! variables of nterest:
lf the results dre as t1w pn:dicb, we wll hin:e more confidence
in thE'
For cOTlsider the fational tneory: potental nit-
ators ot \varfare calculate he cos!s and bendits of attacking other
sta tes" and rilese cak'ulatlofis can be influenced bv cn:dible threats o
real.,tiofi, The most dirt'C't test oi tnis heorv be ro asscss
threats ol ,var, dedsions to are assocakd h'Hh
suen factors as (he o militarv forces behveen the ptential ato
t.,cleer and the defendt'r nr tht' nh,rests ai "take fm he (Huth
1(88), even cases in v.;hch threats Me
ssued corLstifutes a set oJ implimtions of the theorv, thev
,re on1y pan oE the observatiollS thi'lt cnuld be gathered (uli usei
"lone mal' cad to selectoll :;inee stuatons in Vd1ich hrets
tnemsclvcs are deterred vn)Uld be exduded ITorn lhe data seL 1-lenee it
might be tvorthwhile aIs(, to colled data Oll an additinnal
variable a ser nf bdsed on a
measurement of whether
incentives t do so,
Insofar as suffLlent datd ()fl dcterrenee in interntonal
is it could al50 be helpfuI to test a different nne wiih
similar motivational rOl" a differenl dept:'udent variable
under different conditions ,,,heh s still an implication
of the sume For
ex-
r""'TH1'li"''H to see undcr smul,lted "threats" are dc-
t0rred rather han iKcentudted power and firm b,lrganing
(Jr coulJ exam!r'le v-'hether other actors in e sit:.
sueh as flrms eompeting for milrket share or 01'-
{;:rnilies (or USt' deterrence
ane! 110\'1/ successful are undel' rndt'ed, econo'
miss working in the fieid of industrial orgilni7A1t!on have used nor-
\1ajC'f ('ompont'TIts nf l{t>seilrch Design 25
v fr'l ...... ) , \\'hich deterrt:n()'<
C(}(lper,h.l \' t: l. " , .
sludy 5uch pmblcms 35 cnef\' mio market> dmi
and TimJe jqK,n, Ci\'en l.he elose similr1ty 't>tween. tite
theories, emprical e\"idence supportng game lheorv' s predKholls
about firm behavior wauId nCTCa5t' he phnlsibilitv o reJ,ltt'cl hypoth-
t.>SeS aboul stilte behavor in ntemation,)j politcs, Uncertainty \A:ould
rema in abuut the apphcablity of concluskms tmm Ol1e doman to an-
otiler, but the issue is important ennugh lo \<V<1rrant attempts lo g,1111
inSight and evidence \vherever they can be found. ,
In data forever without doing anv arhlJysis wouki
precJude rather than faciJtate complctioll o useful rescarch, In PfiK-
rice, iimited time and resoufees wil! ahvays cOHsl:rn
efforts, AHhllugh more informatiol1, addtional eas,:;,s, extra nlcrvic\''''s,
anothcr afia other relevant fonns of data collection \vil! al-
improve rhe o OUT inferencps to sorne dcgree, prornis-
l'lOtential ;;chola!':: can Ix' ruined bv too much inforrnaton as
as too little. Insisting on rcadng another bnok or slill
one more data sd wit110ut \vriting ti w(lrd s a prescripton fur
unproductive.
OUT third guideline is: !lltlximi::c tile o' om mc:vUn'111cn!,;, 'v'a
liditv refcrs ro what we think \-ve are measuring, The unlLm-
pioyment rate nM)' be a indiz-ator of the state uf the economy, !:lut
the hvo are not In 1t is cask'Sf lO mdxirnize vahd
ildhering to the and nol alknving unobserved or unmeasur-
in he wa,', ff an informant lo our queshO!l
we knO\\' he Si/id tilat he was O
that, \ve have a ';aJid 111CaSurement VA1dt he mCimt s
rm altogdher different that canllo! be measured ,,,th <1
high (lf confidence, For in countres w1th
mdy be a W2y o
poiitkal staternent fm sorne rol" it is <l ,vav nf
donft kno\'\',"
Our fourth gudehne is: msurc tlwf daia-cof1eclion mcillOds re rdiaMe.
ReliabHitv mean!; that app1ving the same in the same WiW
wm' the me,lStm:. \'\/11en ; relable pmcedure is
applied at imL'S and nothing h<1s in the meal1tlme
to the "me" state o the object \NE' ,lrc the san,e
result 'will be obscrved, Rd"blc measures dls<'
\Ve can cht."Ck b\' HH!d<:;,t.H'ing q\ktntlty
wheth1.. .. r .-1-rt' <a11L
thl> qHt.'''..tn!'', .1t Jt:H:nt ,juring
lHay th",' tu
in be
26 . Hle SCi:nce in Social Science
suJts when applied by dlffert'nt resean::hcrs .. and this outcome
ot coursc" uJon theTc bcing c'plicit procedurE's that can be fol-
lo\\'ed. i1
Our final guddine s: al! data ami should, as possible
be Replicability ilpplies not cmly lo data, so that we 'can
whether oul' mensures ilre rehable, but to rile cntre reasonin<> process
used in producing condusorL". On the basis of our resea:rch
ne'v resean::her should be able to duplicate OUT data and trace the
?y v\'hich we reached OUT condusions, Replicability s important even'
t :me repHcates our On!y by reporting the study in
surloent detall so thilt it can be repikatcd s jt possible to cva!uaie the
foUO\-'i'ed i1nd methods used,
Replkability of data may be difficult or impossible in some kinds of
rL"Search: intervieweL"S may die or disappcar, and dired observations
(jf E'vents by or partidpants cannot be repeated.
Rephcabtbty has also come to mean diHerent things in different re-
sea,n::h traditions: In, quantitatl've resean::h, scholars tocus on repH-
cahng I'he analys!s arter starting \vith the &ame data, As an\,one ;,vha
has ever tTied tu the quantitativc rt."Sults ni even .
publisht>d \vorks knows 'l/veR 1t 15 usually a lot harder than it should
be and always more valuable than it seems at the outset {see De\vald
et al. 1986 on repHcation in quantiti1tive research)'
The <1na10g)'" in traditkmal qualitative research s provided bv foot-
notes and biblingraphic essays. Using these tooIs, succeeding <
should be ,lble lo locate the SOUrces used in published vvork and tnake
their m"<.'n evallations of the inf0'rences daimed from this information.
For fesearch based on dred replication is more diffkult.
On0' $Cholar could borro,,,, another's field notL"S or tape recorded inter-
VleiVS to Sf'e lhev support the condusions made by the
nal llWestl
b
Btor. Smce so much o rhe data in field reseal'ch involve
mpressions, and othel' unrecoroed partkipatory inful'- ..
mahon, thlS of results lLsing {he S ..lme data is not often done,
However, some important advances might be adtieved ir more schol-
ars tried this type of i1nd it '\'GuId pmbably also encourage '
others lO keep more neld notes, Occasionallv, an entire re- :
induding has been replicated. Since we
cannot
quite
back in the cal1not be but can be
dH.ldl,ne nonethe!ess, Perhaps the most extensve replkation of
/\'n is t US(I ni tTh'nv n cY.1t,"r to t>'<Jruft
!ron, tr;n'"-"("'ripts ,lt intervlev,/& fr {lV(?
_ how "fkn tlw same judgme!1L H they do no! prodw,:e reliaol" mt'a-
5llnL'S, thel1 "le ca!! mke the cnding mIes more preciS<., ,md try agan, E,entually; se!:
al mIes CAn often be gt'neratoo so tila! he app!kation f same procedure by diJ'fere!.lt
roders will yield the same result. '

Major Compnents o Resean::h Desgn 27
','c sh:dy is thc ",-wiologic11 study ofMiddlett'wl1, Indiana,
by Roberl and Hden LV1KL Tht,r fir;;t "I\1iddlettl\-vn" 5tudv \\';1::-
. in 192Y ;md was in a book publislwd n' 1937.
fifty yearr; after the original stmlv, a long series ni books and
are published !hat n:plicatt' these NignJ studies bee
aL,1983a, 1953b ;md tlle dtatins therenJ, AH qualtative
net.--d not be this extensve, but this major resean:h projed
serve as an exemplar for 1Nhat is possiblc,
research should attempt io ache,-e as mudo replicability as pos-
SChOJ,'fS should alvvays record tbe exad methods, mIes, and pro-
used tn gather nformaton and draw inlerences so that an"
can do the <arlle thing and draw (one hopesJ the same
sion. Replicability also means that scholars \\'ho use unpub-
or private records shnuld endeavor lo ensure t11dt tuture seho!-
wiH have access to the material on similar terms; taking advantagc
access withour seekng access for otheTs predudes repli-
cation and calls into ques!ion the scicntific qualtv jf the \-York. Usuallv
our work wiH not be replicated, but we have the responsibilty lO ilt:t
H someone may .vish to do so. Even if tIte work is not replkated,
providing the rnaterals for such repiication will endble readers io un-
derstand and evaluate \vhat '.'ve have done,
Data
data pwblems by !lew ano better data s almos! "l-
an improvement on trying to use t1awed data in heHer
the former appmach is nut always possible. Soda 1
scientists nften find \vth el,lta and lttle ellanee
to ilr1ythng better; thus, han: to make the of what
the)'
lmprovng tite use o previousJy coIlected data 1S he lTlai.n topc
nmght in 011 statistkal methods and s, indee:::L tite chid contri-
bution oE nferentii;11 statstics lo th(' sodal sdences. The precepts on
this topic that are so dear n he stud! of inferental statistics al::>o
apply io quaIHative research. remainder (lf this book deals wth
these precepts more hlllv, f-'lere .ve merely <1 brief outline o
tne guiddines lor he use o' collected data,
First, whenever possible, \\'e shouid use data to gerwrate nterences
that a.re "ul1biased:' th"l corred cm average. To understand this
very idea fmm staistical imagine,. the S<1m
methodology (in quan!ita!ive or qualitatn" research) for analyzing
and drawing condusions {mm data across Inimv data sets. Because of
smaU errors in the data or in the application o the procedure, a single
appli'1tion of this methodology would probably never be exactly
\-\"111 be ('()rrt}ct tdkvll an aver-
a-'Pllcdhr'''''' ........,'",',, ji nt) single appH,,-atifHl ; correcL
not systematica!lv tilt Ih" oukom<..' in 0111: drectou
.-h'",'-,.,,, unmas0d ferentes depends. (){ Cllurse., br,tll un -he
nal cUectioT1 uf the dat; ami ib later U:-'10; and, ilS \\<e pointt::d out be-
fore, ir is always Oest t,l anticpate problems befoH: dala COHectiOl1 be-
HOVVt,ver, \.\'e mcntion thest' issues brietlv here becauS(' when
using the data, we need to be particularly to anal}'ze whether
sources oC bias were ovel"iooked during dala collectinn. One such
source., wnich C<ln 1e .. ti lo biased infenmres, is hat o( seledion bias:
choosing in d manneT that syslemabcally distorts the
fmm \vhich were drawn. Although M1 obvious exam-
pie is delberately cases whkh support our rheory, selec-
tkm bias calO occur in much more suotlt' wavs. Another diffcultv can
resdt from omtted variable bas, w!ch lo the exdusion of :some
control variable that mght infiuence a seeming causal comwctioll be-
tween Ul,lr explanatory variables and b,lt \vhkh \-ve ",'an to explan.
VVe (iSCU5S hese ilnd numerou5 other ptfalls in prool1dng
unbiased in chapters 2-6.
The second 15 based 0n the statisticaJ 01 "'effi-
. n efficient use of data im'olVi .. 'S maximizim: the information
llsed for or Ci'lusal infcrence. re-
nor onl)' using aJI our d:ta, but aIso usng aH relevant infor-
mllt10n in toe data lo nferences, For eXilmple, if tlw data artO'
into smaE \Ve should use l tila!:
Ine smal ler will llaye
of assodated with thenL bul if toe',' are,
in ob5erv"ble implications of the thenn,", fhe.- "vil con-
..,in sorne inforrnation whiell can be brought lo oear 'on inference
1.3 TJ1E\l1"S Of Tms VOl.l'ME
vVe conclude t!ti" o\'ervt'\v
thernes in
73J
In ths
whle, lrH:st 11,,\1:'
fnd Ji the
he {OUT importan!
h,ve discussed here
!1m! {ftil
to be worth ..
Theme:; uf Thb \tdunw
_h1b,,t1,H facts. In
!:ion, as ilS hmv data . " 11en:, we
want tu strcss thnt tlwnrv "nd em:nncal rcse.1n::h must be ttght!y COI1-
nected. har ddes real ViOr}; fOT us lus , . rOl' em-
prica! inyestgation;. no . . im'l-:tig,ilton OH,'
out ro gtllde lis chOICe oi qUt'stmns. TI1eoI} ,md data ,.O!l:,ttOD
are bth i1spects o the bv which \Ve s0\7k to dt"'-"1de
whether (l tneory should be provisonalJy viewi;'d inle or
as it 15 in both cases to the that charit(terzes all inlerencl:
We should ilsk nI' anv t!wo"v: \Vhat are ts observable lmphcatlOl1s:'
We zsk abuu! my Are he (,bsen'a-
nons rele\'i1J1t ro ihe imrlications of our and, sn, v,h,lt do the)'
liS ro nier aboul the (orrectness of the In ilnv
.. studv, the implcations of the ilnd Hit' obsen'iltion uf
fads need tU vdtn one another: socil sderKe condusons cannot
be rdiable ii are nol based on ano data in
one another ,me! ilnd
mplications of a theorv.
The :;cholar who seuTches fol"
is
ence: as t!Wcfl as ,,,,,;;,;;,IlI,'
Sde!KC seeks (o inGeilSe ti",
ro the inrormaon used in the
what ai first to be a
callsfdvariabJe nr il few variables, the
is ven:
une Of '" variables ,ve al50 han:'
he social sciences in dna even more so in !)i:lrtirular
areas. Ths may be becanse
do not know how to ncrease
it or becanse n"tllre
flot ro be in d cmwenicnt
ion ur for both (Jf these reasons. Ancas
are oiten titose in \Nhch
,1 h\,.lst nf
hule. in such ('"se<;, uur
with more
ol' il.ny-
Y3riab!es: \Vi;' use a lot in
"buuid bc to
There fe ,',Hl0tlS W<1,'" in VI/hic!, we edIl ,ur (lver
,1 n'St>1tch IV"V is t" ncn'hC Hit' lHitnbvr o
:Ir r",':,;::: (,t
1hose }\s w" have dcscribed "00ve, his 13Sk CDn invo!vc
The Scicm::e in Social Science
n) rnprovng the theorv so that it has me>fC observable mp1ic;1tions,
(2) impn)ving the data so more of these implications are indced oh-
SerVE.'t.1 and used to eva}uate the theory, and (3) impnwlng the use of
the datD 50 that more o these implicDtions are extracted frcon) exstng
dat,. None of these, nor the general concept nf maximizing lcverag;
are lhe same as the concept o parsimony, whch, as we explained in
section 1.2,2, is an assumption about the nature of the \vorkl rather
than a mle for designing research.
Maximizing leverage is so important and so genera] that we sfmngly
recummend {hal researc}crs routnely li::;f aH po:;sibfe obst'rmbe mpJicatitms
thtir tila! be obscrced in t/reir data Ot in oha data. It
may be possble to test some of tbese new implications in the original
data set-as long as the implcation does no! "come oul of" the data
but is a hypothesis ndependentJy suggested by ibe theory or a differ-
ent data But it is beHer stilJ to fUffi to other data. Thus \ve should
also consider implkations that might appe,u in (lther data-suco as
dilta about other units, data about other of the units under
study, dala from different of aggreganon, and data fttJm orher
time such as predictions ab::mt the near futul'\c>'-and evaluate
the hypo!hesis in those seUings. The more evidence 'Ne can find in
varied crmlexts, the more powerful our explanation become5, and the
more ccmfidence we and others should in aur
At first thought some researchers ma)' object to the idea oi collect-
. ob::rvable implications aH)' 01" at any level 01' agsreg
a
-
tWl1 difierent fmm that for whkh the theorv was For exam-
Ut'berson (985) to qualitative the statistkal idea
"n" .. ,.,,,",",,,h, dala to make in-
wam cross-lE'\'el
lha! \ve can use data lo make ncorred
about ndivduaIs: if we are interestetl in then
individual'> is generalhi a better if we can obtain
datn, t'''ovvever, ir the "ve seek tn make is more rilan a
C<l.st hypothesis, our theory may implicarons at
ieveis (lf analysis, and we wiH oHen be able to us .. : data fmm aH
these leveIs to provide some our theory Thus, even
ir Vv'te an: priman!y interested in an leve! ni \ve cm
". 15 process> of rCds{)ning freHn
ro mdJ\,'dUal-h:ve! proC('l'St5 15 Heitlwr nllr .1
b $.1n unhWH.Hl(lte of \vrd in tl-u:- nf
)bin;f
Y
t1 ;': n (:oflclUt'"d in hb artidt! ggregate
ft'l n:'\'t"--nn: :1bout individuab bit qUiHititatl'd: sil-ca; scicntb-ts and statisti ..
nn\\', Iha! sme niofmaton ,bout ndiFdu"ls dOt,$ "xi::;! ,'1
gate levds <JI ,lna])'S!::;. <Uld numy metlwds of unbased "ecoJogkal" inferenre have been
devellx"<l,
Themes of This Vo!ume . 31
, t'Jtt'fl g,n len::r"gt' alxml uur then.)' > bv ;lt t[Ie' dat.:l
ihese othe. levds,
example, if h'C develop a tneory k) explain n:nJlutions, ,ve
suui.'llulook for observabk" mplcatjons of that theory n01 only in over-
aU outcm:nlC'S but also such phenom12'na as the to in-depth
intenriews 01' revolutionaries, the reacions al' peoplc in smaH commu-
niHes in minor parts nf the country, and offidal stalemenl<; by partv
leaders. We should be wiHing lo take whatever inJormaton \ve can
acquire 50 long as it helps liS leam about Lile veracHy of OUT lf
'\.\" can k>st our theory by examining outcomes of revolutions, fine. But
most cases ver)' nformaton exsts al: tha! evel, perhaps just
ene or a few observatons, and ther values are rarely unamhguous or
measured without error, Manv different theores are consstent \v1th
the existence nf a revolution. nly by deeper in the prcsent
case, or bringing in reievant informaton exishng in olher ca5{;."5, i5 it
possibie ro distinguish among prcviousiy indistinguishable IhJories.
Tile only issue in USillg inJrmaton ai other level" and from other
50UTces to study a theory designed at nn aggregate level is whether
these new observations contain SO!Ht' nJormation that is relevant to
evaluating implkations of OUT theory. If these new observations help
to test 01.11" thenrv, lhev should be llsea even if thev are not the implca-
Hons o Por example, we may m;t care at aH about tlle
views of revolutionaries, but if their ansvvers to our questions are coo-
sistent with our theon' of re'volutlons, then the theorv itself wil! be
more lkely to be and he colIection of information
\,,'ill have !:leen usefuL 1n ill1 observation at the must
(X'Ctunmce ni a predicted revolution. foI'
one observed implkation of the theory, and be-
cause o the small amoun! 01' infom,ation in t, t should not L1 pn'\i-
leged OVer other observable implicatons. \Ve need io colled infoffilil-
non 011 as many observable mplications of our theory as possible.
133 Rc;orting
AH knowiedge and aH quantitative and in Cjualitative
research-is uncerain. measurement is as
s quanttativ, but the sources of error may ditEer. TIle qualitative in-
terviewer conductng a long, in,depth intervie,y with a respondent
, ... hose background he has studied is less l1kely to mismeasure the sub-
ject's real poltical than IS d researcher conduding a
strudured interview with a randomly selecled respondent abut
'whom he knows nothing. (Although the opposte is aIso possible if,
for instan ce, he reties too heavily on an infurrnant who is not trust-
the ;:Uiq'V researdwr :; h':,;s likelv te generalize
(rom the CSeO' intniewed to th\! broader
poptt<!lticm than 15 the in-depth ff'Se.1fCneL Neither s hnmune from
uncertainties of measurement or tht, undt'dyng pmbabilistc na-
tr' o the ,,,mId.
AH good social sdentists-whetner in [he quanlitatin: or qu"litative
traditons-report estimates o rhe uncertaintv of their inferences. Per-
haps the single mos! serous problem v\'th researrn in po ..
Jitic .. l ::.cience s the pervdsive failure lo prm-ide reasonab!e estimates
(lE rhe ni the investigator's inferences (see King 1990), We
in almos! anv situation, no matter how lim-
ited the follml/ing the in this but \VE' shouid
avod forging s,veeping condusions from weak data. The roinr s not
!hilt relab!e inferCl1ces are in but
racher that "ve snould report a reasonab!e 0stimate oI he
we have in ead of our lnferences. Neustadt and rv1ay
ue"u,,,, vdth <11'(',15 in which quantitative estimares
are propose a uscfuJ method nf encouraging pt,lkymakers
C\<vho are en faeed witl1 the necessitv oi condusions about
\-v'ha! to follow out of inadequate data) tn the uncertaintv
ol their mndusions, They ask "How mucn of \Tour o,,\'n monev ,vHuid
you wager on ie' Ths sense 's vve also as!.::. ":At what
odds?"
1.3.1 El]:!' a St}(.'iaf
md R{YIJ Hypothest's
The of caus.cl inferencl's means that good scentists
them. VVhen (lId l\ ca uses B, someonc v'lllo
social sden tist" asks that cormection is a true
causalone. H s eas}' to ask stlch qUe5ticms abut the research of others,
but it 1S more important tu ask them about our own research. There are
man}' reasons lvhy \Ve might be skeptical nf a causal ilcconnL plausible
though it sound at f-si We rcad in the nevvspaper that rhe
Japanese e,]1 !ess red meat and have fewer heart attack.s than Ameri
can$. This observation alome is in additiol1, rl.E' explana-
han-too much leads to the high Tate of heart dsease in the
United 5tatt's-is plausible. The skepticaJ social scientist asks about the
accuraey of the data (how do we know ;lbour e:ting hilbits? ,vha! sam-
pIe vvas used? are heart attacKs dassf",'<.l similarlv in ]apan and the
Unik'll States so that we Jre comparing smihlr As-sum-
ing that the data are accurah.', lvnat else mlght explain the effeds: Are
tl'\eI'e orher variables (other dietary difference5, genetic features
f
H'-
Themcs of Ths \'lume ;;3
n-
ti\' dnd <.'ftecL' rt is hrd tu imagine he,," not
a hear! aUt1ck might ,-aUSe c,ne lO ea! less red mea! bui t s
. Pernar::; penpk lose th",ir appctile for h<lmburgers and
in lf this were tt-\\: case, tbose W!lO did not 11,1\,<: " heart atiack
whatever reason) would live longer "nd ei1t less m'at This fad
roduce the "ame rdatiol1ship that lcd the H..'St7drchers lo con-
Hwi mear \Vas the cuiprt in heart
is not our purpose lO caH su eh studics nin queston-
we wish mere!)' to illuslrate huw scientists appnhlch tw
oi causal with ilnd il cunL'ern (or alternati\:c
that n1dV h,wt' been overlooked. inferenct'
. bero
mes
a ' condusion becomes the occ2tsion ror
further research tu refine ilnd h:st iL Through succeSSIV\"
!o come i:lnd doser !o accurate nfercnce.
Descriptive Inference
SOCIAL SCIENCE RESEARCH, whether quantitanve 01' qualitative, in-
volves thc dual gods of descrbing ilnd explaining. Sorne scholars set
out to describe the \''lorld; others to t'xplain. Eaeh LS essental. \Ve C3n-
no! construd mcaningful causal explanations without
dcscription, in um, losz's mos! of it:;, inlerest unless linked to
some causal rdationsnips. Description often comes frst; it is hard tu
dcvelop explanations before \ve know something abour the \",orld <lnd
what needs ro be explained cm the of \""hat But the
relanonship behveen descripticm and explanaton is interactive. Sme-
times nur explanatons lead us lo ook for dscriptions of different
parts 01' the \vorld; conve1'sdy, our descriptions milV lcad to nev"
causal explanations. .
and expianation bt)th depend upon rules of sdentific
In this we toens 00 description .md descripti\'c in-
1e1'ence. Description ls far frorn mt."C'nanical 01' unpmblemat:k since ir
involves selection from the nfinte number of fads tnat could be
corded. There dre several fundamental of scientifk description.
One s that it invol\'!;:'S part of the descriptve task is to infer
informatlo!1 about unobserved facts [rom tnt' facts ,';'e have
Another involves dislingl.lishing thar whkh is
arle abtmt he observed and th"t whic1 is n(Jnsvstcmltic.
As ShOllId be we ,vith who denl GTi1te "mere'"
description. Even f explal1ntion-connectng causes
the ultimate goa.l, descrjption a central role in an explanation, and
it J.s fundamentaHy important in and nI" tseH. It s not description ver-
sus explanatian tha! distinguishes scientfk research from other re-
search; ii lS whether systematk is conducted accanlinn to
vaIid pmcedures. Inference, 'ivnether descriptve or causal, qmmtita-
tlve or qualitative .. ls the nltimnte uf al! social sdence.
tematicaHy coHecting fads ls a very importm1t endca\,,'or witho1.l1
t-vhkh sdence ,v(mld no: be possible but whkh deleS not by HseU con-
stitute Good ardtlval work or wdl-donc summaries of hston-
cal mil)! mak' gCh.Xl but neither are suffkient
to constitute sodal science.
In this cnapte1', we distinguish description--the coUection oi faets-
fmm descripthre inference:. In section 2.1 we discuss the relationship
_ -Genera! Knowledge ,md Particular Facts . 35
nl scnolar::;hip: discovering
knowledge and particular fads. \Ve are then
to explai.n in
d
morl' detail tite of inference n s"ction 2.2..
Our approach in thz' i'vmainder of the bOOK is ro present ideas both
vcrbaHv and thnmgh \'ery simple algebnlk mo{iel::; 01' rescarch. In
. we nmsder he nature o these models. \Ve then discuss
. <l>I'",UM"
irR"'ldels for data coHection, for summarizing histoncal dc,taiL aud lor
descripti\'e inferencf' in secnons 2.4, and 2.6, respedivdy.
we provide StlmE' spt'Cific critera for iudging descriptive infen:nccs in

!,/"",,"'",\,!H .:..../
2.1 GENERAL K!\lOWUmGE AND I'A1U1U.iLAR FACTS
TI'le world tha! sodal scientists study s madi'! up of particulars: indi-
vidual v<Jters, partlcular government specifk tri bes,
groups, shltes, and natons. Good sodaJ science attempts t
SO beyond titese particulars to more general knO\vledge. Genem!za-
tion, h\vever, does no! elminate the impnrtance oi the particular. in
mct, the very purrose o mvng fmm the partcuhlr t the is
ro imprve our understandJlg of both. The specific entlties of he
soda ( \.vorld-"()f, more precisey., spedfic facts about these entties-
provide the basis nI! 'which genernlizations must rest. In addition., ,ve
almost leam more aoout a spt:cific case: by studying more gen-
eral condusions. If we wsh to kmn\' \ .... hv the foreign minister of Brazil
it ,vin help to leam ministers in Brazit
fnreign ministers in oiher have resgned, or \vhy peopll'
in general resign frm11. or I;'ven nongovernmental joos.
E:1.ch o these v,m lIS understand different of facts
ano principies of human behavior, but they are very important even ir
our one and only goal is to understand the most n:.'\.--:cnt BrazBiilD
f<treign minister resigned. For eXitmple, by' st"lJdying other ministers,
we rnight learn that al! the ministers in Brazi.l resigned to protest the
ac:nons of the presidenL something ,ve m.ight nt have 1'('aiized ex-
amining onl}' tbe actions of the foreign minister.
Sorne scKal science resenren tries lo say sOJ1.1.ething about a dass of
events or tmlts vdthout snying anything in particular about a specific
event 01' unit. Studies nf using mass L'Xplain
the oecisiol1s of pi..'{)ple in general, no! the vote of an]' particular
indh-'iduaL Stlldies of finance explain the of
monev on electoral outeomes across di! districts. 1'",lost
such ;tudies ,vould nui mcnHon the Seventh Congressional Dstrict in
Pennsylvana (}[ any other district except perhaps, in passing 01' lS
exceptions to a general rule. These studies foilow the injunction of
36 InferenCt.'
and Ir'une zlYH]i: F)T'()!>cr
thOllgh tites" "ludie" !:1ilV nor sL'ck hl UIhJ,:rst;nJ any particular
trIce they should not 19nrt.'--;s sometimcs s unfor!unatelv done
rhis tradition-:-,the requirement th!t rhe facts abou he
tncts that go mo general analyss mus bt' ilccuratt',
Other research tries t:) tell tl" SilIllethng abou! a partcular
stance. H focuses on the Frenen Revolution or sorne other
event and attemprs to pmvdc an expbn"tion of hu\\' or vvhv
event came aDout. Resedrch in this tradition \v(mld be .
cerrain!y unnteresting ro most of fhe usu<1l re"ders of such ..
without names. A politcal scientist wrile dfcctvelv
uf ilcmss the set of -
M specific districts or candd,ltes bu! m"gine
Rnbert Caro s dSCtbSll1!1 09B3l i he 1948 Senare fiKC in Texas w1th-
out , Johnsnn and Cok" 5tt'venson.! Particular events such as
th,,> fn:nch Revolufion or the Democratic Senate primnrv in Texas in
1114S m,,}! inaced be of intrinsk interest: he}' our and
lt t,hey' .vere prenmditions for event'i (such as the
OJ1!C Wa:!"S or \ve ma}' n0<.."(1 to know about them
t:} unders!imd those {"rer knnw!edge about
hon, ,rebeIlion, or 1NJr in \'\'111 pmvide invaiuable
!Or any more rocused study of he cau5l.S o' the French Revolution
111 particular.
We 'will consider these issues
daimed alternath'(:' tn m!en:l1CI! bectll m 2. l. 1, Ihe
of uniquencss lid complcxly o the oi
and thc area of case grudes (section 2J .3).
In the .human some historienl and anthmpological rcseareh-
ers darm to seek knovv!edge through what thev caH "in-
terpretatinn." nI' historcal de-
taiL They also seek !o place the 0\;ents they descrb,e in an infelli"ible
within vvhkh the ()f actims bt:comes b .As
FerE'}ohn COldstein and Keohmwl has ;.vnflen, "We want
l Nnr lB t\', dSD1b5 Caro in anothuf busincss: d
frnrn that {t[ tnf: SiK'ld scit"ntJ.st %york S0111e (ji" the Srne
\Ydtltt.1: \Vh;H h\ads h faHuft' in dection
. iVbat tile rno0ey <.UhJ {n,;;nce in t:ll'-([nfdl \Vh.Jt
mntwat0; ",:mpaign cO!1rrbu!o!'3' TI1<' disnJsson OCU5eS (In ,1 parliculi\f (anlida(y in J
alstncL bu! the maHer .:tl1d the ovedar with
p{)htu::al
Ct'flt'ral Knowlt;:'{ige anJ Particular Facb
thcpnes lo o' eh'nts
<'in ,KCOU'\t o the n2:"i,lES ftlf nr ot sudal ih.:t10n,
to know no! onlv ,""llar ca1.lsed the agent to pedorro some act
the fnr taking the adion," Geertz OQ73:17)
",,"rites that "it is not in nur nterest tn blead, human behaYor
:m,1perties tha! us beiore begin e>.amne it." ,
\\'ho "mterpret"tlOfI seek to IlIummate the m-
aspects human Dehavior by Vcrstdwl1 ("em-
understanding the meaning (lf actons and interactions fmm
o'\'\'n points o vi!.',>v" ! 1975:81 n, interpretvsts
o explan lhe for intentional adnn in .< ro the
se! of in vl/hien if g embedded. llwy Jbo
<-<Thf' m()st obvious are ({}'"'
scope: al1 interprelative "ccoua! provde ma'\ll11i1!
nI intelligibli!'y tn a set of sodal pradces, and an interp:c-
accouni of a set of should b;, VV! h
practct's or tradHions f he i , .
PerItaps the single most important operatlCmal recommenLatlOn 01'
s that should leam a dea! abnut a
prior to fonnu!ing Fr only \Nilh a deep
immersion and understanding of a subject can a
the hypotheses. Far
of workng-dass black

!hat
that
38 . Dscriptive Interence
lt is ..-:w(a! to ,1 culture' deeplv bdurc frmulahni'; hvpoth-
or a research project ro fnd ;m \Ve
only \'lsh tu Bdd that evaJuatillg the vr<Kity of dams based on meth-
ods such observation can bt' accomplished through
lile ot SClt'lltllC vvhich ,ve describe, Finding the right
.Jns\vers lo th<: \vrong questions b a futile lnterprewtion
on . is often a rkh souree oi nsightful hypotheses. For
stance, RIChard FeMo':,; dose obserl/atlons of Congress (Fenno 1978),
madt: he caHs "soaking and poking/' llave made major
n)ntnbuhons ti} the ()f that nstitution, pareularly by helping
to bener ior "Soaking and pokng,"
Putnam in a study oi lti'lhan reglOns (1993:12), he
to marina te in the mmti,}} of an institution--to experience irs
customs and its successes and its as those who Uve
every day do. This mmersion sharpens OUT intuitions and pnwidti.'S
mnumerabJe about no\\' thc institution tits and how it
adapts to ts envimnment." [\ny dcfinition oi science thal dnes no!
dude ro(lm for ideas tne gem:ratio of hypotheses 15 as fooI-
ish as an account that does not eare aboul
truth.
Yet on<:<.: . have been tormulated, demonstrating their cor-
1"ectness ('Yllh m estima te of uncertaintv) valid scientific in-
ferenc(.'S. The procedures for inference folowed bV ' sodal
"","."L"'''''''', must incorporare the standards as those'
followed other i:md researrhers. TIlat
that insightfui
tlOn or other we aIso nss!
that sdence b essential foI' accura!e inteI'pretation. H we could under-
stand human behnvior on1y through we wouid never be
abje lo our hypotheses or c,'i.dence for them
our experience. Ou1" condusons \vould never go bevond the
status of untest:ed and our interpretations remain
personal rather than sCentific.
. ?ne ,of best and most famous in t.he interpretative
dlhon 115 Chfford Geertz'15 analysis of GHbert discussion oE the
difference a h.vitch and a wink Geertz (1973:6) '\Tires
Consider ... wo boys rapidly contractng the of their C'ves. In
ooe, ths s ;m invohmtary twitch; n the other, a conspiratorial to a
frwnd. 11)(> hyo movements are, as movements, dennes.!; from an r.am-a-
mera, "plwnomenalstic" nbsef"ation o thCIl1 ilIon.:, one could flor tdl
which Wib twiren ami whkh \VilS ,,,,in k, Of indeed \vhether both or either
was twitch or 'wnk. Yet the difference, hmvever tmphotogmphable, jx ....
General Knm'\'ledge ,md Partmlar Facts . 39
, a t\'\.Y1tch ::nd ..1: \\'1'nk is .;1$ ) h{:tvc
, t,he frst llf he knws. '1"11,; \dnkt'f i;,. ,me:!
communicating in a and "recial way: (l '; delibt'r;ltely, C.l to
:;;omevnc in Filrticu!;.L (3) lO mp;rt d parlicul,lf mt.'ssage. (4) ilCC()!\:lng lO ,1
establisht.'d mde. and (5) witllout 01 Ilw rest "f thc Gnu'
As Ryle points out, dw winker has done 1\\'0 things, contracted ni!'
eyelids and \vinked, .",hilc the t;vitcher h"s done unl)' one, cuntracted. his
)10m un purpse v,hen there exisls a public
code in ''''hich dong s(' counts as d conspimtnri;J signa] C' wlnking.
an importmt conceptual pont. Without th0 con'
" given meaning by a uf communication, thc
study of "t'yelid contracting by human be-
\yould be rneanngles5 for studens of social relalions. In Ibis ex-
limpIe, che vvhch fmm months of "soaking and
fUg" imd cultural 15 to the propeT qucstion oi
;whether eyelid conudetion even could be "hvitches" or ".vin"s." The
m.agnifkent importance of interpretdtion by this is
it provides new of looking al tite world-ne\v concepts lo
considered and hypothe:St.'S to be evall..lated. Without deep immer-
\",12 might not even think of the right theories to
n""""",nt example, if we did not think o the dfference
n"'lt"i'I""n twich(,"S and winks,. everything \vemld be losL l.f interpreta-
tlCI'fl--Clr an:vthing t:!.se--helps us arrive ai ne,,\! coneepts or hypothe-
then it is ando interpretabon, and similar
hil\'e been proven
made" rdevane theoretkal such as that behveen
the researcher thcn need15 to cuallmfe he hypothe-
It 1S in su,:h evalua!ion !.hat the logk o
. sc::lentific inference is That lhe best >';ay uf determin-
meaning of eyeld contrdctions is through tite meth-
descrihed in this book If distinguishing a tw1tch from wink were
pivota.!, we could desih"1\ a research ro do so. lf, for
mstance, \Ve belJeve that particular contractions are winks im-
bued W1th meaning, then other similar nstances mus also
be since a soph::lsticated device such as his (a
once developed, 15 used Given this
record instance n \\'hieh this actor":; eyelid
contmcts. observE' vvhether tht: (,ther actor is !ooking al the right
time, and ,,\'hether he rt:.'Sponds. V ...re cC'uld E'ven a series o.t ex-
periments to if individuals in th1S culture are accustomed ro com-
municating in this fashion, Understanding the culture, carefully de-
40 O('Scriptive Inference
the t'\'nL i1nd ',winf.' il
tions ,viIl aH help us ask the rjght and C1.'e11 gve us
tional confidence in out condusions. Bu! onlv \vith tlw methods of
entifc .nference H'll we be able t' the h\:pthesis and
hthether 1! is corred.
Geertz's wink is best expressed as d causal
hvhich we define precsely in seLi:ton 3.1); the hypothetical
fed o the wink on the other polHical ador is tite other aetor's
he eyeHd contradion minus his response ii tlwrc were no
ment {a lid no other lE the eyelid contraction were a \vink,
Cimsi11 effect would be if ir \Vere a twitch, the causal
fect h'ould bt' Z('TO. if v/e decided t' estmate this c)Usa!
thus nnd out whelher it \>vas i'l vvinK nf a tvl/irchJ, al! he problems
inicrenc\:" disellssed ilt in the rest uf botik would nE'ed in
underslood it \ve \vefe to arrive at tite best nft'rL'nce with rf><,n,'f't
[he lnterpretatiofl ni lhe oDserved Qeha,'ior.
lf \\'ha w\:" interpret 'lS winks wen, llvoluntarv
(iUI" to derve causar abt,ut evelid contractn
the bnsis or i'l thL"Ory nf voluntary sodal vould Dt'
tinely ,ve would noi be ab!e to ,."... ,,.,,,,.,,,
know Ll
DL'Signing reseiln-::h lo winKs and
tn be , part of most poltkilI ;;dene!:' ,.p,,,,,,,,,,.,-,,
methodologcaI lssue arises in much oi the
ka! sdentists work We are often
polie)' d!:'dsion makers send messages ro eacn otiler,
il 3 3 pont <l statement
dt appeallng to a dt.lIncstic audience? of cultura! norm$,
nmycntions in and of he hislory'
particular adors, as wen as dose obsen'ation 01' or
the comrrnmicatioTl, \\fill aH help us make such 3n Or
consider the puzzle in resarch: Voters in
Unitet1 seem to be by fll't tunling out O(
polI:;, But lvhat the lOw turnout mean? Does it reHect alienation
\vlth the poiitical system: A
ing v\ith he costs being
dates or recent campaigns? Could
rhe minimum age of 01' il
For dw;;:k<:' tha! \Ve e[l!lId dl1
Cjtit'fL'nt tl1t;l'!'V in which M\ c\'dtd <ou!l'action Vi,l!' nnt ,1 wink but stm had i\ Glusal
eifa:! Oil nther a<:ton>. rol' e'clmpe, the twtch <,ould bave mi:;mtl:rpreled. lf ,ve
WHe a!S\) mlefL>sted in wrt.'ther he wih he eydid mtractfm inleuJcd tu wink,
We' would to look for othH conseqU..'Ilces of ths 5iHne theurj',
Generdl Knowkdge and P,uticular Facts . .+1
g tu h) the The of J nt..)t .
.,\".nh or.l di}11nmatc messge. can kan man\' ihings, Th, "o-
n:S6,feher shuuld alv:.lvS work hard to as!.: thc ight que,;
d then careful!v design scientiric research In fnd llUi what lhe
. s ad did in fad mean.
\\'ould also like lo brdly <,(kiress the extreme daims (l ,1 few
nf n!erpretation who argue ha! lhe o some n'setm-:-h
to be feclings <ind meanings with no observi1blc conscquences,
liaren)' a fair characteriza!ion of a11 but a small rnnority o re-
in ,this traditinn, out the d.1ims Me mad\:' suffkt'l1tJy 'r'-
hat they seem worth addressing cAplk:itly. Likt' the over-enthu-
daims oi CMI} Pisitivsrs, \vho tuok the untenahlc har
,-"'rh-,pnt,,, had no in scientific
rescilfch. For exam-
on that which s (w"rt ,md n
observable' aets is l\ilh'l" in Ih,,' le3sL The tn
Ihen, is lu tmdl'r-
11M the actor'" et has for him.
u<!,,,tl',,,,, mal' be corree! that 50cld! sdentists who f(){,'us on nvert
a lOI, bu hOVl are we to kncnv if we
see? for exmnple, if two theorles uf self-conception have identi-'
nMnifestations, then 110 observer will have sufiicient in'
obser\'atiot1,
quantHica-
,md methods are
i)adequate to the task dstinguishing h'\'o theores whoUl diHering
omsl'..'tluences, On the otIter if the hvo theores l1a\'c
some manifestations that diJfer, then rhe rneth()(ls "ve de-
scribe in this booK pmvde ro dstinguish behveen them,
In practicE', ethnogmphers (md aH other sodal scientists) do
Jook for observable behavior in order ti) among their 1heo-
nes. mal' lmmeTse themselves in the bu they al! rely on
various forms of o[,sert'afl1ll. Any further "understanding" of the cul-
tural context comes din'Ctly fmm thc'Se or other comparable observa-
tion!'>. re]evant observations s not On the con-
the appropriate observatons 15 most diJfi-
cult part of a resemeh pmjed
t
especiaHy (and necessarily) for those
ureas of inquiry traditionally dmninated. by qua li t., ti ve research.
Dt.'$Criptive lnference
2,11
Some qu,,!tatvelv nrit::,nted resea.n:'hers 'Nould the positon
general knowledge 15 either necessdry or useful (perhap5 eyen
bId as toe IXl.sb fUf ,1 en:nt Ther positi
tha! the events or unts stuJyare "unique," in Ont' 5<:.'ns<:\ !hey
right. There was onIr orw French Revolution and there s onI)'
Thailand, And no on0 who has reaJ the biographkal aecounts oI'
li\/eJ thmugh the 19605 can Joubt tbe fact tbat there was on!v
Lyndon 8. Johnsnn. But hey go furiher Explanatinn, accordi
their position, is limited to har unique event or unit: not \vhy
tions happen, but vd,}' the French Revolution happened; not ,vhy
ITlocratization sonw!imes seems ro bu! iArhy it llgs in Thailand;
candidales win, but LBJ wun in 1948 or 1964. ""'c",,,.,
"Uniqueness," however, ls a mis!eadng term, The Frenen
tion and Thailand and LB! are, indeed, unique, AH phenomena,
events, are [11 sonw sense The Frencn Revolution
was; but 50 was he eiecton in the Seventh Dist,kt
in 1988 and so "vas Lile voUng decision of every one
the mi1lions of voters \\'no voied in the presidential dectlon tha!
Vevved holisticaI1y, every aspect oisocia!
and in some to
c\'ents. inherent s part
ir drl.('s not dstingush sltuations amenable to sdentHic
about whkh dre no! ,",U:;;'iVI't:,
shmved in theories of dinosaur extincton in chapter Leven.
studied paying attention to
implkations (if theores to ao::ount io! them.
The real that tbe ssue 01' uniqueness mises is tbe problem
o complexity. The point 1S not \vnether events are inl1erently unique.
but \vhether tne features (JI' social reality tliar ,ve want to unde:r:-
stand can be abstrac.ied fmm a mass 01' f,lds, One of the first and mosl:
difficult tasks oi in the social sciences is this ad of simplificll-
tfm. H is il tasI<. that makes as vulnerable t) tIle cntidsm of oversimpli-
and 01 omtting sgnificant oE tite situation. Neverthe-
such simplkatlon i5 inevitable al! researclleTs. Simplification
has been ao oi every knovvn \york--qwmtita-
tive and qualitatlve, zmthmpological and economic in the sodal sd-
}nd in the natural and physical scences-and \v111 pmbably aI-
General Knwledge amI I\rticu!ar Fa"is ' 43
simplfy.
lm:leed, rhe
]
nd tfilf in tttc ,
bcttiX'cn tnckc<;:! ,md tite ab,;: mct .
mi'! )J dt'SCT'iption, no matter how tl'uck. dnd no ex-
1.. t f'> .... ors jnlo tt n,'l1:W:;
nO matter uO\\' many ('xp ana ory oC. '"
capturng the full "bJoomng re, lit y oi' lh
i
:
is no choe'e bu! to 5ystematlt.
sr.,"}) to useful As "n eronomc histonan has
t ' '} t11!' "xtreme o
MISll:' on " .... ".
p --enn' 1S
the H:'TV .. .. "
tu ihc lmiesssness (lf balladcl'rs"
p(lssible, smplify their _
attain <111 of toe ridmess or.. and euI-
sdentsts mav use cmly a few o the of stJmc
" '1' . l'nf"rt'l""'"'' N rich, unstrw.::tured
even,s 1 ,- .",;0, , '" '- ....,
thc historie,.! and cultural context nf the phenomena
v;ant tu deal in a simplified and sdentife' is usu-
avoiding tha! afe simply v,:nmg, Fe1-v
trust the of a about revolu-
elections if that knew little and
Reyolution o, the
belie\'e th"t.< where
(u",,,p'r;; and
;:l\'!,eSE;E'S of evenrs as \veH as "b(lUt
ant to be nnd
her mav ,/arv trorn
bu! both are likeiy in' be
bdng opposed to
tJp ti
Much 01' what politkal scientists do is describe
events svstematcallv. care ,1bout lhe
ln-
filc
Union, thl? n.'actions "uf the in wuntries to the ,
ized \var to drive lraq from Kuwit, the resu!ts (lE the
.' l. l' del tc A1'd h<>v relv on nohhca] so-
gresslonal eiectlons In tr1e ,r'lhe ,J a ",,y, ,rt, <") " . 1:"
cntis!s fUf th,) tened d more d\Vart'ne$$
the reiatiol1ship bd\\'een these ana other reh?,'dI ('venb--en
nuy i.md is found in journalistic accounts. Our
scriptions oi events should be as pn>cise and systemdtc as
This means thi ,vhen ,-\Ce are "ble to find valid quantitalivc 111
(Ji wndt \Ve ,vant to ,ve should use them: Whilt proportio
Soviet nticize gnvernment polky? 'v\lhat do pubik
km in Jordan and Egypt n:veilI about Jordanian and
attHudes tO\.vara the Gulf \var? Whi pern>r1tilge of con;ressonaI
cumben!s wen: reeleded: "
H precisol1
f
ir cines noi neCf'SS,ll
accuracy, SlKt' qllimtitative indixes that do nol
dosely lO the ('oncepts or e\.enb tha[ '\-ve pur:x)rt [O measun: can
lo serous nk>i1SUrerrH'l1! error and probJems Enr ctlusnl inference
section 5, l.!. Similarly, tlwre dre rnore and less precise ways ro des
evenls tha! cannot be quantifietl D::;;:iplned qualitativc rL"St'il
try lo ana!yze consttutiOl1S and L.1\vS ra!her han n";'n""
porl \vnat observers abmJt them. In doing case studes ot
ment researdlers dsk heir nformants trenchanL well-spec
questiuns to \vhkh ans\vers 'Nm be relntvelr unambiguous imd
ioHmv 1.1 p on oH-hano n?marks made i.1.t1 '
relevan! Case siudll.:,'S are for
scription, "nd are, fundamental to stxit! sdence. H 1S
less !o Stc'!:k lo ,vhat we have not described ",,,ith a
of precision,
o comp!ex events s no
(Ir nt-ernational
zmd cmmter others' 1s
ne! expectatitms may playas importi1nt a par
in fnr sta te benirvor, 1\ pllrpnrted
o world th8f ssumes he absenn; interaction
fp,Ktions \viH be rmJd. ;ss usdul lhxm a carefu!
OH evcnts that we have reason to believe
interconn\!cted,
f otten (lverlooked "d.\'antages (lf the in-depth
is that the dl'\'elopmen!: 01' gnnd is (:"W'
lo z:ood dt:S(Tptitm r,1tht'f than \<;ith l. Fr;,m-
e s!';dv Jmund 'an "",plan"pfV tucstkm mdV kd lO more
desGiptiun, en,!! if the studv 15 ultinhlhA:'
in its attempt lo provde e\'Cn a single yalid infereI1Ct'.
ve case shdies \ve yeld \'a!d cilusal nfer-
in thc Test o this bnok are used,
the\' oftcn do not meet the stan-
!\!mplasue here so muer. as it 1S a w,y\, of
non in descriptivc case in su eh a vvny that t
un l,:.-':;t' is \'(.1sL Sorn{'> <.Jf tht.' \vorks
E.:kstt'in (1975), L.iph"r! (19711. ,md Colli<>r (lqqn.
46 Descriptive lnference
bt: used [,Ir desGptin: ('j "::cns,ll in{en"l1('l', i\luch \'luablc ,,'
"bout doing con,}';lrlVe t'd"'e
often igmxl!d,
2.2 lNFElU,:'\;CE: TllE SCIENTfFlC Pt:RPOSf 01'
DATA COLLECTION
lnference is the process oE using the facts \Ne knmv to leam aoou!
we do not kmnv. The facts \Ve do no! knmv are the oE
research questions, theories, anel hypotheses, The fact::; .ve do
form our (Iuant:tative or data or ol-;sef\'iltions,
In
partkular vve must sornehmv avoid being oven'\'
the mas5ivc Di and actual obs0rvanons
rhe world, solution to that problem lk>s ......,,,'-',""'1'\1'
the search knmvledge. That the bt.>st sdennfic way
organize fact" is as implkations of some (Ji'
sis, SLientific simpliHcation involve5 tIte productive choice of a
(or hypothesis! io ('\-'aJuare: the theory tnen guides U5 lo the
of those [ets that are implieation:; oI tneo!): Organizing facts in
o observabie implications ol a tl1ro!)' pn1duef."S
tant and beneJidal results in designing and conducting
\-vith tnis criterion for the sek-ction of fae!s, we can qukkly
that more oDservations of the implkations of a theon' \viH onlv
evaltmting tne in question, Since more o(tls
cannot hure such data are neVi"r and the pnlCt'SS o
Viii;' need not have a
nor musr our theof\r remain throughout
ad. As \'\'ith and tlle egg, some is
befare dat" colh:dicm and some data are ){ftore any
l;"'vtruv,l/c O}1 research teH us tbat we use OUT data lo 'test
E:mm the dala may be as important " goal
evaluating prior theories and h:'Ptneses, Such iearning involves
organizing our data into observabIe implications of the ne\
This is (omrnon early in many researdl
somepreliminary data have t1en collectt't1.; after the
data coUa'tion Hum eontllmes in ordef to evaluate
new throry; VVe should ah','itvs trv to continue to colled
after the reorganizatinn in lo 'test thc ne .. \' ill1d thus a
using he same dala to evaluare the that ve usted lo develop
. For Cnon,bs O'it4; emO!1stratt;,'C! that vrhUlily ever)' tlSt'ful
1 nferen ce 47
en1pha:-;.is un dS obSCr\\lhlc \.)1
S makes the cummon gmtmd betweeIl the quantiLltive and
styles ni research much clearer In f"cl, once we get past
G;ses or units or records in the usual ,'en' narrO\v or even
sense, W fealize that most studi2S poientii\lly pro-
very lurge number ot observable irnplcations fO the theries
- ated, yet many ni these observations ma)' be overlooked by
Organizing tht: data into a list of [he obsen-
ilLctlJlV"" oE a thenry thus helps rcyeal tnt: e&.senta! sdentifk
of r\uch qualHatin' rcsearch, In a sense, we are asking the
\'Irnn 1:;; studving a evenr-il
aSK: "H nw
me out the wa}' it die!, ",hat else
world?" These addtional obs,,'fvahle
dedsions., bu! thev migh! also be found in other
dedsion being shKled: fnr when ii \Yas hmv t
E:, ht.)w it \-vas jus!ified, Tne cnKj,,! maxim ro gude b'Jth Hit'-
and data is: search fOf mOl"{' observable impJka-
ir L'i productive to
be observed,
tems ipr vvhch data
eme additonal datum v,-m
to evaluate a then to
time, and dort constraintsJ it i5 wor!h
or otller observ"ton might be
'abie ot this {or sorne pther rdc\'aniJ
be obvlous thar it v'l'ill not
par of the simplification
data 1nlo observable implications of a
the data. VVe can think ilhout the r,,1;\-' materia! of real-
into "dasses" thar afe made up (jf "unts" or
v,rhkh are" in tum, made uF' of "attrbules" or al'
N The da.ss might be "\'pters"; the units might be l sample
vorers" in s('vera distrcts: and he attrbutes or
Desoiptivc InferelKC
runnel tt?sts, t wuuId be lrrde\<anL sine'e L'Ydl
dust can aplane to V"lcgh more and tbus use more e
'uel, modds of this sort H;: importanl to he aidime industry ilnd
bet'n huilt (and swed millions o doH.lrsL
AH models range beh\<een restrictiy{: ami unrt'strctive '\ '
stridive models are dearer, more par::iimonious! ilnd more
but they are also h:ss redistc lunless the '.vorlJ is
ous), Modeb whch Me unrestrictive are detailed, contextua!,
rnore realistk, bu! lhey dre also leso, cIcar and harder tl1 estmate
precision (sce King19B9: section ') VVhere on his continu
dwose to G.mstruc! a mt.xJd ,,1',"""'1,n,'!C
be anJ on the
C01.mt will
actual judidal
tiOl1, the
induded,
Vv'hHe
be an abstl'adion or
Sillee undersl:auding
book s as nmch ivhat is lef!
researchers often use verbal
in our discusson
verbal models, JUSI as \vith models of
studies of the French Revoiutton, our
research not be confllsed \vith
are dear statements of
otherwise.
In addition, we often
us to dscover de,ls that we vvould no! have
vVe assume rhar readers have had no
braic
in tl1ese "nd
Just because quantitaJive are probably more familiar
our dnes not mean that are any htter al
the of scientific inference, these models do twt
more to than ro qua1ihive r,""f.'p,}'{'!>
the are useful abstracticms (li ine fl>search to ;vhich th(>y
applif.:'(l, 1'0 ease their intmauction, '.ve introduce aH i'\Igebraic
wth ,'crba! followed abo'>:
notation.
skipped without
:\ Form.1 \Jodd uf Data Cne,,'tion < SI
Ul DAT,\ C(}LLfC,'TlU'.
pn!st:ntation pi descriptive ami CllLsal nfr-
(l primary ni social "cience 'NlI d(c-
ior he dMa t be cdiectt'd dnd for these
model is quite simple, but it is a pt)werrul [.)ol for analv/ing
oi inferencc, Our aigebr<lic model will not be <l" formal as
hui makes our ideas dearer and casier ro
collcdiali, \ve reft.'f to a v:idl' range oi indud-
partidpant obs(:f'\'at{)ft lnlensn' intervicws,
survevs, historv recorded 1'rom SG:.llTeS, ran-
"v,,>",,,rnnn' - Z'onlent and anv other
vu' al!
h07C the data {Uhl lv.
r
:: ce carne ro
of information tllar we should
observable impHcatons of our llenIT. !t muv help us
nw research question, but it \vi!l be nf no \.1st: in
CjUt?StiOl1 if H is not an ohservable mplicatiol1 nI' the ques"
eek 10 answer,
data \vith lit::; .. and 011e ex-
annual income of eaeh oi tour people, The data might be
simpJy four numbers: , 521. ana
ihe more case, wc could label the ncome or four
1, 4) as ltL, .'12, }!:\, iind JM- One variable
tor t\VO unstructured intervl'\\'S takt' on the Yalt1E.'S "par-
," "cooperan"e," or "ntransigent," ,:ma migh he labeled .lit
In these cxamples, the '1\7riaNc 1S ti; the U/lits are the ndividual
he are the values nf tbe ,'ariablcs fuI' eiich unt
for d ollars Uf o The y s a
suppose we ;lrc nterested n
since 1945, Before \ve colled om data, we ncd ro decide ",hat
VVe could seek to understand the size
issu area Of
nil.tionl in 1990; changes in the size o!' intemationa!
activity sinee 1945; or cnanges in the size distribution of
Descriptve lnference
organi?atiol1lf dcti\Ylty Vdriablc5
"divity (uuld ndudc the number C>f nmtries
ing to ntem"tionaJ orgimizatinn:, ai a time, tne number of
by infi'rnatonaJ organ:z,ms, Uf the sizv:,; 01' budgets
staffs, In these the un!::; oi ,1Ila.lvsis woukl indllde '
tional orgimizatins, issue Meas, cmmtry rnemberships, and time
ods su eh as years, five-Yt'ar periods, or decades. At the data
slage. no formal mIes <1pply as to \vliar "ariab!es tu colket how
unlls here snould be, whether the units must outnllmb"'f tne
or how "veH vara bies shonld be measured. The oHlv rule j
judgT1wnt as 10 ,",vllar \yill pmve ro be mpurtimL \l/t'
clearer ide,) nI' now the dat.a will be used, the rule becomes fnd
I'mmv implkatitms {lf a theorv as As \Ve
sized in chapter i, rese1rch ca;, be used both tu
priori hypotheses or to not previotlsly
but if he latter l<eh' data must bt' coHected
el/aJuate these
!t shnuld be ver}' dear {mm our disn.lssiol1 thal most 'works
"case studics" h,1Vt' l1umcrotls \'ariables measured over manv
!ypes of unts. Although rarelv uses
handful o cases, the total nmnber 01 observations 15 general.
mense. H s therefore t."Ssent1, to between !.he number
cases and the number of observations. fnrmer mav be of soma
but onl\' the l,ttter s o .
of the number of obsen'alions comes fmm
\vhere n IS he r1umber of persons to be
\ve apply it much more genera!!)'. lndeed, \)ur defin:ition o an
vation" coincides vvith Hanv 0975:85} defi
\'\'hat he c"Hs a "caste." As Ecksteil; argues, NA studv nf six
dection.s in Britan mev but need be, an 11 "" { studv. It
dlso an n '" 6 . H can diso be al1 tI := 120,000,000 It
,,11 \vhether the subject (lf 1S svstems, eJecions
voten;." The "ambiguity atxmt what constitutes 'individual'
on be dispelJed not lookl1g af concrete entities but
the measurt's made of fhem. On this a. 'case' can be defined
as a [or <md interpret only a
gle measure on any pertinent variable." The onl)' difference in
usage is that since Eckstein's artide, schola.rs have continued to use
Summ .. rizing HistorCal Detdl 53
to refe!' ) J. {un case stlldy, has i1 faidy
Tht'ret'el't', wherever poc;sible vve use lhe \",ni
t'l'riters du n:..'Servc he Wfd "obseryation" tu rdt;'r lo
of one 01' more \';rabk's on exactlv one unit
1pt in the rt's uf this d,i1pter tn shp\\, Ix),,,, . 11,:('
nd unHs can inemase the darty pi nur thmkmg abou! re-
even when i! may be jDilppropriate ro on quan-
to summarize the inform,lton at our disposaLTlw
\ye is; tI O\;' CilH ,ve milke de.;eriptve infercnces alxmt
aS it really \<\'i'lS" without getting ost in a sea (, lTelevant
. how can \\'0 S(lrt oul he essential from the
data are the frst in aoy :mm-
of the data. Summarcs JesL'1ibe \Vhlt mil}' be il CUHount
but are not dnxt!v rdated to infcrenee. Sinee \\'e are ulti-
in generalz:1.ttln and exphm;11ioll, ;1 summarv of the
to be explained is usually J good pl;,ce to start but is not a suffi-
o sodal sdence seholarship.
is necessary. We ean l1ever tell
()f events; t would be
whkh events "vere o.m"tnKf
his
be irrelevat h,
h"lr or whe!1wr he friz'd tO
Cood hisloricaJ writlng aHhough it mil\' Hot be
verbal nf " weHer uf historcal
SUmJmln;l!l,g historieal detail is a stab-
o data in abbreviited formo liS
he appropriilte of the data .in a
For example, Orle statlstic 5 t:he mean, or a\'crage:
i!:= + y: + .. , +
, Form,,11v, for " 5<?t o' n units un whch il \\lri,ll:>1<' lf is me;\sUfL'Cl (V;" ,,1,el. ,.1
Ir 15 a unctinn dined as 'ol!ow!>: h '" [(!I'.) '"
54 . Descriptive Inferem:e
;"vht):!'t" '.,.th is a convenient 'lvay (JI + +
statstk is {he Jabt'led U",y,:
ihe &,mple mean of the tour ncomes 1mm the example in
$22,000, 521 and 554,292) is 526,573, The sampie
mum ls $...114,292. \:Ve Gm summanze the originai data C'onta
numbers "vith these two numbers representing he sample
maximum. \Ve can "iso other :xlmple characteristics,
minimum. mode" or \/ariara:e.
Each summary in this modd reduces aH the data
this simple example, or our knmvledge of sorne asped of Eu
history in the other! ro a single number, Communicat!ng v;ith .
res is often eascr and more meanngful lo a reader than
data. Of course, f \Ve had four numl..Jt:rs in a data
then t would make Hule sense h) use five different summaries;
he four original numbers vvould be simpler, Interpreting a
tistic is generaHy esicr han understandng tht.: entire d"ta sd" out
onlya few.
lO5t: information dt>scribing a Iarge sel of numbers
WhaJ rules govern the summary nr historkal detail? The first
thilt summarlt'S should OH the ou/comes hat TUl' !I.'ish [o
H we were intt.:rested in the 01 the ,werage
tional v;e would not be l,vise ro focus 011 the Uni
but if vve were nmcemed about the size distributkm of
l1at:ional from big ro :;ma11, he United
be one of the unirs nn "vaich
Unitcd is not a
tant o:n0. In s!i:ltistcal terms .. tu
ganiza!ion, \ve wouid exanline mean
bershps, etc.), but to undersi;md the
to exarnine the varial1CC. A strorh.l, obvious precept is
mus! the al aur In quan
terms, ihis rule means thar we should ahvavs use [ewer summary
tistics than units in tht: original data, ,ve rouJd as easily
sent aH the origina.! data \vithout summarv at all:' OUT -
should a150 he sufficicnHy simple tha! it can be" understod by .
No phenomcnon can he summanzed perfecHy, so stal;d
adeqtl.1cv mus! depend on our purposes and on tht: imdiE'nce
e pint is dosdy rd"teo to tlw cnnn:'pt of ndetermnan! research
we dscus& in section 4. L
we
Dcsuipn: Inierence 35
paper nn and aBian(t.>:; l,jatd in-
In Sl1ch ,1 p,-,:,cr summanes ot the d;,1
numbers might be justified; ('\/en for an expert
te indicators mght be incomprehensible sume iur-
rv. For a ",eture n the to an under,;raduatt' dass,
might be 5uperur.
'e inr'erenc(; 1S th(' prncess ,111
011 the hasis o a sd uf oDsery;1t\ms, For
in ,'ilr;ions in the distrie! "ole for
Lilbnm. and Social Demoaatic in 13rtal1 in
hav\;' some to ho\ven,e
observe is 650 district eJections tn the House o Com-
in thilt
\ve might think Ihat we \\'01'0 direct!y tht' dec-
of the COl1servatil(es bj recording ther share of the vote
t and their oyeraH "Dare ot seats, But a c0rt .. in degrce o ran-
or unpredctability is inhercnt in politics, as in al! ni sociallfe
i of scientific inql.liry.7 5uppose that in a sudden fit of absent-
m:ss (or in deference to sodal 1he Brtish Parlian,ent
to electlol1s 1979 and suppose
thal hese were nf one another. Even
"""!p,,,,',nn support foi' rhe Conservatives rernaned constan!,
replkation \Vnuld no! produce the same flumber of votes
in each dstrcL The \\'eather
cl(enrs might happen In the nternational nr SGm-
might reaeh the mdSS media; even f these had no long-term
"".hUU"',,"' the'' could afect the weeklv results, Thus, nlJmer<)Us
l'l"iUIFatr)r\i events cuuld effed slightly sets ni election rdl.lms.
.," Our observatioI1 uf any one election v/01.!ld not be a perfect mensure o"
5trength after aH.
As another example, suppse we ilIt:: nh:rested in the of con-
Bct between lsrads (polke and rcsidents) and Palestinans in commu-
nitres on the IsmeH-rn:cuped 'vVest Bank of the Jordan River. Offidal
repons by both sdes seem suspect nr are censort-d, so ,ve decide to
conduct our own study .. ve (',m ascertain the level
conflct in diffcrcnt communities ntensh'e in!er:ieh's or
Dest.Tptive lnfurence
tion in farnHy (ir group If do this tor \\'eek in eZlcn
mtmh', uur condusiol1s "bout the h'\'d uf l:on'!ict ll1 ('a eh onc wll
a {unction in part of \Vhdtevef d1dnct' events occur the \veek we
pen to vsL tven if \ve conduct tlll' study o\'er a veal', \\'e stilI wiH
perf,,'ctiy knm." tht.' true lt'VE,j nf contlict. \.'\'('n tlwugh nur unccrtainty
abl'ut t \vill dn1p,
In these examples, tfte variance in lhe Conservittve vote 3CHlSS
tricts or the variance in conflrt between \Vest Bank communities
be conn:ptualized as arising from tiNO separare fadors: sysfematic
twnsystemafic differences, diHerences in our voter
nelude fundamental and chMilcteristks uf the d
such as differences in in il1come, in campagn
or in traditional for cach of the In hypothetkal weekly
{1f the sanw would per-
but the nnnsvtematic such as rurmmt variations due
the ,ye3!her, ,,-'o11d vary In our \'\iest Bank dif-'
\vmld indude the deep cultural differcnces bet\veen Isradis
and Palestinians, mutual kmrwledge of e<1ch nther, and
pattems ni resdential housing 11' '.ve could star! our
vatlol1 week a dozt'n differen! times, thesc ""'CH''''''
tween communities would contim.l0 to ;tffL'Ct the observed level of 1201\-
mct differences, stlch as terrorist incidents or
instances of Israel brut;,Ity, WQuId noi be predktable ilrHl
\vemld nnly aHect the \vt"ek in whieh they happened to oceur, \"'/1tl1
inferentia! techniqth's .. vve can leam about tne na:-
differences even with the arnbiguty that occurs in
one ser ni' real data due to nunsvstem.1tc, or differences,
{he
is no! more
and our attention shnuld not bl'
''''''P\,',,,,. distinguishing be!v\'een he
1:1"'015 an essental task oE sodal sdenc, One to think about nfer-
enee is to regard the data set \ve compile as one oi many possible
data as the actual 1979 British retums constitute
only one of many sel:; of results for dfterent hypothetkal
on whkh electicms coulel 1ilve been or as our one ,veek
of observation in eme small s one of manv possible weeks,
In inferem:e, we seek lO understand rhe t whieh
our observations reflett either or outliers, tild the
1979 British eiections :uITcd during a tlu epidemic hilr
thmugh 1.vorking-dass houses but tended to span' the rkh, our obser-
vations might be rather pOOl" measures of underlying Conservatlve
Des.:riptive ln:,rence 57
be(\lUSC thl' chan('c et:D1t'ct in thf'
tend ro ()','erwhdm Uf dis;1r; tlw jf our
w0ek bad nCluned immt'j;:tdy atter ihe Israel nvs(m
Leoan01L \\"0 wixlld similarlv t";ped that are
ve o! whi1t USUdlly happt:ns on thc West Bank.
poltical .vorld s theoreli(',l!y c"pable nf producing multple
ets tor everv pmblem but does no! foHow Hw o
sdenlists, \lVe are usuallv on11' fmtuna!e enough In bsern' (1ne
d,:t,l, ror purposcs of a modeL we wiii lel ths une set ni data bl'
eme \'ariablc lf {::oa\', the vote fur Labor) mcasured
t1 ::: 650 tmHs districts): ;, ,11;, ' , , , yry (for example, v: rnght bl,'
peoph: \'oting fpr Llbor in Jstrct ), Tlw set D
we abe! Ir i5 a nvfi:cd mriablc. v(lues \far}' UH'f the 11 umb,
a:ddtion, wc: dcfnl' ' as a rrmdom i'firi;)/ilc oenmse t hlres ran-
aeros:; hypothetkaJ rcp!catons of the same electinn, Thus, ys is
the number of people v()tng for Labor in dstrict S, and Ye; is tlle Tan-
dom \.'aTble representing he vote many h:T
1
ohetical cledions
tilar could havE' heen heId in dislrid S l.mder essentially the Silme (on-
finans, The observed votes for he L<1bor par!y in th", une \ve
observe, 1f, 1t2, ' ' , , l/I!' dffef ilcmss consttuendes because of sys('em-
ane ilnd ;miom faetofS, Tha! to distinguish the t\vo forms (lf "vilr-
" ",ve otten use the emi r,',lli::cd ,'{"iahle !o refer to ,I and nmdorn
"""FIt1,",> to rcfer ro y
same lo our qu"llrative We
wuld han' no nr dcstre of quantifvng the h?\'el of tension be-
tViietm Israels and in part becatlse s a
catee! iSSllC that 1nv01v0s the feelings of numerous indidduals, o1"g,\-
nizational C'onflicts, ;md othcr fcatuft's,
in nis sit.uation, li'; s a realz(x'l variable which stilnds he total cor-
Hict observed ,:h;rim; PUl' \vcck in he liflh cnmmunitv, sav ElBireh,"
The random y; represents both w!1al ,ve Ei-Bin:h
and whaL Vil!: couid have observed; the randomness comes fronl tIte
variation in chanee evcn!s OV'r the vveck::o ,VE' Cdldd have
eh osen lo observe,'!
One uf the "111-
dmn variilblcs bu! st,md,ud, ermi-
although in distingush {mm
nonsystematc components in OUT data, in il case u'e vdsh lo
the san1e to f1Il nligbt :-;tudv
a Noh:' thnt nnt exactiv n\'cr Jitfer-ent bt.,th
ch:m:c eVPl15 ,na Jifferenef'S migh! accmm! ror ntserved Jiff",rellce", We
therefor", <:re!e he more ideal ;:.im,,!io! in which v.'e imagne nnming he wnrld ag"in
wlth sys!em<ltc feature':l hcld cnstant "na cllanee factors ",,wed tu \'wy.
58 Descriptve lnferenn"
t;:kv Jrtd it" %t\1turt:\sJ Fur
we l11ight ",ish ti' klK'\\' the ralue of the Labor \'ote
clistrict :; ({he average Labor q1te '(5 across " 1"r<2;e number o h
thdic11 in this ,:!istridl. Slwe b a fe,1ture o'f
dectoral the ex:pected vaIne is of considerable 1:n.
terest to sod"l scientists, In contr;;1st, the Lloor vote in one nb;;erved
election, ys, is off considerably less long-term inten:st since it 1;; a
tion of fea tu res mI;! random error. it1
The py'nt>r!p,1 \'j!ue tone fedture o the
titth VVt'st Bank El-Bireh, is
where n,} is the \.','!U('
acTOSS an jnfinite number o hvpothetkal
observe in ::;, ElBireh, The
mu ,vith a 5) the answer to tlw
Cillcul;:lion (nlevd o' conmct between Palestinians ano Israel s) far
community 5. Ihis is of om model {or 3 fea-
tun, of the Tandom variable y" One might use file observai lever
,lf"" as al1 l'Stimate uf fi5' but beca use }j-, canfains man)' chance
elements with information aboul ihis
estmators
.i\nother
exist bee sedion 2,.7},
off these random \',riables
h, knO\'\,' 15 the of cnnHct in he iTiynlY;c West Bank
One estm,1tor of Ji might tht: average oi the observed leve15 uf con-
flk! aeross aH the communities studit't1, y, but other estimators tm tMs
feature exist, too' {Note that the &anle smnmary uf data in
our discuss(ll1 of
for the 1'ur1'os0 oi inference.) Other ",,,,,,,,,,,,..,::tt'Ir
o the random varulbles indude [he variance ami i1 vanetv of
causa! introduced in sec!.ion 3,1. .
StiH anoHwr feature 01' these randnm vnnables that might
be oi interes! is th0 variation in the level of nmtlict v:ithn a commu-
, ()f ti" tna \' bt> nf tn<!lendous the t!1 dtstnct ::; h"1 tito!
ill\cJ thus ooln Ule mndom .Hhi sys!emal..: (ji Ih" <,veni be
'''"''''''.'>''1<', Nt'verthde5S" \Ve shouid try lo Ihe mndnm rom 'w WS-
Lkscriptive lnfenmce
the Sdme random produce divergen results, This
the size of th0 nonsysh?n1<1tic comJ)t.1nenL
con11-:1unt\V t he (jnsreatj
23l
"tter ,knotes lhe n:sult ni
t(J the random m a \Ves!
be ,,,'orse. ln an\' l'\'cnt, both
researchcrs,
these isstles we distin::ui.;h hvo
me""s of nmdom \'araton
n
Thest' two re extremes on
Although l1umbers of sdmlars can be !cmnd
are comfor!able wirh each extn!lTk', mo.;r scentists have
somewhere bet\vt"t'J1 the two.
n variul1:S pdrts of
::: A I)ctermiis!1c V'irld, Random vilritol1 i" ha[ p{\rton ni
Hit: workl r which we 11,H'" En The division
abe ;md S[Odla;tc \','1iatml s and dppend5 on n:l1at
,'arables are ilvahli;;' (l!1J indm1<,d in the Cven tlw
variable;;, ti"" W<.lrlJ ;s
In the in ..
: (lY'91b} ror iH1 cd'
Ecnnomists tend t.' i, t'\' ht'lV.1S ':.>tzis!..-unt< dr=t
3
lO
2. 1 is cornmo.n in the filAd engu1(hring
"<Judit Y control/" nave even dehaloo hb di"tinction in tl:w ldd o 'l"antum
m'Chancs, Early P,:;'rspective:2 $ubscrbed to lhe "hidden variable t]wmy"
6 Descriptive lnfcrcnce
i1: tlSSUnlt>: under
femain UnkJH''''n,
!cl1\X' o<.'\:uro- ',\'hen ttwse unknO"'H explilneltorv "fiables in
tiye 2 becnme tlle for the rmd(1m vJriaton in
tive 1, Bec;w:;e nI tlw lack o dltV ObSCiyable mplic1tions with which
to distinguish bet'ween t!wm, d choice bdh'een !lw two pCispecti >
011 bith <Ir beJk( r,llnier han on emprica! vCfication,
As iJnothcr ('xumple, wilh buth distingushng whether
el partcular 01' sodal l"Vnt IS the resuli of a
From tJw
,,5
ser d da!a (nr t','en anuther case) to check fur the
etfect eH' pilttt'n1, il is nory dfficult lO makt' he judgment
From [he e\Jremc version of 2, ,ve can do no mort' th"n:
:m l'vvnt ,1;, ",tochastc (lr
or rrelevant. A more realistic verson 01 ths per-
! 's corred nr i!1correct attribu!:ion of a
but it alkl\vs us some latitudt' in
what ,vil! remain
aH observal:ions being
tht:n tn cvidenc0
rcsul! torces,
random occurrence or
::>uC'p,'u f"r future research,
anei
ti ve rest'archers. research is (ltten but
mos! use as sodal sdence \l:len ir Is ,lIso
thc randOlTl vriables from which observations are
ate'el nnd to lo estmate their
the
data coIledons. Indef'ti, (me lTlark. of a
the n...,>sohtuon uf tht> rntmt":rnus :rcrn.!n.g ("(Hlt.rH . nf thb i!llfxH"LJr
,md !lo; jo. he' nature ni thc wMkl. H,w:ever, lhb in
ks., uS('11 ro mw:h pi Ih", oi is unllk.:'ly !o
thc off or prilctice of resEiltch in the sodal scienres.
Descriptivc lnfercncc
klHd uf :::.cicncc the
In bnoky FinJing uf
vvith some kinds o bu! ir is 11\)
As a11
suppnse th we aH! nterestd in he ouiconH:S (lE CS-SovieT
betwet:?n 955 "nd 1 q9{) OUT ultim,lte purpuse s to
question: un .. whi1! conditions .md to \yhat t"xt'nt
summits lcad to in(reibed ('ooperltitm? Answering thal
a number o dificuH iSSU0S of causal
nrnong a
rest fi ('t
<,venle; O a cnmbinatnn of these measurement
whicn summits \Ven' oHoWf'J inncilsed su-
about tht: nm-
in
and [he l'xtent !o lvhich
Gmdt1l1s in
<>v,..,,,,rt,.,hn1l,,, on both
have bt'i?l1 fulJIled,
leve! of in ea eh y'ear, and [O assodate it
he pres.ence or absence uf " surnmit
as ,vdl as \,'ilh our (lher ,
V'/hat ,ve Obsc've (even our ndices of ,.""""."",...,h",
tbe
VCc1rs sl,llTlmit med-
, \>vhether thc and
ft'lated lO unI' a{lther,
t could be rhaz the a::is(Kidtun
ciue tn funda-
nr bad luck under
lmidentified explanatory variables
sl1ch tmdentified variables incJude 'Neilther fluchwtions
the Soviet shifts in the bll-
aH of ,.vhich could d(Cnun:t ior in
If identified, these variables are altemati'le
viuiabies that could be collect{"(l Of
1he :!ktt of "",1,,,,,,,,,,,
i':(lt.lld tn the CfH1V("ning f surnn'lit )1t:'etlngs"
(',1St', instead o 5ummil mt'\?ting:' explainng cooperaton, antdp<ltcd cCJofX'faon \vnuki
t'xplain adual C'oojJ<'r;tion--han:lly ; startling nndng if adors are ratnnaE
62 Desaiptve Inferenct'
to nS5CSS tla:.ir nt'luence on rhe SUm!Ttlt lf t
variables may be trt,1tt"d dS e\'ents that (ould
for the observed high Ot'gree of sllperpower cooperation. Tb
evidence the possihll:' th random 0vents midentified
variables! aeCollnt for lh.: observed coopc:ratiol1, we
look al many orher vea ro,. Sine/! nmdom events ana procesS{'S
definiran no! hev v,m bl' extremelv unlike1v t
differential cooperation in y'(;:.ns with ana
mil". Once ilgain, ,,\"\:: are lea to the conc!usion that only repelted
in different CO!'('.xts in ls (dSe) enable us to decide whe!:h
a Liue lo lhe transienl:
or randorn proct'sses,
fmm nonsysternatk pmcessL'S i5
uf social a flu epidemic thdl
voters more ht,>;wihr than middle-dass ones is an
eVf!nt that in one , replcatio
!-he 1979 dection would dccn:;lse the vote. Bu! a persi.5tent
em (Ji" dass dfferences n the incdence (lE a disab!ng illness woui
effect the an'rage level ()f Labor
ba5is of the vctor's or an accidental
a debate a randorn factor that couid have
fected the likeHhood of bt?tween the USSR and the
States rile Cold War. But f thc' most effedive
to vote!'s had bell w
c0i1s1srent vktories of
il factor
\v'hen
constants, appea15 ma]'
behavior, but that fad does
mean thi'it campaign do 110t It 15 the
off appeals 011 an electon outcome thar s if
is variable, it is VVllcn
Dad wCJ.ther ah\',H's lO f,,'wer votes fuI'
favoring conciliatory :miidcs) ..
In short, surnrnanzing historienl s an important ,,' ,t.;>rl'Y'"u1
udging Descriplve !nferences
our \.1It1, but ,ve ,'tfs.c dtC\"Scrip-
disting111shing bet\\'een ranLl(}I11
. Knowing wlut happened on a
occasilD is no!' suHkicnt
Jf HO iD fxtratt filt"
CRliERJ.\ :Ol{ DESCRlI'TlYE l'iFERENCE5
s ..:dion, \Ve introduce threc explicit criteri,j th, are com-
in statistics for judging methnds uf infcrences-
ami fileh rcles on he r;mdom-
introduccd in st:don 2.6 hu! has d.in'cj and
for and
\"Ve save
research that is
"kme for toe remainder oi this
off infer!:'IKt? imd l,ve wi!! esb-
are sometlTh.'S t(lO and smnetinles too smalL A.eross 3-
do ,.ve the ansvver 1m !1Z'emgc 7 If
" s said tu be unbiased. Ths prop-
about ho\\' far removed fmm he
()f the methnd might hut correct
o
'. times one sometimes the other. Bias occurs when there is a sys-
tema tic error in the measure Ihar shifts the estimalc mure in une direc-
Hon than <1l1other ove1' a sd of LE in our of conHict
in West leaders h3d cn'dted
the studv"s results lo fur\ht:f ther
llien the leve! 'oi conflict 'Ne observe in every cornmunity \-\,(luld be
biascd tOlA'ard greater confIct, on average_ If tne n'plications of our
hdped side "nel nul the othL'r
fuI' C,msen'd!1,'l"S wen' more nductant lo n,[c <'11
l"e"OHs!. (JI' oul" cstim,1tes mgh! bt' bilSd
fmm '/ote cmmters \\'no favor one pany the
Ir, hnwe\'er, tl1(' rep!icated dections \".'ere 1eld on ",,rious days
in mannel" Lmreldted to rhe variable we are nterested in., any
measl.lremen! v\'ould flor
01' al1o!her might {a'/or (me
counts dne h, r"ndom
of estima tes wl1uld bE' unba"ed.
lr he Britlsh electnns ;Y'::,re
n,ethnd
int the eIectinn c'"d,.w,"
crcumstances [ha! jnduded tnt:'sc
on the that 13 and cines mi! jusi exfst
he data alone. H makL"S littie sens,e ro &lV thar a data set
evel1 though it b-e fiHed y,'h many errors.
In this we our ddinition of "
tistical bias" in ;111 estimafor from bias" in an ch'fmnJ
of he larter are
to '\1ote--:} no! uncomm\.m ,"ubstanbn' bias uf
electoral As '.ve mav \vi3h to estimare he
vote o the actual electora! ""'er,;""
but \Ve
are
bias in the \Vhichever mean "ve are
!lnve a sta!sticalh' unbiased estlmator,
ihat \ove use ror (1,;'<;crU,f1VP 111tl",'PI1ICF><;
tnat are ''''''+'.''''''''''''"-''
snnre up
to find <'5t1m211,,"3 tnat are
VVciner'5 uf education and
labor in India 0991}. In tryng to expIain the evel of commitmel1t
to compulsoJ'y education in india ta toat in atoer countrk"S,"
=/1
th,,"d
then hH'
attcndal1i::e WdS f,H lll1C ,Ll\< CJoser "crunv
figuI\: to tTHKh hn\'('f.
for
IV('
ti' use tlw i1'>[age il"
da!;}, Ci ", lhe
{ur ,1\'e[-
i> " ;m estimatm pi u, s a
eXBmplt' 'titan ,1PP<':;US in formnl s\;\tlsti..::':' lc,s, bu!
are sme.}
1
Dt?scrlF'tivc 1 nrerence
V\'t: d{J not ha ve' an dpp(lrtunity to out' ID'
nmnGer u essentialIy identG1J applications,
ness is o nterest. but we would ke mure cunfdt!.lKE' thi1t the
estin-ub' \ve 15 ck'St: ro the right 011E', Efficency provdL"S a vva
dis!inguishing amung unbased estimators, ndeed, Hw dfcency
ron can "Iso hlp distinguish among "lterrlatjve
sm,,!! amount uf t'ias, tAn estimator \vitn a bias ShCl\lld
be roh:d out ('ven \Ylthour irs
is d rt'ative c'nnr'PT>r
\"ilriance of lhe t:stimator across
biased estm\Jrs, tlh>
ter) tite esnmatoL A 5m,,11 variance is better becilllse our one esti
,vil! clser to the troc vahw. We are nui
prnr,plnr, for aH estim,:or wth il
,mee in !h15 situaton wiH make it that the estimatl::'
near the tfue vallle most of the estmates ,vouJd bt'
dllstered armmd the \alm:L As vve describe bdow, we a
in the case oi a SIl'hin (;im(}Unt of and \V
\.A/e are interes!ed in """.U'''
Palestlnians and lsr,lelis in
mid simiar obst:'rvatons
fve conununites, I! be ob,'ous t!u!
<1 re better (ha 11 a
into enen pi the
We \vil!
enablc:s as ro
"' Ji estimator 01' u with
level 01' conmct found {rom
studies in communities on [he \Ves! Ba.nk. H
both estrnalors ne unbiased !f the same mndel applies, the.
estimator has a vmance pf V(I' al. That
,ve vrould have chosen ""har .ve thought \ ..!as a "typcal"
bz..' b\' rtHl."-..:(H11 ..... Ihv
'25, the Uf
hn1trs
(Le" less diicient) thitn tlw estm,l!c when 1; = 2:" HelKe,
ve the that Hl0Ti: obSer\\lhlH15
are tht' conJitions undcr whkh a more
communitv would ilS
That is, although W(' should
mon, observations tlw resources necessary te> clJed
con-
being ('qua!. our sor)';v" tl\i the rmm:
becdllse tand thus In
b sllen th,,! as the iltHl,b"f o obsen'.,-
dt'Creases to zero, and :he estimare
to estima te d
thar
fur no otJwr re3S0n, than thilt we
make corrE'ctons for such l'dctorS
number of obsen:ationsl. \\l(, are then faced \",ith d tTade'
th"t has additonal nbservations internal t
cases in \vhich eaeh contans une ob-
oi on1)' (lnc
lo our 15-observatnn re-
vvhich are easier to understand
we could firsr sdect our communit)' very
of thc
the others, VVe mght ask l fe\v residen!s or look nt neW5-
",->,."".-1'", io see whdher it .vas <1n average community or \<\'het!wr
l[ /L hu it ,l::' numher nf urdts n.:n.\l<;c .. thl:;':'
cshrnatox Hui irnprovc (OT in.:k:-t\:l change id aJO. An cst1rnah)t' C'ln dtS-t) tR"
out hk'lSt>d, F0f examplt\, y,-, S/n is biasezt hut is consisten! beG1U5f' Sh ",comes 70m
as H appmaches nfnity,
dnd tht'l1 \\"t:'
e5timate d the
tht, most dificult pdrt nf the
h'':: be crefu! that nt)t in. ()ncc \ve
confident that b,,:; is \Ve could ''''cu" on ncreasing
Tu do ths, we rnight man:,' \\'t.'eks in the community
ducting rmmenms studies, Vv'e couJd in!erview commu
edeT";, ordinarv
in a
pf he \vorld. further that there is a choice 01' tWt1
il case or a iictatistical
countries of the !t \vould seem better to
Bu! Jet US Sil)' that ro t is necessal'v (ful'
H:ilSl:.l l1S1 h.l use data to a UN agene)' {rorn tl1\:'
emrncnts. Th('5e lwmbel's Me known tn have little re!ationship
actual of since tht:,' WFfe in the
Office imd based cm ccl11sideriltions o
that \ve tuuld.
make the crrectioHs to the
thi
\\'ould ,ve choose?
hvo 01' enree. Or 'c\'C
data best <lBs.ver our
To take still another example, suppoS(' \Ve re studying the Eur<r
pean Community and \Van! to estmate the degree o
tion of an th(;' ('n ti re that wiH I'esult
of i'v!inisters. We
number oi mies forma!!y
industrial sector in ende thcse rules in terms of their strin-
gene)', ;mJ then estim"lc the '1V\crage uf a nll,,, H vV\.' giher
data on HIO rules \vith smilm a strngency, the vanal1ce of out'
!udging Des.::rptn: 69
ir thc are bt."
d<1ta un une rule as he \"stim,ltor f()r strin.-
as a
wen,[, ihis prcedurc
tu lhe retll
us h) lht' f\)rHul rule as
th\:.' Sic'I.:tnr under
l1\'t'stigatioIl of rule ,lpplicatf)n, !kl\\'eVt'r.
var,.tll1n in the extent lO whcl nominal rules
He!Kc,. measuH.'S uf forma! rules might be
are actually en-
b,
nsLmcc\ in fanJr o' in
a case, \\ t.' ,.,.'(luid {aee the bias-effdencv traJe-,1fT once ol
mk,' 5ense io Ca!T\' "\JI thn'c uI' fnur nteltsiv,' (,be studics o
mp!enwnlilton lo' the hctwecn formal
and actual Onc, would be tu suh-
an e:-;t1milt01' based 011 tlwsc fhree or four based
les;; efficient-for the cstmate', bi1sed cm ltlCl il(1wi?vvr.
he more crt'ative, if tu use (he ntensiVt,
work lor the three or rour cases to corred tht.' bi.as ni om OU-GlSe ind -
cator, and then In use d COH(y('ted version (ir ihe J nO-case indicdtor S
our e:-;timator. In this \'>i0 \-,-'oule! be
o11r intensive case stuJies \.vith
think should bt: follo\.veJ muen more
sodal s,:erKe,
for case stud es m.ldc by hose v\' hu kmn\' a
\\'orld h'el! b onen t h,,' one in tl-n:
studies mav upn num1:x'rs lhi1! are no!
he ni'\\'e researcheI' \In dar base (who
be unwar2 of the in which decton statishcs dfe in
a partinlbT h,cale i'md dssumes" thar thev han' som<:' re]
re!atkmship to the votes JS
the materiIs and !11JV 1,1(' abk lo
com::ctions, ln sedions W(' wll
how sueh choices mght be mih,1e more
Our fonnal o this in he bOl< below shows pn.:-
how to decide wha! the results o the trade-oH are in the e)(<1I1'I-
pIe of British electoral constituencies. The decison in anv
wjJ be beHer when logic lik0 thar sho\\"n in he
this lssue will ,llmost
\vorth more
thal somet mes exist bld.& imd The mean
the first two observatons in any larger set nf unbiased obser\'ations 15
Descriptive InfercJ1ce
Formal The L,rianct: ul t11<' ::,anlple
y' 1S as l'CY), and H1t' ruies for cakulating v,riances
d"m \:"r1aI:>1.::s in the sirnple Cil9' uf fimd(im thi?
if \ve dssume thni rw ,'"ridrl' lCroSS hypothetcal
li.caton uf each distrct eh'dinn s lhe same as t'verv ()ther
dl1d is delh,teJ OC. tlwn tlw variancf' ni he samp{e mean is
.,
,:-.: (-
cstimator has v
!;'sttmatm has \'ariance Un!ess ,ve
random-t'rror correctins to reduce the variance
the statistkal
rut it doe'S n:ducc
If \ye id nt d",) u::,,,, the dficiency criterion, we \vould
no formal criteria for choosng one ove1' the other.
Stippose "re are interesred in \vhether the Democrats ,youM
ucigng Descriptve lnferenct's 71
fandol1l sdectiort we Ch(1Se survev rcspondt'nts inm1 ai! "dul
of v:hch has m ot sdl,ctio!l,l Sup-
. , t some!.mt: else a!so did a similar with 1 cliYd'l:<
we indude these additional observaticms wth ours tu crCalt' a
based on 1,020 respondents? H the ne\\' observations
andomJv sdeded, just as the first tVH.'n!V,. it should be an eas)'
to ne'lude the additionaJ data \vitb mus: with tbe new obser-
{he estimator is stm unbiased and no"v mueh mOfe efficienL
.;uppose tha! 990 nf he 1 ,non obsen',i(ns \VeTe
dr"wn from the US i1nd he otiler ten \Vere
ale Dlembers o Congres5 \vho Wt;[Z; induded in
[he random had been dri'nvn. further
we found out th"t hes!: additional observations were induded .l1
data but did Bol know which OHes \'\'ere ,md thus could Bot
'.Ve novv know a tha M cstimator bibed on
respondents would produce a overestimate of tbe likeli-
i::haf a Deuwcrilt vvould '.vi n the natiomvide vote, Thus, induding
additiunal observations would bias the overaH esti-
but it \vould also substantiaHy its effidency, V,,'helner
uld ndude the observations on whether the
in bias i5 the incrense in slatisbe"l oHh"u",,,\{"L'
it secms dear that Hw estmator b,1sed on the
wil] produce cstimates
fre<.juently than the esUm3tor bascd OH
The bia s n trl'd uced wouId be small
samplc estimalor even though in \ve would proba-
addition, v,'e know the direction ni the bL:lS n this
and ('{luId even partally corree! for U
ff quant:itatve data are avalable and we are ab1e to '"r-
such pwbiems as these, \ve can usuaIl;," mak" a dear decison,
enm if the qualitative nature of the rest:'arch makes e\',1I'uat-
ihis trad("-of{ difficult or mpossible, understanding H should help
us make more reHable inferences.
Fonnal Comparisons oi Bias and Effidenry. Consid:'r t\VO estima-
10m, one a bv someone ,yth il vvlto i5
here're that \Ve
bdieve is unbiased out less effcient ana is done by an im-
partial investigator, As a fnrrnai O1o(le of this examplc,. SUpp05t: "ve
\'vish tn L'Stimafe,u and the large-n study produces estimator d:
72
( :::
r + y: '1
i
!
where di5tricts 1 dnd 2 are iH'erage
ilnd ::') ::::: 11,
VVhkh estirndtor ",h(luld \.ve
us or twst stmator is nol
Flrst, vve wii! asscss bias, We c:m 5ho\\' tha the first cstimatnr <1 i$
biaseo ro the usu,11 calculatinn:
FU,I::: - o.or \11
.:::....,; ,
,
v"',,, cm ,bo sho\\' har he senmd estimd,r < 15 unbiased
lar cakulaton:
/1 +,u
c:::
d
Judgnf.; Dc'S("Tiptive lniercnces 73
the t}t'tc,rt:-:- ()]
U.nb15ed, ()n an lniirte nUD1ber s,.1f n ..
ljciltil1S, h.lr he im'stgi!tO!' wth a prtYnccptiol1, d would tlw
>,/1'ong iU1S\;'!', albdt ,mI\- Sil, E"timah,r " wouJd ttw
>.',>-1',/"''''''''-'.' critt'riO!1 leHs a difIen:nt sh lry To
late the variance of each t'stilTldtor:
-;l;,Vr - tU)l
JI'''''''
1
:;::
This "ariance is the Silme as he vilridn' n( he silmpJc mean be-
cause (Un unes not has zt'rt! vilri:mcd aeross "amplle'i, Smi,
vve cakuI::te the varidlKE' of e as
eSE <'!fkient than ti beClUSl' ::::,/2 ls 325
Ths should be intuitivelv dear dE
lhe nlormiltion in tlw data set.
VVhich shotikl we choose'J Estlnutor ti IS biast'd bu! mnre effkient
1;.S bu! et"fic}cnL !n this p1rticuL1r
estmator d. \Ve \vClu1d thus be wllng to
sinee the sacrifi<.:e io; [,lirlv smali )'
,< -u,
more estm,ltur. ;\t sorne
pOln t tol' a Ii ttJe bias
snce we end up guaranteeng hd! estmates v;il! Dt' farther froln the
truth. The formal \vay to tb!.? biaso{'.ffciency trade-off is to .
calculare the lIlean squan.' error (i'vlSE), \vhiel< s (l oi bias>
and didency, If g is an estimator for sorne pal'ameter ;' Une Gfl'ek
leHer t.-lSE is ddined as
:::::: -+
=: varianee +
bias
square error s Hms the sum of tht' variance dnd the
l The idea s tu chouse the estmator
\-vith the minimum mean square errOr snce it shows precisely how
an cst1matur wth sume bias can be preferred f t has a smalle"r vnri-
anee,
Por our eX2lJ'nn'lp tne t.vo
a 1<:' as follows:
-+ (10001
and
Thus, fol' most values (It (jc, :\'iSHd) <: .lnd we ,vouId
d as an estimator lo c.
In tne'l"n', \ve should unbii1sed estimates thut are as
use as muen
rCil1 reseuren s!11ations ,ve
trade-off beh'lleen bias and
is quite sillient
this
Causality and Causal lnference
\,rE HAVE OISCLSSED two 01' sodal sdt.'IKC rCSl.'iH'ch: SUI11m,1riY-
.lnd dcseriplhe inferelKCS
the workl no systt'mJtk dnd nn!1sysh:nvItc cnmpOHcnts
2.6), M.lny students o sodal and poltical
stop aL Ihis pint, esc!1ev"'ng causal statements and Dskng th('ir se-
e('ted and wcll-rdered facts ro !or tnemseln:s,"
Like historians, :5CiJi scientists m,>ed in summarize histqrCal detal
and ro make descriptive inferences. ror SO!1C socia! scientifk
b witll,)Ut causal lflfert'l1d'>, Th,ll
just as causal inference 15 mpossible \vithout nfer-
e.nce, descriptive inference ilIone 15 oEten anei incomplete,
To say this., however, is nu to daim tllilt a11 social scientists mus, in al!
i their 'work, seek t (1I::,v1se causal f the
studv. Sometimt.'s causal inference rs too diffcult; n m1nV orher
, nference s the ultimate (lE the
endea vr.
Of course, we should \vhctht'r thc
of a pmject or Many sodal
sdentists are uneomfortable wth causal inference, are so Vfa!"v o
the that "corrdation IS nlJt causation" thilt thev \vill noi state
causa! hypothL'Ses or dra,'\' causal tn [hei,
as "studylng assodation and not causJtion," Others m<1h:
causal statements with case, labeling l.l!1cvJlualvd or
ulations as "explanatons" on the hasi" ni indeterminate research
"igILS,l We believ(; lhat each of these positions evades the pmblem 01'
causal nlerence.
In vlew' o' :n pver "mere
it 15 not surprising thi){ students f events to dress thcir \\'nrk in
of iJ:--. inferlt f
\\"ork. r\t its cure, reoJ 1;.: aasf'd (:in (-:H..b31 jnferen(0s. V\'e
in the hterature ar,out H nOf\CciU9","d 1n
itIl GlS'-""5, arf"
H f.hlres to ('xplaln nt)t due ro p-uor 01' lat k
0i milginilt0t\, bul rathef lo lhe nil!ure uf lh,' Jitlw!t hut ha! ti".?\,
,\re examlning, oi nferiority are C(,o<l de!:'cripton o mportarH
"venls s helter than bad explant1tin nf anything.
76 . Causality and Cau&.l Infc.rence
tigatiOl1 eHh!!r renders the rrdc\'imt ur
PV th .. , mIes uf scll.'ntifk interencL Our uncerlinty'
aboui cnusa! will never De climindred, Bul this uncertaint)' ..
sh()uld nol th; we "void a!:tcmph ,lt causal Rther
we should dnl\'; caus1i nierences they appropriate bu!
,1!sO prO\ide the rcacler with the best and most honesl estimate of
t.mcertanty of t1dt inference. rt is dppropriate tn be bold in dra\'v
Gntsal inierence;; as long as we are c,ultious in ddailing the u
I;nty of the il1ference, H is importdnt further, thnt GlUsa! hypot
be as as possbk the mies of
4-6 s to
cIrcUmStances under v.'hieh causal inf:rence is appropriate and
mak" for qualita!ive n::sean:hers to incrcas [he
ihal: thcir re<;t?ilrch wil! reliable evidence <lbout their
in sl'ction 3.1 vve appro-
or qualitative and tlwn in section 3,2 \Ve
sf'vc-ral ahl'rnatve notlons oE catlsality in the Hterature ana
dL'nHmstrate that thev do not cont1kt \vHh "ur more fundamental defi-
niton. In s('Ction 3.3 \\'(: disL'USS rhe assumptons abour
"vorld and the to make reliabk' GHlsal inrerences.
(T teria we "".".,,,,,,,,,'t
",ve cnndude
strw:t GlUsal
3.1 DEflNING CAUSAUT)
\Ve define
of the dlGl uSEd ro learn about
,15 a theurcfica!
"h'Y,""'"" fmm our data, iFor prob!cms of causal
see chapters 4-6.) In sedion 3,1,1 h'e give om definition or
causa!ity in full ddail, \vith a sim,ple quantitatve example, and
in secnon 3,1.2 \ve revisit our definltion altmg wth a more sophisti-
cated
3.],l rile
Our theorerl:al ddnition of lTl0st ,md
lo a As sectipn"\ d unit s une o{ tlw many
elements to be obsefved in a study, sm:h as a person, country, year" or
lOUf point t}f departl1!' in th" ,,('dio!'! is Holhmd's ;.rtide 0986) un causa!ity ana
//
:;;tatus [elf <l Dt'mnuatic c:mdid"te ir tlw LS, House ni
on tht> rr\)porbnn n( H1is
il Democrati<: l:allddate thv , Ld th,.'
\'aridble be Democratk proporton of eh\.'
fO the House, The kev caw"ll explan,ltnry \'ilri"ble s then dichot-
cHher lhe DemcKrat is ;111 incumbenl ur IwL (Fm simpldty
out this 5edion, vve nni)' (onsider districts where tht'
candidate los: the las cledion.J
var<.bl,"
varia bles"
dividl' the
is sOlTletmes ('alled the "outCO!l1C
are ofkn referrpd to as "ndepcn-
variablt,s into th!:'
variable" caned the "caus"
thl' ''('ontrol variables," causal \'driabk
(tU two or more \',llll'S, which are oflcn denoted "trt't1l!rWl1f
and "control group,"
No"" consder onlv the Fourth Dstrid in \;e\\, Yurk
ill1d imagine aH eleco!l in ISlqg with a Demelt:ratk innuub':'I ami one
Rcpublkan {nonincumbentl thc Democratic can-
didate receh,'ed frKtion pf the vote in !his election nlw '*
denotes the Dslrc.i in Nt'w York imd Ihe to
the fae! that rhe Demn(Tat is <1n ,'d b thel1 a valuc of tlw
dep",ndent variabit>. ':ro ddinc the causal eHect Ll tI:nw"tnrl ljlhmtitv},
imagine that \Ve go back in time lo he siart o the electin
ana rernains the same. !hat he lJenKKratic in"
no! lo mn fOl" ilnd [he lJemuCfatk
nominales another candidate the winner lit' the primary
eledionL \Ve denote the tradinn of th" ,,(,te that the Dernocratic (non'
innllTtbentl candidate would rece!vc'
cratic candidate \\'ho is a Non"lncumbent),'
This conditon IS (he essence behind this definiton
causalty, and the difierence betwi'cn the adual vote dnd the likdv
whi1t he G1Ils 'Rubin'" \4udd" Hnlbnd bi1SC5 hb; "n (he work ol' :,eh..,;,
Lx}n:lld Rubin's 097-L \.v()rk nn the subj"ect \Vdt..: Hh'''':!
he ,,10.0 dks Ari',totk Hume. Mf),
INI:: e'xlt'nJ H"lbnJ's ddintion o ;l ,<;uSll
nt?"(t'SSJfY ;,;,Ct.."> nu
iUl,d lh*2>rs,
b.\' u(.,lng
N f"nUll(l t,,"x.k'n-
to a unii in.,! nnc to partH,n cft'>('t:>.
"!id non"YSl'em,k cllmpilnenb,
5ee and Kng 04<Jl rOl' detals o ths example. M"rc ! and IV (MI
"tana for tn .. -"treMmeot" anO "c'ontrn!" group 0f inf ,\ny two f.'xperiment;!ly
Ca usa lit Y and Causa! lnft?rence
\"uh:
t
1:1 this sllu;ltipn :;-..
Qt:.llo\v. \Ve ITlust y,pry ('a-fetul in
although they are obviau"ly COtmter t the
they mus he reasonabie dmi it should be possibl,: fUf the w
tua] 0\'ent to nave occurred under precisdy sta?d crcumstam:
key part uf defning the appnlpriate count"rfadual eondition is
fying what \ve are holding constan! "",hile \Ve aTe ['1'!An,m'
the \'aJue of the treatment variable. In the present example,
causal (or treatment) variable is ncumbencv status, and it
fmm "rKmnbent" ro "non-incumnent'" During this
We hold constant up to the moment the '-"'H''''''''
nomination dedslon-the relatve of the Demt>-
crats and Republicans in elections in ths district, the llature o
the nomnatton :'1(l('ess, tht: (lf the
triet, and the economk ilnd polticaJ
l70t cnnt.nl for nI he
and 01' tu'
else that folkHvs the party nomnauo!1.
Cm!ScqllCt1aS of our treatment
ot indude name
Si) lorth. H ,Ve dd
and hence
and as a
total. In faer,
could milke one
fect at 311.
4
f'vlore tlw causal effcd of in he rClUrth Dstnct
in Ne\v proportion of the vote re.::eived the Democratk
cand.id,e that is aHributable to
difference behveen
that \v!l become deilr <:.nnr'nv
admlnistered in fact ni in O COUise, the dedskm In (HIt: value ni an .'Yl,H"",,-
or" variable a (rcalmen! ,md the orher <l cntrd is ii th!s "",,,,,,,,,,,c
used al al!.
4 ,lon E1"t;r n983:34-36) has daimetl "he rtw'"nllrw'
counteractual tn
((",m!s for t""th Ih" ilppanmi eXiPWn;rOl,
Elster is
c'!mt.>r''i<'!n,, irrliporLlnt in
rh.'llce Elsk','" argument 15 mOr(' ('(lijen!. we thnk. as i) se! ni valuilble against
careJ",s$ use of lhan as d critique of their fundanwntal delnitonal mpm"
mnl.'e in causal rea,solng.
Ddining C1USi1nty 79
Causal Effect fm unit i) ,:;c ...
onlv in t1worv since in til1\' on" r,,'al
\Ve might observe eifhcr pr or hut oolh.
chis simple defintion of GlUsallty demonstrates tha! \Ve Ci1n
hope to know a causal effect fm certain. t:\,Iland 09So} refers lo
problem as Ihe fundamental l:aw:a! ,2nd it is n-
n'"'l'1>':I17,'H problem since no I1Mtter hw the
no matter ho\<: much data \Ve no matter !ww pt'rcepti\'c
obsern,rs" no mi1tter lIow dili;t'rlt the reseafch and nn
n,atter hmv muen control \\0 we will never knovv
for certain. most nf the empiric,11 issues o
desgns !ha! \ve discu:s in ths book nvolH' this
problem, and mosl uf OUf constitutc aHcmprs to
it.
Our working definition o differs fmm Holland's, sinee in
,\i."rtion 2.6 we han: that soda! sdence needs to parti-
tite world into systematic and nonsvsternatic components, and
detinition dc'{;'s not make th...<; disunction lo see the
oE this partitioning, thnK abut !'"ha! \.vould if \Ve
rtCnm he! CJ98 eiection campaign in the Four!h District in Nev"
,vith a DemocnHic ncumbent and ,A
sHgh!!y different total vote would fea-
tures o e]ectlon
canojda!e's rall\" or on election
We can therefnre
rhe v,llues of the [k'molratic
this same eiecton.
OH id en-
featurt"S might indude l
Uf positon on an ssue. n
had \.veilther during one
or th results oi some investigatlve
n ":ariabJe that would
vote acmss hypothetical replications o'
:-:- Vie can for district 1 t'tv h4"" Tf h'1 the
'Th(' reason for this lS thM Ho![md j" a sta!sbda!1 who conws dose lo
f'",'"'''''''''',''' 2" \vhich is in seccrn
L,,"t"'<'nti'cm ut' tu the rm+,[,,'f'I1 (d CiltlSill
hui ibb b
dUh),rent unib ti'> ;.:.,(,h:l' thc' Pr, f"t'taining tne
definition nf ctluSdHty in nne, in his
tvhereas ours O\'e"r oI'
tne experimt.>nt fur just a single un!t Holland 19S;947).
I
,
;\5 rott:d (S()C ,t ':), t
\'<Jliablc" :nCL' it hilS il
'-'<inaNes not
Demo(Tiltk candidate as
votes that ""oHld be rcceived in
era tic nonncumbent as
V\'t" nm ..: ddim,' the miliom owsaf ior dstrict 4 ,1') the
bC!\Yt'en thest? t\Yo r,mdom variables,
wc switch notation from district 'f LO unt :
is a causa! dfect that vares 0\'\'1'
but Iso m,mv
ff \Ve could obser\'\' ,,\,oc
{,ict .f al he 5ame time,
DnlOcrntic incumber1l'
rcalized cmsa!
we ca11nol
the causal effed in
urwl'serued reaHzi:ltinn of the randm causa! effect 3.1
nther ,Kf('5S m<1n)' of the same
tion in dstrkt 4 lNith d Democratk incumbent,. and anoss manv
!hetkal of same dectinn but w;1h a Democriitic
renli7ed ciiusal dfect becnmes a nmdom
il makes our definihon of
iltic
As in mur<' d,'ail in 22,. this m (ontu"ing. A "'ran-
dmn vari."b!c' cpnl;ins S<lmc and t'lms is nHt
\rn"""" .... " this H".,","""_'
peHn FGll10mic ur tlw um:erLlin lnSlUelKC nf ,1
change in ",lector.1l Systt'HL Tlw !O (lar repreSt'ntatcnl 15 tlMi e"eh of hese
"random" procesl,cs have systemalc :md pmbabllistic components,
Definin::,
, '
E"1fen: .. rIH ..<l!1:1 ,JfhJ \'arT,l.H(,e:-; an,,'
'-.:f {")nd0111 \:arlables in 2.2}, it
us to plrtiton a inferenct' pwblem into
tic c"mpOlwnts, Althnugh mi1l1V
,'ar.1bl..:' be o( interesL the most reln':mt fpr dur run,
ex.
u1T
r'le s w ?l/can (dH:"i[
bV this, v,'e return lo our NcV\' Ymk dl'ctlon
tnat th:' \'ariable rdcrs to he vote r,Ktion r<:cclved
nUBlO\'r
the Democrat (inm1bent ur nunncumbentJ iKt\ISS "
replications (,f the Silmc clcdnn. vVe dcflh'
o this random \utc imdon
cffcc! of ncmnbenc\' n unit i is ti
fffect tiJf unir i)
. .l ce
v,,'here in the first Hne uf this eqmttloH, 1) (beta) lo this
causal effect. In tle second 'we indiGlte har tbe mcan
[O unit is just the mean '-<llue) of tite randm
nd in the [hiTd and fourth Unes \ve show no\\' ro cakul,e nI:' mean,
The last Hne 15 ;mother ,vay vvriting thc diference in he mean!:' (,l'
lhe t\\'o sets oE e!ectons, <1\'cr;ge ,,( lile dffcn:na!
behvt:ell t\VO random variables the dflen:no: o tht:c
To summarize in words: rhe (i1!iSi,l is [he bdaten the
temafic component of obsenlfltms madc whcn lhe e:rplanaf"ry nniablc
82 . Causality ilnd Causa!
,"1ftC ;;lue anJ I1::'
u{lr}tibl{ L?! {ohJther ;\"7lru.,',
'fhe 1.1st lnt: or' 3.3 i5 similar lO }, 1, dnd as
rhe Fundamental o C:ms"l Jnference sUB exists in this
,,(ion. lndeceL the this v<av IS 0ven m,lre
ble because t:ven if we could gd amund the Fmblem
a reaHzed ca1.lsal we would still have al! the usua pmblems
inference, induding the pmblem of sepRrating out systematic
!1onsystematic cOHlpllnents of the fandom causal eHect From he
\Ve use HoUand's phrase, the Fundamental Froblem uf Causal
,-"nce, t tefer h lht' (ha! he idc.'lltified a:, ereN tE to rhese
dard 01' vvhch \\'0 llave added to his
In the hox on pagt' we prov;de a more genera! notat:ion 1m
V.'m prove useful thrtlughout tlw !;'st o ths hook.
oher of thesc 1'0ndom causal effects
he nf inteF'st in v"riol!" cirnlmsti1nn's, Fur wis
kno'.\' {he varancc in the (rt'lizcd) causal effeds uf '
status on Democratic vote in unt i, as \vith the variance
he vote itsejf that we described ln equation 23 in section 2,6, To
latE' the YilrialKE' oE the causal we
(vari;,U1ce of the G!Usal effect in unt i1
in fO tl)(.,
varianct> , ), new ncumbenis
,,'sh LO know the variation in the Glusal dice! oi incumbene)! so
can hnw 'wJ!l be to that of ,.,ro""',,
from
underst,md that Ihis vMlanu.: in
o' he ,',>,orld and is not
Ji
\Ve our precise definitjon of in section 3,1.
sume of the in that 5c{.-:-tion are subt1e and quite soprusticated,
iHustrated oul' \yith a running example fron':
quantitative i\?search. This heJ''Xcd as communcaie tile con.,
\ve wished to stress '\vlthout aIso having ro aUend to the
detilil 2nd that L good qualitativ.l:
rescaTe;', In ths our ddintion oE cilus,,11-
ity again, but this tirne vta a quaHtative exarnple.
Potical scientists \vould lcaro a lot if they could rE'run history 'l\tith
everything consh'mt save for one investigator-controlled explanatory
Detlning C'lUsaJitv ' 1'3
, F\.''I'( une \)f tht} l'lucshcH1> th.;Jt in""
plitics and has to cid '\\'Hh the
particubr law or passc,; a tax biU hal is 1n-
to have a particubr to xlrtieular invest-
increase revenue by a certain arnounL and Cl'!1sumption
Does it this dfect? V'le can ot-sene wh,lt after
, is to sce if the ntended o.lflscquences ap:H'ur; bui even
do, t is never certaln that they rC5ull (rom the 1<1\V, The change
.. polC)l might hav\': hilppened lf we could rerun
'wth and 'wthout the rte\v reguiation" \Ve ,yould h,we
more in estimating the causal E'Hect of t.his la\\',
we Gmnot do !his, Hut he l.\'ll research tC)
us dl1 approximate ans'wer to our
nm\' rhe !oHowlng extended
In the vVi1ke (lf the coHapse uf the Soviet
in rhe ex-Soviet republics and in Eastem
uted ne\\' governmental fonns.
realize-in a political they dre ntroducing
constitutlons, nmsttutions that chey wil! have the intended
oi creaung stable democratk One uf the consttutionnI
is betvveen nnd p1'esdential iorms ni
V'/h1Ch 1S more lkely tn lead lo il srabl' :.; the
of consderable debate among scholrs in the feld (Um:i
'tz 19L/:'; Ujphart '!'}93), The debate is not lile leas! be-
(lf t1e numerous of svs-
. fems and the of that migh!
and interact 'Nith Hs choice sw:h as the nalure 01 rhe elec-
Ir s not om ro a !horough o
lilese choices bu rather a of the choice in
order (O define a causal effec! in the context of this example,
,'>'e highlght the distinction between systematc and \1on-
svstematic of a causal effed .
. The debate about presidental versus parlamentary' 1n-
irolve:s varled fearures of the two s\'sterns. We wiH focus on two: the
extent to \vhich each represents the varied inte1'ests oE the dH-
zenry and encourages strong and dccisve The is
th,:t pa1'liamentary do ) better of n:prcsenting the tull
rmge of sodetal group5 nd nterests in the 51nce there
aH' many seats to filled, and can be fmee! repre-
St,'ntath.f.:s clte'cred fmm v,1rnus gnmp5 !n cnntrdst, the all-or-nothinf;
Chari:K1er 01 presdentia! mean:; (hat som(' groups \vin feel left
out of tIte govemment, be disaffected, ,1nd cause greatcr in.<;tability. On
th,,> othel' hand, parliamentary systems---especiaHy if they adequately
represent th!.' fun range of social groups and interests--arE' Hkeiy to be
84
id
h)\), (\H1 h:,Jd Jrh:1
JI1\.' purros,,' uf this ",cdion 15 to d precise ,"-I'!.,."t-i
a causal dfe'ct Tp d" S,L hat we could ns!itute a
degree o demoCTalic stablty {perhaps ,1I:'1ual survivilJ or
dernocracy, attempted mups, Of other indcators o instablity},
'he Silnie (ountry and dI the samc time, nsttute a presidential
aiso ts stablity over the sm1\C period \vith the Sil
sures, The /'cal::ed mus,,! would be tht: difference bdween
gree of observed undeT
a The
m;:my
o the realized
he av
nf this pmblem to
out ilnd Jeavcs the lT\t'dn causal dfect to nelude onlv ""ch-" .. ,
tures. Svsternatic ft:atures include indedsivencss
",,,,,,,t,'n-, or disaffection among minorities in a
temnJic tealures m;;1 ndude t11e sud den ni l president
hnws the nto chaos, TIw taller event would not
it \-','ould appear in one
o t:he
(me oi these
,,,eh,,,,,,, would be nlercsled in ts mean causal dfect on
stabiltjt; however, his om: one
uf this Given this situiltion, pol:itkal
may be interested in more than the causal dfecL They
\ds11 ro understand tvhat tite maxmum and minimum causal
or at least the paritmec oE the causal might be. For
mav be tlm! reduces dcmocratic stabilitv on in
ciis!ndions JR i hemsdv('s dvhated. Jfgue lha! ,1
can du ti And that
nn"<l,:w,r iD be .,1 q,'m:mc
o svsk'm, On tllC other h,1i1d, the \'uhwr;;bliy o
tu !he' vganes of tiw Jwalth ami persontlity oi ,; single individual s a
tic efk"Ci hal r;&;e; the likt'lihood th; j(1fHi' nOl1s}'stl'matk featufe 'wll ,'pF'Ii'ar,
in tl1jS it D1av be thdt
noliticdl le"der;.; \'voukl h, chousc 3rt o."\tlon h"t
r sHghtl
v
\'H) but h ..1S a Icnver \'ar1'-
<-,' 'and thU5 minirrd:lcs tht' i.:hance ni a
.1.1. \l,/e defincd in of ?l
-is lhe diffcrcnce bd"'l.'ert the
variable \\--!len t!w
In this sedion, we use (lur ddnitinl! o ", ro ' ___ . '{,
dnd appi1rcnHy (()mpllCi1tmg H.1ea:-, \'V\'
pv nrher abou '\:allsal
dl'hJ
"fe
a b t11c diec! \1f " lWW
on n:ducng the Unted curren! acccmnt defcit \vth
lo OUl' definition uf the eHect herc is
curren! aC(Cllmt defdt \vith thc taA
... ",rp(1 to the same si!uation (ilt the time and for
me v/ith the exo.:,ption th<'lr the trealy ,vas not in effecL
causal mechanism her' would indude, in tum, the
and rntification of lhe tax ul the event,
mee!ings of the r(:le,'ant actors vvith: majo: multination,11 , ,a
actions ro reduce the!r total iax
(such as its transfer pricing mIes Uf mO\'tng manubduring
plants between countresJ, further actons bv Giher _ and
workers to lnke 01' the rnovements ot .. c1nd labor, bt"
<, - h'" t'n-}l "'te'" on th\" "1"1,
hveen countries, and su un, unhl \,' ",e d , L,,, - ''', ",
,mee of payments Det'\ve-en he United Siates Japan. ,
From the standpoint of through WhlCh causahty
&n E'mphasis on causal mechanisms makes inhltive sense: an)' coher-
86 CausaIity and Causal lnferenct'
C,Jus.al of dc\in; ern
anah/ses. H has in dl1ferent forms, "pro.:ess
ng" ttvhich Wt' disellss in sectiuu "historie,,! anall'sls:' and
taBea case stuJies:' of the derails of w('ll-done case
involve iJent:ifying these causal mechanisms.
However, identifyng the causal mechanisms rE'(juires C<lUStl
enee, tlsing the methods dscuss{'d belcn\'. That is, to demonstrate
causal status ni ('aeh ptential linkage in su eh a positcd mecha
the \,>,ouid have ro define and rhen estmah' he causal
OUT more' oi ca
fUf ci)ch link in the ch,in ni causa! evens.
our ddinition uf prior lo Lhe id",!!
tion 01' causa! nlL'Ch,misms, Furthennore, there exsts in the
cia! sciences an infinity oE G1Usai between any hvo lnks in
chain o CHIS,! mechanisms. lf we that an
causes a dependent a "causal mechanisms" approach
re{luire 115 lo dentfy a lis o' eausallinks betvveen he two
This defnition would {liso us tn a series of caus<11
ages, to causdlit:v ror each pair 01' conseculive variables in
nd to the behveen anv tw of these
<1bles and the (lf' variables. This
al no time it
definition of causalitv rOl' au\' one Cduse and Ol1e
In our of the effect of a' , versus parliam
ano, in rurn,
couId monitor the attitudt'S or
differ under tbe tvvo experimental
oi rile under eaeh sj'stem.
he h;'Tothesized
under a ".,.'oc"",,,,,
[he plhieh a cause has its
for ,1 i.HId is a verv userul opcral'iol1aJ
procedure. ldentifying causa! mecnanisms can gl'le u$ .
nore over a tlieory by makng observatlons at l
Altemative Defintions oE Causality
t1t""\\" causal to \"<c should not
0:)11[115(' a d",fnip!1 oi causality wirh the ntm.:idintionaL albeit (,fb.:'n
. operi1tonal pmeedure uf idl'ntifying nwch,misms
''lv1ultipk Causalily"
Chark'S Ragin, in a recent v\'ork argul"S for il methodol-
many explan,nrv variahles dnd fe\\' e>bservatpn5 in order
(me can t;;lke nto account what he calls "multiple GlUSil'n." That
js, "The phenomemm under has altemiltive JdcTmi-
nants--\vhat \Hll OH4:'\1 tu as thte' ot nf
causes.'" This 15 tlw pmblem referred f\ ilS gen-
1982:1n In slruations cau-
these dutnors argue tha! Hie S<1me outcome can bt' GlUst'd
combinations ni difieren! variables, l
Under nmdHions in vvhich dHerent vdri"bles cm 1V
t.hese variables ha\'(' cilusdl status, is correct tilar some statlstL<,l
(or ndE'\'anl qualtative researen
investgator lo lw existence of "multipie
could fail to alert an
.. but
models can handle situations lke tbes,,' (50me o
discussesJ,
the fundamenta! features o "multipie are
compatible ,vith out' deHnition o Gmsillit!,. They are ,11so no
qUilntitti\'e Ihan resean:h. Thc de;) cont;lins no ne\\,'
reatures or fnr ccmsder the hy-
pothesis th<l! a level nt income
nona1 attainment nd educated
both 15 insuffident. In this case, ;ve need io compare
causal variable: respondents who h.nve high educdtonal attamnent
ano highly educated parents, the t\VO wllo have one but not the
other, and the ;TouP wjth mdther. concept oi cau-
sation" puts greater demands on our data since we no';- cat-
variable
)\\ 'c-r
f
th
btXi:1U5t:" diFtxi.rn fll\;:''lSSary and suffkient con-
3i'./,n,,,',u< ",hen we aJlrwv for tlw
NCllnsdcr the cLrn th;,.)t F>nf>t UHnrnUn1(dtJun surcr
Juhng c["jsis hhcEhOt)d oi This (J dart jt iJt.'D
tiCk>;; ,1 G\lh,1 Vilr,!:Jle (:><1()r (ommnn(i\linn) ,no ilSSHts h{ thi$ vilri;bl,' n.::n::;:;,vs (n ..,
probabilit\, of il outn:mw (w,u). It cann! be tr;ms!at('(l into a daim "prmt tl:w
J?ssaryani! conuitons [ur "",,1', however; i! is iITcducibly
(ji {lUT C<lUSal tYLH
definitlon of For our
income for the pt'r,,,,P1L al
cxperit:ncing c)ch Di the four conditions,
13ut what happens if different GlLlsal
vdlues I rhe dependen! variable" FUf e\mple,
\vhether o, not (me gradudted from
cilusal variable in a popula!ion o ',<'b",r,,'
both groups ((luId quite earn the Silme inconw
pendent variableL Om' n,ason might be that this
attendance')
nf irllme Of hose eduG.!ed and
earn" particular lt.'vel o income
who no educaton might
their {OUT VCM!> o dditYOH'
on thE' In ths situabon \vouldn't \ve lea lo conclua
educatlon" has no causal 011 ncome levpls
>-'.'no win become 1yorki2'TS?
OUT' definition 0f
che condirof1, In the
son
bad; in time tour
nstead nf jn
Uf her income "
levels uf income rOT this une
causal dfect in he 'Ve have imagined that
causal effcct is zero, But this croes no! me"n that "college
no on N that the average '
{1 j and (2) is 2em, In therc s
"tIte causal effl'Ct of (:nHeQc
no! define a causal diee! without al lcast teVO
tlons nect:! n! be rhe hvo Usted !'tere, but
of CdUS11 cond!ions ls lo
lIitte vd!h someone \vithmll '" but with thc same
ob seniority as the college graduate. In one sen;:;', this is u
since the non-L"ollege graduate wou!d have to do something for
nut
lhtil h3d a jnb
Put dlternatlve (\yunh:rt'cfual 1S effct
to that ni mme, wilh senoritv bdd
FailuTe lo hold :icniority consLmt in the e,lUsal mJi-
would any 1\,se.1[cl1 Jesign t,l yeld esHm"es uf pur i]rst
nsh;,)d uf !'tis revised ont', H the laHer wefe he
intluenced caus"l fadors do",s not
pmblematic T!'tt'
ver\' Hnmnn situatinns is h) define tlll'
makiflO un edeh eiluSill effet't ven; VVe demonstrilte in
b r
5 that researchefs need flor "all" causal on a
variable to pnwide estimates uf the Orle causal efft'ct of in-
(even t that lVt>n,," A resean:Jwr can hiell'; nn ,miv the
{}ne effeet nI interes. estdblish firm and tlwn m,)"e on t(1
chal nI;}\' be o nieres! (see :;,2 imd ,:::;:1). l'
bctwecn wh,,[ he refer,; to
\vhkh dffer when an
tn wltt'n it is dCcrC,1Sl.'d.
le 15 nter
tne CilU'i<li infllH::nce f x; tan \'drabltd un ' jil
dependent var.1ble!. tor (>ne has a!so lo considc'f wht:ther shitb lo
a V!U0 nf Xl rrm either dr0 ... :tion h.1v0 the same ctlllsequences or
y,. , f tht, causa! relariol1snip bet1.vcen Al
\ve kncnv ft 1S than detenninisHc \.ye vdth
pn'mise 0987: :;) th,, ni he cnm-
rn('t+tnd are nnt in
examirH:'tl tlcc-nunted t"nr ir l"ut
kx::t i1 uf intuffl\,,"lton on ev,,'ry ilnd
oc! cmbrh1,m of "al m.'!, ,,1' w expi;m,,(ory Filrblt>s., lhe wor1<:l stl! wnuki pn>-
cluero hes<' d,lta lo some probabi!"tk pro,,;;ss (,b defhwd in sec!i"n 2,6), Thb
1I
90 Causality and Causal Inerem::e
As ,111 (1f Lcbt'rsnn's point, imagine th,.t the Fourth
gression,11 Distrid in l\f;'w York had no irh"umbt:'nt in 1998 ilnd Ihil
Democratk candidate n.::ceivt'd SS per'nt of the vote.
would definE' thE' causal effc'Ct uf incumbenc',' as lhe incn:asc
vote if the vv'inning Democrat in 199B runs as an incumbcnt in he
e!edion in the vcar 2000, This cffed vvould bt, dsymmctrk" if the
sence of au inn;mbent i.n the subsequent e1edion (in ycar :::002)
the vote to rdum to 55 TIw eH0i:1 might be "asymmetric"
ur example, tht.' ncmnbtmt Dcmocrt raised moneyand ntpro\'t:'d
Demtx:ratic campdign as ,1 fesuIt, if nn incu
v,:ere in 2002, he Democrabc canddilte might
than 55 of the voh::,
UebersoI1's b dever "ud ver)!
\'icw" nis does nt cCHls!itute a
pHes
causal
derined 011 the bass of E'vents occurring
in the 1998 election in the fourth Disfrkt in York Om d
s the difference in the cmnponent of the vote in t!lis d
\vith an ncul1.bent in election and wthut "n incumbent in
same election, and distrkt.
In contrast, LieberS\)!,; involves no hypc)thetical
aud therefore Cdnnot be d causal detlnitiou,
onlv '.vhat would actuaHv o('cur if the
tv,' real t:,lections .!Tom t
to nonincumbent in hvo other lections. empiriGll
this wuld nvolve mrmerous problem" of infenmce, We
cuss manv o these problems of causal inference in chapters 4-tL in
present \ve might ask \vhether the estimaled
larO'er on1v beeause we failed to account for a number of
l , h '
registerL,<1 ctzens in fourth District Dr, did t e surge m su
for irte Democrat in the elec'1ion in \vhkh <he or he ,va" an iDcum
n as d
nr mlking tol(" ..lrn d/da tht'" S:lmt)
di 5cwnlifk rnt'erem:e th,1 we in !lis hOOK, However, hi\; apprtMch can "ti!
"<11uabli:: ,15 ,1 lo:m o foro1al heory bee secton it enbk'5 he
a imd Hs mpiic,'ltion; in a way tha! be muen more
out iL
01
than it shtHJld ..,' JiS(tlrded
tlw DemoCTdt lo"! dl' tirst elcci,'n:
us Lebers(1{l's con'pts \}[ "svmmetrical'" and "asvmmctri('al"
c;v are imDortant to 'in the cm1tt'xt ot in-rencc,
, , ,
, ,they slu,')t,lld l1t be confust'J with a thlretical detintlm u
in sechon 3,1,
fon ESTJ:'vJATf"<G
do Wt' avoid the FundamentdJ Prublem o( Causal lnfcnmce aBd
ih", problem l)t {!'Pm nons\'st.t,'mdtk \:omplJ"
The iuU nswer lo his question wiH consume bu!
we providc .In pvervf;w herc (1f whi 1" in crms f he rwo
'bIt: assumptons !hi1t enaHe tlS 10 arounJ fundi1mcntal
km 'n1('Sc dTe IiIJt \\'E' discuss n seclnn 33,1 )
and tDJlditional 3.3,2), l1ke any
cther attempt to crcunwent the Fundamental Problem uf Cusallnft'f-
enee" ahvays im'olve Sllmc untes!able ssumptons, H s he
viiit\' ni aH fesearchers tn make the substantive implications of this
" sppt in their r("Such extn:mely c!eilr and visible to read-
Causal inferences should no! appe<1f like , TIl\? ibsumptions
ami slmuld be wi!h wh,,!ever sde
cm be mustered, bul it mus! be
wth
yalu<-"S of our ,'ariabl" eaeh time-as a tTue ::;olu-
tinn lu !lit' Fundamental Problcrn o Causa! lnterence woukl
we can a!tempt to makc " second-bcst
experment in two different !!lit:; that are" Two mi!:>
,1fe hamogew?eus when fhr ,'({ues f [he
each mlit are Ihe same <rilen lur ;lt/rble itlkrs OH fl
oalue. (That is, and For example, if In' nbserve
(an incumbent) in district 1 ,uld X-::::: O {no incumbenO in district 2, <111
assumption (Jf unIt rneans that ,ve can US{' the observed
pmpnrnons nt [he votl' in !\vo sepdriltc districts fnr about thc
causal effect 3.. ,vhieh we dSSUlne is the same in both dstrids, Fnr a
dat1 set \vith unit s the that
al! units wth the ;,ame v,11u<:' (jf the variables have tl1.::'
5."lme expeeted v,1Jue of he depenJent var,lble, course., thb is unl)'
an assumption and t can be wrong: thc hvo districts might differ n