Regression Discontinuity Designs

RegressionDiscontinuityDesigns
EITM2011
Chris Berry
ChrisBerry

TodayssAgenda
Today
Agenda
II.
II.
III.
IV.

MethodologicalOverview
Methodological
Overview
DiscussionofExemplaryPapers
PractitionersGuide
ii
G id
Stataexamples

RegressionDiscontinuity

METHODOLOGICAL OVERVIEW
METHODOLOGICALOVERVIEW

Important References
ImportantReferences
RDmethodswereintroducedbyThistlewaite andCampbell
(1960) See Cook (2008) for an historical perspective
(1960).SeeCook(2008)foranhistoricalperspective.
Recentapplicationsinpoliticsincludeanalysesofthe
incumbencyeffect(Lee,2008),electoralcompetitionon
policy (Lee Moretti and Butler 2004) the effect of gender
policy(Lee,Moretti,andButler2004),theeffectofgender
onlegislatorbehavior(Rehavi nd),thevalueofaseatinthe
legisalture (EggersandHainmueller 2009),theeffectof
mayoral party ID on policy (Ferreria andGyourko
mayoralpartyIDonpolicy(Ferreria
and Gyourko 2009).
Recentimportanttheoreticalworkhasdealtwith
identificationissues(Hahn,Todd,andVanDer Klaauw,2001),
optimal estimation (Porter, 2003), tests for validity of the
optimalestimation(Porter,2003),testsforvalidityofthe
design(McCrary,2008),bandwidthselection(Imbens and
Kalyanaraman 2011).
GeneralsurveysincludeLeeandLemieux(2009),VanDer
General surveys include Lee and Lemieux (2009) Van Der
Klaauw(2008),andImbensandLemieux(2008).
Todaysdiscussionborrowsfromallofthem

RegressionDiscontinuityBasics
g
y
ThebasicideabehindtheRDdesignisthatassignment
tothetreatmentisdetermined,eithercompletelyor
,
p
y
partly,bythevalueofanassignment(orforcing)
variable(thecovariateX)beingoneithersideofafixed
threshold.
Assignmenttotreatmentbycovariatevalue,assignallunits
withXi c totreatment

RD
RDestimatesthelocalaveragetreatmenteffect(LATE)
estimates the local average treatment effect (LATE)
ofthetreatmentatX = c
RDislikearandomizedexperimentatthecutpoint X = c
TheRDdesignisgenerallyregardedashavingthe
h
d
ll
d d h
h
greatestinternalvalidityofallquasiexperimental
methods.Itsexternalvalidityismorelimited,sincethe
estimatedtreatmenteffectislocaltothediscontinuity.

RDScatterplot:PositiveTreatmentEffect
Outcome
(Y )

Cutting Point

Assignment Variable (T)

RDScatterplot:NoTreatmentEffect
Outcome
(Y )

Cutting Point

Assignment Variable (T)

RD Setup Formally
RDSetupFormally
Startwiththeusualpotentialoutcomesframework
TheunitleveltreatmenteffectisYi(1) Yi(0),whereYi(1)isthe
outcomethatwouldoccuriftheunitwereexposedto
treatment; Yi(0)ifnot.Thefamiliarproblemisthatwecannot
treatment;Y
(0) if not. The familiar problem is that we cannot
observethepairYi(0)andYi(1)simultaneously.
AssumeabinarytreatmentvariableTi andacontinuous
assignment (or forcing) variable Xi
assignment(orforcing)variableX
Inasharpregressiondiscontinuitydesign:
Ti =1{X
1{Xi c}
c}
Wherecisacutoffsuchthatallunitsabovereceivethe
treatmentandallthosebelowdonot

Defining the Causal Estimate

DefiningtheCausalEstimate
Underfairlyweakassumptions,theeffectofthe
treatment is identified by:
treatmentisidentifiedby:
lim E Yi | X i c lim E Yi | X i c
0
0
lim
li E Yi (1) | X i c lim
li E Yi (0) | X i c
0
0
Whichis:RD =E[Yi(1) Yi(0)|Xi=c].
Thisistheaveragetreatmenteffectatthecutoffpoint,a
particularLATE.
This inference is possible because the functions E[Y(1)|X]
ThisinferenceispossiblebecausethefunctionsE[Y(1)|X]
andE[Y(0)|X]are(assumedtobe)continuousinx.
p
q
y
g
Someextrapolationisrequiredbecausebydesignthere
arenounitswithXi =cforwhomweobserveYi(0).

RDComparedtoOtherObservationalMethods
Rememberthetwoassumptionsneededtoidentifytreatmenteffects
b h
d d
d
f
ff
fromobservationaldatausingregression/matching(Kosukes slide#31):
overlap andunconfoundedness.
Yougetthesetwoassumptionsforfreeinarandomizedexperiment.
You get these two assumptions for free in a randomized experiment

Ingeneral,unconfoundedness isnotconsideredaparticularlycredible
assumption,andtheothermethodswerestudyingthisweekcanbe
thoughtofaswaysformakingitmoreplausible.
RDisspecialinthefollowingways:
Unconfoundedness issatisfiedbydefinition.WhenXc,Tisalways1;when
X<c,Tisalways0.AfterconditioningonX,thereisnovariationleftinT,so
it cannot be correlated with any other factor RD is a special case of
itcannotbecorrelatedwithanyotherfactor.RDisaspecialcaseof
selectiononobservables.
Bycontrast,overlap isclearlyviolatedinanRDsincewecannotever
observetreatmentandnontreatmentforthesamevalueofX.The
continuityassumptionisrequiredtocompensateforthefailureofoverlap
i i
i i
i d
f h f il
f
l
(moreonthisbelow).
NoteasubtlebutimportantdifferencefromIV:theassignmentvariableis
p
j
allowedtohaveadirectimpactontheoutcome,notjustonthetreatment,
butnotadiscontinuousimpact.
Theestimatedeffectoftheassignmentvariableontheoutcomemayreflectatrue
causaleffectoraspuriouscorrelationduetocorrelationwithunobservables.

RDComparedtoanExperiment

RDisoftendescribedasaclosecousinofarandomizedexperimentorasa
localrandomizedexperiment.
Coughey andSekhon argueagainstthisconceptualization,forreasonswewill
seelater,butitisworthunderstandingwhytheanalogyismade
l t b t it i
th d t di
h th
l
i
d
Consideranexperimentinwhicheachparticipantisassignedarandomly
generatednumber,v,fromauniformdistributionovertherange[0,1].
Unitswithv0.5assignedtotreatment;unitswithv<0.5assignedtocontrol
g
;
g

ThisrandomizedexperimentcanbethoughtofasanRDwheretheassignment
variableX=vandthecutoffisat0.5.
Becausetheassignmentvariableisrandom,thecurvesE[Y(0)|X]andE[Y(1)|X]are
flat And we know that they are flat
flat.Andweknowthattheyareflat.
TheATEcanbecomputedasthedifferenceinthemeanvalueofYoneithersideof
thecutoff.
Becausethefunctionsareflateverywhere,theoptimalbandwidthistouseall
the data
thedata

NotethattherearetwomainwaysinwhichanRDdiffersfromarandomized
experimentinactuality
ThefunctionalformofE[Y(0)|X]andE[Y(0)|X]neednotbeflat(orlinearor
monotonic)andmaynotevenbeknown.
t i ) d
t
b k
Itmaybepossibleforunitstoaltertheirassignmenttotreatmentbymanipulating
theforcingvariableinawaythatisnotpossiblewhenitisassignedatrandomby
theinvestigator.

ExperimentasRD
Outcome
(Y )

Cutting Point
0.50

Assignment Variable (T)

(Random)

EstimationBasics1
Wehavenowdefinedacausaleffectasthe
differenceoftwofunctionsatapoint.Howdowe
estimate that? There are 3 general approaches
estimatethat?Thereare3generalapproaches.
Approach#1:Comparemeans
In
Inthedata,weneverobserveE[Y(0)|X
the data we never observe E[Y(0)|X=c]
c],thatisthere
that is there
arenounitsatthecutoffthatdontgetthetreatment,
butinprincipleitcanbeapproximatedarbitrarilywellby
E[Y(0)|X c ].
E[Y(0)|X=c].
Thereforeweestimate:
E Y | X c E Y | X c
Thisisthedifferenceinmeansforthosejustaboveand
belowthecutoff.
Thisisanonparametricapproach.Agreatvirtueisthatit
p
pp
g
doesnotdependoncorrectspecificationoffunctional
forms.

Estimation Basics 2
EstimationBasics2
NotethatIsaidinprinciplewecanestimatemeans
arbitrarilyclosetothecutoff.Inpractice,thisdependson
y
p
,
p
havinglotsofdatawithin ofthecutoff.Supposeyou
dont.
Approach #2: OLS (with Polynomials)
Approach#2:OLS(withPolynomials)
TheoriginalRDdesign(Thistlewaite andCampbell1960)was
implementedbyOLS.

Y T X
where isthecausaleffectofinterestand isanerrorterm.
Thisregressiondistinguishesthenonlinearanddiscontinuous
jumpfromthesmoothlinearfunction.
OLSwithonelinearterminXisseldomusedanymore
b
becausethefunctionalformassumptionsareverystrong.
h f
i
lf
i
Whatarethey?

Estimation Basics 3
EstimationBasics3

Supposetheunderlyingfunctionsarenonlinearandmaybeunknown.In
particular,supposeyouwanttoestimate

Y T f ( X )

wheref(X)isasmoothnonlinearfunctionofX.
Perhaps the simplest way to approximate f(X) is via OLS with polynomials in X
Perhapsthesimplestwaytoapproximatef(X)isviaOLSwithpolynomialsinX.
Commonpracticeistofitdifferentpolynomialfunctionsoneachsideofthe
cutoffbyincludinginteractionsbetweenTandX.
Modelingf(X)withapthorderpolynomialinthiswayleadsto

Y 01 X 02 X 2 ... 0 p X p

T 1TX 2TX 2 ... pTX p

CenteringXatthecutoffpriortorunningtheregressionensuresthatthe
Centering
X at the cutoff prior to running the regression ensures that the
coefficientonTisthetreatmenteffect.
Commonpractice,forwhateverreason,seemstousea4th orderpolynomial,
thoughyoushouldbesurethatyourresultsarerobusttootherspecifications
(
(moreonthisbelow).
hi b l )

EstimationBasics4
OLSwithpolynomialsisaparticularlysimplewayofallowinga
flexiblefunctionalforminX.Adrawbackisthatitprovidesglobal
estimatesoftheregressionfunctionthatusedatafarfromthe
cutoff.
Theremanyareotherways,buttheRDsetupposesacoupleof
problems for standard nonparametric smoothers
problemsforstandardnonparametricsmoothers.
Weareinterestedintheestimateofafunctionataboundarypoint.
(Forwhythisisaproblem,seeHTVorImbens andLemieux.)
Standardnonparametrickernelregressiondoesnotworkwellhere
p
g

ThisleadstoApproach#3:LocalLinearRegression
Insteadoflocallyfittingaconstantfunction(e.g.,themean),fitlinear
regressionstoobservationswithinsomebandwidthofthecutoff
Arectangularkernelseemstoworkbest(seeImbens andLemieux),
butoptimalbandwidthselectionisanopenquestion
Aseriousdiscussionoflocallinearregressionisbeyondthescopeof
,
p ,
j
((1996))
thislecture.See,forexample,FanandGijbels
But,really,werejusttalkingaboutrunningregressionsondatanear
thecutoff.

RDPitfall:MistakingNonlinearity
forDiscontinuity
Consequences
Consequencesofusinganincorrectfunctional
of using an incorrect functional
formarepotentiallymoresevereforRDthanfor
othermethodswearestudyingthisweek
Misspecificationofthefunctionalformmay
generateabiasinthetreatmenteffect
Themostcommonsituationofthistypeiswhen
Th
t
it ti
f thi t
i h
anunaccountedfornonlinearityinthe
conditionalmeanfunctionismistakenfora
discontinuity
Eachofthe3estimationmethodsdealswiththis
issue in a different way
issueinadifferentway

Nonlinearity Mistaken for Discontinuity

NonlinearityMistakenforDiscontinuity
Outcome
(Y )

False discontinuity

Cutting Point

Assignment Variable (T)

Polynomial Regression
PolynomialRegression
Outcome
(Y )

Cutting Point

Assignment Variable (T)

LocalLinearRegression
Outcome
(Y )

Cutting Point

Assignment Variable (T)

CompareMeans
Outcome
(Y )

Cutting Point

Assignment Variable (T)

CompareMeans:SmallerBandwidth
Outcome
(Y )

Cutting Point

Assignment Variable (T)

Manipulation
Ifindividualshavecontrolovertheassignmentvariable,thenwe
shouldexpectthemtosortinto(outof)treatmentiftreatmentis
desirable(undesirable)
(
)
Thinkofameanstestedincomesupportprogram,oranelection
Thosejustabovethethresholdwillbeamixtureofthosewhowould
passedandthosewhobarelyfailedwithoutmanipulation.

Ifindividualshaveprecisecontrolovertheassignmentvariable,we
f i di id l h
i
l
h
i
i bl
wouldexpectthedensityofXtobezerojustbelowthethreshold
butpositivejustabovethethreshold(assumingthetreatmentis
desirable).
McCrary(2008)providesaformaltestformaniupulaiton ofthe
assignmentvariableinanRD.TheideaisthatthemarginaldensityofX
shouldbecontinuouswithoutmanipulationandhencewelookfor
discontinuities in the density around the threshold
discontinuitiesinthedensityaroundthethreshold.
Howprecisemustthemanipulationmustbeinordertothreatenthe
RDdesign?SeeLeeandLemieux(2010).

ThismeansthatwhenyourunanRDyoumustknowsomething
y
y
g
aboutthemechanismgeneratingtheassignmentvariableandhow
susceptibleitcouldbetomanipulation.

ExampleofManipulation

Anincomesupportprograminwhichthoseearningunder$14,000qualifyfor
A
i
t
i hi h th
i
d $14 000
lif f
support
SimulateddatafromMcCrary2008

Assessing the Validity of an RD

AssessingtheValidityofanRD
Itisimpossibletotestthecontinuityassumptiondirectly,but
we can test some implications of it
wecantestsomeimplicationsofit
Namely,allobservedpredeterminedcharacteristicsshould
haveidenticaldistributionsoneithersideofthecutoff,inthe
limit as we approach smaller and smaller bandwidths That is
limit,asweapproachsmallerandsmallerbandwidths.Thatis,
thereshouldbenodiscontinuitiesintheobservables.
Againthereisananalogytoanexperiment:wecannottest
whether unobserved characteristics are balanced but we can
whetherunobservedcharacteristicsarebalanced,butwecan
testtheobservables.Rejectioncallstherandomizationinto
question.
AsubtlepointintheRDcontextisthatafindinga
A subtle point in the RD context is that a finding a
discontinuityinobservablecovariatesindicatesaviolationof
thecontinuityassumption,notaviolationof
unconfoundedness which is satisfied by definition (More on
unconfoundedness,whichissatisfiedbydefinition.(Moreon
thisbelow)

The Role for Covariates in RD

TheRoleforCovariatesinRD
Inprinciple,covariatesarenotneededfor
p
p ,
identificationinRD,buttheycanhelpreduce
samplingvariabilityintheestimatorandimprove
precision
Thisisastandardargumentwhichalsosupports
inclusionofcovariatesinanalysesofrandomizedtrials

Addingcovariatesshouldnotaffectthepoint
estimateoftheeffect(verymuch).Ifitdoes,
there is a problem.
thereisaproblem.
Thewiderthebandwidththemoreimportantit
maybetoincludecovariates.

Graphical Analysis
GraphicalAnalysis
GraphicalinspectionisanintegralpartofanyRD
analysis.
3typesofgraphsshouldalwaysbeproduced,where
assignment variable is graphed against:
assignmentvariableisgraphedagainst:
1:theoutcome
2:othercovariates
3:densityofcases
3 d it f

1shouldshowadiscontinuity;2and3shouldshowno
discontinuity
Ifyoucan'tseethemainresultwithsuchasimple
graph,it'sprobablynotthere
Ifyouseeadiscontinuityin2or3,beworried
If you see a discontinuity in 2 or 3 be worried

BandwidthSelection

ForLocalLinearRegression

Bandwidthselectionrepresentsthefamiliartradeoffbetweenbiasandprecision

Whenthelocalregressionfunctionismoreorlesslinear,thereisntmuchofatradeoffsobandwidthcan
belarger.
g

Therearetwogeneralmethodsforselectingbandwidth
Adhoc,orsubstantivelyderived(e.g.,electionsbetween4852%areclose)
Datadriven

ForPolynomialRegression

Choosingtheorderofthepolynomialisanalogtothechoiceofbandwidth
Twoapproaches

UsetheAkaike informationcriterion(AIC)formodelselection:AIC=Nln(2)+2p,where2 (shouldhavea

h t) i th
hat)isthemeansquarederroroftheregressionandpisthenumberofmodelparameters
d
f th
i
d i th
b
f
d l
t
Selectanaturalsetofbins(asyouwouldforanRDgraph)andaddbindummiestothemodelandtest
theirjointsignificance.Addhigherordertermstothepolynomialuntilthebindummiesarenolonger
jointlysignificant.

Optimalbandwidthmethods(Imbens andKalyanaraman)
(
g
;
andLemieux))
Crossvalidationmethods(LudwigandMiller;Imbens

Thisalsoturnsouttobeatestforthepresenceofdiscontinuitiesintheregressionfunctionatpointsotherthenthe
cutoff,whichyoullwanttodoanyway

Inbothcases

Inpractice,youmaywanttofocusonresultsfortheoptimalbandwidth,butit'simportantto
testforlotsofdifferentbandwidths.Thinkoftheoptimalbandwidthonlyasastartingpoint.
Ifresultscriticallydependonaparticularbandwidth,theyarelesscredibleandchoiceof
bandwidth requires a substantive justification
bandwidthrequiresasubstantivejustification.
Inprinciple,theoptimalbandwidthfortestingdiscontinuitiesincovariatesmaynotbethesame
astheoptimalbandwidthforthetreatment.Again,followthepracticeoftestingrobustnessto
variationsinbandwidth.

FuzzyRD
Supposetheprobabilityoftreatmentchangesdiscontinuously
h
b bl
f
h
d
l
atathreshold,butnotfrom0to1.Thisisasituationfor
applyingFRD.
NotethatthefuzzinessinFRDcomesfromthechangein
h h f

f
h h
probabilityoftreatment,notfuzzinessaboutthethreshold

InsharpRDdesigns,thejumpintheoutcomeatthecutoffisthe
estimate of the ca sal impact of the treatment In a FRD design
estimateofthecausalimpactofthetreatment.InaFRDdesign,
thejumpintheoutcomeisdividedbythejumpinthe
probabilityoftreatmentatthecutofftoproducethelocalWald
estimate (equivalent to a local IV estimate) of the causal impact
estimate(equivalenttoalocalIVestimate)ofthecausalimpact.
(InSRD,thejumpisone,sothedivisionisinconsequential,but
thisdemonstratestherelationshipbetweenSRDandFRD).
TheimportantpointtorememberisthatfuzzyRDisnumerically
The important point to remember is that fuzzy RD is numerically
equivalent(andconceptuallysimilar)toIV(seeMostlyHarmless
sec.6.2)
AnadditionalgraphisneededwhendoingFRD:probabilityof
An additional graph is needed when doing FRD: probability of
treatmentbyassignmentvariableshouldshowadiscontinuous
probabilityatthethreshold

RegressionDiscontinuity

DISCUSSION OF PAPERS
DISCUSSIONOFPAPERS

DoVotersAffectorElectPolicies?
byLee,Moretti,andButler(LMB)

Motivation
Motivation:2fundamentallydifferentviewsoftheroleof
elections
Convergence:Competitionforvotesdrivescandidatestoseek
middlegroundpolicies,compromise(medianvotertheorem).
Voters affect policy choices of politicians
Votersaffectpolicychoicesofpoliticians.
Divergence:Votersselectcandidates,whothenenacttheirown
preferredpolicies.Voterselectpolicies.
Whichofthetwoweseeinpracticedependsonwhethercan
Which of the two we see in practice depends on whether can
makecrediblepromisestoimplementpoliciesthatarenotat
theirownblisspoint(crediblecommitmentsarefacilitatedby
repeatinteractions)

Thegoalofthepaperistoexaminewhichphenomenonis
moreempiricallyrelevantforUSpolitics,specificallyvoting
intheHouse

Theory (vastly simplified)

Theory(vastlysimplified)
Consider2parties,DandR
Rsblisspointis0,Dsblisspointisc(>0)
p
p
( )

TheprobabilitythatDwinstheelectionisP
IfDwinselection,policyxisimplemented;ifRwins,yis
implemented
P*representstheunderlyingpopularityofpartyD,orthe
probabilitythatDwouldwinifx=candy=0.Anincreasein
P*representsanexogenousincreaseinD
P
represents an exogenous increase in Dsspopularity
popularity
Whendx*/dP*anddy*/dP*>0,wesaythatvotersaffect
candidatespolicychoices
*denotedequilibrium
* denoted equilibrium

Whendx*/dP*anddy*/dP*=0,wesaythatvotersmerely
electpoliticianswithfixedpolicies.Thatis,anincreasein
P*doesnothingtotheequilibriumpoliciesoftheparties.
P
does nothing to the equilibrium policies of the parties

Estimating Framework
EstimatingFramework
Therollcallvotingrecordoftherepresentativeinthe
district following election t is
districtfollowingelectiontis
RCt =(1 Dt)yt +Dtxt
WhereD
Where Dt istheindicatorforwhetherDwon.Thatis,
is the indicator for whether D won That is
onlythewinningcandidatespolicyisobservable

Theexpressioncanbetransformedinto:
p

This simply
y parameterizes the derivatives from
prior slide as 0.
Italsoallowsanindependenteffectofparty,1.

EstimatingFrameworkContd
WecannotobserveP*soequation(2)cannot
be directly estimated
bedirectlyestimated
ButsupposewecouldrandomizeDt.ThenDt
wouldbeindependentofP*
o ld be independent of P*t and
and t.Then:
Then

EEverythingunderlinedinredcanbeestimated
hi
d li d i
d
b
i
d
fromthedata

WhyThisWorks
Theelectcomponentis
1 isestimatedbythedifference
is estimated by the difference in voting
records between the parties at time t
ThefractionofdistrictswonbyDemocratsint+1
y
isanestimateof
Because we can estimate the total effect, ,
of a Dem victory in t on RCt+1
RCt+1, we can then
net out the elect component to implicitly get
the affect component
p
RandomassignmentofDt iscrucial.Without it,
equation (5) would reflect 1 and that Dem
districts have more liberal bliss points

GraphicalEstimateofEquation4

20

Graphical Estimate of Eqn. 5

GraphicalEstimateofEqn.5

45
4

Graphical Estimate of Eqn. 6

GraphicalEstimateofEqn.6

0.50

Statistical Results
StatisticalResults

Robustness, Etc.
Robustness,Etc.
No
Nootherobservabledistrictattributeschange
other observable district attributes change
atthediscontinuity
Resultsholdformanyothermeasuresof
Results hold for many other measures of
representativevotingrecords
Resultsrobusttoallowingforvarioussortsof
R l
b
ll i f
i
f
districtheterogeneity
Results(smallaffectcomponent)stableover
time

RegressionDiscontinuityDesigns
and Popular Elections: Implications
andPopularElections:Implications
ofProIncumbentBiasinCloseU.S.
HouseRaces
byCaughey andSekhon (CS)

Basic Argument
BasicArgument
Closeelectionsarenotlikeotherelections
Strategicpoliticalactorshavestrongincentivestotargettheir
resourceswheretheywillhavethegreatestmarginalimpact

Thereisanincumbencyadvantageeveninveryclose
elections
Theincumbentwinsdisproportionatelyandhasgreaterfinancial
resources

Thisfinding,alongwithothercovariateimbalancesatthe
cutpoint,callsintoquestiontheLMBincumbency
advantageresultsand,moregenerally,theassumptionthat
outcomesincloseelectionsareasgoodasrandomly
t
i l
l ti

d
d l
assigned
NotethatCScritiqueLee(2008),notLMB,buttheimplications
for the incumbency advantage results in both papers are the
fortheincumbencyadvantageresultsinbothpapersarethe
same

TheLee(&McCrary)TestsforManipulation

A graph like (A) led Lee, and separately McCrary, to conclude that there is no manipulation.
However, (B) and (C) begin to suggest another story. Remember, the concern is with the
incumbent partys vote share, not the Democratic vote share.

DensityoftheAssignmentVariable

Key Takeaway: The candidate of the incumbent party is about three times more likely to win
election by half a percentage point or less than to lose by a similar margin. The density of this
variable appears to diverge rather than converge in the neighborhood of the cut-point.

Covariate Imbalance
CovariateImbalance
BasedoncorrectingsomeofLeesdataandaddingsome
newvariables,CSfindimbalanceatthecutoffinthe
following:

Democraticmargininthepreviouselection
thepartiesrelativecampaignexpenditures
1stdimensionNOMINATEscoreofthecurrentincumbent
whethertheDemocratic(Republican)candidateisthecurrent
incumbent
b
numberoftermstheDemocrat(Republican)hasservedinthe
U.S.HouseofRepresentatives
whethertheDemocrat(Republican)hasmorepolitical
h th th D
t (R
bli ) h
liti l
experiencethantheRepublican(Democrat)
CongressionalQuarterlysOctoberpredictionofwhich
candidate will win the race
candidatewillwintherace

CovariateImbalanceGraph

SensitivitytoBandwidthSelection

PotentialMechanisms
Notlikelybeoutrightfraud,becausesignificanceoflagged
voteshareisincreasingovertimeandwebelievepotential
for fraud has been decreasing
forfraudhasbeendecreasing
Controloverrecountsdoesnotappeartobethekey
becausetheyrarelyhappenandevenmorerarelychange
the outcome
theoutcome.
Butwedontneedanexplanationbasedonvotecounting.
Differencesbetweenwinnersandlosersinincumbency,
money political experience and other pre election
money,politicalexperience,andotherpreelection
resourcesareevidentfarbeforeanyvotesarecast,
counted,ormanipulated.
Thesedifferencescanbeseeninelectionsexpectedtobe
These differences can be seen in elections expected to be
closeexanteandinthosethatwereinfactdecidedbya
narrowmargin.
Thesefactscontradicttheideathatresources,
These facts contradict the idea that resources
expectations,andallelseshouldbebalancedintheclosest
elections.

Regression to the Mean Explanation?

RegressiontotheMeanExplanation?

Disturbingly,themostseverelyimbalancedcovariatesarethoseclosely
related to the partisanship of the current occupant of the House seat
relatedtothepartisanshipofthecurrentoccupantoftheHouseseat
thatis,thelaggedvalueoftreatmentandthesecondorderlagofthe
outcome.Onesuchcovariate,notshowninthebalancetable,isthe
generalpartisanswingamongallHouseracesinagivenyearrelativeto
the previous election In good Democratic years such as 1948 1958 1964
thepreviouselection.IngoodDemocraticyears,suchas1948,1958,1964,
1974,and2006,closeelectionsareoverwhelminglyconcentratedin
normallyRepublicanseats.Conversely,Democratheldseatstendtobe
closelycontestedinbadDemocraticyears,like1946,1966,1980,and
1994 Because the incumbent party is much more likely to prevail in close
1994.Becausetheincumbentpartyismuchmorelikelytoprevailinclose
races,closeDemocraticvictoriesaremuchmorelikelytooccurinbad
Democraticyears,andcloseRepublicanvictoriestooccurinbadyearsfor
Republicans.Totheextentthatbadelectionsforonepartytendtobe
followed by more normal ones this pattern suggests that the apparent
followedbymorenormalones,thispatternsuggeststhattheapparent
incumbentpartyadvantagemaybecontaminatedbyregressiontothe
meaneffects.

Lessons from LMB & CS

LessonsfromLMB&CS

Thisisacautionarytale
LMBarevery goodscholars.
Theydidalmosteverythingright
Theydonotdoalottojustifyfunctionalformorshowrobustnesstodifferentbandwidths

Remember,theLMBresultsforDemocraticvote(eqn.5)arenotimplicatedinthis
critique.Thisisthebestpartofthepaperanyway,IMHO.

Whatcanyoulearnfromthisexchange:
Trytofindproblemsinyourdesignbeforesomeoneelsedoesitforyou
Identifyandcollectaccuratedataontheobservablecovariatesmostlikelytoreveal
sorting at the cutpoint
sortingatthecut
point.Thismaynotbethecovariatesthathappentobesittingin
This may not be the covariates that happen to be sitting in
yourdataset.
Laggedvaluesofthetreatmentvariablearealwaysagoodidea.Inelections,thepartythat
currentlycontrolstheoffice.

Automatedbandwidthselectionalgorithmsdonotguaranteegoodresults.Theyare
g
g
g
y
justastartingpoint.
ForRDpurposes,whatconstitutesacloseelectionappearstobecloserthanthe
4852%bandwidthwidelyuseduptonow.CSgetmostoftheirresultsusing49.5
50.5%.
GiventhecurrentfetishwithRDinpoliticalscience,understandthatitisnotafact
ofnaturethatcloseelectionsarerandom.Rememberthiswhenyousee(orsetout
towrite)thenextRDpaperoncloseelections.

RegressionDiscontinuity

GUIDE TO PRACTICE
GUIDETOPRACTICE

Summary of RD Assumptions
SummaryofRDAssumptions
Thetreatmentisdeterminedatleastinpartby
e t eat e t s dete
ed at east pa t by
theassignmentvariable
y
Thereisadiscontinuityintheleveloftreatment
atsomecutoffvalueoftheassignmentvariable
(selectiononobservablesatthecutpoint)
Unitscannotpreciselymanipulatetheassignment
variabletoinfluencewhethertheyreceivethe
treatment or not
treatmentornot
Othervariablesthataffectthetreatmentdonot
change discontinuously at the cutoff
changediscontinuouslyatthecutoff

Internal & External Validity

Internal&ExternalValidity

The strength of the RD design is its internal validity,

arguably the strongest of any quasi-experimental design
External validity may be limited
Sharp RD (SRD) provides estimates for the subpopulation
with X=c, that is those right at the cutoff of the assignment
variable.
The discontinuity is a weighted average treatment effect
where weights are proportional to the ex ante likelihood
that an individuals realization of X will be close to the
threshold.
Fuzzy RD (FRD) restricts the estimates further to compliers
at the cutoff (more on this below)
You need to justify extrapolation to other subpopulations
(e.g., treatment homogeneity)

Threats to an RD Analysis
ThreatstoanRDAnalysis
TherearethreegeneraltypesofthreatstoanRD
g
yp
design
1. Othervariableschangediscontinuouslyatthe
cutoff

Testforjumpsincovariates,includingpretreatment
valuesoftheoutcomeandthetreatment

2. Therearediscontinuitiesatothervaluesofthe
assignmentvariable
3 Manipulationoftheassignmentvariable
3.
M i l ti
f th
i
t i bl

Testforcontinuityinthedensityoftheassignment
variableatthecutoff

StepsforSharpRDAnalysis
1.

Graphthedatabycomputingtheaveragevalueoftheoutcomevariable
overasetofbins.

2.

Estimatethetreatmenteffectbyrunninglinearregressionsonbothsidesof
the cutoff point.
thecutoffpoint.

3.

Thebinwidthhastobelargeenoughtohaveasufficientamountofprecision
The
bin width has to be large enough to have a sufficient amount of precision
sothattheplotslookssmoothoneithersideofthecutoffvalue,butatthe
sametimesmallenoughtomakethejumparoundthecutoffvalueclear.

Witharectangularkernel,thesearejuststandardregressionestimatedwithina
binofwidthhonbothsidesofthecutoffpoint.Notethat:Standarderrorscan
becomputedusingstandardleastsquaremethods(robuststandarderrors).The
p
g
optimalbandwidthcanbechosenusingcrossvalidationorothermethods.

Therobustnessoftheresultsshouldbeassessedbyemployingvarious
specificationtests.

Lookingatpossiblejumpsinthevalueofothercovariatesatthecutoffpoint
Testing for possible discontinuities in the conditional density of the forcing
Testingforpossiblediscontinuitiesintheconditionaldensityoftheforcing
variable
Lookingwhethertheaverageoutcomeisdiscontinuousatothervaluesofthe
forcingvariable
Using various values of the bandwidth with and without other covariates that
Usingvariousvaluesofthebandwidth,withandwithoutothercovariatesthat
maybeavailable.

Steps for Fuzzy RD Analysis

StepsforFuzzyRDAnalysis
1. Graphtheaverageoutcomesoverasetofbinsasinthe
caseofSRD,butalsographtheprobabilityoftreatment.
2. Estimatethetreatmenteffectusing2SLS,whichis
numericallyequivalenttocomputingtheratiointhe
estimateofthejump(atthecutoffpoint)intheoutcome
variableoverthejumpinthetreatmentvariable.

Standarderrorscanbecomputedusingtheusual(robust)
2SLS
2SLSstandarderrors
d d
Theoptimalbandwidthcanagainbechosenusingoneofthe
methodsdiscussedabove.

3 Th
3.
Therobustnessoftheresultscanbeassessedusingthe
b t
f th
lt
b
d i th
variousspecificationtestsmentionedinthecaseofSRD
designs.

EvaluatinganRDPaper
(P ibl Y
(PossiblyYourOwn)
O )

Doestheauthorshowconvincinglythat

Treatmentchangesdiscontinuouslyatthecutpoint
eat e t c a ges d sco t uous y at t e cutpo t
Outcomeschangediscontinuouslyatthecutpoint
Othercovariatesdonotchangediscontinuouslyatthecutpoint
Pretreatmentoutcomesdonotchangeatthecutpoint
Th
Thereisnomanipulationoftheassignmentvariable(bunchingnearthe
i
i l ti
f th
i
t i bl (b hi
th
cutpoint)

Arethebasicresultsevidentfromasimplegraph?
Aretheresultsrobusttodifferentfunctionalformassumptionsaboutthe
p
assignmentvariable
Forexample,parametricandnonparametricfits,differentbandwidths,etc.

Couldotherpossiblyunobservedtreatmentschangediscontinuouslyat
the cutoff (bundling of institutions)
thecutoff(bundlingofinstitutions)
Forexample,18th birthdaymarksadiscontinuouschangeineligibilitytovote,
butalsoeligibilityfordraft,sentencingasanadult,andlotsofotherthings,
whichmayormaynotberelevantdependingontheoutcomeinquestion

External validity
Externalvalidity
Arecasesnearthecutpoint differentfromcasesfarfromthecutpoint inother
ways?Dothesedifferencesmakethemmoreorlessrelevantfroma
theoreticalorpolicyperspective?

RegressionDiscontinuity

EXAMPLES IN STATA
EXAMPLESINSTATA

Regression Discontinuity Designs

RegressionDiscontinuityDesigns
EITM2011
Chris Berry
ChrisBerry