External Validity

The External Validity Question

Theory What you think

Cause Cause-effect construct

Effect
construct construct
 The External Validity Question

Theory

Cause Cause-effect construct

Effect
construct construct

Can we generalize to other persons, places, OR

times?
Program Program-outcome relationship
Observations Program Program-outcome relationship
Observations
What you do What you see What you do What you see

Observation Observation

Program Program-outcome relationship

Observations Program program-outcome relationship
Observations
What you do What you see What you do What you see

Observation Observation
 How Do We Generalize?
Model I: Sampling
Specified persons,
places, and times

Population
 How Do We Generalize?
Model I: Sampling

Population

Draw Sample Draw

sample sample
 How Do We Generalize?
Model I: Sampling
GeneraLize Generalize
back back
Population

Sample
 How Do We Generalize?
Model II: Proximal Similarity

Our
Study
 How Do We Generalize?
Model II: Proximal Similarity

Settings

Times Our People

Study

Places
 How Do We Generalize?
Model II: Proximal Similarity
Less
similar

Settings

Less Our Less

similar Times People
Study similar

Places

Less
similar
 How Do We Generalize?
Model II: Proximal Similarity
Less
similar

Settings

Less Our Less

similar Times People
Study similar

Places
Gradients of
similarity
Less
similar
 Threats to External Validity

Interaction of selection Maybe it is just these

and treatment people.

Interaction of setting Maybe it is just these

and treatment places.

Interaction of history Maybe it is just these

and treatment times.
How Can We Improve External
Validity?
How Can We Improve External
Validity?
Population

Random sampling
Sample
How Can We Improve External
Validity?
Population

Random sampling
Sample

Replicate,

Places
Our
Study
Replicate,

es
Replicate

T im
People
How Can We Improve External
Validity?
Population

Random Sampling
Sample

Replicate,

Places
Our
Study
Replicate,

es
Replicate

Ti m
People

settings

Use theory times Our people

Study

places