Estimating Trustworthiness

Should you trust a studys findings?,

and other questions to trouble your sleep!

What not to do!

Skim the methodology section and skip directly
to the findings and conclusions
Accept the findings and conclusions at face value
Assume that any published study is trustworthy
Ignore the studys epistemology
Impose the criteria for trustworthiness used for
one type of research for another.

Terms and Concepts

Overall Term

Postpositivist

Truth value

Internal validity

Consistency

Reliability

Applicability

Generalizability

Neutrality

Objectivity

Strategies for Strengthening Credibility

Problems of
credibility

Strategies
Postpositivist
Researchers Use

Strategies
Interpretive
Researchers Use

history & maturation

hold time constant

design studies with
short duration

trace changes over prolonged engagement

observer effects

rely on standardized
instrumentation

document & construct

random sample

persistent observation

(Threats to validity)

selection effects

history of fieldwork
prolonged engagement
member checks

random assignment to
treatment & control group

mortality

substitute one subject

for another

document changes in
group studied

Strategies for Strengthening Consistency

Problems of
consistency

Strategies
Postpositivist
Researchers Use

Strategies
Interpretive
Researchers Use

Could the findings be

consistently repeated if
the study were
replicated?

repeated samples

make the researchers

positionality explicit

Test-retest reliability
construct/leave an audit
use multiple researchers trail
to code data (inter-rater
document & construct
reliability)
history of fieldwork
prolonged engagement
member checks

Strategies for Strengthening Applicability

Problems of
applicability

Strategies
Postpositivist
Researchers Use

Strategies
Interpretive
Researchers Use

Are the findings

applicable to other
settings?

stratified random
sample

construct thick
descriptions

use of inferential
statistics

use member checks to

understand participants
interpretations of the
setting

Strategies for Strengthening Neutrality

Problems of
neutrality

Strategies
Postpositivist
Researchers Use

Do the findings reflect the Blind assignment to

researchers personal and treatment & control
theoretical biases?
group

Strategies
Interpretive
Researchers Use
triangulation of
methods, sources, and
investigators

masked coding of data

practice reflexivity

use of inferential
statistics

use peer debriefing

invite outside researcher
to do confirmability audit

Sowhat is good research?