Principles of Assessment Validity & Reliability

Adapted from:
http://blog.wonderlic.com/reliability-and-validity-in-testing-what-do-they-mean

https://fcit.usf.edu/assessment/basic/basicc.html

Validity

Validity is more difficult to assess than reliability. There are various ways to
assess and demonstrate that an assessment is valid, but in simple terms,
validity refers to how well a test measures what it is supposed to
measure.
There are several approaches to determine the validity of an assessment,
including the assessment of content, criterion-related and construct validity.

Type of Validity Definition Example/Non-Example

Content The extent to which the
content of the test matches
the instructional objectives.
Criterion The extent to which scores on
the test are in agreement
with (concurrent validity) or
predict (predictive validity) an
external criterion.
Construct The extent to which an
assessment corresponds to
other variables, as predicted
by some rationale or theory.
A semester or quarter exam
that only includes context
covered during the last six
weeks is not a valid measure
of the courses overall
objectivesit has very low
content validity.
If the end-of-the-year math
tests in 4th grade correlate
highly with the statewide
math tests, they would have
high concurrent validity.
If you can correctly
hypothesize that English
Language Learners will
perform differently on a
reading test than English-
 speaking students (because of
theory), the assessment may
have construct validity.

Reliability

If you are focusing on the reliability of a test, all you need to ask isare the
results of the test consistent? If I take the test today, a week from now
and a month from now, will my results be the same?

If an assessment is reliable, your results will be very similar no matter when

you take the test. If the results are inconsistent, the test is not considered
reliable.

Type of How to Measure

Reliability

Stability or Give the same assessment twice, separated by days,

Test-Retest weeks, or months. Reliability is stated as the correlation
between scores at Time 1 and Time 2.

Alternate Create two forms of the same test (vary the items
Form slightly). Reliability is stated as correlation between scores
of Test 1 and Test 2.

Internal Compare one half of the test to the other half.

Consistency

If a test is valid, it is almost always reliable. If a test is reliable, it may

not provide a valid measure.