Reliability and Validity of Measurements

4/24/12

Click to edit Master subtitle style

Definitions

Reliabilityconsistent, reproducib le, dependabl e

Validitymeasures what it says it measures

4/24/12

Reliability

Measurement Error Reliability Coefficients Types of Reliability

4/24/12

Measurement Error

Observed score = true score + error Measurement Error = true score observed score Reliability estimates measurement error

4/24/12

Sources of Measurement Error

Systematic- consistently wrong in the same amount Random- chance rater measuring instrument variability in what you are measuring
4/24/12

Reliability Coefficients
True score variance True score variance + error variance

as error increases, coefficient increases coefficient ranges from .00 to 1.00 < .50 poor .50 to .75 moderate > .75 good
4/24/12

Types of Reliability: TestRetest

Get the same results every time you use the test Intervals between testing- long enough to avoid fatigue or remembering the answers but not so long that natural maturation occurs Intraclass correlation coefficient (ICC)

4/24/12

Another way to Test- Retest

Alternate forms: different versions covering the same content (SAT, GRE) correlation coefficient is used

4/24/12

Types of Reliability: Internal Consistency

Are all questions measuring the same thing? Split-half: correlation of two halves of same test (odds and evens) SpearmanBrown prophecy Cronbachs alpha: essentially an average of the all the possible split-half reliabilities, can be used on multiple choice. 4/24/12

Types of Reliability: Raters

Intra rater: stability of one rater across trials

Inter-rater: consistency between raters

4/24/12

Use ICC for both twotype s

Validity

Validity vs. Reliability Types of Validity

4/24/12

Validity vs. Reliability

4/24/12

Generalizability

External validity- the test is valid if used with the intended population The test is reliable if used in appropriate context and as directed for its given purpose

4/24/12

Face Validity

appears to test what it is supposed to measure weakest form ok for ROM, length, observation of ADLs

4/24/12

Content Validity

covers the entire range of the variable and reflects the relative importance of each part based on expert opinion, needs to free of cultural bias Test of function- 20 questions on brushing your teeth, 1 question each on mobility, bathing, dressing VAS vs. McGill Pain Questionnaire
4/24/12

Criterion-related Validity
target test compared to gold standard

concurrent

predictive
examines whether the target test can predict a criterion variable

target test is taken at the same time as another test with established validity

4/24/12

Construct Validity

ability of a test to measure a construct based on a theoretical framework what would you include for a test on wellness?

4/24/12

Ways to establish construct validity

Known groups Convergent comparison Divergent comparison Factor analysis

4/24/12

Remember

The reliability and validity of a test measurement is not the same thing as reliability and validity of a research design.

4/24/12