Reliability

Summarized by
Moh. Kusen
Definition of Reliability
a measure of the degree to which a test gives
consistent results. A test is said to be reliable if it gives
the same results when it is given on different
occasions or when it is used by different people
(Richards, Jack C. and Schmidt, 2002)
Reliability refers to the consistency of a measure. A
test is considered reliable if we get the same result
repeatedly. Unfortunately, it is impossible to calculate
reliability exactly, but there are several different ways
to estimate reliability.
http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm
Types of Reliability
Equivalent form reliability (parallel form
reliability )

Inter-Rater reliability

Test retest reliability

Internal consistency reliability

Equivalent Form Reliability (Parallel Form Reliability )

Parallel-forms reliability is gauged by comparing two

different tests that were created using the same
content. This is accomplished by creating a large pool
of test items that measure the same quality and then
randomly dividing the items into two separate tests.
The two tests should then be administered to the
same subjects at the same time
http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm
Inter-Rater reliability

This type of reliability is assessed by having two or more

independent judges score the test. The scores are then
compared to determine the consistency of the raters’
estimates.
http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm

The degree to which different examiners or judges making

different subjective ratings of ability (e.g. of L2 writing
proficiency) agree in their evaluations of that ability.
(Richards, Jack C. and Schmidt, 2002)
Test-Retest reliability
To gauge test-retest reliability, the test is administered twice at
two different points in time. This kind of reliability is used to
assess the consistency of a test across time. This type of
reliability assumes that there will be no change in the quality or
construct being measured.
http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm

An estimate of the reliability of a test determined by the extent

to which a test gives the same results if it is administered at two
different times.
(Richards, Jack C. and Schmidt, 2002)
Internal Consistency Reliability

This form of reliability is used to judge the consistency of results

across items on the same test. Essentially, you are comparing
test items that measure the same construct to determine the
tests internal consistency. When you see a question that seems
very similar to another test question, it may indicate that the
two questions are being used to gauge reliability. Because the
two questions are similar and designed to measure the same
thing, the test taker should answer both questions the same,
which would indicate that the test has internal consistency.
http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm
Internal Consistency Reliability

A measure of the degree to which the items or parts

of a test are homogeneous, equivalent or consistent
with each other.
(Richards, Jack C. and Schmidt, 2002)
Ways to Reach Reliability
Do not allow candidates too much freedom
Write unambiguous items
Provide clear and explicit instructions
Ensure that tests are well laid out and perfectly legible
Candidates should be familiar with format and testing
techniques
Provide uniform and non-distracting conditions of
administration
Use items that permit scoring which is as objective as
possible
Provide a detailed scoring key
Train scorers
Identify candidates by number, not name
Employ multiple, independent scoring
(Hughes, 1996: 36-42)
 Factors Affecting Reliability
Student related reliability Test administration
a) Temporary illness reliability
b) Fatigue a) Street noise
c) A bad day b) Photocopying variations
d) Strategy c) Poor light
Rater reliability d) Temperature variations
a) Human Error e) Chair and table condition
b) Subjectivity Test reliability
c) Lack of attention to a) Timed test
scoring criteria b) Ambiguity of test items
d) inexperience www.yesdil.com