AKLAN STATE UNIVERSITY

College of Industrial Technology

TEACHER EDUCATION DEPARTMENT
Kalibo, Aklan

IMPROVING A
CLASSROOM-BASED
ASSESSMENT TEST

KENNETH KIM L. VILLOCINO

Discussant
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Popham (2011) suggests two

approaches to undertake item
improvement :

 Judgmental Approach
 Empirical Approach
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Judgmental Approach
This approach basically makes
use of human judgment in reviewing
the items.
 Teachers
 Teacher’s peers or colleagues
 Students
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Teacher’s Own Review

Five suggestions for the teachers to
follow in exercising judgment(Popham):
1. Adherence to item-specific guidelines
and general item-writing
commandments
2. Contribution to score-based inference
3. Accuracy of content
4. Absence of content gaps
5. Fairness
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Peer Review
Time is provided for this activity and it
has almost always yielded good results for
improving tests and performance-based
assessment tasks.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Review Checklist
a. Do the items follow the specific and general
guidelines in writing items especially on:
 Being aligned to instructional objectives?
 Making the problem clear and unambiguous?
 Providing plausible option?
 Avoiding unintentional clues?
 Having only one correct answers?
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

b. Are the items free from inaccurate content?

c. Are the items free from obsolete content?
d. Are the test instruction clearly written for
students to follow?
e. Is the level of difficulty of the test appropriate to
level of learners?
f. Is the test fair to all kinds of students?
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Student Review
The judgment is based on students’
experience in taking the test, their
impressions and reactions during the testing
events. The process can be efficiently carried
out through the use of a review
questionnaire.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Item – Improvement Questionnaire For Students

1. If any of the items seemed confusing, which is
ones were they?
2. Did any items have more than one correct
answer? If so, which ones?
3. Did any items have no correct answers? If so,
which ones?
4. Were there words in any items that confused
you? If so, which ones?
5. Were the direction for the test, or for particular
subsections, unclear? If so, which one?
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Another techniques of eliciting student

judgment for item improvement is by going
over the test with his/her students before the
results are shown.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Empirically –Base Procedures

Tests developers refer to this
technical process as an item analysis
as it utilizes data obtained separately
for each item. An item is considered
good when its quality indices, i.e.
difficulty index and discrimination
index, meet certain characteristics.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

For a norm-referenced test,

these two indices are related since
level of difficulty of an item
contributes to its discriminability.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

For a context of criterion-

referenced interpretation or testing
for mastery, an item with a highh
difficulty index will not be considered
as an “easy item” and therefore a
weak item, but rather an item that
displays the capability of the learners
to perform the expected outcome. It
therefore becomes an evidence of
mastery.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Empirically –Base Procedures

 Difficulty Index
 Discrimination Index
 Distracter Analysis
 Sensitivity to Instruction Index
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Difficulty Index
An item’s difficulty index is obtained by
calculating the p value (p) which is the
proportion of students answering the item
correctly.
p=R/T
Where p is the difficulty index
R = Total number of students answering the
item right
T = Total number of students answering the
item
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Difficulty Index
Item 1: There were 45 students in the Item 1 has a p value of 0.67.
class who responded to item 1 and 30
answered it correctly.
sixty-seven percent (67%) got the
item right while 33% missed it.
P = 30/45
= 0.67
Item 2: in the same class, only 10 Item 2 has a p value of .22. out of
responded correctly in item 2.
45 only 10 or 22% got the item
P = 10/45 right while 35 or 78% missed it
= .22

For Normative-referenced test: Between the two items, item 2

appears to be a much more difficult item since less than a fourth of
the class only was able to respond correctly.
For Criterion-reference test: The class shows much better
performance in item 1 than item 2. it is still a long way for many to
master item 2.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Discrimination Index
The power of an item to discriminate
between informed and uniformed groups or
between more knowledgeable and less
knowledgeable learners is shown using the
Item-discriminatory Index(D).
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

For a classroom test, the discrimination

index shows if a difference exist between the
performance of those who scored high and
those who scored low in an item.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Direction of the results

a. Positively discriminating item – Proportion
of high scoring group is greater than that of
the low scoring group.
b. Negative discriminating item – Proportion
of high scoring group is less than that of low
scoring group.
c. Not discriminating – proportion of high
scoring group is equal to that of the low
scoring group.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Formulas
D = Ru/Tu –Rl/Tl Where D is the item discrimination index
Ru = number of upper group getting the item
correct
Tu = number of upper group
D = Ru-Rl Rl= number of lower group getting the item
T correct
Tl = Number of lower group
T = number of either group
Pu =is the p-value for upper group (Pu/Tu)
D = Pu – Pl Pl= is the p-value for lower group (Pl/Tl)
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

To obtain the proportion of the upper and lower

groups responding to the item correctly, the teacher
should follow these step:
a. Score the test papers using key to correction to obtain the
total scores of the students. Maximum score is the total
number of objective items.
b. Order the test paper from highest to lowest score.
c. Split the test papers into halves: high group and low group.
 For a class of 50 or less students, do 50 -50 split. Take the upper
half as the HIGH Group and the lower half as the LOW Group
 For a big group of 100 or so., take the upper 25 – 27% and the
lower 25 – 27%
 Maintain equal numbers of test papers for Upper and Lower
group.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

d. Obtain the p value for the upper group and p – value

for the Lower group

P (Upper) = Ru/Tu P(Lower) = Rl/Tl

e. Get the Discrimination index by getting the difference

between the p-values
 GUIDELINES FOR EVALUATING THE
DISCRIMINATING EFFICIENCY OF ITEMS
Discrimination Index Item Evaluation

.40 and Above Very Good

.30 -.39 Reasonably good items, but possibly subject

to improvement

.20 -.29 Marginal items, usually needing

improvement

.19 and below Poor items, to be rejected or improved by

revision
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Distracter Analysis
Another empirical procedure to discover
areas for item-improvement utilizes an analysis
of the distribution of responses across the
distracters. Especially when the difficulty index
and discrimination index of the item seem to
suggest its being candidate for revision,
distracters analysis becomes a useful follow-
up.
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

It can detect differences in how the more

able students respond to the distracters in a
multiple – choice item compare to how the
less able ones do it.
Item Diffic Discrimi Group Alternatives
N= ulty nation
40 Index Index (D)
(p)
A B C D Omit

1 .38 -.35 Upper 2 10 *5 3

Lower 2 0 12 6

2 .45 -.50 Upper 2 *4 10 4

Lower 5 14 1 0
IMPROVING A CLASSROOM-BASED ASSESSMENT TEST

Sensitivity to Instruction Index

It signifies a change in student’s
performance as a result of instruction. The
information is useful for criterion-referenced
test which aim at determining if the mastery
learning has been attained, after a designated
or prescribed instructional period.

Si = p (post) – p(pre)