Test and Item Analysis of Multiple Choice Test

Questions Report

by

C. R. Adjah

July 2007

1
 Table of Contents
Page
Table of Contents 1
List of Tables 2
List if figures 3

Section
1 Introduction 4
1.1 Statement of purpose 4
1.2 Methodology 4
1.3 Report structure 5

2 Test analysis 6
2.1 Distribution of students with correct option per 6
question
2.2 Distribution of percentage scores 7
2.3 Descriptive statistics 7

3 Item analysis 9
3.1 Difficulty index 9
3.2 Discrimination index 10
3.3 Item reliability 11

4 Conclusion 12
Bibliography 13
Addendum A
Addendum B

List of Tables
Table Table Name Page
1 Mean, Standard deviation, Skewness and 7
Kurtosis per question
2 The mean, median, mode and standard 8
deviation of percentages
3 Difficulty index 9
4 Discrimination index 10
5 Cronbach’s alpha 11
6 Cronbach’s alpha on deleting an item 11

2
 List of Figures
Figure Figure name Page
1 The approach 4
2 Report structure 5
3 Histogram of students with correct options 6
4 Frequency histogram of percentage scores 7

3
1 INTRODUCTION
This is a report on the test and item analysis of a 20 multiple choice test
questions taken by 25 students.

1.1 STATEMENT OF PURPOSE

The purpose of this report is to provide a descriptive statistics and item
analysis of 20 multiple choice test questions taken by 25 students.

1.2 METHODOLOGY
The approach followed is as shown in Figure 1.

Figure 1: The approach

Steps Description

4
 The data collected from answer scripts of the
students were captured in an excel
Data tabulation spreadsheet

For each student, the chosen options captured

as A, B, C and D were recoded into 1 for a
Recoding of data correct option and 0 for an incorrect option in
an excel spreadsheet.

Calculation of The score per student was calculated and

student score sorted in descending order according to
percentages

Grouping of 13 of the students were then grouped in an

students upper group and 12 in a lower group.

An analysis of the data was carried out using

SPSS (Statistical Program for the Social
Sciences) to determine the mean, standard
Analysis of data
deviation, mode, median, difficulty index,
discrimination index and the Cronbach’s alpha.

A histogram of number of students with

Histogram correct options per question was drawn.
1.3 REPORT STRUCTURE
The report is made up of the four main sections:
• Introduction
• Test analysis
• Item analysis
• Conclusion
These sections as illustrated in Figure 2 are subdivided into subsections by
their headings.

5
Figure 2: Report structure

2 TEST ANALYSIS
2.1 Distribution of students with correct option per question
The number of students with the correct options chosen per question were
determined and a histogram drawn. This is illustrated in Figure 3.

Figure 3: Histogram of students with correct options

6
 Q1
HISTOGRAM OF NUMBER OF STUDENTS WITH Q2
CORRECT OPTIONS PER QUESTION Q3
Q4
Q5
Q6
25 Q7
Q8
20 Q9
FREQUENCY
Q10
15 Q11
Q12
10 Q13
Q14
5 Q15
Q16
0 Q17
Q18
QUESTION Q19
Q20

It is shown from the histogram that between 21 and 23 which represent 84%
to 92% of the students chose the correct options for questions 1, 2, 5, 11,
14, 15 and 16. Between 8 and 13 representing 32% to 52% of the students
answered questions 4, 7, 8, 9, 10 and 19 correctly.

2.2 Distribution of percentage scores

The number of students that fall within a percentage score is represented by
a histogram as illustrated in Figure 4.

Figure 4: Frequency histogram of percentage scores

7
 HISTOGRAM

6
20-30
5
FREQUENCY 30-40
4 40-50
3 50-60
2 60-70
70-80
1
80-90
0
90-100
PERCENTAGE SCORES

It is shown that 14 learners representing 56% of the students obtained

scores above the mean with 44% of the students have scores below the
mean.

2.3 Descriptive statistics

The mean, standard deviation per item is shown in Table 1.

Table 1: Mean, Standard deviation, Skewness and Kurtosis per question

Std.
QUESTION N Sum Mean Deviation Skewness Kurtosis
Q1 25 21.00 .8400 .37417 -1.975 2.061
Q2 25 22.00 .8800 .33166 -2.491 4.563
Q3 25 17.00 .6800 .47610 -.822 -1.447
Q4 25 12.00 .4800 .50990 .085 -2.174
Q5 25 21.00 .8400 .37417 -1.975 2.061
Q6 25 17.00 .6800 .47610 -.822 -1.447
Q7 25 11.00 .4400 .50662 .257 -2.110

Table 1: Mean, Standard deviation, Skewness and Kurtosis per question

Std.
QUESTION N Sum Mean Deviation Skewness Kurtosis
Q8 23 12.00 .5217 .51075 -.093 -2.190
Q9 25 13.00 .5200 .50990 -.085 -2.174
Q10 24 8.00 .3333 .48154 .755 -1.568
Q11 25 23.00 .9200 .27689 -3.298 9.641

8
Q12 25 19.00 .7600 .43589 -1.297 -.354
Q13 25 15.00 .6000 .50000 -.435 -1.976
Q14 25 21.00 .8400 .37417 -1.975 2.061
Q15 25 20.00 .8000 .40825 -1.597 .593
Q16 24 22.00 .9167 .28233 -3.220 9.124
Q17 24 15.00 .6250 .49454 -.551 -1.859
Q18 25 8.00 .3200 .47610 .822 -1.447
Q19 25 13.00 .5200 .50990 -.085 -2.174
Q20 25 16.00 .6400 .48990 -.621 -1.762
Valid N
22
(listwise)

The mean percentage score calculated is illustrated in Table 2. Also in the

table are the Median, Mode and standard deviation of the percentage scores.

Table 2: The mean, median, mode and standard deviation of percentages

Mean 65.24
Median 65.00
Mode 65.00
Standard deviation 21.60

3 ITEM ANALYSIS
3.1 Difficulty index
Illustrated in Table 3 are the p-values of each test item. The p-values indicate
the proportion of students who got the test items correct.

9
Table 3: Difficulty index
Difficulty index
QUE #Correct #Answered p REMARKS
Q1 21 25 0.84 Unacceptable item
Q2 22 25 0.88 Unacceptable item
Q3 17 25 0.68 Acceptable item
Q4 12 25 0.48 Acceptable item
Q5 21 25 0.84 Unacceptable item
Q6 17 25 0.68 Acceptable item
Q7 11 25 0.44 Acceptable item
Q8 12 23 0.52 Acceptable item
Q9 13 25 0.52 Acceptable item
Q10 8 24 0.33 Acceptable item
Q11 23 25 0.92 Unacceptable item
Q12 19 25 0.76 Acceptable item
Q13 15 25 0.60 Acceptable item
Q14 21 25 0.84 Unacceptable item
Q15 20 25 0.80 Acceptable item
Q16 22 24 0.92 Unacceptable item
Q17 15 24 0.63 Acceptable item
Q18 8 25 0.32 Acceptable item
Q19 13 25 0.52 Acceptable item
Q20 16 25 0.64 Acceptable item

From the table, the p-values of Q1, Q2, Q5, and Q14 are greater than 0.80
and therefore can be termed to be unacceptable test items. Q11 and Q15
with p-values above 0.90 are very easy items and should not be reused in
following tests. All other test items are acceptable as their p-values fall
between 0.20 and 0.80.

3.2 Discrimination index

A measure of the extent to which students who do well on the overall test
differentiate from students who did not do well on the overall test items was
determined as the discrimination indices. These discrimination indices
determined are shown in Table 4.

10
Table 4: Discrimination index
Discrimination index
#U #L
REMARKS
QUE (UPPER) (LOWER) D
Q1 12 9 0.23 Acceptable item
Q2 13 9 0.31 Acceptable item
Q3 13 4 0.69 Acceptable item
Q4 7 5 0.15 Unacceptable item
Q5 13 8 0.38 Acceptable item
Q6 11 6 0.38 Acceptable item
Q7 8 3 0.38 Acceptable item
Q8 10 2 0.62 Acceptable item
Q9 10 3 0.54 Acceptable item
Q10 8 0 0.62 Acceptable item
Q11 12 11 0.08 Unacceptable item
Q12 12 7 0.38 Acceptable item
Q13 11 4 0.54 Acceptable item
Q14 13 8 0.38 Acceptable item
Q15 12 8 0.31 Acceptable item
Q16 13 9 0.31 Acceptable item
Q17 10 5 0.38 Acceptable item
Q18 5 3 0.15 Unacceptable item
Q19 10 3 0.54 Acceptable item
Q20 10 6 0.31 Acceptable item

Even though the discrimination indices of the test items are all positive and
therefore can be considered to be desirable items, Q4, Q11, Q18 with
discrimination indices less than 0.20 indicate that these test items are poorly
constructed items and unacceptable (Measurement and Evaluation Center,
2003).

3.3 Item reliability

Cronbach’s alpha which is the indicator of the overall test reliability is shown
in Table 5.

Table 5: Cronbach’s alpha

Cronbach's Cronbach's N of Items

11
 Alpha Based on
Standardized
Alpha Items
.804 .812 20

The high Cronbach’s alpha value of 0.812 indicates that the overall test is
reliable. Deleting a test item either increases or decreases the Cronbach’s
alpha. These changes are reflected in Table 6.

Table 6: Cronbach’s alpha on deleting an item

Scale Mean Scale Cronbach's
QUEST if Item Variance if Alpha if Item Comments
Deleted Item Deleted Deleted
Q1 13.0455 15.093 .802 Acceptable
Q2 13.0000 14.571 .791 Acceptable
Q3 13.0909 14.372 .791 Acceptable
Q4 13.3182 15.846 .821 Unacceptable
Q5 12.9545 15.474 .804 acceptable
Q6 13.0909 15.420 .809 Unacceptable
Q7 13.4091 14.253 .795 Acceptable
Q8 13.3182 14.513 .799 Acceptable
Q9 13.3182 13.656 .783 Acceptable
Q10 13.5000 13.405 .777 Acceptable
Q11 12.9545 15.474 .804 Acceptable
Q12 13.0000 15.333 .804 Acceptable
Q13 13.2273 13.898 .787 Acceptable
Q14 12.9545 14.617 .789 Acceptable
Q15 13.0909 14.468 .793 Acceptable
Q16 12.9545 14.617 .789 Acceptable
Q17 13.2727 14.113 .792 Acceptable
Q18 13.5000 14.833 .804 Acceptable
Q19 13.2727 13.827 .786 Acceptable
Q20 13.1364 14.504 .795 Acceptable
Q4 and Q6 showed an increase in Cronbach’s alpha value if deleted. This
indicates that this question needs modification or deletion as a test item in
order to maintain the reliability of the test.

4 Conclusions

12
All test items discriminate well except for Q4, Q11 and Q18. In the case of
Q1, Q2, Q5, and Q14 with difficulty indices above 0.80 is an indication that
they are quite easy test items and may need a review. Questions 11 and 15
with difficulty indices above 0.90 are very easy items and should not be
reused in subsequent testing. However, based upon the Cronbach’s alpha
values, all the test items can be considered to be reliable and acceptable
except for Q4 which needs modification or deletion in order to increase the
reliability of the test.

Knoetze, J. (2007). Test data. Retrieved July 16, 2007 from

<http://www.jknoetze.co.za_2007/testdata.xls>
Measurement and Evaluation Center. (2003). Test Item Analysis & Decision
Making. The University of Texas at Austin. Retrieved July 16, 2007 from
<http://www.utexas.edu/academic/mec/research/pdf/itemanalysishando
ut.pdf>

13
Varma, S. (n.d.). Preliminary Item Statistics Using Point-Biserial Correlation
and P-Values. Educational Data Systems Inc Morgan Hill CA. Retrieved
July 16, 2007 from
<http://www.eddata.com/resources/publications/EDS_Point_Biserial.pdf
>

14
 ADDENDUM A
Coding and grouping of students
Key C B D D B C D A C B A C B D A A C D B C
St
No Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 #Corr #Ans % Grp
11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 20 100.00 U
16 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 20 20 100.00 U
2 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 18 20 90.00 U
3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 18 20 90.00 U
25 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 18 20 90.00 U
13 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 17 20 85.00 U
20 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 17 20 85.00 U
14 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 16 20 80.00 U
5 1 1 1 0 1 1 0 1 1 0 1 1 1 1 1 1 0 0 1 1 15 20 75.00 U
4 1 1 1 0 1 1 0 1 1 1 1 1 0 1 0 1 1 0 0 1 14 20 70.00 U
12 1 1 1 1 1 1 1 0 0 0 1 1 0 1 1 1 1 0 1 0 14 20 70.00 U
8 1 1 1 0 1 1 0 0 0 0 1 1 1 1 1 1 1 0 1 0 13 20 65.00 U
9 1 1 1 0 1 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0 13 20 65.00 U
18 1 1 0 1 1 0 1 0 0 0 1 1 0 1 1 1 1 0 1 1 13 20 65.00 L
23 1 1 1 0 1 1 0 0 0 0 1 1 1 1 1 1 1 0 1 0 13 20 65.00 L
10 1 1 0 0 1 1 1 0 0 0 1 0 0 1 0 1 1 1 1 1 12 20 60.00 L
21 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 0 1 11 20 55.00 L
22 1 1 0 0 1 1 0 0 0 0 1 1 1 1 0 1 0 1 0 1 11 20 55.00 L
17 1 1 0 0 1 0 1 0 1 1 0 1 1 1 0 0 0 9 17 52.94 L
6 0 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 0 1 10 20 50.00 L
7 0 1 0 0 1 1 0 0 0 0 1 1 1 1 0 1 0 1 0 1 10 20 50.00 L
15 0 1 1 1 1 0 0 1 0 0 1 1 0 0 1 0 0 0 0 0 8 20 40.00 L
1 1 1 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 6 19 31.58 L
24 1 1 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 6 19 31.58 L
19 1 0 0 1 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 4 20 20.00 L
65.64
21.60
Upper 13
Lower 12
ADDENDUM B

15
 Discrimination
Difficulty index index
QUE #Corr #Ans p #U #L D
Q1 21 25 0.84 12 9 0.23
Q2 22 25 0.88 13 9 0.31
Q3 17 25 0.68 13 4 0.69
Q4 12 25 0.48 7 5 0.15
Q5 21 25 0.84 13 8 0.38
Q6 17 25 0.68 11 6 0.38
Q7 11 25 0.44 8 3 0.38
Q8 12 23 0.52 10 2 0.62
Q9 13 25 0.52 10 3 0.54
Q10 8 24 0.33 8 0 0.62
Q11 23 25 0.92 12 11 0.08
Q12 19 25 0.76 12 7 0.38
Q13 15 25 0.60 11 4 0.54
Q14 21 25 0.84 13 8 0.38
Q15 20 25 0.80 12 8 0.31
Q16 22 24 0.92 13 9 0.31
Q17 15 24 0.63 10 5 0.38
Q18 8 25 0.32 5 3 0.15
Q19 13 25 0.52 10 3 0.54
Q20 16 25 0.64 10 6 0.31
M 65.64
MDN 65.00 % FREQ
MODE 65.00 20-30 1
STD 21.60 30-40 2
40-50 1
50-60 5
60-70 5
70-80 3
80-90 3
90-100 5

16