Sei sulla pagina 1di 4

Multiple choice item construction:

Avoiding constructions that reduce validity


and reliability Cynthia Brame, Assistant Director
Vanderbilt Center for Teaching

Anatomy of a Multiple Choice Item References


Test questions consist of a stem and alternatives, one of which is the answer and the
remainder of which are distractors. Steven J. Burton, Richard R. Sudweeks,
Paul F. Merrill, and Bud Wood. How to
A 54-year-old woman, G0P0, with a BMI of 20, smokes and Prepare Better Multiple Choice Test Items:
works as a convenience store clerk. She is seeing you Guidelines for University Faculty, 1991.
because she has been having urine leakage. Which of the Stem
following in her history is a known risk factor for urinary
incontinence?
Derek Cheung and Robert Bucat. How
A. BMI can we construct good multiple-choice
Distractors B. occupation type items? Presented at the Science and
Alternatives
C. parity Technology Education Conference, Hong
Answer D. smoking status Kong, June 20-21, 2002.

Test items can be described in terms of validity, or the degree to which they measure the learning
outcomes they purport to measure, and reliability, or the degree to which they consistently Thomas M. Haladyna. Developing and
measure a learning outcome. To increase validity and reliability, test writers should avoid validating multiple-choice test items, 2nd
constructions that help the test-wise and constructions that test skills not central to the edition. Lawrence Erlbaum Associates,
stated learning outcomes. 1999.

Guidelines for writing the stem


Thomas M. Haladyna and S. M.
The stem should: Downing. Validity of a taxonomy of
multiple-choice item-writing rules. Applied
 be meaningful by itself and should present a definite problem.
Measurement in Education, 2(1), 51-78, 1989.
 contain only relevant material.
 be negatively stated only when significant learning outcomes require it.
 be a question or a partial sentence.
– A question stem is preferable. Susan Morrison and Kathleen Walsh
– Stems with beginning and interior blanks should be avoided. Free. Writing multiple-choice test items
that promote and measure critical thinking.
Journal of Nursing Education 40: 17-24, 2001.
Guidelines for writing alternatives
Alternatives should:
National Board of Medical Examiners.
 be plausible. Constructing written test questions for the
 be stated clearly and concisely. basic and clinical sciences.
 be mutually exclusive. Available at www.nbme.org
 be homogenous in content.
 not include “all of the above” and “none of the above”
 be presented in a logical order (e.g., alphabetical, numerical) to avoid a bias toward
certain positions.
 be free from clues about which response is correct. Specifically, the alternatives
should all
o have grammar consistent with the stem.
o be parallel in form.
o be similar in length.
o use similar language.
Other Guidelines
 As long as all alternatives are plausible, the number of alternatives can vary
among items. There is little difference in difficulty, discrimination, and test score
reliability among items containing two, three, and four distractors.
 Avoid complex multiple-choice problems (i.e., alternatives such as 1 and 2; 2 and
3; 1 and 3; 1, 2, and 3).
 Keep the specific content of items independent of one another.

Additional information
 The National Board of Medical Examiners provides an excellent tutorial on
writing multiple choice items; it is available at
http://download.usmle.org/IWTutorial/intro.htm. They recommend using two
questions when reviewing items: “Is the item front-loaded? Can you cover the
options?” These questions encourage the question writer to place key information
in the stem and to construct items that an informed test-taker can answer without
choices.
 The guidelines presented above help test-writers avoid constructions that tip off
the test-wise or that target skills that are not central to the learning outcomes.
o Test-wise examinees are alert to cues that indicate the correct answer.
These cues may take the form of grammatical clues, “clanging” (i.e., the
use of different forms of the same word in the stem and the correct
answer), convergence (i.e., the use of elements of the correct answer in
multiple alternatives), or logical cues (e.g., length of the answer or the
use of “always” or “never” in incorrect answers). Being attentive to these
cues and following the guidelines above can help in the construction of
items that are more valid evaluations of the desired learning outcomes.
o Test items that contain irrelevant material, wordy alternatives, or
negative constructions have reduced validity and reliability because they
test, in part, examinees’ reading ability and ability to hold information in
short-term memory.
o The use of complex multiple choice and “all of the above” and “none of
the above” alternatives reduces item reliability, in part because it allows
examinees to use partial knowledge to arrive at a correct answer.
Multiple choice item construction:
Avoiding constructions that reduce
validity and reliability Cynthia Brame, Assistant Director
Vanderbilt Center for Teaching

Bloom’s Taxonomy of Educational Objectives References


In 1956, Benjamin Bloom and colleagues proposed a framework for categorizing
educational goals, ranking those goals from lower-level cognitive functions
(knowledge and comprehension) to higher level (application, analysis, synthesis, B.S. Bloom. Taxonomy of Educational
and evaluation). In 2001, Anderson and colleagues proposed a revised taxonomy, Objectives, Handbook I: The Cognitive
briefly summarized here. This framework can be a useful way to consider the level Domain. New York: David McKay Co,
Inc., 1956.
of your MC questions, and can be a tool for adapting them to higher order
cognitive functions if appropriate for your learning goals.

Creating Not appropriate for MC L.W. Anderson, D.R. Krathwohl, P.W.


Airasian, K.A. Cruikshank, R.E. Mayer,
Evaluating P.R. Pintrich, J. Raths, and M.C.
Application of knowledge Wittrock. A Taxonomy for Learning,
Analyzing Can be tested with MC Teaching, and Assessing: A Revision of
Bloom’s Taxonomy of Educational
Applying
Objectives. New York: Pearson, Allyn, and
Understanding Bacon, 2000.
Recall of knowledge
Remembering Easily tested with MC

Descriptions and verbs*


Evaluating: Students are asked to make judgments about data, arguments, or
other, potentially conflicting information. Verbs to target this level: Evaluate,
argue, criticize, recommend, choose, defend

Analyzing: Students are asked to separate material or concepts into parts,


understanding facts and the inferences they may allow. Verbs to target this level:
Analyze, compare/contrast, interpret, differentiate, prioritize, deduce

Applying: Students are asked to use a concept in a new situation or to apply an


abstraction without prompting. Verbs to target this level: Solve, apply, calculate,
construct, predict, manipulate

Understanding: Students are asked to demonstrate comprehension. Verbs to target


this level: Explain, compare, classify, describe, contrast, give examples,
summarize, identify

Remembering: Students are asked to recall previously learned information. Verbs


to target this level: Define, label, select, identify, list, state, describe, name

*Note: These verbs are meant as suggestions to help with adapting questions to
higher level Bloom’s levels, but are not essential and do not always neatly map to
a single category.
Using vignettes to test higher-order learning objectives
The National Board of Medical Examiners recommends that vignettes contain
four or five of the following: National Board of Medical Examiners.
Constructing written test questions for the
 Age, gender basic and clinical sciences.
 Presenting complaint Available at www.nbme.org
 Patient history/family history
 Diagnostic studies
 Subsequent findings
 Site of care
 Duration of symptoms
 Physical findings
 Initial treatment

Questions associated with the vignettes can ask students to analyze test results,
predict the results of treatments, propose a diagnosis, evaluate the best treatment
option, etc., allowing these questions to target multiple types of higher level
learning objectives.

Potrebbero piacerti anche