Score Scaling and Referencing

Now that weve discussed the measurement process we can go over some common methods for
giving meaning to the scores that our measures produce. These methods are referred to as score
scaling and norm and criterion score referencing. Each is discussed briey below, with examples.

Score Scaling

Score scales are often modied to have certain properties, including smaller or larger score intervals,
dierent midpoints, and dierent variability. A common example is the z-score scale, which is
dened to have a mean of 0 and standard deviation (SD) of 1. Any variable having a mean and SD
can be converted to z-scores, which express each score in terms of distances from the mean in SD
units. Once a scale has been converted to the z-score metric, it can then be transformed to have any
midpoint, via the mean, and any scaling factor, via the standard deviation. Equations for these
transformations are shown below. Methods for carrying out these transformations are discussed
again in Chapter 5.

To convert a variable Y from its original score scale to the z-score scale, we subtract out Y , the
mean on Y , from each score, and then divide by Y , the SD of Y . The resulting z transformation of
Y , labeled as Y z, is:

Y z = Y Y Y . (1.1)

Having subtracted the mean from each score, the mean of our new variable Y z is 0, and having
divided each score by the SD, the SD of our new variable is 1. We can now multiply Y z by any
constant s, and then add or subtract another constant value m to obtain a linearly transformed
variable with mean m and SD equal to s. The new rescaled variable is labeled as Y r:

Y r = Y zs + m. (1.2)

The linear transformation of any variable Y from its original metric, with mean and SD of Y and
Y , to a scale dened by a new mean and standard deviation, is obtained via the combination of
these equations, as:

Y r = (Y Y ) s Y + m. (1.3)

Scale transformations are often employed in testing for one of two reasons. First, transformations can
be used to express a variable in terms of a familiar mean and SD. For example, IQ scores are
traditionally expressed on a scale with mean of 100 and SD of 15. In this case, Equation 1.3 is used
with m = 100 and s = 15. Another popular score scale is referred to as the t-scale, with m = 50 and s
= 10. Second, transformations can be used to express a variable in terms of a new and unique metric.
When the GRE was revised in 2011, a new score scale was created, in part to discourage direct
comparisons with the previous version of the exam. The former quantitative and verbal reasoning
GRE scales ranged from 200 to 800, and the revised versions range from 130 to 170.

Norm referencing

Norm referencing gives meaning to scores by comparing them to values for a specic norm group.
For example, when my kids bring home their standardized test results from school, their scores in
each subject area, math and reading, are given meaning by comparing them to the distribution of
scores for students across the state. A score of 22 means very little to a parent who does not have
access to the test itself. However, a percentile score of 90 indicates that a student scored at or above
90% of the students in the norming group, regardless of what percentage of the test questions they
answered correctly.

Norms are also frequently encountered in admissions testing. If you took something like the ACT or
SAT, college admissions exams used in the US, or the GRE, the admissions test for graduate school,
youre probably familiar with the ambiguous score scales these exams use in reporting. Each scale is
based on a conversion of your actual test scores to a scale that is intentionally dicult or impossible
to understand. In a way, the objective in this rescaling of scores is to force you to rely on the norm
referencing provided in your score report. The ACT scales range from 1 to 36, but a score of 20 on
the math section doesnt tell you a lot about how much math you know or can do. Instead, by
referencing the published norms, a score of 20 tells you that you scored around the 50th percentile
for all test takers, which isnt great if youre hoping to get into a good college.

These two examples involve simple percentile norms, where scores are compared to the full score
distribution for a given norm group. Two other common types of norm referencing are grade and age
norms, which are obtained by estimating the typical or average performance on a test by grade level
or age.

Criterion referencing

The main limitation of norm referencing is that it only helps describe performance relative to other
test takers. Criterion score referencing does the opposite. Criterion referencing gives meaning to
scores by comparing them to values directly linked to the test content itself, regardless of how others
perform on the content (Popham & Husek, 1969).

Educational tests supporting instructional decision making are often criterion referenced. For
example, classroom assessments are used to identify course content that a student has and has not
mastered, so that deciencies can be addressed before moving forward. The vocabulary test
mentioned above is one example. Others include tests used in student placement and exit testing.

Standardized state test results, which were presented above as an example of norm referencing, are
also given meaning using some form of criterion referencing. The criteria in state tests are
established, in part, by a panel of teachers and administrators who participate in what is referred to as
a standard setting. State test standards are chosen to reect dierent levels of mastery of the test
content. In Nebraska, for example, two cut-o scores are chosen per test to categorize students as
below the standards, meets the standards, and exceeds the standards. These categories are referred to
as performance levels. Student performance can then be evaluated based on the description of typical
performance for their level. Here is the performance level description for grade 5 science, meets the
standard:

Overall student performance in science reects satisfactory performance on the standards and
sucient understanding of the content at fth grade. A student scoring at the Meets the Standards
level generally draws on a broad range of scientic knowledge and skills in the areas of inquiry,
physical, life, and Earth/space sciences.

The Nebraska performance categories and descriptions are available online at

www.education.ne.gov/assessment. Performance level descriptions are accompanied by additional
details about expected performance for students in this group on specic science concepts. For
example, again for grade 5 science, meets the standard:

A student at this level generally:

Identies testable questions,

Identies factors that may impact an investigation,
Identies appropriate selection and use of scientic equipment,
Develops a reasonable explanation based on collected data,
Describe the physical properties of matter and its changes.

The performance levels and descriptors used in standardized state tests provide general information
about how a test score relates to the content that the test is designed to measure. Given their
generality, these results are of limited value to teachers and parents. Instead, performance level
descriptors are used for accountability purposes, for example, to assess performance at the school,
district, and even the state levels in terms of the numbers of students meeting expectations.

The Beck Depression Inventory (BDI; Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) is an
example of criterion referencing in psychological testing. The BDI includes 21 items representing a
range of depressive symptoms. Each item is scored polytomously from 0 to 3, and a total score is
calculated across all of the items. Cuto scores are then provided to identify individuals with
minimal, mild, moderate, and severe depression, where lower scores indicate fewer depressive
symptoms and higher scores indicate more severe depressive symptoms.

Comparing referencing methods

Although norm and criterion referencing are presented here as two distinct methods of giving
meaning to test scores, they can sometimes be interrelated and thus dicult to distinguish from one
another. The myIGDI testing program described above is one example of score referencing that
combines both norms and criteria. These assessments were developed for measuring growth in early
literacy skills in preschool and kindergarten classrooms. Students with scores falling below a cut-o
value are identied as potentially being at risk for future developmental delays in reading. The cut-
o score is determined in part based on a certain percentage of the test content (criterion
information) and in part using mean performance of students evaluated by their teachers as being at-
risk (normative information).

Norm and criterion referencing serve dierent purposes. Most comparisons of the two note that norm
referencing is typically associated with tests designed to rank order test takers and make decisions
involving comparisons among individuals, whereas criterion referencing is associated with tests
designed to measure learning or mastery and make decisions about individuals and programs (e.g.,
Bond, 1996; Popham & Husek, 1969).