State Assessment Sample Design in 2000

The 2000 state assessment sample design utilized a multistage probability sampling approach for all
participating jurisdictions. The assessment tested fourth- and eighth-graders in the subject areas of
mathematics and science. The first stage sampled public schools from the participating jurisdiction. After
designating assessment sessions for the selected schools, the final stage sampled students from those
schools and assigned them to sessions.

Before sampling began, the state assessment defined its target population for the state or jurisdiction and
created a school sampling frame. A stratification of schools within the sampling frame enabled the school
sample to represent a variety of student groups. Next, schools were selected and assigned sessions.
Within the sampled schools, the student sample selection identified which students would participate in
each assessment session.

48
Target Population of the 2000 State Assessment

The NAEP state assessment targets public school students enrolled in the fourth or eighth grades. All 50
states and the District of Columbia are eligible to participate in the state assessment, as well as territories
and Bureau of Indian Affairs (BIA) schools. The state assessment also includes separate jurisdictions of
Department of Defense Education Activity schools. One jurisdiction covers the department's domestic
schools (DDESS), while another jurisdiction covers its international schools (DoDDS). Participation in the
2000 assessment included 41 states, the District of Columbia, DDESS, DoDDS, and three U.S. territories,
as detailed on the web page Number of Schools and Enrollment of All Students and Testable Students for
Participating Jurisdictions.

Sampling parameters for the grades 4 and 8 public school frames, state assessment: By

jurisdiction, 2000

Grade 4 Grade 8
Jurisdiction Number of schools Number of students Number of schools Number of students
Total 50,500 3,593,000 25,900 3,542,000
Alaska 380 10,000 300 10,400
Alabama 790 59,200 520 56,500
Arkansas 540 36,200 360 37,000
Arizona 820 65,100 480 62,700
California 5,210 474,000 2,130 425,500
Colorado 850 52,900 390 53,700
Connecticut 610 42,300 240 38,800
Delaware 90 9,100 60 8,700
Florida 1,750 174,300 950 183,900
Georgia 1,090 104,800 440 104,300
Hawaii 180 14,500 60 14,000
Iowa 750 35,800 420 38,800
Indiana 340 18,100 200 19,400
Illinois 2,400 152,800 1,580 149,800
Indiana 1,150 77,600 490 76,200
Kansas 800 34,700 440 36,600
Kentucky 830 51,200 440 51,600
Louisiana 820 59,000 490 58,900
Massachusetts 1,080 76,900 430 70,700
Maryland 820 62,600 280 61,400
Maine 390 16,700 240 17,300
Michigan 2,120 134,500 1,000 132,100
Minnesota 1,120 65,600 820 74,100
Missouri 1,130 69,300 680 70,640
Mississippi 490 41,000 340 40,600
Montana 460 12,100 290 13,200
North Carolina 1,210 99,300 600 94,500
North Dakota 320 8,400 230 9,200
Nebraska 880 21,500 510 23,600
New Hampshire 260 16,900 130 16,200
New Jersey 1,310 101,000 660 92,600
New Mexico 400 25,000 180 26,000
Nevada 280 23,000 110 22,400
New York 2,270 218,700 1,070 203,600
See notes at end of table.

49
Sampling parameters for the grades 4 and 8 public school frames, state assessment: By
jurisdiction, 2000 (continued)

Grade 4 Grade 8
Number of Number of Number of Number of
Jurisdiction schools students schools students
Ohio 2,190 139,500 1,030 146,100
Oklahoma 940 49,200 620 50,200
Oregon 770 42,500 360 42,900
Pennsylvania 1,810 140,410 760 139,700
Rhode Island 190 12,300 50 11,600
South Carolina 570 51,700 270 52,300
South Dakota 400 9,700 290 10,900
Tennessee 950 73,100 560 66,800
Texas 3,540 290,400 2,070 297,900
Utah 440 35,690 170 38,500
Virginia 1,090 88,200 420 87,900
Vermont 240 8,200 130 8,300
Washington 1,280 78,400 700 82,000
Wisconsin 1,150 60,900 570 66,600
West Virginia 530 21,900 220 22,900
Wyoming 220 7,100 100 8,100
Other jurisdictions
District of Columbia 100 6,200 30 4,000
DDESS1 40 3,300 10 1,700
DoDDS2 100 6,300 60 4,600
Guam 20 2,600 10 2,400
Virgin Islands 20 1,700 10 1,900
1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for

Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Assessment.

50
Sampling Frame for the 2000 State Assessment

Both the National Center for Education Statistics (NCES) and Quality Education Data, Inc. (QED), an
educational research company, supplied school data files to create the 2000 state assessment's school
sampling frame.

Creating the school sampling frame required comprehensive lists of public schools in each participating
jurisdiction. The lists needed accurate school information such as

• possible stratification variables (urbanization and minority enrollment.);

• school grade span and enrollment data; and
• school identification information (school name and school address).

The Common Core of Data (CCD) public school file, maintained by NCES, served as the primary school
sampling frame. This database covered elementary and secondary schools in the 50 states, District of
Columbia, and outlying areas. It also included Department of Defense Dependents Schools (DoDDS).

QED maintains a national education database school file that lists public and nonpublic schools within the
United States. This database provides DoDDS for the school sampling frame.

51
Number of Schools and Enrollment of All Students and Testable Students for
Participating Jurisdictions in the 2000 State Assessment

The table below shows the distribution of schools with fourth- or eighth-grade in the 2000 state
assessment's school sampling frame. For all participating and non-participating jurisdictions, the table
lists the number of schools containing each grade and their combined grade-specific enrollments.
Estimates of grade-specific enrollment for each school results from dividing total school enrollment by the
number of grades in the school. This information comes from the 1997-1998Common Core of Data (CCD)
public school file.

Distribution of schools and enrollment for participating jurisdictions by grade, number of

schools, and number of students enrolled, state assessment: By jurisdiction, 2000

Fourth-grade schools Eighth-grade schools

Total Total Total Total
Jurisdiction schools enrollment schools enrollment
Total 42,410 3,001,500 21,300 2,906,100
Alabama 790 59,200 520 56,500
Arizona 820 65,100 480 62,700
Arkansas 540 36,100 360 37,000
California 5,210 474,000 2,130 425,500
Connecticut 610 42,300 240 38,800
Georgia 1,090 104,800 440 104,300
Hawaii 180 14,500 58 13,997
Idaho 335 18,085 185 19,410
Illinois 2,403 152,801 1,579 149,770
Indiana 1,146 77,584 491 76,223
Iowa 753 35,808 † †
Kansas 796 34,705 436 36,600
Kentucky 828 51,186 437 51,580
Louisiana 819 58,981 489 58,943
Maine 389 16,698 236 17,287
Maryland 819 62,576 275 61,420
Massachusetts 1,079 76,931 429 70,665
Michigan 2,124 134,525 1,003 132,145
Minnesota 1,122 65,576 819 74,086
Mississippi 489 41,021 341 40,603
Missouri 1,130 69,339 678 70,636
Montana 458 12,087 290 13,161
Nebraska 876 21,491 508 23,550
Nevada 284 22,973 112 22,363
New Mexico 403 24,805 179 25,942
New York 2,266 218,649 1,072 203,577
North Carolina 1,210 99,310 595 94,498
North Dakota 317 8,380 225 9,151
Ohio 2,188 139,463 1,031 146,057
Oklahoma 944 49,187 620 50,221
Oregon 771 42,479 357 42,931
Rhode Island 185 12,260 52 11,555
South Carolina 567 51,660 273 52,288
Tennessee 949 73,129 551 66,817
Texas 3,541 290,379 2,070 297,939
See notes at end of table.

52
Distribution of schools and enrollment for participating jurisdictions by grade, number of
schools, and number of students enrolled, state assessment: By jurisdiction, 2000
(continued)

Fourth-grade schools Eighth-grade schools

Total Total Total Total
Jurisdiction schools enrollment schools enrollment
Utah 444 35,685 173 38,464
Vermont 238 8,168 132 8,267
Virginia 1,092 88,245 417 87,860
West Virginia 528 21,948 216 22,935
Wisconsin 1,145 60,884 572 66,644
Wyoming 223 7,083 96 8,077
Other jurisdictions
American Samoa 22 1,172 22 1,120
District of Columbia 106 6,164 32 4,019
DDESS1 40 3,316 14 1,707
DoDDS2 96 6,317 59 4,620
Guam 23 2,593 7 2,366
Virgin Islands 24 1,728 7 1,893
† Iowa did not participate in the eighth-grade state assessment.

1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for

Education Statistics, National Assessment of Educational Progress (NAEP), 2000 Assessment.

53
Stratification of Schools in the Sampling Frame for the 2000 State Assessment

Before school selection begins for the state assessment, the school sampling frame undergoes a
hierarchical sort of schools based on selected demographic variables. This stratification process ensures
that the state assessment represents a variety of school and student groups. Stratified sampling also
improves the precision of the NAEP estimates by allowing separate estimates of population parameters
for each stratum and removing the variation between strata.

Stratification variables provide information about a school, its students, and its environment. The state
assessment uses such stratification variables as:

• urbanization,
• minority enrollment,
• achievement data, and
• median household income.

54
School Stratification Variables for the 2000 State Assessment

The school sampling frame undergoes two levels of stratification. Primary stratification breaks down the
school sampling frame by selected demographic detail. Implicit stratification provides a measure of control
over additional school variables after the primary stratification occurs.

Primary stratification variables for public schools, listed in hierarchical order, are as follows:

• small or large school district class,

• school size class,
• urbanization classification, and
• minority classification.

Implicit stratification variables for public schools include achievement data and median household
income. If available for a participating jurisdiction and grade, achievement data becomes the implicit
stratification variable. If no viable achievement data are found, the median household income of the
school location's ZIP code serves as the implicit stratification variable.

Prior to the selection of the school samples, the public schools are sorted by the four primary stratification
variables in an order such that changes occur on only one variable at a time. Serpentine sorting
accomplishes this feat by alternating between ascending and descending sort order on each variable
successively through the sort hierarchy. For example, if schools are to be sorted by urbanicity (urban,
suburban, rural) and size (enrollment):

• Urban schools are sorted in ascending order of enrollment.

• Suburban schools are sorted in descending order of enrollment.
• Rural schools are sorted in ascending order of enrollment.

This sorting pattern places the large urban schools next to large suburban schools and the small
suburban schools next to small rural schools. A traditional sort places the large urban schools next to
small suburban schools and the large suburban schools next to small rural schools, which is less
desirable for variance estimation purposes.

The implicit stratification of public schools by either achievement data or median household income
maintains the established serpentine order. This final sorting stage results in implicit stratification of either
variable.

55
Urbanization Classification for the 2000 State Assessment

The National Center for Education Statistics (NCES) type of location variable defines the urbanization of
the school's location. The type of location variable has seven urbanization categories:

• Large Central City: A central city of a metropolitan statistical area (MSA) with a population greater
than or equal to 400,000, or a population density greater than or equal to 6000 persons per
square mile;

• Mid-Size Central City: A central city of an MSA but not designated as a large central city;

• Urban Fringe of Large City: A place within an MSA of a large central city and defined as urban by
the U.S. Census Bureau;

• Urban Fringe of midsize City: A place within an MSA of a mid-sized central city and defined as
urban by the U.S. Census Bureau;

• Large Town: A place not within an MSA, but with a population greater than or equal to 25,000 and
defined as urban by the U.S. Census Bureau;

• Small Town: A place not within an MSA, with a population less than 25,000, but greater than or
equal to 2,500, and defined as urban by the U.S. Census Bureau; and

• Rural: A place with a population of less than 2,500 and defined as rural by the U.S. Census
Bureau.

Each urbanization classification within a participating jurisdiction must have a minimum of 10 percent of
the jurisdiction's eligible students. Some jurisdictions must have their type of location categories collapsed
together until this requirement is met. Specific rules govern the collapsing of type of location categories.
Initial collapses merge the central city categories, urban fringe categories, and/or town categories. If
needed, additional collapses merge together all central city and urban fringe categories, or both town and
the rural categories.

For the 2000 state assessment, all public school jurisdictions requiring urbanization classification
contained no missing urbanization classification values.

Note that the urbanization classification variable is more detailed than the type of location variable used
for reporting data. The reported type of location variable contains only three categories: urban, suburban,
and rural.

56
Minority Enrollment Classification for the 2000 State Assessment

A school's minority classification depends on its urbanization classification and the percentages of its two
largest minority student populations. Minority student populations include Blacks, Hispanics, Asians and
Pacific Islanders, and Native Americans. Urbanization classifications with small minority populations do
not require minority stratification. Urbanization classifications with larger minority populations are divided
into three or four minority enrollment classes, described in Cases 2 and 3 below.

Based on its urbanization classification and student race/ethnicity percentages, each school falls into one
of three minority classification schemes. These schemes are as follows:

• Case 1: An urbanization classification with less than 10 percent of students in its largest minority
group and 7 percent of students in its second largest minority group. No stratification by minority
enrollment occurs within this classification.

• Case 2: An urbanization classification with greater than or equal to 10 percent of students in its
largest minority group or 7 percent of students in its second largest minority group, but not more
than 20 percent in either group. This classification sorts schools by the combined percent student
enrollment of its two largest minority groups, then divides the schools into three groups (Low,
Medium, High) with similar numbers of students per minority classification.

• Case 3: An urbanization classification with greater than 20 percent of students in both its largest
and second largest minority groups. This classification designates the minority group with the
highest concentration as the primary stratification variable, and the other minority group as the
secondary stratification variable. After sorting schools by the primary stratification variable, the
schools split into two groups with approximately equal numbers of grade-specific students. Each
resulting group sorts its schools by the secondary stratification variable, then splits into two
subgroups of schools with approximately equal numbers of grade-specific students. This
procedure creates four minority classifications: low primary/low secondary; low primary/high
secondary; high primary/low secondary; and high primary/high secondary.

The minority classifications exist solely for creating efficient stratification design at this sampling stage.
They also reduce sampling errors for achievement-level estimates. The analysis and reporting of state
assessment data, however, do not use these minority classifications.

Methods of imputing missing public school minority enrollment data include:

• Assigning the average minority enrollment percentages of schools within the school district;

• Assigning the average minority enrollment percentages of schools within the school's five digit
ZIP code; and

• Assigning the average minority enrollment percentages of schools within the school's three digit
ZIP code prefix.

For the 2000 state NAEP assessment, all schools in Idaho had missing minority enrollment data. The
Common Core of Data (CCD) public school file contained no student race/ethnicity data for all Idaho
schools, preventing any imputation of missing minority enrollment data.

57
Achievement Data for the 2000 State Assessment

Where available, achievement data served as the implicit stratification variable for public schools.

Each participating jurisdiction with its own educational assessment program submitted the results for
each school. These school achievement results varied by jurisdiction, grade, and testing subject. The
achievement data files were processed in order to select one achievement measure that could be used in
stratification for each jurisdiction and grade. A jurisdiction that used achievement data for stratification
assigned schools with missing achievement data the median achievement result within the school's
urbanization and minority classifications.

For each combination of jurisdiction and grade where achievement data were not available or usable,
median household income served as the implicit stratification variable.

58
Usable Achievement Data for the 2000 State Assessment

Usable school achievement measures met the following criteria:

• An achievement measure reflected the assessment's targeted grades and subjects.

• An achievement measure came from the most recent educational assessments.
• An achievement measure appeared for at least 70 percent of schools within the jurisdiction and
grade.
• An achievement measure adequately differentiated schools (e.g., district-level measures or
pass/fail indicators tended not to make effective sort variables).

For the 2000 state assessment, usable achievement data included average school test scores for fourth-
and eighth-grade mathematics and science. Third or fifth grade mathematics and science data could
substitute for fourth grade data, while seventh grade mathematics and science data could substitute for
eighth grade data. Achievement data came from the 1996, 1997, or 1998 educational assessments.

Thirty-eight participating jurisdictions in the 2000 state NAEP assessment submitted school achievement
results. Of these 38 jurisdictions:

• 30 jurisdictions used school achievement results for stratification for at least one grade.
• 29 jurisdictions used school achievement results for stratification of their fourth-grade schools.
• 28 jurisdictions used school achievement results for stratification of their eighth-grade schools.

59
Implicit Stratification Variables for Jurisdictions from the 2000 State Assessment

The table below shows whether the 2000 state assessment's participating jurisdictions and grades used
school achievement data or median household income as their implicit stratification variable.

The table includes only those 41 states that required stratification. It does not include any jurisdiction that
required the sampling of all of its schools. District of Columbia, Guam, Virgin Islands, and the Department
of Defense schools are the jurisdictions that required the sampling of all schools. Note that Hawaii and
Rhode Island required sampling of all eighth-grade schools, while Iowa did not participate in the eighth
grade state assessment.

Alaska, Colorado, Delaware, Florida, New Hampshire, New Jersey, Pennsylvania, South Dakota, and
Washington did not participate in the NAEP 2000 state assessment.

Implicit stratification variables for state assessment, by grade and state:

2000

Fourth grade Eighth grade

School Median School Median
achievement household achievement household
State data income data income
Alabama * — * —
Arizona * — — —
Arkansas — * — *
California * — * —
Connecticut * — * —
Georgia * — * —
Hawaii * — — —
Idaho * — * —
Illinois * — * —
Indiana * — * —
Iowa — * — —
Kansas * — * —
Kentucky * — * —
Louisiana — * — *
Maine * — * —
Maryland * — * —
Massachusetts * — * —
Michigan * — * —
Minnesota — * — *
Mississippi * — * —
Missouri * — * —
Montana — * * —
Nebraska — * — *
Nevada * — — *
New Mexico * — * —
New York — * — *
North Carolina * — * —
North Dakota * — * —
Ohio — * — *
See notes at end of table.

60
Implicit stratification variables for state assessment, by state and grade:
2000 (continued)

Fourth grade Eighth grade

School Median School Median
achievement household achievement household
State data income data income
Oklahoma * — * —
Oregon * — * —
Rhode Island — * — —
South Carolina * — * —
Tennessee * — * —
Texas — * — *
Utah — * — *
Vermont * — — —
Virginia * — * —
West Virginia * — * —
Wisconsin * — * —
Wyoming — * — *
* This variable was used for implicit stratification.
— This variable was not used for implicit stratification.
SOURCE: U.S. Department of Education, Institute of Education Sciences,
National Center for Education Statistics, National Assessment of Educational
Progress (NAEP), 2000 Assessment.

61
Median Household Income for the 2000 State Assessment

For each combination of jurisdiction and grade with neither available nor usable achievement data,
median household income served as the implicit school stratification variable.

A school's median household income resulted from matching its ZIP code against a geographic-based
median household income file derived by Donnelly Marketing Analytical Services from 1990 Census data.
Any school still missing an income value after the match received the average median household income
calculated for the three-digit ZIP code prefix or county of its location.

62
School Sample Selection for the 2000 State Assessment

A systematic random sample selects grade-eligible schools from each participating jurisdiction's stratified
frame of public schools. The school sample design uses a selection probability proportional to the
estimated grade-specific enrollment of the schools. As a result, large schools are more likely to be
selected than small schools.

Each cooperating sampled school administers one or more assessment sessions. The number of
sessions administered depends on the school's estimated grade-specific enrollment.

Numerous features highlight the detailed school sample selection process:

• estimated grade enrollment and measure of size,

• district size and school size,
• school sorting and sample selection,
• sparse state sample option,
• selection of schools in small jurisdictions,
• control of overlap of school samples for national educational studies,
• selection of new public schools, and
• school substitution and retrofitting.

63
Estimated Grade Enrollment and Measure of Size for the 2000 State Assessment

A school's estimated grade enrollment approximates the number of students enrolled in the assessed
grade (either fourth or eighth grade). The measure of size standardizes a school's estimated grade
enrollment for calculating the school sampling probability.

Dividing a school's total student enrollment by its number of grades results in its estimated grade
enrollment. Schools with missing total student enrollment values receive estimated grade enrollments
equal to 20 students. The value 20 was chosen because it leads to a measure of size that yields a self-
weighting sample of students within the school.

A school's measure of size depends on its estimated grade enrollment. The purpose of this measure is
twofold. First, selecting schools with probability proportional to size (PPS) sampling reduces the variances
in NAEP. Second, sampling very small schools less frequently than their size would dictate reduces the
cost per student of data collection.

The table below provides the formula to calculate a school's measure of size.

Calculation of a school's measure of size, NAEP state

assessment: 2000

Estimated grade enrollment (EGE) Measure of size

EGE < 10 30
10 = EGE < 20 3 x EGE
20 = EGE < 65 60
EGE = 65 EGE
EGE unknown 60
SOURCE: U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics, National
Assessment of Educational Progress (NAEP), 2000
Assessment.

64
District Size and School Size for the 2000 State Assessment

District size and school size are two stratification variables used in the systematic sorting of the state
NAEP school sampling frame. The classification of schools by district size and school size occurs before
school sample selection. Both district size and school size have two possible values: "small" and "large".

Small school districts contain less than 20 percent of the jurisdiction's student population, while large
school districts contain 20 percent or more of the jurisdiction's student population.

The small school size class represents all schools with an estimated grade enrollment of less than 25
students, while the large school size class represents all schools with an estimated grade enrollment of
25 students or more.

65
School Sorting and Sample Selection in the 2000 State Assessment

The state NAEP school sampling frame undergoes a systematic sorting before school sampling begins.
This sorting ensures that the sampled schools represent a variety of population subgroups. The sorting
variables, listed in their hierarchical sort order, are as follows:

• small or large district status,

• small or large school size class,
• urbanization classification,
• minority enrollment classification, and
• achievement data or median household income.

For each participating jurisdiction, a sample of schools comes from the sorted school sampling frame, and
each school's probability of selection is proportional to its measure of size.

The table below shows the mean, minimum, and maximum number of schools sampled by grade and
school type within the 2000 state assessment's participating jurisdictions.

Mean, minimum, and maximum number of schools sampled, by grade, state

assessment: 2000

Mean Minimum Maximum

number of number of number of
Schools sampled schools sampled schools sampled schools
Fourth-grade schools 112 22 209
Eighth-grade schools 99 7 150
SOURCE: U.S. Department of Education, Institute of Education Sciences, National
Center for Education Statistics, National Assessment of Educational Progress (NAEP),
2000 Assessment.

66
Sparse State Sample Option for the 2000 State Assessment

Jurisdictions with student populations largely concentrated in small schools find standard NAEP sample
design requirements burdensome. These jurisdictions have to select large numbers of schools to reach
the required student sample sizes. They bear a large burden in school recruitment and assessment
administration, but they do not qualify for reductions in student sample sizes. The Sparse State Sample
Option addresses these problems while maintaining adequate sampling standards for representation and
precision.

Any participating jurisdiction with at least 120 schools selected in the full school sample qualifies for the
Sparse State Sample Option. This option selects a proportional sample of schools that reduces both
school and student sample sizes. The new sample meets the following criteria:

• The number of selected schools equals at least 115;

• The number of selected schools for each assessment subject equals at least 80;

• The sampling probability of each school equals at least half of its probability in the full sample;
and

• The sample retains the jurisdiction's largest schools and their student sample sizes.

The last two conditions imply retaining all of a jurisdiction's large schools and at least half of its small
schools. In practice, jurisdictions exercising this option see their school samples drop to 115 schools.
Student samples within the schools retain at least half, and often more, of their original sizes.

67
Selection of Schools in Small Jurisdictions in the 2000 State Assessment

The table below contains the participating jurisdictions that had all schools selected in the 2000 state
assessment by grade. The symbol § indicates that the jurisdiction had all schools selected for the state
assessment for the indicated grade, while the symbol ~ indicates that the jurisdiction did not have all
schools selected for the state assessment for the indicated grade. The table only contains those
participating jurisdictions that had at least one grade where all schools were selected.

Jurisdictions with all schools selected, state

assessment: By jurisdiction, 2000

Jurisdiction Grade 4 Grade 8

Hawaii ~ §
Rhode Island ~ §
District of Columbia § §
DDESS1 § §
DoDDS2 § §
Guam § §
Virgin Islands § §
~ The jurisdiction did not have all schools selected
for the state assessment for the indicated grade.
§ The jurisdiction had all schools selected for the
state assessment for the indicated grade.
1
Department of Defense Domestic Dependent
Elementary and Secondary Schools.
2
Department of Defense Dependents Schools
(Overseas).
SOURCE: U.S. Department of Education, Institute of
Education Sciences, National Center for Education
Statistics, National Assessment of Educational
Progress (NAEP), 2000 Assessment.

68
Effect of Sparse State Sample Option on Jurisdictions in the 2000 State
Assessment

Eligible jurisdictions for the Sparse State Sample Option in the 2000 state assessment included:

• Grade 4: Alaska, Arkansas, Colorado, Idaho, Illinois, Iowa, Kansas, Kentucky, Maine, Michigan,
Minnesota, Missouri, Montana, Nebraska, New Hampshire, New Mexico, North Dakota, Ohio,
Oklahoma, Oregon, South Dakota, Vermont, Washington, West Virginia, Wisconsin, Wyoming.
• Grade 8: Alaska, Kansas, Minnesota, Montana, Nebraska, North Dakota, Oklahoma, South
Dakota.

Of the 26 jurisdictions eligible for the option among fourth-grade schools, 14 jurisdictions exercised the
option. Of the eight jurisdictions eligible for the option among eighth-grade schools, four jurisdictions
exercised the option. Note that Alaska requested the Sparse State Sample Option for the 2000 state
assessment, but later declined to participate in the assessment.

The table below shows the effects of the Sparse State Sample Option on sample sizes in those 2000
state assessment jurisdictions that exercised the option.

Effect of sparse state sample option on jurisdictions, by grade, state

assessment: By jurisdiction, 2000

Reduced student
sample as
Original Reduced percentage
school school of original
Grade Jurisdiction sample sample student sample
4 Idaho 131 115 88
Illinois 123 115 94
Kansas 149 115 77
Michigan 121 115 95
Minnesota 132 115 87
Missouri 126 115 91
Montana 199 115 57
Nebraska 213 115 54
New Mexico 123 115 94
Ohio 121 115 95
Oregon 128 115 90
Vermont 173 115 69
Wisconsin 130 115 88
Wyoming 175 115 67
8 Kansas 124 115 93
Minnesota 126 115 91
Montana 133 115 91
Nebraska 147 115 81
SOURCE: U.S. Department of Education, Institute of Education Sciences,
National Center for Education Statistics, National Assessment of
Educational Progress (NAEP), 2000 Assessment.

69
Control of Overlap of School Samples for National Educational Studies in the
2000 State Assessment

A comparison of joint national main and state assessments usually finds an overlap of sampled schools.
Schools selected for both assessments spend a fair amount of time preparing and administering two sets
of educational tests. Concern about school sample overlap arises each assessment cycle. NAEP has a
policy of avoiding national and state sample overlap where possible. This policy avoids excess burden on
individual schools without distorting the resulting school samples with either bias or substantial variance.

To control overlap between state and national main samples, a procedure is used that conditions on the
national NAEP primary sampling unit (PSU) sample. National school selection probabilities (for the
national main NAEP assessment) that are conditional on the selection of national main sample PSUs are
used in determining state assessment school selection probabilities. Participating jurisdictions not
containing national main PSUs do not have their state assessment school selection probabilities adjusted.

The overlap control procedure reduces the variance of the state samples, although it leads to a greater
degree of sample overlap than using unconditional national selection probabilities. However, this degree
of sample overlap is less than would be achieved without any overlap control at all. The procedure
additionally recognizes the impact of within-PSU sampling in noncertainty PSUs in some jurisdictions.
Schools necessary for the state sample are not subject to overlap control; such schools are self-
representing in the state sample. Excluding such schools on a random basis adds undue variance to the
state estimates.

The state assessment's conditional selection probabilities generated by the overlap control procedure
tend to be smaller than the unconditional state selection probabilities for schools selected for the national
main sample. Unconditional probabilities obtained by integrating over the national sampling process equal
the required state assessment probabilities without any overlap control implementation. Such probabilities
mean that a school's unconditional probability of state assessment selection stays the same regardless of
overlap control implementation.

More information about the sample overlap procedures appears in the Technical Report of the NAEP
1994 Trial State Assessment Program in Reading (Mazzeo, Allen, and Kline 1995).

70
Number of Schools Selected for Both the State and National Main Assessments in
2000

The table below lists the number of schools selected for both the 2000 state and national main
assessments. Note that for the 2000 national main assessment, 573 non-new schools were selected at
the fourth-grade level, 480 at the eight-grade level, and 319 at the twelfth-grade level.

Number of schools selected for both state and national main

assessments: 2000

State National main assessment schools

assessment Fourth-grade Eighth-grade Twelfth-grade
schools schools schools schools
Fourth-grade schools 5 4 3
Eighth-grade schools 6 34 4
SOURCE: U.S. Department of Education, Institute of Education Sciences,
National Center for Education Statistics, National Assessment of
Educational Progress (NAEP), 2000 Assessment.

71
Selection of New Public Schools for the 2000 State Assessment

A supplemental sample of new public schools reflects additions to the target population that occur after
the creation of the sampling frame. New public schools include:

• recently opened schools not listed on the original sampling frame; and

• schools existing on the sampling frame that changed grade spans to include an eligible grade.

Because public schools are organized into districts, and district personnel (not school personnel) will be
knowledgeable about the existence of new schools, a sample of school districts is selected in order to
compile lists of new schools. A school district level file constructed from the Common Core of Data (CCD)
public school file forms the district sampling frame. The CCD public school file, maintained by the National
Center for Education Statistics, contains information about elementary and secondary schools in the 50
states, District of Columbia, and outlying areas.

The district sampling frame separates school districts into two groups:

• Small districts: Districts that contain at most three schools and no more than one fourth-grade,
one eighth-grade, and one twelfth-grade school; and

• Medium/large districts: Districts that contain four or more schools, two or more fourth-grade
schools, two or more eighth-grade schools, or two or more twelfth-grade schools.

The selection of new schools excludes small school districts. School recruitment during the state
assessment identifies any new schools in the small school districts.

The district sampling frame yields a sample of school districts to contact regarding the existence of new
schools. The school district sample includes all school districts from those participating jurisdictions that
sample all eligible schools. For each of the other participating jurisdictions, the school district sample
includes a combination of 10 medium and large school districts.

To create this school district sample for a participating jurisdiction, the medium and large school districts
in the jurisdiction are first sorted by student enrollment, from the smallest enrollment to the largest
enrollment. The student enrollments of the sorted school districts are summed, starting with the district
with the smallest school enrollment. Summing is stopped at the school district that causes the cumulative
student enrollment to exceed 20 percent of the jurisdiction's student enrollment. All school districts in the
sorted list above (but not including) that school district forms the smaller district sampling group, while all
other school districts form the larger district sampling group.

Two school districts are selected from the group of smaller school districts with equal probability. Eight
school districts are selected from the group of larger school districts with probability proportional to
student enrollment. For jurisdictions with less than eight school districts in its larger district group, all
school districts from the larger district group are selected, as well as enough districts from the smaller
district group to make a total sample of 10 districts.

Each selected school district receives a list of its public schools that appear on the original school
sampling frame. These districts, in return, provide information about either new schools or new grade-
eligible schools not included on the lists. These new school data form the sampling frame for the selection
of new schools.

72
From each participating jurisdiction's new school data comes a sample of new schools. The following
formula gives the probability of the selection of a school within a jurisdiction:

⎡ sampling rate × measure of size ⎤

minimum ⎢ , 1⎥
⎣ P(district) ⎦

where

• sampling rate equals the sampling rate used for the particular jurisdiction in the original school
selection sample;
• measure of size equals a standardized measure of the school's estimated grade enrollment (see
Estimated Grade Enrollment and Measure of Size for more information); and
• P(district) equals the probability of selection of a school district within the particular jurisdiction.

Newly grade-eligible schools within the new school sample keep the measure of size calculated during
the original school sample.

The new school sample includes two additional groups of schools. First, it adds all new and newly grade-
eligible schools found during the field period for medium and large school districts in participating
jurisdictions that sample all eligible schools. Second, it adds all new and newly grade-eligible schools
coming from small school districts that have a school selected in the original sample.

73
Public School Sampling Frame and New-School Sample Sizes

The table below shows the number of new schools from participating jurisdictions for the 2000 state
assessment. Jurisdictions not listed in these tables did not have any new schools selected as part of the
sampling procedure.

Public school sampling frame and new-school sample sizes, grades 4 and 8: By jurisdiction, 2000

Fourth-grade Eighth-grade
Common Common
Core Core
of Data Fourth-grade Total fourth- of Data Eighth-grade Total eighth-
(CCD) school new school grade school (CCD) school new school grade school
Jurisdiction sample sample sample sample sample sample
Total 5,220 62 5,280 4,429 64 4,490
Alabama 117 3 120 114 4 120
Arizona 116 0 120 114 0 110
Arkansas 123 1 120 113 0 110
California 111 1 110 110 2 110
Connecticut 114 1 120 108 1 110
Georgia 107 2 110 107 2 110
Hawaii 111 0 110 59 2 60
Idaho 115 0 120 95 2 100
Illinois 115 0 120 117 0 120
Indiana 117 0 120 108 0 110
Iowa 141 0 140 † † †
Kansas 103 3 110 105 0 110
Kentucky 125 0 130 115 0 120
Louisiana 117 4 120 114 2 120
Maine 156 0 160 113 1 110
Maryland 110 1 110 107 2 110
Massachusetts 116 0 120 107 0 110
Michigan 114 0 110 113 2 120
Minnesota 115 1 120 115 1 120
Mississippi 116 2 120 112 2 110
Missouri 115 2 120 119 2 120
Montana 115 0 120 115 0 120
Nebraska 115 0 120 115 0 120
Nevada 113 4 120 65 3 70
New Mexico 115 0 120 100 0 100
New York 109 0 110 108 1 110
North Carolina 112 0 110 110 1 110
North Dakota 207 2 210 150 0 150
Ohio 115 1 120 109 1 110
Oklahoma 137 0 140 127 0 130
Oregon 116 0 120 115 0 120
Rhode Island 114 6 120 52 3 60
South Carolina 110 0 110 106 0 110
Tennessee 116 0 120 112 0 110
Texas 113 1 110 115 0 120
Utah 111 1 110 101 0 100
Vermont 113 0 110 107 0 110
Virginia 112 1 110 109 0 110
See notes at end of table.

74
Public school sampling frame and new-school sample sizes, grades 4 and 8: By jurisdiction, 2000

Fourth-grade Eighth-grade
Common Common
Core Core
of Data Fourth-grade Total fourth- of Data Eighth-grade Total eighth-
(CCD) school new school grade school (CCD) school new school grade school
Jurisdiction sample sample sample sample sample sample
West Virginia 157 2 160 111 2 110
Wisconsin 115 1 120 113 0 110
Wyoming 115 2 120 83 12 100
Other jurisdictions
American Samoa 22 0 20 22 1 20
District of Columbia 106 15 120 32 13 50
DDESS1 40 0 40 14 0 10
DoDDS2 96 1 100 59 1 60
Guam 23 4 30 7 1 10
Virgin Islands 24 0 20 7 0 10
† Iowa did not participate in the state assessment at grade 8.

1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

NOTE: Nonparticipating jurisdictions are not included in this table. These jurisdictions are Alaska, Colorado,

Delaware, Florida, New Jersey, New Hampshire, Pennsylvania, South Dakota, and Washington. The table

includes a new school from a small district that was discovered during the data collection period (Maine grade

8). Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2000 Assessment.

75
School Substitution and Retrofitting in the 2000 State Assessment

Substitution for sampled schools that do not participate in the assessment provides a means to reduce
the nonresponse bias that may result from school nonparticipation, and to maintain an acceptable yield of
assessed students. Substitution cannot eliminate school nonresponse bias. However, to the extent that a
nonparticipating school can be replaced in the sample by a school, not originally selected in the sample,
that has similar characteristics, nonresponse bias is likely to be reduced. Thus the 2000 NAEP state
assessment included a procedure for associating a potential substitute school with each school in the
sample, where feasible.

An automated substitute selection mechanism assigns each sampled school a substitute school, if
possible, before the field period begins. Schools and their substitutes share the same jurisdiction and
urbanization classification. If possible, substitutes for public schools come from public schools in other
school districts. This different district strategy arises from the fact that public school nonresponse may be
due to district-level refusal.

A distance measure between a sampled school and each potential substitute school determines how well
the schools match each other. This measure incorporates differences between the paired schools among
four component variables:

• Estimated grade enrollment;

• Achievement data or median household income;
• Largest student minority enrollment; and
• Second largest student minority enrollment.

Each difference was squared and standardized to the population standard deviation for the component
variable across all grade-eligible schools and jurisdictions. The distance measure equals the sum of the
four resulting squared standardized differences.

For example, compare the following two schools from the NAEP 2000 state assessment:

School A:

• Estimated grade enrollment = 49

• Achievement score = 221
• Largest minority enrollment percentage = 29
• Second largest minority enrollment percentage = 24

School B:

• Estimated grade enrollment = 49

• Achievement score = 223
• Largest minority enrollment percentage = 25
• Second largest minority enrollment percentage = 21

Using the population standard deviation calculated for each component variable in the NAEP 2000
assessment, the distance measure between schools A and B equals 0.40. School B would be assigned
as a substitute school for school A if no other school had a smaller distance measure.

The automated substitute selection mechanism chooses the potential substitute with the smallest
distance measure as the substitute for a given school. The mechanism assigns only one substitute school
to a sampled school. Once a school has been assigned as a substitute, it can not be used again. For the
first pass, the maximum acceptable distance is 0.60, and public school substitutes must come from a
different school district than the sampled school. Some sampled schools do not receive an assigned

76
substitute school in the first pass. Either the sampled schools outnumber the substitute schools, or the
distance measure exceeds 0.60 for the remaining substitute schools.

For the second pass, the mechanism raises the maximum acceptable distance measure to 0.75. It also
lifts the different district constraint for public schools. These steps produce a small number of additional
assigned substitutes.

Note the selected distance measure cutoff points of 0.60 and 0.75 started with the 1994 Trial State
Assessment. Before that assessment, statisticians reviewed a large number of school listings, calculated
the distance measures between school pairs, and agreed on the cutoff points at which substitute schools
appear unacceptable. The selected cutoff points have been used since the 1994 assessment. More
information can be found in Technical Report of the NAEP 1994 Trial State Assessment Program in
Reading (Mazzeo, Allen, and Kline 1995).

When substitutes were assigned, it was sometimes the case that there was just a single school that was
a suitable substitute for two sampled schools. Thus the sampled school that had the smaller distance
measure to that potential substitute (school A) received it as an assigned substitute, while the second
sampled school (school B) received no substitute. Once the field period began, on occasion it transpired
that school A participated, while school B did not. In a case such as this, the substitute school was
reassigned, and became a substitute for school B. In this way substitute schools were assigned to as
many sampled schools as possible.

77
Number of Substitute Schools in the 2000 State Assessment

The links to the right connect the user to tables that provide the number of original and participating
schools in the 2000 state assessment samples, together with weighted school participation rates, within
each jurisdiction by grade and assessment subject. These tables also show the number of substitute
schools in each jurisdiction associated with nonparticipating original schools and the number of those
substitute schools that participated.

The table below includes information about the number of substitute schools provided in each
participating 2000 state assessment jurisdiction by grade. In jurisdictions where all of the eligible schools
were included in the original sample, there were no schools available to serve as substitutes. This is
indicated by a zero entry in the table.

Number of substitute schools provided in each

jurisdiction by grade, state assessment: By
jurisdiction, 2000

Jurisdiction Grade 4 Grade 8

Total 4,454 3,072
Alabama 116 96
Arizona 111 77
Arkansas 106 73
California 112 110
Connecticut 111 65
Georgia 109 102
Hawaii 53 0
Idaho 105 44
Illinois 112 112
Indiana 117 99
Iowa 137 —
Kansas 113 87
Kentucky 124 97
Louisiana 115 96
Maine 127 69
Maryland 110 92
Massachusetts 114 98
Michigan 113 113
Minnesota 108 95
Mississippi 102 76
Missouri 116 109
Montana 108 76
Nebraska 111 78
Nevada 89 12
New Mexico 92 21
New York 107 94
North Carolina 110 105
North Dakota 92 70
Ohio 112 104
Oklahoma 132 106
Oregon 115 90
See notes at end of table.

78
Number of substitute schools provided in each
jurisdiction by grade, state assessment: By
jurisdiction, 2000 (continued)

Jurisdiction Grade 4 Grade 8

Rhode Island 66 0
South Carolina 101 81
Tennessee 114 99
Texas 112 110
Utah 112 20
Vermont 94 25
Virginia 110 91
West Virginia 149 61
Wisconsin 116 106
Wyoming 81 13
Other jurisdictions
American Samoa 0 0
District of Columbia 0 0
DDESS1 0 0
DoDDS2 0 0
Guam 0 0
Virgin Islands 0 0
— Not available; Iowa did not participate in the state
assessment at grade 8.
1
Department of Defense Domestic Dependent
Elementary and Secondary Schools.
2
Department of Defense Dependents Schools
(Overseas).
SOURCE: U.S. Department of Education, Institute of
Education Sciences, National Center for Education
Statistics, National Assessment of Educational
Progress (NAEP), 2000 State Assessment.

79
Distribution of school sample, grade 4 mathematics state assessment: By jurisdiction, 2000

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Alabama 87.3 94.4 117 3 100 14 8 108
Arizona 87.5 87.5 111 3 95 12 0 95
Arkansas 86.7 86.7 117 3 99 12 0 99
California 75.9 75.9 110 3 81 26 0 81
Connecticut 100.0 100.0 112 6 106 0 0 106
Georgia 99.1 99.1 109 1 107 1 0 107
Hawaii 99.1 99.1 109 0 108 1 0 108
Idaho 74.0 74.6 105 4 76 22 1 77
Illinois 73.7 73.7 109 5 78 25 0 78
Indiana 71.0 71.0 114 3 80 31 0 80
Iowa 70.5 70.5 129 2 90 36 0 90
Kansas 71.3 71.3 106 3 79 23 0 79
Kentucky 91.5 94.4 115 5 101 9 3 104
Louisiana 100.0 100.0 117 8 109 0 0 109
Maine 86.0 86.0 129 1 108 19 0 108
Maryland 100.0 100.0 110 1 109 0 0 109
Massachusetts 99.0 99.0 113 7 105 1 0 105
Michigan 72.5 84.8 107 7 73 27 12 85
Minnesota 82.7 82.7 102 9 77 16 0 77
Mississippi 98.1 98.1 115 5 108 0 0 108
Missouri 96.4 96.4 108 3 101 4 0 101
Montana 75.2 77.1 87 5 59 20 2 61
Nebraska 97.3 97.3 87 6 79 2 0 79
Nevada 100.0 100.0 112 3 109 0 0 109
New Mexico 93.0 93.0 109 2 100 6 0 100
New York 71.4 71.4 106 1 76 28 0 76
North Carolina 100.0 100.0 110 3 107 0 0 107
North Dakota 88.4 88.4 154 6 131 3 0 131
Ohio 81.9 81.9 111 6 86 18 0 86
See notes at end of table.

80
Distribution of school sample, grade 4 mathematics state assessment: By jurisdiction, 2000 (continued)

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Oklahoma 100.0 100.0 116 2 114 0 0 114
Oregon 72.9 74.0 106 2 77 26 1 78
Rhode Island 100.0 100.0 119 7 112 0 0 112
South Carolina 96.6 96.6 108 0 104 3 0 104
Tennessee 96.7 96.7 112 4 104 4 0 104
Texas 96.9 99.0 110 8 99 3 2 101
Utah 100.0 100.0 109 0 109 0 0 109
Vermont 70.3 70.3 88 1 61 20 0 61
Virginia 100.0 100.0 109 3 106 0 0 106
West Virginia 100.0 100.0 134 11 123 0 0 123
Wisconsin 66.6 68.8 107 2 68 37 2 70
Wyoming 100.0 100.0 98 4 94 0 0 94
Other jurisdictions
American Samoa 100.0 100.0 16 0 16 0 0 16
District of Columbia 99.3 99.3 121 10 110 0 0 110
DDESS1 100.0 100.0 40 0 40 0 0 40
DoDDS2 100.0 100.0 87 1 86 0 0 86
Guam 96.6 96.6 27 2 25 0 0 25
Virgin Islands 100.0 100.0 23 0 23 0 0 23
1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2000 Mathematics Assessment.

81
Distribution of school sample, grade 8 mathematics state assessment: By jurisdiction, 2000

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Alabama 81.8 91.3 115 3 91 20 11 102
Arizona 76.1 76.1 108 4 79 19 0 79
Arkansas 87.4 87.4 110 3 94 10 0 94
California 71.5 71.5 109 3 76 29 0 76
Connecticut 99.1 99.1 108 4 104 0 0 104
Georgia 99.0 99.0 109 6 102 1 0 102
Hawaii 91.2 91.2 59 6 51 0 0 51
Idaho 78.4 78.4 91 9 66 2 0 66
Illinois 74.8 74.8 111 8 78 25 0 78
Indiana 72.8 72.8 107 2 76 27 0 76
Kansas 70.6 70.6 105 1 74 17 0 74
Kentucky 94.5 95.5 110 8 96 5 1 97
Louisiana 100.0 100.0 112 8 104 0 0 104
Maine 83.0 84.4 102 2 82 9 2 84
Maryland 98.1 98.1 108 1 105 1 0 105
Massachusetts 99.0 99.0 107 7 99 1 0 99
Michigan 71.1 80.7 110 7 75 28 10 85
Minnesota 73.8 73.8 104 17 64 15 0 64
Mississippi 97.9 97.9 110 7 101 1 0 101
Missouri 92.3 94.2 115 5 102 6 2 104
Montana 73.9 75.0 92 2 64 16 1 65
Nebraska 98.5 98.5 92 8 83 0 0 83
Nevada 100.0 100.0 65 2 63 0 0 63
New Mexico 91.0 91.0 95 4 83 0 0 83
New York 69.9 69.9 109 3 74 29 0 74
North Carolina 99.0 99.0 108 3 104 1 0 104
North Dakota 90.5 90.5 108 1 95 3 0 95
Ohio 90.7 90.7 108 12 87 7 0 87
See notes at end of table.

82
Distribution of school sample, grade 8 mathematics state assessment: By jurisdiction, 2000 (continued)

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Oklahoma 99.4 99.4 115 1 113 1 0 113
Oregon 74.7 74.7 111 2 81 20 0 81
Rhode Island 100.0 100.0 54 3 51 0 0 51
South Carolina 91.2 92.2 106 3 94 4 1 95
Tennessee 88.8 90.8 107 2 93 11 2 95
Texas 92.9 95.7 111 2 101 8 3 104
Utah 100.0 100.0 99 3 96 0 0 96
Vermont 81.9 81.9 92 0 76 1 0 76
Virginia 100.0 100.0 107 2 105 0 0 105
West Virginia 100.0 100.0 108 4 104 0 0 104
Wisconsin 65.3 73.4 110 1 70 36 9 79
Wyoming 100.0 100.0 81 10 71 0 0 71
Other jurisdictions
American Samoa 100.0 100.0 15 1 14 0 0 14
District of Columbia 100.0 100.0 45 10 34 0 0 34
DDESS1 100.0 100.0 13 13 0 0 13
DoDDS2 100.0 100.0 54 3 51 0 0 51
Guam 100.0 100.0 8 1 7 0 0 7
Virgin Islands 100.0 100.0 6 0 6 0 0 6
1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2000 Mathematics Assessment.

83
Distribution of school sample, grade 4 science state assessment: By jurisdiction, 2000

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Alabama 87.0 94.4 117 2 100 15 9 109
Arizona 86.8 86.8 112 2 95 14 0 95
Arkansas 85.1 85.1 113 3 93 13 0 93
California 75.9 75.9 110 3 81 26 0 81
Connecticut 100.0 100.0 113 6 107 0 0 107
Georgia 99.1 99.1 109 1 107 1 0 107
Hawaii 98.4 98.4 108 0 106 1 0 106
Idaho 74.9 74.9 104 1 78 22 0 78
Illinois 73.1 73.1 108 5 77 25 0 77
Indiana 70.0 70.0 113 3 78 32 0 78
Iowa 70.6 70.6 129 4 89 35 0 89
Kentucky 91.6 94.4 116 5 102 9 3 105
Louisiana 100.0 100.0 116 8 108 0 0 108
Maine 85.5 85.5 131 4 107 19 0 107
Maryland 100.0 100.0 111 1 110 0 0 110
Massachusetts 99.0 99.0 113 6 106 1 0 106
Michigan 71.0 83.5 106 7 71 28 12 83
Minnesota 83.0 83.0 103 9 78 16 0 78
Mississippi 98.0 98.0 114 6 106 0 0 106
Missouri 96.5 96.5 109 2 103 4 0 103
Montana 75.9 77.5 89 1 66 19 1 67
Nebraska 95.7 95.7 87 11 73 3 0 73
Nevada 100.0 100.0 112 3 109 0 0 109
New Mexico 92.9 92.9 107 2 98 6 0 98
New York 72.2 72.2 109 1 79 28 0 79
North Carolina 100.0 100.0 110 2 108 0 0 108
North Dakota 88.9 88.9 150 4 129 5 0 129
See notes at end of table.

84
Distribution of school sample, grade 4 science state assessment: By jurisdiction, 2000 (continued)

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Ohio 81.7 81.7 111 7 85 17 0 85
Oklahoma 98.6 98.6 123 1 120 2 0 120
Oregon 73.1 74.2 108 3 78 26 1 79
Rhode Island 100.0 100.0 117 7 110 0 0 110
South Carolina 96.6 96.6 107 0 103 3 0 103
Tennessee 96.7 96.7 112 3 105 4 0 105
Texas 96.9 99.0 111 10 98 3 2 100
Utah 100.0 100.0 110 0 110 0 0 110
Vermont 74.7 74.7 88 0 66 16 0 66
Virginia 100.0 100.0 111 3 108 0 0 108
West Virginia 100.0 100.0 134 8 126 0 0 126
Wisconsin 65.2 67.3 108 2 67 39 2 69
Wyoming 100.0 100.0 98 5 93 0 0 93
Other jurisdictions
American Samoa 100.0 100.0 17 0 17 0 0 17
DDESS1 100.0 100.0 39 0 39 0 0 39
DoDDS2 100.0 100.0 85 1 84 0 0 84
Guam 95.6 95.6 26 2 23 0 0 23
Virgin Islands 100.0 100.0 22 0 22 0 0 22
1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2000 Science Assessment.

85
Distribution of school sample, grade 8 science state assessment: By jurisdiction, 2000

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Alabama 82.3 91.8 115 4 91 19 11 102
Arizona 75.9 75.9 109 3 80 20 0 80
Arkansas 87.2 87.2 108 3 92 10 0 92
California 72.1 72.1 110 5 76 28 0 76
Connecticut 100.0 100.0 107 3 104 0 0 104
Georgia 99.0 99.0 108 5 102 1 0 102
Hawaii 91.2 91.2 58 6 50 0 0 50
Idaho 78.0 78.0 87 8 63 2 0 63
Illinois 75.0 75.0 111 6 80 25 0 80
Indiana 73.0 73.0 106 2 76 26 0 76
Kentucky 94.4 95.5 108 7 95 5 1 96
Louisiana 100.0 100.0 112 8 104 0 0 104
Maine 83.4 84.8 103 1 84 9 2 86
Maryland 97.2 97.2 107 1 103 2 0 103
Massachusetts 99.0 99.0 106 6 99 1 0 99
Michigan 71.5 81.0 109 5 76 28 10 86
Minnesota 73.3 73.3 103 22 59 15 0 59
Mississippi 97.9 97.9 111 8 101 1 0 101
Missouri 92.3 94.2 113 3 102 6 2 104
Montana 73.3 74.4 91 4 61 15 1 62
Nebraska 98.4 98.4 94 5 87 0 0 87
Nevada 100.0 100.0 65 1 64 0 0 64
New Mexico 90.6 91.3 96 3 84 1 1 85
New York 70.7 70.7 108 3 74 28 0 74
North Carolina 98.1 98.1 109 4 103 2 0 103
North Dakota 90.7 90.7 106 1 93 4 0 93
Ohio 90.7 90.7 108 11 88 7 0 88
Oklahoma 100.0 100.0 115 1 114 0 0 114
See notes at end of table.

86
Distribution of school sample, grade 8 science state assessment: By jurisdiction, 2000 (continued)

Weighted percent of school Number of schools in the original Number of substitute schools for
participation sample nonparticipating originals
Before After Total
Jurisdiction substitution substitution Total Not eligible Participated Provided Participated participated
Oregon 73.9 73.9 109 2 78 21 0 78
Rhode Island 100.0 100.0 54 2 52 0 0 52
South Carolina 91.2 92.2 105 2 94 4 1 95
Tennessee 89.6 91.6 109 3 95 10 2 97
Texas 91.4 94.3 110 3 97 10 3 100
Utah 100.0 100.0 98 3 95 0 0 95
Vermont 80.2 80.2 92 0 74 2 0 74
Virginia 100.0 100.0 108 3 105 0 0 105
West Virginia 100.0 100.0 108 6 102 0 0 102
Wisconsin 66.3 74.5 109 1 71 34 9 80
Wyoming 100.0 100.0 74 10 64 0 0 64
Other jurisdictions
American Samoa 95.7 95.7 18 1 16 0 0 16
DDESS1 100.0 100.0 14 0 14 0 0 14
DoDDS2 100.0 100.0 55 2 53 0 0 53
Guam 100.0 100.0 8 1 7 0 0 7
Virgin Islands 100.0 100.0 7 0 7 0 0 7
1
Department of Defense Domestic Dependent Elementary and Secondary Schools.

2
Department of Defense Dependents Schools (Overseas).

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2000 Science Assessment.

87
Assignment of Subject, Sample Type, Monitor Status, and Drop Flag in the 2000
State Assessment

Each sampled school holds one or more assessment sessions. Determining what assessment sessions
each school administers depends on four session characteristics:

• Assessment subject indicates the tested academic subject for the session. These subjects vary
by grade and NAEP assessment.
• Sample type indicates whether or not the session allows accommodations for students with
disabilities or limited English proficiency.
• Monitor status indicates whether or not a NAEP representative monitors the assessment session.
• A drop flag indicates which assessment subject to drop if a school assigned assessment sessions
in both subjects can only handle a single session.

88
Assessment Subject in the 2000 State Assessment

The 2000 state assessment focused on the students' knowledge of mathematics and science. All schools
with 25 or more eligible students participated in both mathematics and science assessments. Students
from such schools participated in either the mathematics assessment or the science assessment, but not
both assessments. All eighth-grade schools with fewer than 25 eligible students participated in a
randomly selected assessment.

89
Sample Type in the 2000 State Assessment

Each state assessment uses the inclusion rules (established in 1996) for students with disabilities (SD)
and limited-English-proficient (LEP) students. Sample type denotes whether or not a session may allow
such accommodations. Sample type 2 (S2) provides no accommodations for SD/LEP students, while the
sample type 3 (S3) allows SD/LEP accommodations.

For an assessment subject that allows accommodations, sample type assignment begins by considering
all schools assigned an assessment session in the subject. After sorting the schools by jurisdiction,
sampled grade, and assessment subject, the assignment of sample type (S2 or S3) to the schools occurs
in an alternating pattern.

For the 2000 state assessment, both the mathematics and science subjects allowed SD, LEP, and
SD/LEP accommodations. The 2000 state assessment student counts by sample type are provided on
the web pages Participating Fourth-Grade School Counts and Participating Eighth-Grade School Counts.

90
Monitor Status in the 2000 State Assessment

A random subsample of selected schools determines which schools NAEP representatives monitor during
the assessment field period. This subsample enables reliable comparisons between sessions
administered with and without monitoring.

The monitoring rate for each participating jurisdiction depends whether or not the jurisdiction took part in
an earlier state assessment. Jurisdictions participating in earlier state assessments have a 25 percent
school monitoring rate, while jurisdictions new to state assessments have a 50 percent school monitoring
rate.

Monitoring assignment starts first by sorting all sampled schools in the following order:

• jurisdiction;
• assessment subject;
• sample type (S2 or S3); and
• school identification number.

A systematic sample, determined by the jurisdiction's school monitoring rate, assigns monitoring status
(monitored or not monitored) to the sorted schools.

For the 2000 state assessment, American Samoa had a 50 percent school monitoring rate. All other
participating jurisdictions had a 25 percent school monitoring rate.

91
Drop Flag Assignment in the 2000 State Assessment

Schools assigned sessions in both assessment subjects occasionally turn out to be smaller than
expected. As students may only be assessed in one subject, these schools are only large enough to
administer one assessment subject. For these schools, the drop flag indicates what assessment subject
session to drop. Schools assigned only a single session ignore the drop flag.

The drop flag assignment starts by sorting all sampled schools in the following order:

• jurisdiction;
• assessment subject;
• number of assessment sessions (in descending order);
• sample type (S2 or S3); and
• school identification number.

An alternating pattern assigns an assessment subject to each sorted school as its drop flag.

92