|
The schools and students participating in NAEP assessments are selected to be representative of all schools nationally and of public schools at the state level. Samples of schools and students are drawn from each state and from the District of Columbia and Department of Defense schools. Because each school that participated in the assessment, and each student assessed, represents a portion of the population of interest, the results are weighted to account for the disproportionate representation of the selected sample. Read more technical information about weighting adjustments made at the school and student level.
National participation
To ensure unbiased samples, NAEP requires that participation rates for original school samples be 70 percent or higher to report national results separately for public and private schools. In instances where participation rates meet the 70 percent criteria but fall below 85 percent, a nonresponse bias analysis is conducted to determine if the responding school sample is not representative of the population, thereby introducing the potential for nonresponse bias.
Before substituting new schools for originally sampled schools that declined to participate, the student-weighted national school participation rates for the 2009 reading assessment were 97 percent for grade 4 (100 percent for public schools and 73 percent for private schools), 97 percent for grade 8 (100 percent for public schools and 72 percent for private schools), and 83 percent for grade 12 (86 percent for public schools and 52 percent for private schools). Weighted student participation rates were 95 percent at grade 4, 93 percent at grade 8, and 81 percent at grade 12. The nonresponse bias analysis for private schools at grades 4 and 8 showed that, while the original responding school sample may not have been fully representative, the potential bias was reduced by including substitute schools and by adjusting the sampling weights to account for school nonresponse. At grade 12, the school participation rate for private schools fell below the standard for reporting.
Because the grade 12 weighted student participation rate in public schools was below 85 percent (80 percent), a student nonresponse bias analysis was conducted. That analysis showed that the responding student sample differed from the original student sample with respect to race, relative age, and student disability status. After adjusting the sampling weights to account for student nonresponse, the remaining bias was small, with the nonresponse adjusted estimates for the three identified variables differing from the unadjusted estimates by plus or minus 0.1 percentage points.
State and district participation
Standards established by the National Assessment Governing Board require that school participation rates for the original state samples need to be at least 85 percent for results to be reported. In 2009, all 52 states and jurisdictions, along with all 18 districts participating in the reading assessment at grades 4 and 8 met this participation rate requirement. Because the weighted school participation rate in Illinois was 84.5 percent, a school nonresponse bias analysis was conducted for the grade 12 public school sample in that state. That analysis showed that the potential nonresponse bias was effectively reduced by including substitute schools and adjusting the sampling weights to account for school nonresponse. The school participation rates for the remaining 10 states all met the required standard with weighted participation rates ranging from 92 to 100 percent.
Student nonresponse bias analyses were conducted at grade 12 for Connecticut, Florida, Illinois, Iowa, Massachusetts, New Hampshire, New Jersey, South Dakota, and West Virginia, where the student response rate fell below 85 percent. Across these nine states, after nonresponse adjustments, the percentage difference between nonresponse adjusted sample and eligible student sample was 1.3 percent or less. Based on the student characteristics available, there does not appear to be evidence of substantial bias resulting from student nonresponse.
The tables below provide participation rates before substitution.
Learn more about the sampling design.
|