National participation
To ensure unbiased samples, NAEP requires that participation rates for original school samples be 70 percent or higher to report national results separately for public and private schools. In instances where participation rates meet the 70 percent criteria but fall below 85 percent, a nonresponse bias analysis is conducted to determine if the responding school sample is not representative of the population, thereby introducing the potential for nonresponse bias.
Before substituting new schools for originally sampled schools that declined to participate, the weighted national school participation rates for the 2009 mathematics assessment were 97 percent for grade 4 (100 percent for public schools and 73 percent for private schools), 97 percent for grade 8 (100 percent for public schools and 72 percent for private schools), and 83 percent for grade 12 (86 percent for public and 52 percent for private). Weighted student participation rates were 95 percent at grade 4, 93 percent at grade 8, and 80 percent at grade 12. The nonresponse bias analysis for private schools at grades 4 and 8 showed that substitution and weight adjustments reduced the potential bias. At grade 12, the school participation rate for private schools fell below the standard for reporting.
Because the grade 12 weighted student participation rate in public schools was below 85 percent (79 percent), a student nonresponse bias analysis was conducted. That analysis showed that the responding student sample differed from the original student sample with respect to race, relative age, and student disability status. After adjusting the sampling weights to account for student nonresponse, the remaining bias was small, with the nonresponse adjusted estimates for the three identified variables differing from the unadjusted estimates by plus or minus 0.1 percentage points.
State and district participation
Standards established by the National Assessment Governing Board require that school participation rates for the original state samples need to be at least 85 percent for results to be reported. In 2009, all 52 states and jurisdictions, along with all 18 districts participating in the mathematics assessment at grades 4 and 8 met this participation rate requirement. Because the weighted school participation rate in Illinois was 84.5 percent, a school nonresponse bias analysis was conducted for the grade 12 public school sample in that state. That analysis showed that the potential nonresponse bias was effectively reduced by including substitute schools and adjusting the sampling weights to account for school nonresponse. The school participation rates for the remaining 10 states all met the required standard with weighted participation rates ranging from 92 to 100 percent.
Student nonresponse bias analyses were conducted at grade 12 for Connecticut, Florida, Illinois, Iowa, Massachusetts, New Hampshire, New Jersey, South Dakota, and West Virginia, where the student response rate fell below 85 percent. Across these nine states, after nonresponse adjustments, the percentage difference between nonresponse adjusted sample and eligible student sample was 1.3 percent or less. Based on the student characteristics available, there does not appear to be evidence of substantial bias resulting from student nonresponse.
The tables below provide participation rates before substitution.
Learn more about the sampling design.
|