David Anderson: Do the NECAP Test Results Mislead?
Secretary of Education Arne Duncan recently lamented, “I think we are lying to children and families when we tell children that they are meeting standards and, in fact, they are woefully unprepared to be successful in high school and have almost no chance of going to a good university and being successful.” Do his remarks pertain to Rhode Island’s 8th grade public school children? Is it the Rhode Island Department of Education (RIDE) that is misrepresenting their skills through their use of the New England Common Assessment Program (NECAP) tests?
Recently released public school achievement results from the NECAP test seem to confirm these worries. The proficiencies reported not only exaggerate the performance of Rhode Island students in mathematics and reading but also claim improving 8th grade reading scores when, in fact, they are declining. This seems inconsistent with the transparency that was promised.
Stakeholders, including Governor Carcieri, have been pushing Rhode Island educators to improve student performance. In response the NECAP 8th grade reading scores are giving them only the illusion of improvement. Illusion is a poor substitute for truth. These advocates of better education have been blindsided by RIDE and the NECAP testing regime.
To see the problem, please consider the 8th grade NECAP results in recent years. They are significantly inconsistent with those of the well-respected Nation’s Report Card- also known as the National Assessment of Educational Progress (NAEP).
The NECAP measure of skills should roughly reflect that of the NAEP. It fails to do so in two regards:
Firstly, the NECAP exaggerates. It reports that roughly twice as many Rhode Island 8th grade children are proficient (at or above grade level) as are reported by the NAEP. We are told that last year 53% of Rhode Island pupils were proficient in math and reading when, in fact, the NAEP trend says it’s only 26%.
Perhaps worse, and what is new to us, is the fact that NECAP reading proficiencies have been increasing over time while the NAEP shows declines. We call this the up-down problem.
You can see this in the graph, which shows the 8th grade reading proficiencies reported by the NECAP and those interpolated or reported for the NAEP. The vertical separation of the plots shows the exaggeration effect while the up-slope versus down-slope represents the up-down problem.
We also reviewed the performance of 8th grade pupils in the best school district, of Barrington, and in one of the worst districts, of Providence. Additionally, using a mapping technique we developed, we have estimated NAEP proficiencies for them.• By considering these cases parents and other stakeholders can get some sense of what’s wrong with our public schools.
All must seem fine to Barrington 8th grade parents when the NECAP is claiming student proficiency percentages exceeding 90%. But not when they see our NAEP estimates showing one-third of their children below grade level.
Parents in Providence, while not pleased by the NECAP results showing only 28% of their 8th grade children proficient in both subjects, would be justifiably outraged if they knew that the more trustworthy NAEP estimates suggest that less than 10% are at or above grade level. If these numbers are correct, Providence schools are extremely dysfunctional.
How could these inconsistent results arise? It’s fairly clear that the exaggeration problem is due to the policies of the NECAP authorities and, indirectly, of the participating states’ departments of education.
As to the up-down problem of erroneously rising NECAP reading proficiencies, that uptrend could be due to a number of causes. Other states have seen officials gaming the tests to produce artificial gains- as happened in California some years ago. While we can’t entirely rule out this possibility, we think the explanation lies more in the realm of good intentions gone awry.
Maybe the NECAP curriculum content is narrower than that of the NAEP and is thus more easily taught? It’s like learning a booklet instead of a book. The one is mastered and the other not so well. Then scores go up for the one and down for the other.
But that is just a hypothesis. Further study of this is needed- probably by outside independent experts. Consideration should also be given to conducting the assessment function through an independent agency to remove concerns about conflicts of interest. That is the practice of Massachusetts’s MCAS test.
Do the authorities of the Rhode Island Department of Education (RIDE) take these inconsistencies seriously?
As to the exaggeration problem, consider that at least one RIDE official apparently thinks that the NAEP standard is too high and should be lowered. She says, “NAEP ‘proficient’ is a very high aspirational standard.”
It suggests that these officials would prefer to lower the NAEP standards rather than elevating those of the NECAP. Lowering expectations of students is no way to build tomorrow’s work force. We should be maintaining or raising standards. Senator Kennedy has introduced legislation to encourage aligning state standards with the NAEP. Instead the gap between the NAEP and NECAP seems to be widening- at least with respect to 8th grade reading proficiencies.
When asked about the up-down problem a RIDE official said that the 8th grade reading test “has been administered ONLY ONCE” in recent years. But how could that be when the RIDE website reports 8th grade NECAP proficiencies for four consecutive years?
It seems that obfuscation is the primary response from RIDE officials while nary a word is said about studying these discrepancies.
We believe that when parents and other stakeholders have a more transparent picture of our public schools, then the needed political pressure can be developed to begin serious reforms. Our preliminary analysis of test results suggests that social promotion is a fundamental problem that exists in every public school in the state. But most interested parties don’t yet see it that way- certainly not with the wool NECAP pulled over their eyes.
David V. Anderson, Ph.D., is CEO of Asora Education Enterprises, and his NAEP estimates were generated under a contract with the Ocean State Policy Research Institute.