Last time I wrote about Florida, it was touting their “silver medal” among the 50 states for growing student achievement in the past 15 years. The Harvard study that handed out the imaginary awards analyzed how much progress 4th-grade and 8th-grade students have made on the national NAEP test.
Second place out of 50? Not too shabby. But how valid is it? Some critics have said the remarkable gains Florida 4th-graders have achieved, particularly in reading, are dramatically overblown because of their student retention policy. Since 2003 most of the state’s 3rd-graders who have failed to demonstrate reading proficiency have been held back, of course the test results for the smaller pool of 4th-grade pupils is going to look better. End of story, right?
Not so fast. A recent Independence Institute guest speaker has gone behind the numbers to figure out just how much the retention policy can explain away Florida’s remarkable gains. In a newly published analysis for Education Next, Manhattan Institute senior fellow Marcus Winters finds the truth lies between the two claims, but closer to those made by Florida’s boosters than those made by its critics:
The evidence presented here shows that Florida’s elementary-school students did in fact make large improvements in reading proficiency in the 2000s. As critics contend, the state’s aggregate test-score improvements on the 4th-grade FCAT reading exam—and likely on the NAEP exam as well—are inflated by the change in the number of students who were retained in 3rd grade in accordance with the state’s new test-based promotion policy. Large test-score improvements are also observed, however, among students whose scores were not influenced by changes in the sample selected.
How did Winters do it? He went back and analyzed 3rd-grade results from the Florida Comprehensive Assessment Test (FCAT), tracked individual student progress, and then made adjustments to match results with the 4th-grade NAEP tests. He found real evidence of significant learning gains both before and after the retention policy went into place.
In the end, even after you factor out retained students, “only D.C. and Delaware made a larger test-score improvement” than Florida between 1998 and 2009. If I’m reading the results right, the average Florida elementary student at the end of this time was about a grade level’s worth of learning ahead of their counterpart a decade before. Positive results for both reading and math are strong.
So according to Winters, what kind of reforms did Florida enact that most likely contributed to the improvement?
- A rigorous accountability system that assigns letter grades to schools based on performance
- A voucher program for students in failing schools
- A tuition tax credit scholarship program for poor students
While research of all these (in addition to the retention policy) has shown positive effects, they simply don’t add up by themselves to account for the significant gains. Nor did Florida’s exceptional amount of direct literacy interventions through Reading First and state programs. Finally, neither a provision restricting class sizes nor a universal preschool program were even around when Sunshine State students registered their biggest gains.
Figuring out exactly which reforms have yielded which effects is a tricky business. But in the case of Florida, Winters’ new analysis makes it that much harder to argue with Florida’s success.