I’ve written about it before, and there is a lot of evidence for the effect: teacher turnover is a bad thing for the academic success of pupils. But is this by definition? Well, most of the time, but results from a new study show that teacher turnover under IMPACT, the teacher-evaluation system used in the District of Columbia Public Schools, seem to have improved student performance on average. The study also shows that the departure of teachers who were identified by IMPACT as low-performing resulted in substantial improvement in overall teacher quality and student achievement.
But the program hasn’t been without controversy, as e.g. also merit pay was involved. When reading the report some things become a bit more clearer:
The high stakes associated with IMPACT have been controversial, both within the District of Columbia, as well as in broader discussions of education policy. There are elements of both sides of this debate in our estimates. While we are unable to identify high-performing teachers who leave DCPS because of IMPACT, our estimates indicate that replacing high-performing teachers who exit with teachers who perform similarly is difficult.
Is this project an example of merit-pay gone well? Imho, it’s hard to tell. It’s rather about keeping the good teachers and eliminating the less effective teachers, inside the program.
From the press release (with a link to the report):
The study by researchers from the University of Virginia’s Curry School of Education and Stanford University’s Graduate School of Education examines the effects of teacher turnover in the D.C. public schools from the 2009-10 through 2011-12 academic years. It is published as a working paper by the National Bureau of Economic Research.
IMPACT is a multi-measure comprehensive teacher assessment system that evaluates the performance of all D.C. public school teachers. Teachers rated “ineffective” are dismissed; those whose performance is rated as “minimally effective” have one year to improve; and teachers who are rated “highly effective” receive large bonuses and the potential for substantial increases in base pay. This system is intended to encourage high-performing teachers to remain and induce low-performing teachers to leave.
In addition to the longstanding belief that replacing exiting teachers is difficult, additional concerns have been raised in the D.C. schools that high-performing teachers may leave because of the stress of high-stakes evaluation.
“While these are reasonable concerns and in some situations this may occur, overall our analysis suggests they don’t hold true at DCPS,” said James Wyckoff, a professor at UVA’s Curry School and one of the study’s authors. “This likely reflects IMPACT’s design to retain more effective teachers and encourage low-performing teachers to leave.”
Teacher quality and student achievement in both math and reading increased substantially after the departure of low-performing teachers — those dismissed by IMPACT, or those who left voluntarily following their first “minimally effective” rating.
“We found that a disproportionate share of low-performing teacher exits are from high-poverty schools,” explained Thomas S. Dee, professor of education at Stanford and an author of the study. “Our results indicate that DCPS is able to accurately identify low-performing teachers and consistently replace them with teachers who are more effective in raising student achievement, particularly in high-poverty schools.”
On average, the exit of a low-performing teacher resulted in an increase in student achievement equivalent to four months of learning in math and reading.
The findings were surprising even when focusing on the high-performing teachers who exited the D.C. schools.
“In other school districts, we usually observe negative effects when high-performing teachers leave,” Dee said. “That doesn’t appear to be the case in DCPS. The turnover of high-performing teachers is a challenging problem but, in DCPS, we find that the exit of high performers generally has small and statistically insignificant effects on student achievement.”
These findings provide the first empirical evidence of the effects of a high-stakes teacher assessment system on the composition of teacher effectiveness and student achievement. As in virtually all urban school districts, there is substantial turnover of teachers in the D.C. schools, and for a variety of reasons.
“We have no way of differentiating their motivations for leaving,” Wyckoff said. “We suspect a good many leave irrespective of IMPACT for many of the same reasons that turnover is high in many urban schools. However, there is no doubt that when teachers identified as low-performers exit, both teacher quality and student achievement substantially improve.”
As encouraging as these gains may be, student achievement in several high-poverty D.C. schools remains low. In 2012-13, IMPACT was redesigned so that more teachers were designated as low-performing and to target more narrowly the financial incentives for high-performing teachers, focusing on those teaching in high-poverty schools.
“Whether these recent changes provide additional gains to students in these schools remains to be seen,” Dee said.
It is uncertain whether other school districts can replicate these effects, because they depend on factors that many have little control over (including the local supply of high-performing teachers), as well as those they do control (for instance, the introduction of meaningful teacher evaluation).
The report can be found at: http://www.nber.org/papers/w21922