December 2010

Student achievement improved and teacher turnover declined in schools participating in the Texas state-funded District Awards for Teacher Excellence (D.A.T.E.) program, the National Center on Performance Incentives at Vanderbilt University has found.

“Our findings suggest that, more often than not, participants in the D.A.T.E. program had a positive experience, student achievement gains and teacher turnover moved in a generally desirable direction and teachers had favorable attitudes towards D.A.T.E.,” Jessica Lewis, research associate at NCPI and co-author of the report, said.

D.A.T.E. is a state-funded program that provides grants to school districts in Texas for the implementation of locally-designed incentive pay plans.

The findings come at a relevant time for Texas and for national policy on teacher pay. In Texas specifically, the state legislature will soon convene to consider the future direction of D.A.T.E., while at the same time Texas and various districts within the state have become recent recipients of federal Teacher Incentive Fund grants.

“The evaluation’s objective is not to recommend any universal approach for design and implementation of incentive pay in Texas. Rather, the report informs policymakers about the implications of various program design and implementation choices,” Lewis and her colleagues wrote.

Overall, D.A.T.E. districts saw bigger declines in teacher turnover than did other districts in Texas during the first year the incentive pay program was in operation, and turnover was related to the incentive awards. Teacher turnover increased for districts with incentive plans that proposed relatively small awards, and decreased as possible award amounts increased. Additionally, the probability of turnover surged among teachers who did not receive a D.A.T.E. award, while it fell sharply among teachers who did receive one.

“Teachers who received a D.A.T.E. award were much less likely to turn over than those who did not, and the size of the award received by a teacher was less important than the fact that the teacher received any award at all,” Lori Taylor, associate professor at Texas A&M University and co-author of the report, said.

During the first two years of the D.A.T.E. program, students in D.A.T.E. schools had greater achievement gains than those in non-D.A.T.E. schools. Additionally, the report found that the difference between students’ passing rates on the Texas Assessment of Knowledge and Skills in D.A.T.E. and non-D.A.T.E. schools decreased when compared with pre-D.A.T.E. scores, indicating that passing rates in D.A.T.E. schools were catching up to rates in non-D.A.T.E. schools.

Teachers in D.A.T.E. schools also held relatively positive views of the incentive pay plans operating in their schools.

“More often than not, teachers believed the incentive pay plans were fair, the goals targeted by the plans were worthy, and that the correct teachers received awards,” Mark Ehlert, research assistant professor at University of Missouri-Columbia and another report co-author, said.

And, while teachers did not perceive negative effects from D.A.T.E., they also did not indicate that the incentive plans were contributing much to school improvements.

The report’s authors point out that while overall outcomes related to student achievement gains, teacher turnover and teacher attitudes were desirable in D.A.T.E. schools, there was notable variation in those outcomes between D.A.T.E. schools. At least part of this was attributable to the design of incentive pay plans. Typically, but not universally, larger incentive award amounts and greater expectancy among teachers for receiving awards were associated with more desirable outcomes.

D.A.T.E. incentive pay plans were first implemented in Texas districts during the 2008-09 school year. The program is currently in its third year of operation with approximately $197 million in annual state funding. All districts in the state are eligible to receive grants, but participation is voluntary.

The NCPI report evaluated data from the first two years of data from the districts and schools that participated in the D.A.T.E. program during the 2008-09 and 2009-10 school years, which was Cycle 1 of the program. The findings focus on the decisions made by Texas districts whether or not to participate in the program, the local design preferences of districts that implemented D.A.T.E. incentive pay plans and the program’s influence on teachers and students (i.e., student achievement gains, teacher turnover and educator attitudes and practice).

While participation in D.A.T.E. was open to all public school districts in Texas, just 16 percent participated in Cycle 1. However, participating districts’ student enrollment (2.4 million) represents almost 50% of the state’s entire public school student population. The Cycle 1 D.A.T.E. districts tended to be districts with a lower measure of wealth, larger student enrollment, a greater share of at-risk students (i.e., minority, ED, and LEP students) and were more likely to have participated in past state-funded incentive pay programs.

The authors stress that there are important limitations to consider about the evaluation’s findings. Districts chose to participate in D.A.T.E. and designed their own incentive pay plans. Therefore, districts and schools that ended up participating in D.A.T.E. may have differed from non-D.A.T.E. schools in ways that can not be easily captured in statistical analyses and those differences could impact outcomes for students and teachers independent from participation in D.A.T.E.

Texas’ D.A.T.E. program and NCPI’s evaluation of the program were implemented in a notably different manner than the Nashville incentive pay program experiment conducted by NCPI released in September, which found that rewarding teachers with bonus pay, in the absence of any other support programs, did not raise student test scores.

NCPI conducted the D.A.T.E. program evaluation under a contract with the Texas Education Agency. Scholars from Texas A&M University, University of Missouri-Columbia and Corporation for Public School Education K-16 were key collaborators on the report.