Working Papers

• Bartanen, B. and Husain, A. N. (2021) Connected Networks in Principal Value-Added Models.

A growing literature uses value-added (VA) models to quantify principals’ contributions to improving student outcomes. Principal VA is typically estimated using a connected networks model that includes both principal and school fixed effects (FE) to isolate principal effectiveness from fixed school factors that principals cannot control. While conceptually appealing, high-dimensional FE regression models require sufficient variation to produce accurate VA estimates. Using simulation methods applied to administrative data from Tennessee and New York City, we show that limited mobility of principals among schools yields connected networks that are extremely sparse. As a result, most principals’ VA estimates are highly localized, which potentially limits their usefulness as a measure of effectiveness. When larger connected networks form, VA estimates contain substantial estimation error. This uncovers a key tradeoff between practical utility and statistical precision: connected networks allow for comparisons among a larger set of principals, but undermine the reliability of VA estimates. However, employing a shrinkage estimator can alleviate estimation error in large networks, partially resolving this tradeoff. We conclude with recommendations for estimating principal effects and a discussion of whether—and for what purposes—current principal VA methods should be used.

Refereed Journal Publications

• Grissom, J. A. and Bartanen, B. (conditionally accepted). Potential Race and Gender Biases in High-Stakes Teacher Observations. Journal of Policy Analysis and Management.

Classroom observations are the largest component of evaluation ratings given to teachers in the multiple-measure evaluation systems states have implemented in the last decade. Using data from the first eight years of Tennessee’s teacher evaluation system, we document race and gender gaps in observation ratings and ask whether these gaps reflect true differences in instructional effectiveness. White and female teachers receive, on average, 0.15 SD and 0.30 SD higher observation ratings than their Black and male colleagues. Gaps persist even conditional on other measures of teachers’ effectiveness, such as value-added to student test scores or student attendance, consistent with potential bias. The Black–white gap is largest in schools where Black teachers are racially isolated and is partly explained by Black teachers’ propensity to be assigned less advantaged students within their schools. Teachers receive somewhat higher ratings from raters of the same race. We find no same-gender rater effects and, beyond score differences associated with grade and subject taught, uncover few explanations for the large advantage women see in observation ratings. Our results suggest the need for steps to address bias in evaluation processes to ensure the accuracy of evaluation feedback and fair, equitable treatment of teachers in evaluation and staffing actions that rely on it.

• Bartanen, B. and Grissom, J. A. (2021) School Principal Race, Teacher Racial Diversity, and Student Achievement. In Press, Journal of Human Resources.

Exploiting variation from principal and teacher transitions over long administrative data panels from Missouri and Tennessee, we estimate the effects of principal race on the racial composition of a school’s teachers. Evidence from the two states is strikingly similar. Principals increase the proportion of same-race teachers in the school by 1.9–2.3 percentage points, on average. Both increased hiring and increased retention of same-race teachers explain this compositional change. Further, leveraging longitudinal student-level data from Tennessee, we find that having a same-race principal improves math achievement but that this effect largely operates through avenues other than the racial composition of the teaching staff.

• Bartanen, B. and Kwok, A. (2021) Examining Clinical Teaching Observation Scores as a Measure of Preservice Teacher Quality. In Press, American Educational Research Journal. 

We draw on rich longitudinal data from one of the largest teacher education programs in Texas to examine the properties of rubric-based observational evaluations of preservice teachers (PSTs) during clinical teaching. Using a variance decomposition approach, we find that little of the variation in observation scores is attributable to actual differences between PSTs. Instead, differences in scores largely reflect differences in the rating standards of field supervisors. Men and PSTs of color receive systematically lower scores, as do PSTs in lower-income and rural placement schools. Finally, higher-scoring PSTs are slightly more likely to become employed as K–12 public school teachers and substantially more likely to be hired at the same school as their clinical teaching placement.

• Bartanen, B., Rogers, L.K., and Woo, D.S. (2021) Assistant Principal Mobility and Its Relationship With Principal Turnover. In Press, Educational Researcher. 

We draw on rich longitudinal data from one of the largest teacher education programs in Texas to examine the properties of rubric-based observational evaluations of preservice teachers (PSTs) during clinical teaching. Using a variance decomposition approach, we find that little of the variation in observation scores is attributable to actual differences between PSTs. Instead, differences in scores largely reflect differences in the rating standards of field supervisors. Men and PSTs of color receive systematically lower scores, as do PSTs in lower-income and rural placement schools. Finally, higher-scoring PSTs are slightly more likely to become employed as K–12 public school teachers and substantially more likely to be hired at the same school as their clinical teaching placement.

• Bartanen, B. (2020). Principal Quality and Student Attendance. Educational Researcher. 

Student attendance is increasingly recognized as an important measure of educational success, which has spurred a body of research examining the extent to which schools can affect this outcome. However, prior work almost exclusively focuses on teachers, and no studies have explicitly examined the importance of school leaders. This study begins to fill this gap by estimating principal value-added to student absences. Drawing on statewide data from Tennessee over a decade, I find that principal effects on student absences are comparable in magnitude to effects on student achievement. Moving from the 25th to 75th percentile in principal value-added decreases student absences by 1.4 instructional days and lowers the probability of chronic absenteeism by 4 percentage points. Principals have larger effects in urban and high-poverty schools, which also have the highest baseline absenteeism rates. Finally, principals who excel at decreasing student absences may not be those who excel at increasing student test scores and high-stakes accountability measures, such as supervisor ratings, fail to identify principals who decrease student absenteeism.

• Bartanen, B., Grissom, J.A., and Rogers, L.K. (2019) The Impacts of Principal Turnover. Educational Evaluation and Policy Analysis. 

Nationally, 18% of principals turn over each year, yet research has not yet credibly established the effects of this turnover on student and teacher outcomes. Using statewide data from Missouri and Tennessee, this study employs a difference-in-differences model with a matched comparison group to estimate arguably causal effects. We find that principal turnover lowers school achievement in math and reading by 0.03 SD in the next year, on average. Effects vary by transition type, with larger negative effects for transfers to other schools but no or even positive later effects of demotions of (presumably lower-performing) principals. Principal turnover also increases teacher turnover, but this mechanism does not explain the drop in student test scores. Replacing an outgoing principal with an experienced, effective replacement can largely offset negative principal turnover effects.

• Grissom, J.A., Bartanen, B., and Mitani, H. (2019). Principal Sorting and the Distribution of Principal Quality. AERA Open.

Numerous studies document the inequitable distribution of teacher quality across schools. We focus instead on the distribution of principal quality, examining how multiple proxies for quality, including experience, teachers’ survey assessments of leaders, and rubric-based practice ratings assigned by principals’ supervisors, vary by measures of school advantage, using administrative data from Tennessee. By virtually every quality measure, we find that schools serving larger fractions of low-income students, students of color, and low-achieving students are led by less qualified, less effective principals. These patterns persist across urban, suburban, and rural settings. Both differential hiring/placement and differential turnover patterns by principal quality across school characteristics contribute to these patterns. Simulation evidence suggests that hiring and turnover vary in relative importance to principal sorting patterns according to the measure of quality examined, and that differential principal improvement across contexts may matter as well. Complementary analyses of national survey data corroborate our main results.

• Grissom, J.A., and Bartanen, B. (2019). Principal Effectiveness and Principal Turnover. Education Finance and Policy.

Research demonstrates the importance of principal effectiveness for school performance and the potentially negative effects of principal turnover. However, we have limited understanding of the factors that lead principals to leave their schools or about the relative effectiveness of those stay and turn over. We investigate the association between principal effectiveness and principal turnover using longitudinal data from Tennessee, a state that has invested in multiple measures of principal performance through its educator evaluation system. Using three measures of principal performance, we show that less effective principals are more likely to turn over, on average, though we find some evidence that the most effective principals have elevated turnover rates as well. Moreover, we demonstrate the importance of differentiating pathways out of the principalship, which vary substantially by effectiveness. Low performers are more likely to exit the education system and to be demoted to other school-level positions, while high performers are more likely to exit and to be promoted to central office positions. The link between performance and turnover suggests that prioritizing hiring or placing effective principals in schools with large numbers of low-income or low-achieving students can serve to lower principal turnover rates in high-needs environments.

• Grissom, J.A. and Bartanen, B. (2019). Strategic Retention: Principal Effectiveness and Teacher Turnover in Multiple-Measure Teacher Evaluation Systems. American Educational Research Journal.

Studies link principal effectiveness to lower average rates of teacher turnover. However, principals need not target retention efforts equally to all teachers. Instead, strong principals may seek to strategically influence the composition of their school’s teaching force by retaining high performers and not retaining lower performers. We investigate such strategic retention behaviors with longitudinal data from Tennessee. Using multiple measures of teacher and principal effectiveness, we document that indeed more effective principals see lower rates of teacher turnover, on average. Moreover, this lower turnover is concentrated among high-performing teachers. In contrast, turnover rates of the lowest-performing teachers, as measured by classroom observation scores, increase substantially under higher-rated principals. This pattern is more apparent in advantaged schools and schools with stable leadership.

• Bartanen, B., Grissom, J.A., Joshi, E., and Meredith, M. (2018). Mapping Inequalities in Local Political Representation: Evidence from Ohio School Boards. AERA Open.

Elected representatives’ place of residence can reveal information about their socioeconomic status, their likely social networks, and potential biases in the constituencies they represent. Using data on home addresses we collected from local elections offices, we investigate the geographic distribution of school board candidates’, including winners’, places of residence across two election cycles for 610 school districts in Ohio. We employ Geographic Information Systems (GIS) to identify census block group and school enrollment zones associated with each candidate’s residence. We document differences among block groups and schools with more and less school board representation, including a robust association between the relative affluence of a neighborhood and the likelihood of school board members residing in that area. We find that more citizens from affluent areas run for school board, and because a large proportion of school board elections feature minimal competition, these higher propensities to run explain disparities in representation.