Evaluating the Effectiveness of Tennessee's Voluntary Pre-K Program

Peabody Research Institute
&
TN Division of School Readiness and Early Learning

Tennessee has been funding prekindergarten programs since the 1990s. The state’s Pre-K program expanded considerably in 2005, when the Tennessee General Assembly passed the Voluntary Pre-K for Tennessee Act. The Act significantly increased the state’s investment and provided greater access to state pre-k classrooms. Since 2005, Tennessee has provided $213 million new dollars for its Voluntary Pre-K Program (TN-VPK), creating 786 new classrooms, and serving an additional 15,000 preschoolers across the state. With this kind of state funding commitment – not to mention the program’s impact on thousands of children – one question remains: is Tennessee’s prekindergarten program effective at preparing students for educational success?

To answer this question, PRI joined forces with the Tennessee Department of Education to commence the first scientifically-rigorous statewide evaluation of the effectiveness of the TN-VPK program, funded by a grant from the Institute of Education Sciences.

The Questions We are Trying to Answer

  1. Does participation in TN-VPK improve children’s academic and behavioral skills when they enter kindergarten?
  2. Does participation in TN-VPK improve children’s long-term academic and behavioral skills after Pre-K?
  3. What are the characteristics of the children who benefit the most from TN-VPK?
  4. What characteristics of TN-VPK teachers, classrooms, and school/system organization are associated with improvements in children’s school readiness?

How We are Addressing Those Questions

We are using several different study components to allow us to best answer these questions. The two primary study parts are described below.

Randomly Assigning Children to Attend TN-VPK

This part of the study is designed to evaluate the long-term effects of the program on children from pre-k through 3rd grade using a randomized controlled trial (RCT). Across the state, the design involves over 3,000 children who were randomly assigned to TN-VPK classrooms in schools that had more children who wanted pre-k than could be served. Some of these children applied for TN-VPK in 2009-10, and others applied in 2010-11.  A map of the school systems involved in this part of the study can be found by clicking here.These 3,000 children are being tracked through the state education database, and we are collecting information each year about their grade placement, school attendance, etc.  Over 1,000 of these children are being directly assessed each year of the study by trained research staff members using assessments of academic skills.  We are also collecting teacher ratings of their social/behavior skills every year from K until 3rd grade. Both those children who were in the TN-VPK program and those who did not get in are being followed and assessed.  When these children reach 3rd grade, we will collect their state standardized assessment scores.

Studying Classrooms in a Representative Sample of TN-VPK

This part of the evaluation looks at specific TN-VPK classrooms for a two year period, and examines only school readiness for kindergarten, and not long-term effects. The research design, utilizing a regression discontinuity design (RDD), includes data collection from TN-VPK classrooms taught by the same teacher two years in a row, and will follow the first year children into kindergarten. Kindergarten entry skills of children having just completed TN-VPK will be compared to those of children having just enrolled in TN-VPK.  TN-VPK classroom characteristics are being observed, as well.  The goal is to determine the TN-VPK classroom features associated with children’s kindergarten readiness.  This part of the project involves 160 TN-VPK classrooms across the state that were selected to represent the gamut of programs involved in TN-VPK, including partnership sites, pilot classrooms, and classrooms in schools that are considered to be high-priority based on their AYP standing.  We divided the state into 4 regions and focus on 1 region in each year of the study.  A map of the school systems involved in this part of the study can be found by clicking here.The first region, involved in 2009-2011, was the Central West region, which includes Nashville.  The second region (West), which includes Memphis, was involved in 2010-2012.  The third region (Central East) including Chattanooga participated in 2011-2014, and the fourth region (East) including Knoxville is currently participating.

A timeline of key study activities for each part can be found below:

Newly-Funded Follow-Up Study

PRI has recently received new funding from the National Institutes of Health to continue to follow a portion of the children in the RCT study through their 7th grade year (2018-19).  The purpose of the follow-up study is to continue the annual data collection without interruption into the middle school years. The longitudinal findings so far can be fairly characterized as mixed—the effects of TN-VPK participation on the achievement measures have faded but, at the same time, significant effects on grade retention have emerged, and effects on other ‘non-cognitive’ outcomes are being investigated. This is not an unexpected pattern and does not mean that further follow-up would be fruitless. Indeed, there is good reason to hypothesize that further positive effects will appear as these children progress through later grades; moreover, it is important to know whether they do.

The Questions We are Trying to Answer in the Follow-Up Study

1. Whether there are effects of TN-VPK on primary cognitive and non-cognitive/behavioral outcomes through grade 7, including:a) Academic achievement, specifically on state achievement test scores and student grades overall and for math and English language arts, the subject areas with the most continuity across these grades;b) Other academic outcomes—attendance, grade retention, special education status, and English language learner designation;c) Problem behavior—school disciplinary incidents, delinquent behavior, substance use, and other such behaviors that tend to emerge in the middle school period.

2. Whether there are effects of TN-VPK on variables that are candidates for mediating the relationship between pre-k and the primary cognitive and non-cognitive/behavioral outcomes: (a) school engagement; (b) approaches to learning; (c) executive function; and (d) peer relations.

3. Whether TN-VPK effects on the primary outcomes are mediated by prior effects on school engagement, approaches to learning, executive function, and/or peer relations.

4. Whether TN-VPK effects on any of the primary outcomes or candidate mediators are moderated by:a) Quality of the schools attended in the late elementary and middle school grades;b) Students’ personal characteristics—e.g., gender, ethnicity, and non-English native language.

 

Links:

Funding:

The five-year, $6 million study is funded by grant #R305E090009 from the U.S. Department of Education’s Institute of Education Science.

The new five-year, $4 million study to follow a portion of the original sample is funded by grant #R01HD079461-01 from the U.S. Department of Health and Human Service's National Institutes of Health.