Introduction
A rapid expansion of state-funded prekindergarten (pre-k) programs began in the 1980s and has continued to the present day. The objectives of the advocacy groups and state legislatures promoting this expansion vary; one common theme is enhancing the school readiness of children prior to kindergarten entry, especially children from economically disadvantaged families. Alongside that relatively short-term goal, however, is an expectation that pre-k will have longer-term effects on academic performance and behavioral outcomes.
This evaluation of the Tennessee Voluntary Prekindergarten program (TN-VPK) is the only randomized control trial of a scaled-up state-funded pre-k program that is longitudinal, to date following the children through high school.
While state pre-k programs vary in many of their characteristics, the Tennessee program is relatively typical in its general contours. It is organized and overseen by the state department of education and serves 4-year-old children from low-income families statewide with local programs in all but a few of the school districts in the state. Since 2005, Tennessee has provided millions of new dollars for TN-VPK, including funding 900 free preschool classrooms serving over 18,000 students and costing $86 million for the 2018-19 school year. With this kind of state funding commitment – not to mention the program’s impact on thousands of children – the question remains: Is Tennessee’s prekindergarten program effective at preparing students for educational success?
To answer this question, the Peabody Research Institute[1], under the leadership of Mark Lipsey ad Dale Farran, joined forces with the Tennessee Department of Education to plan and carry out the first scientifically-rigorous statewide evaluation of the effectiveness of a state-funded program.
Study Design
This evaluation of the TN-VPK program consists of three phases. Phase 1 was funded by IES (R305E090009) (7/09-6/14), Lipsey and Farran, CoPIs. Phase 2 was funded by NICHD (1R01HD079461-01) (7/14-6/21), Farran and Lipsey, Co-PIs, with a predoctoral fellowship (R01HD079461-01S1) awarded to Pearman. Phase 3 is being funded by IES (R305A210130) (&/21-6/24), Durkin, Lipsey, Farran, and Preacher, Co-PIs.
Phase I
Phase 1 of the TN-VPK evaluation had two major components: a randomized control trial (RCT) and an age-based regression-discontinuity design (RDD). During Phase 1, 2990 students from oversubscribed pre-k programs around the state were randomized to pre-k admission or to the wait list. Students in both treatment (pre-k) and control (wait list) groups have been followed longitudinally. The RDD sample was constructed to be representative of the statewide program (155 programs). Students who had attended pre-k the previous year were assessed at the beginning of kindergarten and students in the RDD comparison condition were assessed at the beginning of pre-k. RDD does not allow for longitudinal follow-up.
RCT
The primary goal of the RCT was to determine both short- and longer-term impacts of the statewide TN-VPK program on participating children.
RCT Full Sample
Students were recruited to participate in the RCT Full Sample from 79 over-subscribed schools across Tennessee in which two cohorts of pre-k applicants were randomized to offers of admission or waitlist status during school years 2009-10 and 2010-11. Academic and behavioral outcome data are gathered annually on these students from the Tennessee Department of Education (TNDOE) database through the cooperation of the Tennessee Education Research Alliance (TERA).
Click here to view ITT and TOT conditions for the RCT Full Sample
Click here to view the RCT Consort chart
Click here to view RCT Full Sample measures
Click here to view a map of the districts with participating TN-VPK programs
ISS Sample
In addition to the data collected from school records for the full sample, parental consent was obtained at the beginning of the pre-k year for 1,076 of the children in the RCT Full Sample to allow the research team to collect additional data directly from the children and their teachers at the beginning and end of the pre-k year and annually through third grade. This subsample is referred to as the Intensive Substudy (ISS) Sample. The 1,076 students in the ISS Sample are from Cohorts 1 and 2 of the RCT Full Sample.
Click here to view the ISS Sample Consort Chart
Click here to view ISS Sample measures
RCT Phase 1 Research Questions
- Does participation in VPK improve early literacy, language, and math skills, and classroom behavior by kindergarten entry the following year (immediate effects)?
- Does participation in VPK provide advantages that carry forward to enhance academic performance in later grades by improving achievement and reducing grade retention, absenteeism, disciplinary infractions, and special education placement (longer-term effects)?
- Are there demographic subgroups of children who benefit more from participation in VPK?
RCT Phase 1 Findings
The results through third grade have been described in research reports (TN-VPK technical report from year 1 TN-VPK technical report through pre-k, TN-VPK technical report through first grade, TN-VPK technical report through third grade) and a published summary article (Lipsey, Farran, & Durkin, 2018). Support from the NIH predoctoral grant (3R01HD079461-01S1) allowed Pearman to examine the effects of neighborhood poverty (Pearman, 2019) and the impact of the quality of children’s educational environments in subsequent years (Pearman, 2020).
RDD
The age-cutoff RDD component of this larger study was more limited in scope with a focus on assessing TN-VPK’s impact on selected outcomes measured at the beginning of kindergarten. Moreover, classroom observations and related data were collected for each sampled classroom.
The state of TN was divided into four data collection regions. RDD data were collected in one region per year over four consecutive years. Impact analyses were conducted in the following year on the kindergarten sample and the sample of comparison children, assessed at the beginning of pre-k, who then actually completed the pre-k year and were enrolled in kindergarten the following fall.
Click here to view RDD measures
Click here to view a map of districts with participating TN-VPK programs
RDD Research Questions
- What is the impact of participation in TN-VPK on students’ academic preparedness when they enter kindergarten?
- How do TN-VPK classrooms across the state of TN vary in daily activities, learning environments, and classroom instruction?
RDD Findings
Two technical reports are available examining RDD student achievement effects and variations among RDD classrooms, and findings have also been reported in a published summary article.
RDD Research Report_June 2020_
Phase 2
Phase 2 of our investigation of TN-VPK effects tracked students through middle school and consisted of two parts.
The first part continued to follow the RCT Full Sample through the state education database to track middle school academic performance and behavioral indicators. A key question was whether the rather unexpected adverse VPK effects found in Phase 1 on state achievement test scores, special education placements, and school rule violations faded and perhaps even reversed in these later grades.
The second part was to collect information on behavioral outcomes and plausible mediators of VPK effects on academic and behavioral outcomes. For this purpose a new high-quality subsample (the Follow-up Intensive Substudy (FISS) Sample) of the RCT Full Sample was selected.
RCT Full Sample
We continued to track both cohorts of students in the RCT Full Sample (N = 2,990) through middle school during Phase 2 of the TN-VPK study.
Click here to view ITT and TOT conditions for the RCT Full Sample
Click here to view the RCT Consort chart
Click here to view the RCT Full Sample measures
Click here to see a map showing the districts of participating TN-VPK programs
FISS Sample
The Follow-up Intensive Substudy (FISS) Sample was selected based on the 45 VPK programs in the RCT Full Sample that provided the largest proportions of randomized control cases and the highest parental consent rates during Phase 1 of the study. The 763 students who are in the FISS Sample are all from Cohort 2 of the RCT Full Sample.
Click here to view the RCT FISS Sample consort chart
Click here to view the RCT FISS Sample measures
Phase 2 Research Questions
- What are the effects of TN-VPK participation on other outcomes in midcdle school for the FISS sample, including delinquency, problem behaviors, school engagement, executive function, peer relationships, and conscientiousness?
- What are the effects of TN-VPK participation on administratively reported behavioral outcomes in middle school for the full sample, including attendance and school disciplinary infractions?
- What are the effects of TN-VPK participation on academic outcomes in middle school for the full sample, including academic achievement, grade retention, and special education status? a. Do measures of delinquency, problem behaviors, school engagement, executive function, peer relationships and conscientiousness in early middle school mediate the effects of TN-VPK on student outcomes at the end of middle school?
Phase 2 Findings
Phase 3
RCT Full Sample
We are continuing to track both cohorts of students in the RCT Full Sample (N=2,990) through high school during Phase 3 of the TN-VPK study. In addition to examining the long-term effects of the TN-VPK program, the purpose of this high school follow-up study is to assess the nature and extent of COVID-19 related school disruptions and how those effects interact with prior school experiences (including pre-k attendance), various student characteristics, as well as the extent of school disruptions experienced. These students are attending high school in the midst of the COVID-19 pandemic that caused disruptions in schooling expected to have effects on their academic progress, especially for these youth who are all from lower income families.
FISS Sample
We are also continuing to follow the FISS Sample through high school. This allows us to continue collecting measures of delinquency, problem behaviors, school engagement, peer relationships, and conscientiousness in high school. We are also asking these students and their parents and teachers detailed questions about their schooling experiences during and after the COVID-19 pandemic.
Phase 3 Research Questions
- How do students’ trajectories on outcomes related to academic progress (achievement, GPA, on-grade progress, attendance) change during the years disrupted by Covid-19? a. Are there differential effects for students’ changes in trajectories related to their prior academic history, their demographic characteristics, school and district adaptations made to address COVID risk, and general school-level characteristics such as size, location, and student demographic mix?
- How are the different variations in school and district adaptations to COVID risk related to performance on the ACT given in high school and ultimate dropout/graduation after controlling for student and school characteristics that may be confounded with those adaptations?
- For the FISS sample, how do students’ school/education attitudes, peer relationships, and prosocial and risky behaviors change with the onset and continuation of COVID-related school disruptions? Are those effects moderated by student conscientiousness and executive function or parenting behaviors?
[1] PRI has since been absorbed into Peabody College, Vanderbilt University