The mixed methods research design involves both primary data collection and analysis of primary and secondary data in two school districts, Milwaukee Public Schools and Dallas Independent School District. We measure student access to digital tools, where a student has the opportunity to use digital tools in a classroom or other educational setting, and actual use (or enactment) of digital tools in the educational setting, including measures of intensity, duration and quality.
We have developed a standardized observation instrument to evaluate the nature of digital tools themselves and their implementation, as well as the quality of learning opportunities in digital and blended instructional settings within the instructional core and the elements that contribute to quality. The instrument is based on existing research on the integration of technological tools into classrooms, as well as indicators of quality elements in the instructional core in general and for digital instruction in particular.
We are also conducting structured interviews with district and school-level administrators and support staff to characterize and understand how malleable factors such as organizational capacity and inter-organizational coordination; staffing, training and support decisions, and policy guidance and requirements for implementing the technology initiatives in school districts influences access to and the effectiveness of digital tools in improving student outcomes and reducing achievement gaps.
We are employing econometric methods in analyzing school record data on students in classrooms where the digital tools are being accessed and used, which include student characteristics and educational outcomes. In Milwaukee Public Schools, we are also linking student data with administrative data from contracted providers of the digital tools, which includes detailed (session-level) information on the intensity of student use, active and idle time, course-taking and performance, as well as other measures of digital tool enactment in and outside of the classroom. We will draw on these measures in quantitatively analyzing the relationship between the use of digital tools and student learning and achievement and the malleable factors that contribute to their effectiveness (or limitations) in improving student outcomes and reducing achievement gaps.