June 1, 2005

NECC05_Mandinach.doc

One of the hallmarks of the No Child Left Behind Act (NCLB, 2001) in the United States is the requirement that states develop annual assessments to measure school and student progress and that educators use data to help improve the learning of all students. As a result, administrators and teachers are being confronted with complex and diverse sources of data from which they must make informed instructional decisions. Increasingly school districts are turning toward technology-based solutions that they believe will help them to use data more effectively and there are a growing number of technology-based products that enable districts to provide data to many levels of the system - teachers, administrators, parents, and policy makers - as a means to improve instruction, student learning, and communication. Examining how technology-based tools can facilitate decision-making, and how administrators and teachers use such tools and data to enhance instruction is therefore essential if we are to understand how assessment data can be used effectively to inform educational decision-making. The project described in this paper brings together complimentary evaluation techniques, using systems thinking as the primary theoretical and methodological perspective, to examine the implementation and use of data-driven applications in school settings. The project has two goals: (a) to build a knowledge base about how schools use data and technology tools to make informed decisions about instruction and assessment; and (b) to develop an evaluation framework to examine the complexities of dynamic phenomena that will inform the field and serve as a knowledge building enterprise.