August 1, 2008

Evaluating Educational Technology.pdf

This talk describes some of the roles evaluation research is playing in advancing the effective use of educational technologies in the US. As we look towards a future of sharing our experience with colleagues around the world, this talk is an opportunity to reflect on the rich history of the Center for Children and Technology (CCT) and to think about what we have learned about how to conduct effective research, and to consider how we might improve what we do and how we work. My comments in this paper build on our collective experiences as researchers during twenty-one years of investigating how technology can best be integrated into high-quality educational environments. Our discussion emphasizes the importance of locally valid and locally useful research designs and attempts to define our approach to conducting evaluations. The challenge of combining validity and utility is increasingly at the center of our work at CCT. Specifically, we are seeking to conduct research that will help both the research community and educators to understand how complex organizations, like schools, school districts, state and national educational authorities, finance and implement educational technologies, and how those practices might best be improved. In this paper we argue that effective evaluation must produce both research-based knowledge of what technological applications can work best in various educational environments, and practice- based knowledge of how the technology integration process can best be designed to meet locally defined learning goals in schools. The first section of this paper is a brief review of the recent history of U.S. research related to educational technologies and some of the lessons we have learned from this work. This review points to some of the promising future directions for educational research. In the second section we specifically discuss a role for evaluation in meeting the challenges of helping educators successfully integrate meaningful uses of technology. The third section discusses an evaluation model that stresses collaborative work between research groups, like CCT, and local educators. Our strong concern with conducting research that is not only rigorous and valid but also useful to practitioners grows out of our collaborative experiences with educators working in many different settings. The Center for Children and Technology has been asking questions about how technology can best support teaching and learning in K-12 schools and other educational contexts for over twenty years. Our work at CCT brings me into contact with many different types of institutions: school districts, museums, individual teachers, college faculty members, after-school programs, and many others. These relationships take many different forms, but they always require us to value the needs and priorities of those individuals and institutions that are working with us. Working closely with classroom educators, administrators, policymakers, and curriculum and tool developers has pushed us, as researchers, to reflect on and question our theoretical and methodological groundings, and to be both explicit and modest in stating the frameworks and assumptions that guide us in our work. This work and the work of our many colleagues has led us to our current perspective on what is important about infusing technology into K-12 education. We have learned that when student learning does improve in schools that integrate technology, those gains are not caused solely by the presence of technology or by isolated technology-learner interactions. Rather, such changes grounded in learning environments that prioritize and focus a district's or school's core educational objectives (Hawkins, Spielvogel, & Panush, 1997). At the core of our research agenda is a belief that technology can enhance the communicative, expressive, analytic, and logistical capabilities of the teaching and learning environment by supporting types of communication, analysis and expression by students and teachers that are important in two ways. First, the power of technologies offer more flexibility in undertaking certain activities (like writing, editing or graphing) than would otherwise be possible. For example, advanced telecommunications support dynamic and relevant communication with people outside of the classroom; graphic and image technologies allow students to engage with politically ambiguous or aesthetically challenging visual imagery; and word processing makes revision and reworking of original student work easier. Second, technologies can support the extension of learning experiences in ways that would simply be impossible without technological tools -- such as visualizing complex scientific data, accessing primary historical source materials, and representing one's work to multiple audiences. The increasing democratization of access to technology can also make these learning activities available to all students.