January 1, 2004

ITTF_INTL_intfinal04.pdf

This interim report presents preliminary data and observations from evaluations of Intel Teach to the Future being conducted around the world, and recommendations for building and refining this evaluation portfolio to ensure that findings will be instructive at the local, national and international level. The data presented here reflect the various evaluation processes that are underway, as well as those that are just beginning, in countries participating in Intel Teach to the Future. This interim report is intended to provide formative information for consideration by program staff as they develop and/or extend their evaluation plans, rather than summative information about overall program impact. The report is structured to present the following:

• Preliminary data from the Core Surveys administered in four countries: Taiwan, Japan, India and the U.S. (n=11,986)




• Themes derived from analysis of the evaluation reports submitted to EDC, site visits to four countries, conversations with education managers and evaluators, and EDC's evaluation of the U.S. program. The report also presents programmatic and evaluation recommendations that address these themes.




• Summaries of the evaluation plans and findings from twenty countries.

The preliminary results from the Core Survey suggest that Intel Teach to the Future is having an impact on the teaching practice of participants. Not only did a large percentage of teachers report implementing a new technology-integrated lesson or activity since their participation in the program, but many also report that they are experimenting with a number of the project-based teaching strategies promoted in the training. These preliminary findings also suggest strong relationships between access to technical resources and rates of implementation, and an equally strong relationship between teachers' perception of the relevance of the project-based teaching strategies presented in the training and rates of implementation. Some key findings from the Core Survey administered in four countries include:

• 79% of teachers report implementing a new technology-integrated lesson or activity since the training.




• Teachers who did not have access to computer labs were more likely to report not having implemented (39%) than those who had access (19%).




• Teachers who had classroom computers were more likely to implement technologyintegrated lessons than those who did not, and the more classroom computers teachers had, the more often they implemented.




• The challenges to implementation most commonly cited by both teachers who implemented and teachers who did not were those related to lack of access to technology and lack of time. However, among those who had not implemented, lack of administrative and technical support were also cited frequently as obstacles, while those who had implemented did not cite these as frequently.




• Teachers who agreed that the teaching strategies presented in the training were relevant to their teaching goals were more likely to implement technology-integrated lessons than those who did not.




• Nearly all teachers (94%) who implemented a technology-integrated lesson reported that their students were motivated and actively engaged in the lesson.




• More than 60% of teachers reported presenting lessons to students using technology, conducting research on the Internet, and accessing Internet resources for lesson planning more frequently since the training.

The themes that we identified across countries were organized using a research-based framework for analyzing technology integration in educational environments. This framework describes four factors that facilitate or impede the integration of technology: infrastructure, professional development, administrative support and time. We also identified some key evaluation themes and issues that have emerged in multiple countries. In each category we make programmatic and evaluation recommendations for addressing these themes.

• Infrastructure is consistently raised as an issue by teachers working in a wide range of access conditions. We suggest making it explicit in the training that Intel Teach to the Future is designed to help teachers integrate project-based ICT curriculum within the given ICT environments in which they work, and focusing discussion in the training around the specific access conditions of participants. We also suggest collecting stories of innovative implementation in challenging access environments through case studies and contests. These would provide concrete examples for teachers who find it difficult to conceptualize how to integrate technology into their teaching.




• Teachers across the world have been overwhelmingly positive about the professional development they have received through Intel Teach to the Future. We suggest ways in which the program can build on this initial positive response. Some countries have already developed methods for providing ongoing professional development for their MTs through the use of MT groups who meet regularly to share ideas and experiences. These groups could be the focus of qualitative evaluations that explore the ways in which these groups allow MTs to learn from each other and provide follow-up support to teachers. Based on concerns expressed by teachers in a number of countries regarding their need for guidance in helping students make effective use of digital resources, we also suggest follow-up professional development in "information literacy" or "media literacy."




• The relationship between administrative support and teachers' ability to implement technology-integrated curriculum varies across national and local contexts. In order for the program to be effectively adapted to a range of environments, program staff need to be able to create programmatic structures that involve administrators at the level necessary for participants to receive the support they need to experiment with technology. Case studies of administrator workshops currently in place could shed light on how these initiatives can be integrated into programs in other countries. In addition, hierarchical, multi-level evaluations of educational systems can be designed to provide information to program staff about the administrative and structural issues that their programs need to address.




• Time constraints are consistently mentioned as an obstacle to technology integration by program participants. Although lack of time is a difficult problem to resolve, there are ways that program design can address the issue. At least one program has required that all participating schools have Master Teachers on site. Evaluation of this design element can enable program staff in other countries to understand whether this strategy allows participating teachers to more effectively and efficiently implement technology-integrated lessons. In addition, data suggest that offering incentives, especially in the early stages of the program, is essential for encouraging both MTs and PTs to dedicate their time to the training. Systematic exploration of the importance of incentives will better enable program staff to make decisions about how to allocate resources.




• At this stage of our involvement with the international program, a number of evaluation issues have arisen. First, the core survey has already produced interesting, cross-country findings. However, we have identified some key areas of Core Survey development, administration and reporting that can be improved, such as the degree of standardization of the questions and communication about Core Survey requirements. On a broader level, we emphasize the need to establish realistic and meaningful criteria and indicators for program success. We also encourage the use of targeted case studies to explore various aspects of program design and a variety of educational contexts so that individual countries' evaluations can inform program staff around the world. Looking to the future, we suggest creating standardized instruments for use in pre-service program evaluations and documentation of the knowledge base that exists among program staff.

The interim report then presents summaries of the materials we have received from twenty evaluations being conducted around the world. Some countries with mature programs have submitted evaluation reports, the finding of which we summarize in this section. Other countries are at the preliminary stages of their evaluations, and for these we describe their evaluation plans.