January 1, 2005

Since March of 2003, Education Development Center's Center for Children and Technology (EDC/CCT) has been coordinating the worldwide evaluation of the Intel Teach to the Future professional development program. EDC/CCT's role has been twofold. First, EDC/CCT designs and coordinates the implementation of two global surveys: the End of Training survey and the International Impact survey. Second, EDC/CCT supports the national education managers and local evaluators in designing country-specific evaluations and administering the global surveys. This report presents analyses of the Impact survey and individual country reports and is organized as follows:

Analysis of International Impact Survey
This section analyzes the results of the Impact survey for the eleven countries that reported survey data about the Intel Teach to the Future program from June 2003 through November 2004 (N=16,296). The data have been analyzed by macro-level contextual factors of region and national income level and by participant and school level contextual variables of infrastructure and teachers' familiarity with project-based learning. Key findings include the following:

• Implementation of new technology-rich activities: The survey results indicate that 82.3% of the respondents across the world have implemented new technology activities since completing the training. Using the World Bank's income categorization, the data also show variation in the implementation rate by national income levels. Teachers in mid-level income countries report an implementation rate of 75.5% and the teachers in high-income countries report a rate of 87%.
• Distribution of available resources: The majority of respondents (59.5%) have both lab and classroom computer access, while just under one third (31.7%) report having lab-only access. When analyzed by country, national patterns emerge since most teachers in each country fall into one or the other category. The data suggest that the decision to place technical resources in classrooms is a policy decision, and not solely tied to economic resources, since there are high resource countries, such as Italy and Japan, where most respondents only have lab access, and mid-income nations, such as Brazil and Russia, where large portions of respondents have classroom access.
• Implementation and technology resources: Classroom access facilitates the integration of technology into teaching and learning. The implementation rate for teachers who have lab access only is 71.7%, a figure that increases to 87.2% when teachers have one PC in their classrooms and reaches 94.8% when teachers have access to two to six classroom computers.
• Familiarity with project-based learning: For more than a quarter of participants (28.5%), the project-based teaching strategies taught in the program are new, while half of the participants (53.1%) have some familiarity with project-based learning, and 18.4% are already very familiar with project-based learning. Regional variations in teachers' familiarity with this pedagogy also exist. Teachers in Asia Pacific Area Countries (APAC) are the least familiar with 47.9% reporting no familiarity with the pedagogy. In the U.S. and the Europe/Middle East/Africa region (EMEA), 10.7% and 22.8%, respectively, of teachers had no familiarity with project-based learning, while the Latin America Region data (LAR) showed 35.2% of teachers with no familiarity.
• Familiarity with project-based learning and implementation: Despite differences in teachers' prior experience with project-based learning, the majority of teachers are integrating new technology- based activities after completing the training. The program is most successful, however, among teachers who have some familiarity with project-based learning: 90.6% of teachers with prior experience and 83.7% of teachers who are somewhat familiar with project-based learning are implementing new activities, compared to 75.3% who have no familiarity.
• Challenges: The challenges teachers face during implementation vary according to availability of computer resources and prior exposure to project-based learning. Teachers with technology available in a lab and in their classroom report fewer challenges than those teachers who have only lab access. An even stronger relationship exists between prior knowledge of project-based learning and challenges to technology integration reported by teachers.

  Global Synthesis
  This section highlights some key issues identified through an analysis of the Impact survey data as well as the national evaluation reports submitted by 22 countries. The issues addressed illustrate how the program has been adapted and suggest ways that the program might be modified to ensure its relevance to and sustainability in a wide array of cultures.

• Integrating technology into project-based pedagogy. Impact survey data and local evaluation reports indicate that many teachers come to the training with little or no experience with project- based teaching and learning. These teachers find the training challenging. To best support such teachers, Master Teachers may need to learn how to help teachers make connections between new pedagogical ideas to traditional practice. Another strategy is to use a combination of online and face-to-face formats to provide Participant and Master Teachers with resources on project-based pedagogy that address the theoretical and practical issues involved in applying these teaching strategies effectively.
• Selecting, training and sustaining Master Teachers. A critical component for the success of Intel Teach to the Future in any country is the program's ability to select and train high-quality Master Teachers, and for these Master Teachers to become part of a sustainable professional development network. Local evaluations suggest the original model of recruiting classroom teachers to assume the role of Master Teachers does not work effectively in all contexts. Evaluators have noted that classroom teachers often do not have the flexibility, incentives, technical skills, access to technology, or legitimacy within existing professional development structures to continue as trainers, leading some programs to take a different approach to selecting Master Teachers. As the model for creating cadres of Master Teachers is adapted across contexts, the program may need to systematically define the criteria for what makes an effective Master Teacher and how programs need to train and support people with very different backgrounds and experience.
• Unit plan development. Focusing the training around the creation of a unit plan can help participants connect new ideas to familiar activities, provoke discussions about teaching practice, and give participants a concrete product they can use directly in their classrooms. However, some local evaluation reports suggest that the unit plan template in the program materials is not compatible with the lesson planning formats that teachers participating in the program are required to use in their work. This can make it difficult for teachers to use these plans in their classrooms. The materials could be adapted to provide unit plan templates that are more closely aligned with the existing planning formats of the country or region in which the teachers work. In addition, evaluation reports indicate that some training approaches prioritize completion of the unit plan template over engaging in broader, process-oriented discussions around teacher practice. Especially in countries where project-based pedagogy is new, programs may need to emphasize in the Master Teacher training that the unit plan is intended to promote discussion rather than serve as a requirement to be fulfilled.
• Identifying and communicating how the program is aligned with national educational goals. The degree to which the program is aligned with a country's educational policies and goals, and how that alignment is communicated, influences how participants experience the program. In some cases, teachers' expectations did not match the program goals, and in other cases, participants had trouble distinguishing Intel Teach to the Future from other technology training programs. However, in countries where education managers have worked closely with government agencies, the data show that teachers are recognizing how the training can support their work. Additional information about how national education policies are aligned with Intel Teach to the Future would help education managers ensure that these aspects of the program are clearly communicated to participants and emphasized in the training.

  Conclusions and Recommendations
  The conclusions and recommendations presented in this section are based on the analyses of the Impact survey and country evaluation reports. Recommendations fall into two categories: strategies to improve the global evaluation process and programmatic modifications to address specific challenges. Evaluation recommendations include:

  • Closely examine Master and Participants Teachers' understanding of project-based pedagogy before they come to the training and whether/how this understanding changes after they have completed the workshop
  • Develop profiles of Master Teachers and the educational contexts in which they work
  • Create rich descriptions of how teacher unit plans function in the classroom, how teachers adapt them to their existing practice and how students engage with them
  • Review country education policies, including national goals and professional development processes, teaching requirements, incentives, and assessment practices. Programmatic recommendations include:
    • Work with teachers who have no classroom computers to develop resources and lesson examples that can specifically target teachers working in these environments
    • Deepen the pedagogical components of the training for both Master and Participant Teachers and create channel building activities that offer continued enrichment around helping teachers develop their knowledge of project-based teaching strategies
    • Work with teachers and administrators, as well as government officials, to identify program alignment opportunities across a range existing educational structures (professional development, lesson planning, teacher reward systems) and when appropriate, make adaptations to the program.

  Appendices
  Following a brief reference section, this report includes five appendices. Country report summaries. This appendix is an annotated bibliography of the evaluation reports received from various countries, describing methods, and key findings.

  • International Impact survey
  • International Impact survey frequencies
  • International Impact survey frequencies, by region
  • International Impact survey frequencies, by income level