January 1, 2006

IntelWorkshops2005.pdf

This report presents findings from a formative evaluation of the Intel® Teach to the Future Workshops on Teaching Thinking with Technology, conducted by Education Development Center, Inc.'s Center for Children and Technology (EDC/CCT). The workshops are part of a portfolio of professional development programs supported by the Intel Innovation in Education initiative, and are designed to prepare teachers to use web-based software in their classrooms. Each of the three tools addressed in the training are intended to help students represent their thinking visually and to collaborate around both the creation and the analysis of those representations. More specifically:

• The Seeing Reason Tool helps students map cause-and-effect relationships and analyze complex systems;

• The Visual Ranking Tool helps students order and prioritize items in a list and then analyze and evaluate the criteria for their decisions;

• The Showing Evidence Tool helps students hypothesize and support claims with evidence, and then analyze and evaluate the criteria for their decisions.

This evaluation posed the following formative research questions, with a goal of generating insight into current program implementation and follow-up to inform further refinement of the program.

• How and to what extent does the training shape participants' understanding and use of the tools and associated resources?

• How do participants who have gone through the training, and their students, make use of the workshop resources?

Data sources gathered in the course of this formative evaluation included surveys, observations, communications with regional program administrators, trainers, and teachers, and sample unit plans collected from teachers.

Key findings: Successes and challenges

Successes
This formative evaluation found substantial evidence that the workshops are well received by the majority of their participants and that key concepts about using the tools in a project-based context are being effectively communicated and translated into practice. Specific findings suggestive of program success include the following:

• Participants and students find the online thinking tools engaging, innovative, easy to learn and technically simple to navigate.

• Participants leave the workshop focused on using the tools in a project-based context, and the unit plans they develop are consistent with this approach.

• Many participants report using the tools in their classrooms after the workshop, particularly Visual Ranking.

• Participants are most interested in using the tools to make student thinking visible and to promote comparison and discussion of student ideas, two activities that they value and believe to be stimulating for their students.

Challenges
This formative evaluation also identified some challenges to the program's success, primarily related to the program's goal of engaging and enhancing students' higher order thinking skills. Specific findings regarding the challenges of supporting students' higher-order thinking skills include the following:

• During the workshop, participants do not explore in depth how to use the tools to scaffold student use of specific thinking skills, and their unit plans do not typically include activities or instructional strategies that would provide that scaffolding.

• In observed classrooms, participants did not use the tools to support activities that contribute to the systematic development of higher-order thinking skills, such as sustained collection or rigorous evaluation of evidence, or drawing conclusions about the validity or strength of hypotheses or conclusions.

Two additional challenges were identified in the evaluation.

• Teachers of elementary grades, mathematics teachers and foreign language teachers consistently raised concerns about the relevance of the tools to their work with students. These teachers also exhibit lower rates of follow-up tool use in the classroom.

• A substantial minority of participants reported that it was difficult to gain access to the hardware and Internet connections they needed to support whole-class use of the tools.