March 1, 2008

This report presents findings from evaluation research on the WGBH Teachers' Domain Earth and Space Science Collection, conducted by Education Development Center's Center for Children and Technology (EDC/CCT) from September 2006 through March 2008. WGBH's Teachers' Domain (www.teachersdomain.org) is a free online service designed to utilize public broadcasting's extensive programming resources (broadcast, interactive, and educational) to support standards-based teaching and learning in grades K-12. The digital collections are designed to support teacher training as well as direct classroom use. Materials are selected with the assistance of content experts, curriculum specialists, and classroom teachers, and are often re-edited to address the particular needs and realities of K-12 educational settings. The Earth and Space Science collection contains over 200 science resources organized within the topics Earth and the Universe Earth System, Structure, and Processes; and Water Cycle, Weather, and Climate. Each topic area contains a variety of subtopics with background resource pages, rich media (videos, interactives, images and documents) along with 35 lesson plans that incorporate rich media resources.

Three questions guided the evaluation of the Earth and Space Science (ESS)collection and the corresponding online professional development course:

1. How are teachers integrating Pathways' Earth and Space Science resources into their classroom practices and what is their reported impact on student learning?

1. How effective is the Earth and Space Science Professional Development module in supporting student learning and teachers' classroom practices?

1. How has WGBH improved the approximately 40 NSDL resources it has 'reworked' to incorporate into the collection?

EDC/CCT researchers gathered and analyzed data from surveys and interviews with Teachers' Domain Earth and Space Science collection users, teachers who participated in the Earth and Space Science online professional development course, and scientists who assisted in updating resources to be packaged within the Teachers' Domain site.Additionally, researchers analyzed web use data from the Teachers' Domain site. The findings presented in this report reveal the following:

TEACHERS' USE AND ASSESSMENT OF TEACHERS' DOMAIN EARTH AND SPACE SCIENCE COLLECTION:

• After only a year, the ESS Collection became one of the most frequently accessed of the Teachers' Domain collections.

• Teachers found the Teachers' Domain ESS collection easy to use and the resources to be of very high quality. Some reported that Teachers' Domain is the first site they turn to when looking for rich media educational materials.

• Most teachers discovered Teachers' Domain during individual web searches for rich media content, rather than through Teachers' Domain promotional activities or materials.

• Teachers primarily use the ESS collection to find the rich media resources, such as interactives and videos, to enhance their existing lessons.

• Teachers reported pulling content from the background essays and ideas from the lesson plans to supplement their existing lessons, rather than implementing lesson plans from the site in their entirety.

• Teachers reported that Teachers' Domain materials, particularly rich media, enhance student's engagement with science by showing them real world examples of the science concepts they are learning in class.

• Teachers created lessons that require students to log onto the Teachers' Domain site and make use of the materials directly. They would like the site to provide tools, resources and an interface to support this use of the materials.

EARTH AND SPACE SCIENCE ONLINE PROFESSIONAL DEVELOPMENT COURSE:

• Participants in the Earth and Space Science online professional development course were impressed with the quality of the resources available on Teachers' Domain and anticipated that they would use these materials in their teaching.

• Participants found the workload of the course to be unreasonable for the amount of credit they received and were particularly critical of the online discussion forum.

• Most participants would recommend the course to other teachers, as long as the developers revised the workload requirements.

• Facilitators were generally impressed with the course, although, like the participants, they reported that the time commitment was problematic, particularly the time needed to post and respond to the online discussion forum.

• Most teachers who completed the online course used Earth and Space Science resources, and also explored other Teachers' Domain resource collections.

FEEDBACK ON REWORKED RESOURCES:

• Respondents had a positive experience working with WGBH and appreciated that they were responsive to their feedback.

• Respondents felt the updated resources were of high quality and the accompanying materials (background essays, discussion questions) greatly enhanced the resource.

• Respondents were pleased to have their materials reaching a wider audience. The evaluation revealed that teachers regularly integrate the Teachers' Domain Earth and Space Science resources into their science curriculum and find the rich-media resources to be of the highest quality. However, the analysis also revealed areas for improvement of the site and the online professional development course. Based on the data, EDC makes the following recommendations to the WGBH development team:

• Teachers are always looking for new and interesting ways to use the resources and would further benefit from enhanced networking capabilities within Teachers' Domain that would allow them to connect and share their lesson ideas with other teachers.

• Teachers would benefit from having a tools and an interface on the site that is designed to help them create their own lessons and student assignments using Teachers' Domain rich media resources.

• Enhancing the existing search capabilities would impact teachers' use and ease of navigation throughout the site. As Teachers' Domain continues to increase the number of quality resources, the search function must also evolve to ensure that teachers can search, filter and locate relevant resources efficiently.

• The online professional development course provided excellent background content and teaching strategies, but the workload was overwhelming. Rather than having whole group online discussions, which require participants to click through dozens of posts, both teachers and facilitators would benefit from smaller discussion groups of five to seven people and fewer but more substantive assignments.

• Finally, researchers recommend that Teachers' Domain continue to promote Teachers' Domain directly to teachers and also reach out and solicit feedback from teachers through research and outreach channels. Teachers greatly appreciate the fact that Teachers' Domain seeks out and incorporates their input. These ongoing efforts not only result in enhanced materials for teachers, but also serve to create a sense of community of users around Teachers' Domain.