September 1, 2010

SuplFinalReportSmall.pdf

OVERVIEW

The report presents information about each of the properties, provides logic models describing each program's components and expected outcomes, describes evaluation methods for data collection and analysis, discusses findings for individual properties, presents a discussion of findings across properties, and finally offers recommendations to further the use of technological enhancements to increase the reach of media products targeting early literacy skills.

Methods:

Researchers collected data from multiple sources to inform the evaluation, including print and Web-based documents; interviews with teachers, parents, and property producers; site visits that included observations of adults and children interacting with each evaluated property; and online surveys.

Qualitative data analysis methods were used to review, code and analyze the range of data collected. Print and web-based material was reviewed and catalogued; interview data were transcribed, entered into Atlas.ti, a qualitative data analysis tool, and coded; observations were reviewed and analyzed for common themes as well as property-specific themes; and survey data were analyzed through generation of response frequencies and review and coding of responses to open-ended questions.

Research Questions

The evaluation addressed three central research questions:

• Do the supplemental materials or activities address a gap or missing link in the overall dissemination and communication plan of the product, filling a void that would otherwise go unmet by existing products and services?
• Do the supplemental materials meet the needs of the end user (caregiver, teacher, or child) in a way that provides a novel/innovative/valued experience?
• Do the supplemental materials alter the interactions between caregivers and young children with respect to early literacy experiences?

TECHNOLOGICAL ENHANCEMENTS

Out of the Blue's Super WHY! Technology Add-On

Out of the Blue Enterprises developed an online games extension that built on their existing Super WHY! Summer Reading Camp. The camps provide access to literacy-oriented games, hands-on activities, video viewing that is linked to early literacy skill activities, and computer games that reinforce early literacy skills. The computer games introduce practice targeting a different early reading skill each day. Each game focuses on the particular literacy "power" possessed by one of the four Super WHY! Characters including: alphabet power, spelling power, word power, and reading power.

Overall, responses from camp facilitators were positive with regard to the content, design, and implementation of the Super WHY! Summer Reading Camp with technology add-on. The program was described as engaging, fun, colorful and stimulating for campers, and also offered a chance to engage with technology. Some facilitators stated that the program included games that were too challenging for their campers and others suggested that games requiring more basic early literacy skills were more popular choices for participating children.

Recommendations for this property include:

• Provide a digital tour of the curriculum materials so that teachers can have access, on their own time, to a guided introduction to the program, its characters and the underlying literacy goals.
• Provide teachers with a set of the Super WHY! games on a CD so that teachers do not have to rely on internet access to fully implement the curriculum.

Sesame Workshop's The Electric Company School's Initiative Curriculum

Sesame Workshop's The Electric Company partnered with Success for All to develop a hands-on curriculum using whiteboard technology. The result is a curriculum that allows teachers to deliver Success for All phoneme lessons using videos and games that borrow characters from The Electric Company story line.

Teachers were generally very satisfied with The Electric Company School's Initiative Curriculum, reporting that it was engaging, fun, and encouraged student participation in activities that are otherwise unappealing or less interesting. Some teachers indicated that the pacing of lessons could be faster, and others indicated that there was not always a good match between student grade level and content, and between the Success for All lesson and The Electric Company School's Initiative Curriculum.

Recommendations for this property include:

• Review pacing of the games, which was too slow for some teachers, and also the overall time requirement of 45 minutes for full implementation. Both the game time and overall program time could be shortened.
• Provide more opportunities for interaction during the games, and have games reflect student interaction, such as providing different scores based on different playing levels.

WordWorld's eBook Library

WordWorld's eBook is an electronic book that provides audio and animation as part of the book reading experience. When viewed the eBook will "read" the text out loud, highlight each word as it is read, and encourage page exploration through animations that demonstrate how letters come together to form words.

Parents were positive about their experience reading an eBook with their child, and indicated that the pacing of the highlighted words, the visual images and the animations were all positive qualities of the book. Children also enjoyed and were engaged by the eBook. Parents who were Spanish speaking indicated they would like to know what the story is about and would benefit from having the story text available in Spanish, which would allow them to help their children better understand the English words incorporated into the book.

Recommendations for this property include:

• Provide additional common elements of book design by including a title page, author and illustrator to support children's exposure to these aspects of book reading.
• Provide a Caregiver Guide that is more simplified and offers suggestions for caregivers that can be implemented individually or as a series of activities.
• Provide a Spanish language version of the Caregiver Guide.
• Provide the book's text in Spanish to support parent's understanding of the story so that they can help their children with comprehension.

DISCUSSION AND GENERAL RECOMMENDATIONS

Discussion

Despite the substantial differences in the goals and methods of the three supplemental technological enhancement efforts, some elements are common to all three. The most prominent of these is the focus producers had on supporting literacy skills for underserved children and the adults who care for them, and the use of technology to reach out to these communities in innovative ways. Although the producers used unique technologies to address different child and adult populations (from age 3 through 8; and teachers or parents), all sought to ensure that every child had access to experiences that would support early literacy learning. All three also took advantage of current and innovative technology to support this goal. If we return to the three central evaluation questions when considering the outcome of all three properties, we must consider some other common findings. Our analysis of data from each property indicate that each property produced positive outcomes in response to one or more of the study's three central research questions.

Question one asked whether the supplemental materials or activities address a gap or missing link in dissemination and communication that would otherwise go unmet by existing products and services. Of the three properties, two appeared to meet this need: the Super WHY! Summer Reading Camp with technology add-on and The Electric Company School's Initiative Curriculum both address specific gaps raised by teachers and identified by producers as they sought to develop their technological enhancements. There was no evidence of a gap or need filled by the development of an eBook Library CD; however, it should be noted that parents said they thought having an eBook on a CD would encourage a child to use it more often because the simple act of seeing the CD would be a reminder.

Question two asked whether the supplemental materials meet the needs of the end user in a way that provides a novel experience. All three properties were able to address this issue in ways that reflect their individual goals. The Super WHY! Summer Reading Camp games provided young children with engaging experiences that supported practice of early reading skills. Although not all games were considered academically appropriate for all children, camp facilitators were able to support their children's use of at least two of the four games and indicated that children benefited from the experience and would not have had the same experience without the games. The Electric Company School's Initiative Curriculum exposed students to phoneme practice in ways that were engaging, visual, and fun, and provided some relief from what was perceived as an otherwise less visually interesting curriculum. Lastly, the WordWorld eBook provided children and their caregivers with a reading experience that could be modified to meet their needs (with or without reading aloud) and encouraged parent engagement in the early reading experiences of young children.

Question three asked whether the supplemental materials alter the interactions between caregivers and young children with respect to early literacy experiences. In this case we suggest that only the eBook provided some evidence of altering interactions between a child and caregiver because of the engaging animations, story line, and visual and auditory information about letters, sounds, and the words they make. However, we also suggest that the other properties have the potential to alter informal interactions between adults and students because of their engaging nature and their use of humor, physical activity, and singing to encourage participation. Both The Electric Company/Success for All video segments and the games included in the Super WHY! Summer Reading Camp seek to engage children in age-appropriate activities such as singing, or game-play in ways that reinforce a specific concept (such as the alphabet or a phonic element). Once children are engaged they are more focused on a task, more likely to retain the information that is presented and are more likely to receive positive reinforcement from a teacher (who may not provide as much positive reinforcement in a setting where children are not engaged or focused). In this way the technological enhancements may support altered interactions between adults and children in different settings.

Exploring innovative uses of technology to achieve the goals laid out by the three properties included in this study is a highly challenging undertaking, but it also can be a rewarding process. Technology has provided a broad range of new approaches to reaching out to children, parents, and educators, but exploring which of these approaches is best suited to a particular audience or task will require a considerable amount of experimentation. Efforts such as the Technological Enhancements project will provide guidance and leadership for future work in this area.

Recommendations

Ongoing support for the development of programs and outreach efforts that include digital tools requires a willingness to experiment with new and existing technologies, and with innovative approaches to integrating media into learning experiences for young children and their care-providers. Undertaking such a development process may be out of the ordinary for many traditional public media producers. Yet fostering an environment where this kind of development is encouraged and supported must take place if new ways of reaching new and existing audiences are to be discovered. We recommend that CPB and its partners continue supporting this kind of innovation and exploration by integrating the following into their current and future programs:

• Support communication across producers and public media stations experimenting with digital tools in a range of learning contexts. In this way different properties may be able to build on the experiences of others, which may lead to opportunities for collaboration across properties and will contribute to a shared culture of innovation and iterative development.
• Conduct ongoing needs assessments of target audiences. Although technology may offer innovative ways to reach out to those whom public media is seeking to support (e.g., families, teachers, and learners), access to different kinds of technology is still limited for some populations. Ongoing needs assessment can help producers and stations determine which audiences have access to which technologies, and which audiences are benefiting from newly introduced technological enhancements.
• Recognize that developing new and innovative uses of technology requires room for experimentation and for mistakes. Providing opportunities for producers to experiment and to recognize and learn from mistakes may help to create an environment where innovation leads to new ways of reaching out to children who are most in need of support.