September 1, 1999

AOL-09_1999.pdf

In 1999, the AOL Foundation's Interactive Education Initiative (IEI) funded 54 educational technology projects in schools, community-based organizations (CBOs), and youth-serving institutions around the country, as part of its mission to use online technology to improve the lives of families and children and empower the disadvantaged. The initiative sought to help K-12 learning environments make the most of interactive technology, to construct models and/or identify best practices that can be replicated elsewhere, and to nurture a network of educators and others who can promote effective educational use of interactive technology.

AOL Foundation hired EDC's Center for Children and Technology (CCT) to conduct an evaluation of the initiative's 54 one-year implementation projects. The goals of the evaluation were to document the characteristics of the schools and organizations that received the AOL IEI grant, identify the traits of successful projects as well as factors that impede their success, recognize those projects that exemplify best practices, and assist the Foundation in identifying projects that could benefit from added investment.

METHODOLOGY
CCT used a variety of methodologies - site visits, surveys, case studies, and telephone interviews and questionnaires - to assess the impact of the IEI project at the local level and to understand the projects from the vantage point of the institutions implementing them. CCT collected data through two core strategies. First, the research team gathered baseline quantitative information through surveys given to all 54 IEI sites. Second, the team conducted site visits and phone interviews, and collected formative evaluation reports from project sites to develop a qualitative understanding of the projects from the perspective of the schools and community-based organizations in which they were implemented. The information amassed by these methods we present in the form of case studies. From these, we derived the traits of successful implementation as well as factors that hindered success.

In addition to evaluating projects, CCT also helped sites define their own goals and benchmarks for success.

The attached report has four sections: grantee profile, project profiles, case studies and appendices. The first presents a detailed analysis of information culled from a survey, site visits, and phone interviews with the IEI grantees. After summarizing the research information, in the second section we describe our observations and conclusions about the projects. We also present criteria used to rate the sites into three categories: exemplary, promising, and struggling. The third section gives more complete pictures of a variety of projects on a continuum of successful to less successful implementation.

Key Findings
While much of the money distributed by federal funding agencies for educational technology innovations has tended to support top-down, large-scale collaborations involving multiple partners, AOL's Interactive Education Initiative has nurtured educational technology change at the grassroots, practitioner level. We found that the relatively small amounts of money AOL provided schools, in many cases, managed to bypass central offices and go directly to classrooms, thereby avoiding internal politics, the drag of dividing resources among too many recipients, or the prospect of running counter to someone else's educational agenda. In some cases, these AOL projects made major contributions to their schools and organizations as well as to technology efforts at the district level.

Upon closer examination, CCT found that demographic or technological factors could not account for the success of the projects with any consistency. Rather, we found a variety of characteristics, such as leadership and vision, innovative educational designs, and reflective use of technology, most often factored into the successes of exemplary sites. For example, our research team found a small experimental project, which had little hope of impacting large numbers of students but which used technology in a reflective manner to support an innovative curriculum idea, to be very successful. Conversely, we found a larger project involving many more students and teachers, but which used the technology in a less thoughtful manner to support a more traditional curriculum effort to be less successful.

Recommendations
Based on our findings, we recommend the distribution of small grants that go directly to the practitioner. These grants are extremely effective, probably more so in terms of dollar and impact than the large, highly visible grant made to an institution. We also [advocate/suggest/propose] that teachers with technological competence and a specific set of wishes and goals be targeted for grants because they are more likely to take full advantage of the funds versus novice teachers who require greater support. Grants should be used not only for technology itself but also to free teachers to reflect, study and design new curricula, such as paying for substitute teachers to give the practitioner the flexibility to attend conferences. We recommend that AOL develop clear definitions and criteria about what it considers innovative and should make clear to grantees that the foundation prizes teacher flexibility and creativity because it understands the complex reality of schools, which often interferes with well-considered plans. Instead of using a Web site, AOL should endorse a different model of dissemination, the most effective of which would be through face-to-face or interactive sharing of information. AOL could arrange for teachers from exemplary sites to meet with other AOL grantees and discuss what they've learned during implementation of their own projects.