June 1, 2008

Breakthrough, an international human rights organization, developed the ICED! ("I Can End Deportation") video game in order to educate a wide audience about unfair U.S. immigration laws and practices and deportation policies. In winter 2007, Breakthrough engaged the Education Development Center/Center for Children and Technology (EDC/CCT) to assess changes in ICED! game players' knowledge and attitudes about U.S. immigration and deportation policies due to game play. EDC/CCT developed a 14-item, online, pre/post questionnaire to collect data in order to answer the following two evaluation questions:

1. Does playing ICED! increase players' knowledge about U.S. immigration and deportation policies, as they are addressed in the game, from pre to post?
2. Does playing ICED! influence players' attitudes about U.S. immigration and deportation policies from pre to post? If so, how has it done so? The questionnaire items addressed three broad content areas to which players are introduced in the game: U.S. immigration policies; immigrant detention and deportation policies; and immigrants' civic behavior options as they are influenced by policies and laws. A third evaluation question emerged during our content analysis of players' open-ended responses to a posttest question about whether game play had changed their attitudes:
3. How do experiences during the game influence learning outcomes and attitude changes?

We analyzed questionnaire responses from 6,007 cases. One hundred twenty-nine players completed all pre and post questionnaire items, 99 of whom had complete data sets (in which all survey items were completed) for assessment. Our analysis yielded the following findings, discussed in greater detail in the report:

1. Playing ICED! contributed to an increase in player knowledge about U.S. immigration and deportation policies. An analysis of the paired pre/post knowledge scores for a group of 99 respondents revealed a mean score increase from pre to post game play: average 6 out of 12 correct at pre, 9 out of 12 correct at post (t (98) = -8.02, <.001 similarly="" the="" trends="" in="" much="" larger="" groups="" of="" players="" who="" only="" completed="" pre="" or="" posttests="" suggest="" that="" playing="" iced="" did="" lead="" to="" gains="" learning="" outcomes.="">

2. Over half (56.5%) of the respondents in the matched pre/post group indicated that playing ICED! had changed their attitude about the ways in which immigrants are treated in the United States. While the majority of respondents did not complete the open- response associated with this question, those who did indicated the game had changed their attitudes positively toward immigrants' rights. Our third question about player experiences during game play revealed several broad themes:
• Perspective taking and role playing enabled some players to consider the facts presented in the game from immigrants' points of view;
• Ethical and situational dilemmas encountered by immigrant characters in the game engendered strong written responses to the open-ended question, primarily in two directions: some players argued that the scenarios revealed a strong bias among the game developers, while others wrote that they did not realize how precarious day-to-day living can be for immigrants.

These findings suggest ICED! is an effective tool for educating a broad audience about facts associated with these issues and for influencing their attitudes about U.S. immigration policies. In the final section of this report, we include the following recommendations-based on our analysis of the data-for Breakthrough's future efforts that include ICED!:

• Consider embedding content into problem-based scenarios that require the application of knowledge that players have learned in the game-as well as what they have learned through their own experiences-rather than presenting it through true/false dichotomies
• Provide clear links to the sources for the content presented in the game so that players can explore the topics beyond ICED!
• For future data collection, Breakthrough should consider devices that automatically send players to the posttest, rather than giving them the option to click through.
• When ICED! is used as a tool to complement a classroom curriculum, consider asking an open-ended question about player attitudes toward U.S. immigration policies in the pre and posttests in order to explore whether and how game play influences those attitudes
• Given the unexpected change in some players' attitudes toward due process for immigrants (discussed below: see pp. 10-11), change the questionnaire item in order to allow respondents to describe any changes that occurred. In its current wording, the item is vague and may explain the increase in negative attitude toward due process. An open- ended item for this topic, however, would allow players to explain any issues that arose during game play that might have problematized the issue and led them to rethink their positions.