June 1, 2002

HBa_NECC_02.ppt

For twelve years, the Jason Multimedia Project has offered diverse populations of students and teachers unique opportunities to learn about earth systems, life on earth, and technologies used to study the earth-space system. In that time, the JASON Project has reached a diverse population of approximately 25,000 teachers and 1 million students around the country.

JASON's approach to encouraging student and teacher discovery is inclusive of technology, focuses on scientists doing science in the context of a research expedition, relates science to other subject domains, and provides for interactive learning. To achieve this process while supporting local curricula, JASON provides teachers with multimedia instructional tools that bring together academic standards, the rich research environment of a new curriculum topic each year, and student performance measures to support state standards and assessment initiatives. The following resources compose the JASON's multimedia curriculum model: print curriculum, prologue and update videos, live broadcast, and Team JASON Online.

To help gauge the program's impact on students and teachers, JASON asked EDC's Center for Children and Technology (CCT) to investigate the JASON Project's effect on student learning in specific science content areas study and student science inquiry skills.

Methodological Approaches
CCT worked with JASON teachers who have various degree of experience with the JASON curriculum and serve different population of students. With this guiding criterion, we selected an initial pool of 20 to 30 JASON schools. JASON site coordinators from these schools were asked to send a letter of participation to all potential participants in their area. The entire selection process took two months. With the help of JFE staff, we identified an initial pool of 30 schools, before narrowing down to the final eight school sites selected to participate in the study.

These eight schools reflect the diversity of learning contexts in which Jason is being implemented such as student background and ability and teachers' experience in using Jason. Furthermore, the participants in this study come from 8 schools around the country: Arkansas, Texas, Michigan, Pennsylvania, Ohio, California, New York, and Wisconsin. We conducted in-depth research with the following participants: Administrators (8), JASON teachers (8), JASON students (269), and non-JASON students (108)

Results
Based on site visits as well as teachers' and principals' interviews, we found that the JASON curriculum helps students make concrete their learning of complex and abstract scientific concepts through rich hands-on activities; and make real-world connections.

Hands-on learning

• Keeps students engaged
• Appeals to diverse learning styles
• Involves the creation of tangible products
• Is especially effective with at-risk students.

Making real-world connections

• Makes science real and relevant to students
• Allows students to interact and identify with scientists
• Exposes students to experiences they would never otherwise have
• Helps students ask better questions
• Inspires an interest in science that can extend beyond the JASON experience.

Inquiry Test Results:
Based on the results of a pre- and post-inquiry test that asked students to answer questions by interpreting data and building an argument, we found that:

• Most JASON students (66%) made overall gains (from 1 to 10);
• More than half of the JASON students in each classroom made some gains on the test with the exception of students in one classroom;
• The average JASON classroom gains were all positive (from .44 to 2.45);
• Most JASON students did better in process (66%) than in content (46%);
• Average classroom gains in process skills were positive (from .16 to 1.55) for all classrooms;
• Average class scores for content were negative in two classrooms and positive in seven classrooms (from .28 to .91)
• JASON students who scored at or above average (87%) did much better in process than in content.
• Half of the JASON students in all three grades made significant gains, especially in 6th and 7th grades;
• Students who worked in small groups (72%) on a regular basis in their classrooms also made significant gains in the inquiry test.

In the cases where the inquiry test was administer to a control group, we found that:

• JASON students (52%) did better (1 to 7) than the control group (38%) in the inquiry test;
• JASON students specifically performed better in the area of process/scientific argument building.

Video Assessment Results:
Based on inter-scorer reliability analysis, we found that the overall reliability of the scores given to JASON students' videotaped presentations of their projects was 88 percent. Relative to other research projects using similar assessment techniques, these scorers achieved an extremely high level of reliability (Frederiksen, 1994a).

Based on cluster analysis determined as groupings in which the three dimensions of scoring (understanding, critical thinking, and communications skills) have maximally distinct means and the coding of each assessment dimension on a scale of 1 to 5, we found that:

• Fifty percent (50%) of the sample fell into the High Cluster (3.6 to 3.7). These scores were high and consistent across the three assessment dimensions;
• Thirty-one percent (31%) of the sample fell into the Middle Cluster (2.4 to 2.6). Scores for this cluster were average;
• Nineteen percent (19%) of the sample was in the Low Cluster (1.7 to 2.1).

A further analysis of the videotaped presentations reveals that:

• The overall score of more than half of the students' videotaped presentations were high (3.8);
• Of the students' videotaped performances scoring at or above three (56%) across three assessment dimensions, most of them did better (66%) in critical thinking;
• Ten presentations (31%) from Texas, Long Island, Arkansas, Michigan, and Wisconsin scored high (at or above 3) consistently across the three dimensions of scoring. Inferential statistics from the student's survey data indicate that most of the students from these five states knew the general goals of the JASON Project, and the topic of this year's JASON curriculum;
• Three presentations from Arkansas and Wisconsin scored very high (at or above 4) across the three dimensions. All three projects addressed topics from the JASON curriculum: plate tectonics, cultures and history of Hawaii, and volcanoes.

Based on this first year in-depth study, we are in the process of adeveloping a multi-year large scale evaluation of the JASON Project, which also will be discussed at this session.