The effectiveness of a 5-unit college-level Japanese language course taught with online supplemental material was examined against the same course taught without online material. Participants were 86 students in 8 sections of a first-semester elementary Japanese language course during a 4-year period. Complete data (quiz, midterm, and final exam scores) were available for 39 students in the comparison group and 47 students in the experimental group. The instructor used a conventional textbook and a Kana workbook based on an eclectic approach. The results showed no statistically significant differences on the students' test scores between the comparison group and the experimental group on 9 quizzes and final but showed statistically significant differences on one quiz and midterm (without-online students scored higher than with-online students). Dropout rates were significantly lower in the with-online course.
Computer technology continues to develop. As William Massy (1997) noted "the faculty role will change from being mainly a content expert, professor's job is to profess, to a combination of content expert, learning process design expert, and process implementation manager" (p. 31). In recent years, the Internet has become the leading edge in delivering instruction at a distance by virtue of its ability to incorporate learning in innovative ways. By using the Internet, students can maintain their control over the rate and timing of instruction and homework. In 1986, Kulik and Kulik found that (a) computer-based instruction has a small but significant positive effect on achievement and (b) computer-based instruction substantially reduced instruction time to as little as a third of that required by traditional instruction. Recent research by Aberson et al. (2000), comparing computer-based instruction to traditional instruction, found no statistically significant differences between the online tutorial and the lecture groups' test scores. Their findings were, however, encouraging because the online instruction can be implemented as an effective supplement to traditional classroom. The Pew Internet and American Life Project (2002) reported that 86% of the students attending universities in the United States have accessed the Internet, compared with 59% of the general population. Besides, 79% of college students stated that the use of the Internet has had a positive impact on their overall learning. According to Ryan and associates (1999), "Higher education is moving with deliberate speed to an electronic classroom. Much has been published on faculty experiences with WWW course delivery. However little research exists on the evaluation of these methods" (p. 272). A recent study by Al-Jarf (2004) indicated that use of Web-based instruction as a supplement to traditional inclass ESL writing instruction was significantly more effective than writing instruction depending on the textbook alone.
The use of the Internet is expected in foreign-language classrooms. The benefits of the Internet use in second-language learning is, however, often questioned. Does the use of the Internet show positive effects on language learning? It is not yet known whether the Internet use yields positive results. This study analyzed the effects of the Internet use on second-language learners' test scores in an elementary Japanese class. The results show pedagogical implications for the Internet application in second-language curricula.
Purpose of the Study
The present study investigated and evaluated the effectiveness of a language course supplemented with online material for college-level Japanese learners. The study compared exam scores from students enrolled in traditional inclass instruction taught 5 consecutive days for 50 minutes "without online supplement" material with exam scores of students enrolled in traditional inclass instruction taught for 5 consecutive days for 50 minutes "with online supplement" material at home.
Method
The sample consisted of full-time students from first-semester Japanese language classes during Fall 1997, Fall 1999, Fall 2000, Spring 2002, Fall 2002, Spring 2003, Fall 2003, and Spring 2004 semesters at San Jose State University. The comparison group was subjected to traditional inclass instruction, and the experimental group was subjected to traditional inclass instruction plus online supplement material at home using their personal computers.
Ten quizzes, one midterm exam, and one final exam were given. The quizzes, the midterm exam, and the final exam were identical across semesters. The data from the first 4 classes that were taught without online supplement material (Fall 1997, Fall 1999, Fall 2001, and Spring 2002) were compared with the data from the remaining classes that were taught with online supplement material (Fall 2002, Spring 2003, Fall 2003, and Spring 2004). A series of two-tailed independent t tests were used, and effect sizes are reported.
Because greater replication studies provide additional support for believable generalizability (Robinson & Levin, 1997), an external replication was based on an independent group of students for 4 semesters. Comparison-group data were obtained from students enrolled in the first 4 semesters. Data were collected from regular classes, and students were not aware of the treatment they were receiving so that "novelty effect" was under control. Because comparison-group scores were collected during the first 4 semesters, the "John Henry Effect" (refers to control groups or their teachers feeling threatened or challenged by being in competition with a new program or approach and, as a result, outdoing themselves and performing well beyond what normally would be expected) should have little or no impact on the test scores. All students responded that they had access to the Internet and had knowledge of its use.
The individuals in this study were 86 students who had no prior knowledge of Japanese, that is, who never studied Japanese before. Greater proportion of students in the experimental group were male than in the comparison group (see Table 1). The average ages for the 2 groups are comparable (comparison group M = 21.05 years old, SD = 2.62 vs. experimental group M = 21.40 years old, SD = 2.35). The average number of units each student carrying per semester was 14.26, ranged from 5 to 23, and the averages are comparable between the 2 groups. The majority of students had an English language background.
Procedure
All students received instruction 5 days per week, 50 minutes per day for one semester. Students in the comparison group and the experimental group used the same textbooks and were taught with the same methods by the same instructor, for the same length of time (15 weeks) during the semester. Both followed the same course outlines that assured uniformity in grammar topics treated by the same instructor. Towards the end of each textbook lesson, as homework, both groups were assigned to do the Review Exercises at the end of each textbook lesson.