1 Introduction
Two functions are distinguished by Harmer (Reference Harmer1991) in teaching language. One is language input, when information is stored in students’ brains, and the other is language output, when students apply the information they have learned. The former includes listening and reading, and the latter includes writing and speaking skills. Harmer (Reference Harmer1991) argued for keeping a balance between those two functions in order to promote efficient language learning. Lin (Reference Lin1995) and Liu and Littlewood (Reference Liu and Littlewood1997) noticed that teaching EFL to students in East Asia, particularly in Taiwan, focuses primarily on language input rather than language output functions. As a result, students felt uneasy and anxious during active learning, especially when engaged in group discussions, asking questions, or giving feedback to others in class, which hindered their language learning. Studies in CALL may relate to computer applications that support language teaching and learning. For example, multimedia annotations (Chun, Reference Chun, Ducate and Arnold2006) help students better utilize digital learning material. However, most CALL studies focus on the development of vocabulary acquisition and reading comprehension because of the current state of computer technology (Ariew & Ercetin, Reference Ariew and Ercetin2004; Chun, Reference Chun, Ducate and Arnold2006; Sakar & Ercetin, Reference Sakar and Ercetin2005).
Based on the above statements, this study argues that multimedia web annotation technology can also facilitate teaching and learning activities for improving students’ EFL writing and speaking performance. Thus, we designed an experiment with appropriate online learning activities for teaching and learning EFL that focus on language output. The students participated in learning activities by creating short stories and completing incomplete stories in written or oral forms. Moreover, students shared the stories with their peers, who could study them and give feedback. As learning materials provided to the students were web-based, the learning activities were supported by Virtual Pen (VPen), a multimedia web annotation system. The students could write or record orally the stories or feedback using textual or audio annotation respectively and share their annotations with the support of VPen. The aim of this study was to investigate:
– students’ perceptions about using VPen and learning activities, their attitude toward using VPen, their actual usage of VPen and the relationship between these factors;
– the relationship between students’ actual usage of VPen and their writing and speaking performance;
– the relationship between students’ writing and speaking performance and their learning achievement;
– students’ preferences toward writing and speaking output methods.
The rest of the paper is organized as follows. First, the literature on the functions and interactivity of EFL learning is reviewed. Then, theories of CALL, key features of web annotation systems and the importance of annotations for EFL learning are discussed in order to support the analysis. Following a description of the underlying method of the study, analysis, results, discussion and conclusions are presented.
2 Literature review
2.1 English language learning: functions and interactivity
Harmer (Reference Harmer1991) suggested that in-class language learning activities can be divided into two functions: one is language input, referred to as receptive skills, when information is stored in students’ brains, and the other function is language output, referred to as productive skills, when students apply the information they have learned. He argued that exposing students to language input is not enough; the students need to activate the knowledge they have received by producing language. Language production allows students to rehearse language use in classroom conditions whilst receiving comments or corrections from the teacher and their peers, and self-provided feedback.
Research on language learning emphasizes the importance of interaction in the learning process. According to Yu (Reference Yu2008), classroom interaction can facilitate students’ language development and communicative competence. Brown (Reference Brown2001) and Wu (Reference Wu, Bird and Harris1992) argued that language learning tasks should be interactive and meaningful. They proposed a triangular process of interactions which includes interaction among learners, the teacher and the learning materials. Ellis (Reference Ellis1988) stated that such a process of interactivity used in learning, with students taking an active part in it, can positively affect their learning achievement. On the other hand, Beebe (Reference Beebe, Seliger and Long1983) admitted that achieving students’ active participation in the language learning process is a challenge for teachers, especially in East Asia, including Taiwan. Yang and Chen (Reference Yang and Chen2007) studied this issue in Taiwan and related it to the Taiwanese language environment which does not provide learners with a real and natural English environment. Consequently, Taiwanese students could only learn EFL through regular class teaching, radio broadcasts, television, newspapers, magazines, and so on. Most students studied EFL through repeated recitation and rote memorization. Furthermore, in most Taiwanese high school EFL classes, the prevalent mode of instruction was based on large-group, teacher-dominated grammar-translation methods, and exam-oriented textbook-based lectures. Students thus acquired knowledge in a decontextualized way. As a result, East Asian students learning EFL had (a) little confidence in their ability to speak; (b) initial unease with group discussions, and (c) extreme anxiety generated simply by the thought of asking a question in class (Song, Reference Song1995; Liu & Littlewood, Reference Liu and Littlewood1997). Huang and Lee (Reference Huang and Lee2004) observed a similar phenomenon and argued that students in the traditional Taiwanese language learning classroom did not get used to interacting with other students, sharing individual ideas, or supporting their viewpoints.
2.2 Computer Assisted Language Learning
Computer Assisted Language Learning (CALL) involves computer technologies that support language teaching and learning. The CALL technologies in use for EFL learning assist students with further information or exemplification or provide practice and exposure to extended learning material. According to Warschauer (Reference Warschauer, Fotos and Brown2004), the current phase of CALL development is based on multimedia computers and the internet. Thus, Chun (Reference Chun, Ducate and Arnold2006) argued, the current technologies of CALL include web-based activities that seek to teach a variety of components, the internet as a source of materials, and multimedia annotation systems. Caldwell (Reference Caldwell1998) suggested the utilization of multimedia (text, pictures, audio etc.) in language learning, since it engages students and stimulates their imagination, helping them to give meaningful output. Hwang et al. (Reference Hwang, Chen, Shadiev and Liin press) underlined the usefulness of multimedia annotation systems as they enable students to express their reaction to digital learning material by adding their own annotations to it.
Furthermore, Hwang et al. (Reference Hwang, Chen, Shadiev and Liin press) suggested that multimedia annotation systems enable learners to share their annotations with their peers and/or teacher for further discussion and collaboration.
In this learning scenario the students are no longer limited to viewing content passively on the web, but are free to add and share commentaries and links, therefore transforming the web into an interactive medium (Yeh & Lo, Reference Yeh and Lo2009). According to Huang and Lee (Reference Huang and Lee2004), Lamy and Goodfellow (Reference Lamy and Goodfellow1999) and Sproull and Kiesler (Reference Sproull and Kiesler1991), multimedia annotations support students’ asynchronous interaction, which promotes active language learning. The asynchronous nature of the interaction provides time for students to reflect before they comment or carry out online tasks and it reduces their anxiety while learning the target language.
2.3 Key features of multimedia web annotation systems
Hwang et al. (Reference Hwang, Chen, Shadiev and Liin press) surveyed students about particular features of multimedia web annotation systems to support studying with digital learning material. The students asked for the following features and we provide examples of multimedia web annotation systems which support these features:
– Making textual annotations: Annotation systems should enable students and teachers to emphasize important points of digital learning material by adding textual annotations to it. The Online Annotator for EFL Writing allows the addition of textual annotations for error correction and corrective feedback on web based documents (Yeh & Lo, Reference Yeh and Lo2009).
– Making annotations on multimedia content: As multimedia objects, especially images and video, are presented in many web-based learning materials, annotation systems should provide support for annotating various types of multimedia objects. Collaborative Video Annotation System (Hasan et al., Reference Hasan, Jekjantuk, Chanwirawong, Nantajeewarawat and Iskander2007) supports learners and teachers in annotating multimedia objects such as video.
– Incorporating multimedia objects into annotations: Annotation systems should allow multimedia content to be included in annotations. Reading Toolbox (Sakar & Ercetin, Reference Sakar and Ercetin2005) enables the annotation of EFL teaching and learning material with multiple types of media such as text, graphics, audio and video.
– Collaboration with annotations: Annotations created by peers should be reviewable; additionally, peer annotations should be annotatable. Collaborative Video Annotation System (Hasan et al., Reference Hasan, Jekjantuk, Chanwirawong, Nantajeewarawat and Iskander2007) permits the sharing of personal annotations and collaboration among learners; annotations can be accessed by other learners remotely through the web.
Since existing systems could not satisfy all the needs of the students, we developed Virtual Pen (VPen), a multimedia web annotation system (see Hwang & Wang, Reference Hwang and Wang2004; Hwang et al., Reference Hwang, Wang and Sharples2007; Hwang et al., Reference Hwang, Chen, Shadiev and Liin press). With the support of the VPen system, students could create text annotations, review them and share them with their peers. The VPen system also allowed students to make annotations to multimedia objects and create annotation content in multimedia formats. We implemented the VPen system in a series of experiments to support mathematics learning activities. The results of the experiments showed that the VPen system was both useful and effective, and the students who used the system performed significantly better than the students who did not use the system during mathematics learning activities (see Hwang & Wang, Reference Hwang and Wang2004; Hwang et al., Reference Hwang, Wang and Sharples2007; Hwang et al., Reference Hwang, Chen, Shadiev and Liin press).
2.4 Annotations for EFL learning
A number of studies have focused on the potential of web based annotations for EFL learning. Ariew and Ercetin (Reference Ariew and Ercetin2004), Chun (Reference Chun, Ducate and Arnold2006) and Sakar and Ercetin (Reference Sakar and Ercetin2005) explored whether multimedia annotations facilitate vocabulary acquisition and reading comprehension in EFL learning. The qualitative data obtained from their studies revealed that learners considered annotations highly useful for reading comprehension. They argued that annotation is thought to provide fast and easy access to the meaning of unknown words and to compensate for insufficiently automatic lower level processes and thus allows the reader to attend to higher level processes. Yeh and Lo (Reference Yeh and Lo2009) conducted an experiment with an online corrective feedback and error analysis system called Online Annotator for EFL Writing to evaluate the effectiveness of the system for EFL writing instruction. The analysis of students’ corrective feedback revealed significantly better performance in the group that received corrective feedback with the system compared with the group that used the paper-based error correction method. Hasan et al., (Reference Hasan, Jekjantuk, Chanwirawong, Nantajeewarawat and Iskander2007) investigated the development of a Collaborative Video Annotation System. They argued that the system can facilitate collaborative learning of a foreign language efficiently by enabling the students to upload, annotate and share their personal multimedia collections. Although computers have grown more powerful and multimedia has become more integrated, it is obvious that CALL paradigms have remained focused largely on receptive skills (Todd, Reference Todd, Vyas and Patel2009).
3 Method
3.1 Research architecture
The architecture of this research is shown in Figure 1. First, the study investigates students’ perceptions toward using VPen and learning activities, attitude toward using VPen, actual VPen usage and their relationship. Second, the study explores the relationship between students’ actual usage of VPen and their writing and speaking performance. Third, the study examines the relationship between writing and speaking performance and learning achievement. And fourth, the study researches students’ preferences towards writing and speaking output methods.
Research architecture
3.2 Learning design
The following seven learning activities with elements of ‘playfulness’ and familiarity of context, i.e., the activities related to topics in students’ textbooks or in their daily lives, were proposed for students during the experiment to practice writing and speaking skills and facilitate their interaction. Each learning activity lasted two weeks and took place one after another sequentially.
Activity A: Individual picture sharing. The students read the learning material first. After that, they searched for a picture from the internet that related to the learning material they had read. Then the students had to add the picture to the activity website. Finally, the students were asked to describe a picture by adding annotations to it with textual content, as shown in Figure 2.
Activity A: Individual picture sharing; Activity B: Commenting on peer work
Activity B: Commenting on peer work. The students worked in pairs during this learning activity. First, the students were asked to share the annotations from activity A with a partner. Then each student reviewed their partner's annotation and gave his/her opinions of it by adding an annotation with audio content, as shown in Figure 2.
Activity C: Questioning and answering. This learning activity was divided into two parts. In the first part, students were provided with five different pictures related to a subject and they had to compose at least three questions for each picture. In the second part, students shared their questions with peers; each student had to listen to other students’ questions and answer them. Questions and answers were added to the activity website as annotations with audio, as shown in Figure 3.
Learning activity C: Questioning and answering
Activity D: Complete the story. The students worked individually during this learning activity. Each student was assigned five different pictures where only the first picture had a text, the beginning of a story. The students had to read this and complete it based on the rest of the pictures. The students added textual annotations to the activity website that contained the rest of the story, as shown in Figure 4.
Learning activity D: Complete the story
Activity E: Story relay. The students were assigned to create a story based on another five pictures. The students were divided into small groups with two to three students in each, then they discussed the sequence of the pictures and the outline of the story among group members. Afterwards, students took turns to record the content of the story and added annotations with audio content to the activity website, as shown in Figure 5.
Learning activity E: Story relay
Activity F: Pictures and their description. The students chose between three and five pictures from a school database of pictures related to a sports competition that had been held at the school a week earlier and then added the pictures to the activity website. Then the students created annotations with textual descriptions of the pictures and gave an oral description of the sports event, as shown in Figure 6.
Activity F: Pictures and their description; Activity G: Opinion exchange
Activity G: Opinion exchange. This learning activity was a sequel to activity F. The students were asked to share their annotations from learning activity F with their peers. Then, the students were required to review their peers’ pictures and read or listen to the descriptions of the pictures and the school event. After that, students added their comments, suggestions, or questions as annotations with textual or audio content, as shown in Figure 6.
3.3 Multimedia web annotation system
The main interface of the VPen multimedia web annotation system is shown in Figure 7. Students log into the website using individual user names and passwords. EFL learning materials were uploaded onto VPen and the system provided students with a structured method to navigate through EFL learning activities, as shown on the left-hand side of Figure 7. In order to move to specific sections of content, students click on corresponding links. Content appears on the right-hand side of the window as shown in Figure 7. One of the features of the VPen system is a multimedia annotation tool that is available for students to annotate the learning materials. Students use the comment box provided by the multimedia annotation tool to create text annotation. The comment box allows the addition of multimedia content, such as pictures or audio, into text annotations. Additionally, we developed a recording mechanism in the VPen system so that students are able to record their speech and add it as an audio file into the annotation.
VPen multimedia annotation system
The VPen system provides two modes of working with learning materials, individual and group modes. The individual mode is presented by default after a student logs into the system. The student is able to add and see only his/her own annotations in this mode. In the group mode the student is able to add and see his/her own annotations as well those of his/her peers. This feature enables students to work in groups and interact with each other. To switch from the individual mode into the group mode, one has to check the “group mode” option of the VPen system, as depicted in the left part of Figure 7. A peer's annotations are presented in the VPen system as an annotation that includes the peer's name, while a student's own annotation has no name. Figure 7 is a screen capture from one of the student's interfaces and the right portion of the figure presents an example of student group work when two students have annotated learning material. One student, the owner of the interface, added geometric figures, in this case a rectangle, and attached two annotations – one with textual content and one with an audio file.
This study adopted the VPen system to support students’ participation in learning activities, i.e., to make annotations with text and audio content, to add descriptions to the pictures, create or complete a story, provide feedback to peers, and ask questions or get answers. The teacher trained the students to work with the system during the first week of the experiment because the students had never used the VPen system before and they were unfamiliar with its functionalities.
3.4 Experimental research
The research was designed to investigate: (1) students’ perceptions toward using VPen and learning activities, attitude toward using VPen, actual usage of VPen and their relationship; (2) relationship between students’ actual usage of VPen and students’ writing and speaking performance; (3) relationship between writing and speaking performance and students’ learning achievement; (4) students’ preferences towards writing and speaking output methods. The following subsections desrcribe details including sampling, experimental design, experimental tools, and statistical analysis methods.
3.4.1 Sampling
Twenty-seven third-grade students from one junior high school participated in this study which was carried out over one semester. During the experiment students visited the computer classroom every two weeks to participate in learning activities and to complete their assignments.
3.4.2 Experimental design
The aim of this study was to improve students’ EFL writing and speaking performance with appropriate learning activities supported by the VPen system. We administered a questionnaire survey to investigate students’ perceptions toward using the VPen system and learning activities and their attitude toward using the VPen system. We adopted the Pearson's correlation analysis to test the relationship between: (a) perceived ease of use of the VPen system, perceived usefulness of the VPen system, perceived usefulness of the activities, perceived playfulness of the activities and attitude toward using VPen; (b) attitude toward using VPen and actual VPen usage; (c) actual VPen usage and speaking and writing performance; (d) writing and speaking performance and students’ learning achievement. Then we employed a simple regression method to find the predictive power of (a) attitude toward using VPen to actual VPen usage; (b) actual VPen usage to speaking and writing performance; (c) writing and speaking performance to students’ learning achievement. In addition, the study used stepwise multiple regression analysis to measure the degree to which one-way writing, interactive writing, one-way speaking, and interactive speaking contribute to the prediction of students’ learning achievements. Finally, we conducted one-on-one semi-structured interviews with the students after the experiment to explore some qualitative insights into the individual, personal experiences of participants in the experiment and perceptions toward using the VPen system and learning activities.
3.4.3 Experimental tools
This study employed the following tools: (1) evaluation of students’ prior knowledge (GEPT); (2) counting, categorization and assessment of students’ annotations; (3) evaluation of students’ learning achievement (post-test); (4) a questionnaire survey and one-on-one semi-structured interviews on students’ perceptions toward using the VPen system and learning activities.
All participants in this study took the General English Proficiency Test (GEPT) before the experiment. The study aimed to determine the students’ level of EFL proficiency before the experiment by using the result of the GEPT as a pre-test and to eliminate from the experiment those students whose EFL proficiency level was lower than elementary. All students in this study passed the test with at least an elementary level; thus, all of them were adequate to participate in the experiment.
The post-test measured students’ learning achievement in writing and speaking proficiency and it took place during the last class of the experiment. The post-test items included tasks that required students (a) to listen to a few audio files and answer questions related to them; (b) to study several pictures provided by the teacher and then compose questions or answer questions related to the pictures; (c) to study the first sentence of a story and four pictures and use these to complete the story; (d) to compose an essay about sports day at school. The post-test was scored on a 100-point scale, with 100 as the highest score.
A questionnaire survey was administered during the last class to investigate students’ perceptions toward using the Vpen system and learning activities and their attitude toward using the VPen system. The questionnaire design was based on the Technology Acceptance Model (Davis, Reference Davis1989). According to Davis, when users are presented with a new technology, a number of factors influence their decision about how and when they will use it. Five dimensions were covered in the questionnaire following the recommendations of Chunga and Tan (Reference Chunga and Tan2004), Davis (Reference Davis1989) and Davis, Bagozzi, and Warshaw (Reference Davis, Bagozzi and Warshaw1992):
– Perceived ease of VPen use is the degree to which a student believes that using the VPen system would be free of physical and mental effort.
– Perceived usefulness of VPen is the degree to which a student believes that using VPen would enhance his or her learning performance.
– Perceived usefulness of activities is the degree to which a student believes that particular activity would enhance his or her learning performance.
– Perceived playfulness of activities is the extent to which a student believes that performing a particular activity is intrinsically enjoyable or interesting.
– Perceived attitude to using VPen influenced by perceived ease of VPen use, perceived usefulness of VPen system, perceived usefulness of activities and perceived playfulness of activities.
Responses to the questionnaire items were scored using a five-point Likert scale, anchored by the end-point “strongly disagree” (1) and “strongly agree” (5). Twenty-six valid answer sheets to the questionnaire were received out of twenty seven students. We counted the students’ annotations and categorized them into following variables:
– Actual VPen usage represents the total number of a student's annotations including textual or audio annotations;
– One way writing represents the number of a student's annotations with textual content to complete individual assignments without a peer's interaction;
– Interactive writing represents the number of a student's annotations with textual content that questions peers, answers peers’ questions, and feedbacks to a peer's annotation;
– One-way speaking represents the number of a student's annotations with audio content to complete individual assignments without peer's interaction;
– Interactive speaking represents the number of a student's annotations with audio content that questions peers, answers peers’ questions, and provides feedback to peers’ annotations.
Three raters assessed the content of each annotation; any large rating difference was resolved through discussion among raters. The followings categories were derived from the assessment procedure:
– One-way writing score represents the mean of non-zero scoresFootnote 1 that a student received to the content of each of her/his annotations during a one-way writing activity;
– Interactive writing score represents the mean of non-zero scores that a student received to the content of each of her/his annotations during an interactive writing activity;
– One-way speaking score represents the mean of non-zero scores that a student received to the content of each of her/his annotations during a one-way speaking activity;
– Interactive speaking score represents the mean of non-zero scores that a student received to the content of each of her/his annotations during an interactive speaking activity;
– Writing and speaking performance represents the sum of one-way speaking score, interactive speaking score, one-way writing score, and interactive writing score.
In addition, one-on-one semi-structured interviews with the students and subsequent data analysis was conducted followed the general recommendations of Creswell (Reference Creswell2005). The interviews contained open-ended questions and asked the students about their individual, personal experiences of participation in the experiment and perceptions about using the VPen system and learning activities. Each interview took approximately thirty minutes; all interviews were audio-recorded with the permission of the interviewee and then fully transcribed for analysis. The text segments that met the criteria for providing the best research information were highlighted and coded. Next, codes were sorted to form categories and codes with similar meanings were aggregated. The categories thus established provided a framework for reporting findings from the study that were relevant to the research questions.
3.4.4 Statistical analysis methods
The following methods of statistical analysis were adopted:
1. Cohen's kappa – to evaluate the inter-rater reliability of the assessment. The result of analysis exceeded 0.89, indicating its high reliability.
2. Cronbach α – to assess the internal consistency of the survey. The values exceeded .80 in all dimensions, which shows the results reached high reliability.
3. The Pearson's correlation – to examine how significantly (a) perceived ease of use and perceived usefulness of the VPen system, perceived usefulness of activities and perceived playfulness of activities correlate with attitude toward using VPen; (b) attitude toward using VPen correlate with actual VPen usage; (c) actual VPen usage correlates with speaking and writing performance; (d) speaking and writing performance correlates with learning achievements.
4. Simple regression – to find the predictive power of (a) attitude toward using VPen to actual VPen usage; (b) actual VPen usage to speaking and writing performance; (c) speaking and writing performance to learning achievements.
5. Stepwise multiple regression – to measure the degree to which one-way writing, interactive writing, one-way speaking, and interactive speaking contribute to the prediction of students’ learning achievements.
4 Results and discussion
4.1 Questionnaire analysis
All items in perceived ease of use and perceived usefulness of the VPen system dimensions were ranked with a high score. Most of the students agreed it was easy to use the VPen system and the system was useful during learning activities. A similar finding about students’ perceptions about using the VPen system for mathematics learning was obtained by Hwang et al. (Reference Hwang, Chen, Shadiev and Liin press). Almost all items in the perceived usefulness of activities dimension were ranked highly, showing that students generally thought that learning activities are useful for improving writing and speaking performance. Students mentioned in the interview that learning activities were useful and fun to do. First, because they created content as textual or audio annotations and practiced EFL writing and speaking skills. The students returned to their annotations to recall the content of learning activities and for remediation. In addition, the students shared their annotations so they got feedback from the teacher and their peers or they gave feedback to others for further remediation of the annotations’ content. Second, the pictures with familiar context that were provided from their textbooks and everyday life inspired students to create meaningful output and engage with the material (Caldwell, Reference Caldwell1998).
Only three items were ranked with a low score: (i) “By sharing my opinions to my partners in B, F, and G activities I can improve my English speaking ability” (ranked as 3.5 out of 5); (ii) “In general, sharing ideas in B, F, and G activities can assist my English speaking” (ranked as 3.6 out of 5); and (iii) “In general, I think activities C and E are helpful for my English speaking practice” (ranked as 3.6 out of 5). The interviews revealed that the students missed the provision of immediate feedback from the teacher or their peers during learning activities due to the asynchronous nature of the interaction. For example, when a student created audio content with a description of a picture it was impossible to get immediate feedback and they had to wait until the teacher or their peers had listened to the audio before they gave their feedback, which could take long time. Meanwhile, the student had no idea on what was incorrect in his/her speech (pronunciation, sentence arrangement, vocabulary, etc.) so he/she just continued speaking and recording speech without any awareness of making mistakes. In the question relating to perceived playfulness of activities, five items were ranked high, showing that students thought the EFL learning activities were fun. In the interview the students mentioned that the approach used in this study to practice writing and speaking skills was different from standard curricula. Whilst previously the students had studied EFL in a decontextualized way from a textbook, and the classes were exam-oriented and based on grammar-translation methods (Yang & Chen, Reference Yang and Chen2007), this time EFL learning was creative and enjoyable. However, some of items in the questionnaire were ranked low (3.2 out of 5). The statement, “I can experience an authentic English learning environment when doing the English online activities” was scored low because we could not provide an authentic EFL learning environment with immediate feedback to the students’ text or audio input. In a future study, we will try to overcome this and other shortcomings. The rest of the items were scored low because some students were not familiar with the activity design, which was production-oriented, where students had to create text or audio content; thus some students found participating in activities difficult and they indicated that they disliked these activities. Almost all items in the “perceived attitude toward using the VPen system” question were ranked high. This shows that, in general, the students had a positive attitude toward using VPen. Only the statement, “I frequently use the VPen system to do English learning activities” was ranked low (3.5 out of 5). The interview with the students revealed that all students used the VPen system during class time only. They said they did not use the VPen system after class because they were focused on studying other subjects and completing homework. Furthermore, parents did not allow students to use computers after class. Since the students needed to pass the EFL exam at the end of the academic year to enter senior high school, the parents had high expectations that the students should spend their after-class time in studying, doing homework and preparing for the exam. The parents were afraid their children would spend their time playing computer games instead of studying.
4.2 Correlation and regression analyses
4.2.1 Students’ attitude to using the VPen system and their actual VPen usage
The result of Pearson's correlation analysis showed that perceived ease of use and perceived usefulness of the VPen system, perceived usefulness of activities and perceived playfulness of activities significantly correlated with attitude toward using VPen, as shown in Table 1. The questionnaire and the interviews revealed that the students had positive perceptions toward using the VPen system and learning activities, thus perceived ease of use and perceived usefulness of the VPen system, perceived usefulness of activities and perceived playfulness of activities significantly correlated with perceived attitude to using VPen.
Correlation analysis of students’ perceptions toward using VPen and learning activities and attitude toward VPen use
*p < .05, **p < .01.
The results of Pearson's correlation analysis showed that students’ attitude toward using VPen have no significant correlation with actual VPen usage (.303, p > 0.05). In addition, the simple regression analysis demonstrated that the students’ attitude toward using VPen have no predictive power regarding actual VPen usage (β = 1.521). The students mentioned in the interviews that although they had a positive attitude toward using VPen they could use it during class time only. They were busy studying other subjects and their parents prohibited them from using a computer after class.
4.2.2 Students’ actual VPen usage and speaking and writing performance
The Pearson's Correlation revealed that actual VPen usage has significant correlation with speaking and writing performance (.556, p < 0.01). Also, the simple regression analysis showed that actual VPen usage has significant predictive power (β = .306) over writing and speaking performance. The reason for this finding can be explained as follows. In accordance with the learning activity design, the students created annotations with meaningful textual and audio content based on pictures with familiar context. Then the students were involved in interaction by sharing content with peers, exchanging comments, suggestions, and questions. The creation of content and further interaction facilitated learning (Brown, Reference Brown2001; Ellis, Reference Ellis1988; Wu, Reference Wu, Bird and Harris1992; Yu, Reference Yu2008) as students practiced writing and speaking skills by creating content, asking and answering questions, giving suggestions and feedback, revising and improving their own annotation content based on their peers’ feedback. Since the VPen system was adopted in this study and the students could only create content and share it with their peers by using the system, it can be concluded that the students practiced productive skills every time they used the VPen system. And since practicing EFL skills influence performance, the actual VPen usage had significant correlation with speaking and writing performance.
4.2.3 Students’ writing and speaking performance and learning achievement
The Pearson's correlation showed a significant correlation (.392, p < 0.05) between students’ speaking and writing performance and learning achievement. Moreover, the simple regression analysis demonstrated that students’ speaking and writing performance has predictive power over students’ learning achievement (β = .697). This finding can be explained as follows. The students participated in the learning activities that foster EFL use and rehearsal by creating textual and audio content, sharing it and interacting with peers. The more a student uses and rehearses the language, the better his/her writing and speaking skills will develop. The writing and speaking performance of those students’ who actively participated in the learning activities was high throughout the semester, and therefore they obtained high scores in the post-test as the post-test items related to students’ writing and speaking performance.
Stepwise multiple regression demonstrated that none of the scores of one-way writing, interactive writing, one-way speaking, and interactive speaking have significant power to predict students’ learning achievements. Perhaps this finding is due to a ceiling effect: both the GEPT test at the beginning of the experiment and the post-test at the end of the experiment demonstrated that students had good EFL proficiency. Students passed the former with at least an elementary level and the latter with comparatively high scores (an average of 93 out of 100 point scales). We will address this issue in the future study by adopting different post-test items with suitable item discrimination and item difficulty. This will help us to obtain more precise details of each student's performance in the post-test, making the assessment of the students’ performance clearer and providing more accurate results regarding the predictive power of various variables on learning achievement.
4.2.4 Writing and speaking: students’ preferences
Further investigation was conducted to analyze students’ preferences for output methods to complete assignments. The investigation revealed that the students preferred one-way writing and interactive speaking more than interactive writing and one-way speaking methods (see Table 2).
Methods and rates to complete seven EFL learning activities
The reason for such preference can be explained as follows. The teacher asked students to complete their assignments using any method they preferred, thus students could favor one method over another. The most preferred method was one-way writing since the other methods are not often employed in Asian schools (Yang & Chen, Reference Yang and Chen2007). Teachers often give an assignment that requires writing an essay or composing a text, thus, students get used to the one-way method of writing. Students did not favor the one-way speaking method because this was a new way for them to practice speaking skills; they preferred speaking to the teacher or their peers rather than speaking into a microphone. Another reason was lack of immediate feedback to the students’ audio content. The students completed the assignments by speaking into a microphone and then shared audio with the teacher to get feedback for further remediation, but they could not get it immediately. Therefore, most of the students completed the major part of the assignments using the one-way writing method instead of the one-way speaking method. The students did not wish to use the interactive writing method to complete their assignments. They mentioned in the interviews that it was their first experience of using the interactive writing method and they could not get used to it. The students mentioned in the interviews that it was their first experience of using an interactive speaking method and they could not get used to it. Also, the students mentioned that the interactive speaking method was preferable because it was fun. The students could hear their peers’ pronunciation, sentence arrangement, vocabulary, etc., and compare their own content with others’ for remediation. In addition, the students could point out the mistakes of the others and make appropriate suggestions. Therefore, they completed the assignments using the interactive speaking method rather than interactive writing. It was also found that the students performed worse on completing the assignment for activity G in both interactive writing and interactive speaking. A possible reason for this finding is that it was the end of the semester when the students had a greater work overload and pressure before final exams.
4.2.5 Support of the VPen system for developing EFL writing and speaking skills
The findings imply that the VPen system is helpful for developing students’ productive skills due to the following reasons. First, the system enables students to reflect on the learning material individually by creating annotations and then collaboratively by sharing annotations with their peers and/or teacher for further and deeper discussion about their ideas and thoughts (Hwang et al., Reference Hwang, Chen, Shadiev and Liin press; Yeh & Lo, Reference Yeh and Lo2009). Secondly, the VPen system creates an asynchronous learning environment for students. The asynchronous learning environment provides flexibility so that students have time for reflection before they comment or carry out online assignments. As a result, learning in such an environment reduces student anxiety, and thus influences their active learning (Huang & Lee, Reference Huang and Lee2004; Lamy & Goodfellow, Reference Lamy and Goodfellow1999; Sproull & Kiesler, Reference Sproull and Kiesler1991), especially those from Asian countries, including Taiwan. We agree that other technologies, for example, a discussion board, can also support students’ asynchronous interaction and we agree that a discussion board provides the students with benefits similar to those of the VPen system. But we argue that the interface of a discussion board includes a list of threads and posts which is separate from the interface of the learning material and both interfaces cannot be easily combined; thus, using a discussion board may create obstacles for the students in focusing on the whole learning scenario. On the contrary, the VPen system provides learning material and annotation tools in the same interface so the students can attach their annotations to the learning material. In addition, annotations can be attached to any position in the learning material thus building a connection between the content of the annotation and that of the learning material and giving the students a clear picture of the whole learning scenario with an appropriate explanation of it. Thirdly, the VPen system provides multiple channels for students’ language output by enabling students to attach annotations with both textual and audio content which is another advantage over a discussion board in practicing writing and speaking skills efficiently. Also, using pictures in the annotations which are familiar in the context of the learning material and everyday life stimulates the students engagement and imagination, and as a result they create meaningful output (Caldwell, Reference Caldwell1998).
5 Conclusion
Previous studies related to the use of multimedia web annotation systems focused primarily on the development of vocabulary acquisition and reading comprehension because of the current state of computer technology (Ariew & Ercetin, Reference Ariew and Ercetin2004; Chun, Reference Chun, Ducate and Arnold2006; Sakar & Ercetin, Reference Sakar and Ercetin2005). This study argues that teaching and learning activities to improve writing and speaking performance can also be facilitated by using multimedia web annotation technology. Therefore, we conducted an experiment with appropriately designed learning activities, using the VPen system. The aim of this study was to investigate the students’ perceptions of VPen and learning activities. The relationships between (a) actual usage of VPen and writing and speaking performance and (b) writing and speaking performance and learning achievement were also investigated. The study revealed that the students perceived the VPen system as easy to use and useful for learning. They also perceived the learning activities as useful for learning and fun. The results also showed that actual VPen usage significantly correlated with speaking and writing performance and the speaking and writing performance significantly correlated with learning achievement. The study found that one-way writing and interactive speaking were the methods most preferred for completing assignments.
Based on the findings obtained we recommend employing appropriate learning activity design and the VPen system for facilitating students’ writing and speaking performance and improving learning achievement. In designing learning activities the teacher has to account for students’ preferred method of output. In this study the students favoured one-way writing and interactive speaking methods, thus, the teacher may focus on these methods. In addition, the teacher may facilitate the transition from one way writing to interactive writing because this is beneficial for learning in several ways: (a) the students can study the content of their peers’ annotations, then compare this with the content of their own annotations and find the differences, with a view to further improving the content of their own annotations; (b) the students can find the errors in the content of their peers’ annotations and suggest appropriate improvements; (c) the students can improve the content of their own annotations based on their peers’ suggestions. The VPen system can help students practice writing and speaking skills in an asynchronous learning environment, allowing them enough time to reflect on the learning material or create content. The pictures with familiar context can stimulate students’ engagement and imagination so that they create meaningful content.
Although the findings of this study are promising, there are several limitations to the study that need to be considered. The first limitation concerns the relatively small sample size; for this reason, these findings cannot be generalized to the broader community based on this study alone. The second limitation relates to effects observed following short-term exposure to experimental conditions that have limited relevance to those occurring after long-term exposure to actual “real-world” work conditions. The third limitation relates to the post-test assessment of the students’ learning achievement. The ceiling effect was experienced in this study and showed that the average score of the post-test at the end of the experiment was quite high due to poor selection of the post-test items. These limitations will be addressed in a future study. We would also like to focus on researching the effectiveness of learning activities with the support of the VPen system for improving receptive skills, as the students mentioned in the interview that learning activities with support of the VPen system were helpful in developing reading and listening skills as well. In addition, we would like to study the relationship between the number of times the students read/listened to their own or/and their peers’ annotations and their learning achievement.
Acknowledgements
This work was supported in part by the National Science Council (NSC), Taiwan, ROC, under Grant NSC 98-2631-S-008-003, NSC 98-2511-S-008-008-MY3 and NSC 98-2511-S-008-005-MY3.
