1. Introduction
Storytelling is a genre of spoken interaction that dominates workplaces and social worlds, but it can be challenging to frame these out-of-school agentive experiences in second language (L2) teaching and learning. Recent technologies link academic discourse to new media and culture outside school (Ware & Warschauer, Reference Ware and Warschauer2005) and transform resources for research data collection and analysis (Flewitt, Hampel, Hauck & Lancaster, Reference Flewitt, Hampel, Hauck and Lancaster2014). Beyond elocution, that is, “the art of careful public speaking, using clear pronunciation and good breathing to control the voice” (Cambridge Dictionary Online), storytellers can integrate and transform multimodal resources and digital technologies across contexts to refashion L2 storytelling. Despite a growing body of research investigating multimodal communicative activities in face-to-face (F2F) and digital contexts (e.g. Goulah, Reference Goulah2007; Hull & Katz, Reference Hull and Katz2006; Hull & Nelson Reference Hull and Nelson2005; Neville, Reference Neville2010, Reference Neville2015), L2 researchers have not fully explored multimodal narrative discourses using a range of methodological procedures. In the sections that follow, the author presents a technology-mediated discourse approach to L2 storytelling by synthesizing research on narrative and speech styles as well as on digital and multimodal communication. The results of multimodal narrative discourse analysis (MNDA) illustrate L2 students’ diverse storytelling styles and communicative modalities that go beyond the performance of verbal art.
2. Literature review
Narrative analyses of personal experiences have highlighted temporally ordered storylines, including the abstract, orientation, complicating actions, resolution, and coda, and the narrator’s evaluation and elaboration of the events and actions (Labov, Reference Labov1972; Labov & Waletzky, Reference Labov and Waletzky1967). To assess L2 storytelling, researchers (e.g. Ko, Reference Ko2001; Ko, Schallert & Walters, Reference Ko, Schallert and Walters2003) have adjusted the narrative elements into the plot, elaboration, coherence, evaluation, and resolution. Although narrative structures provide organizational support for storytelling, the structural model may fail to explain the interactional dynamics of storytelling in local contexts of communication (De Fina & Georgakopoulou, Reference De Fina and Georgakopoulou2012). Narrative can be contextualized as discourse to link tellers and audiences in interactions.
L2 narratives are discursively and aesthetically constructed (Duff & Bell, Reference Duff and Bell2002). Research on narratives often involves complex analyses of storytelling styles and social contexts to investigate storytellers’ language learning trajectories and relations with the audience (Pavlenko, Reference Pavlenko2002). In such a context of narrating knowledge, participants perform and manage situational identities (Barkhuizen, Reference Barkhuizen2011). For example, narrators tell, retell and reflect on stories, and researchers interpret, revisit, and shift their understanding of storyteller experiences. Experience-based narratives can be crafted and styled collaboratively in classroom spaces as a rhetorical construction for engaging communities of learners, teachers, and researchers and as a performative mode of inquiry that fosters embodied knowledge and multiple perspectives (Nelson, Reference Nelson2011). In this sense, agentive experiences of telling and interpreting stories are conceived as our sense or awareness of ourselves as social actors rather than as willing individuals in creating speech styles and changing L2 storytelling.
From the discourse perspectives of ethnolinguistics and sociolinguistics, speech styles are alternative ways of speaking (Hymes, Reference Hymes1972) and are creative, design-oriented processes (Coupland, Reference Coupland2007). Crystal and Davy (Reference Crystal and Davy1969) identified several stylistic devices in English, including (a) vocalizations (i.e. non-phonetic sounds); (b) silence and voiced pauses; (c) prosodic variations in pitch, loudness, and speed; and (d) paralinguistic and physiological mechanisms (e.g. nasalization, velarization, labialization, huskiness, and whisper). Social actors use particular styles of English and other embodied styling practices to index social and cultural positioning within interactive situations (Pennycook, Reference Pennycook2007; Rampton, Reference Rampton2006).
Discovering ways of storytelling is a matter of style, genre, and stance expressions (Pennycook, Reference Pennycook2007). To communicate emotions or stances, narrators can use expressive prosodic and phonological features such as exaggerated pitch and volume, lengthened sounds, pauses, breaths, hesitations, and laughter to complicate actions (Wennerstrom, Reference Wennerstrom2001) and make points (Mischler, Reference Mischler2008). Moreover, storytellers can use their bodies to enact character movements and actions (i.e. character viewpoints) or their hands to emphasize certain narrative segments (i.e. observer viewpoints), which also create a discourse context that makes narrative intelligible to audiences (Debreslioska, Özyürek, Gullberg & Perniss, Reference Debreslioska, Özyürek, Gullberg and Perniss2013). Multimodal contextual styling – that is, genre making and breaking for creating and responding to context (cf. Coupland, Reference Coupland2007) – enables dynamic and interactive storytelling.
Recent multimodal environments for digital storytelling allow participants to adjust stories to meet the expectations of different contexts and tasks. A digital story featuring still and moving images can be constructed to engage the public (Godwin-Jones, Reference Godwin-Jones2012). Applying Labov’s narrative structure to analyze a digital story, Hull and Nelson (Reference Hull and Nelson2005) argued that youths often relocate stories within specific multimodal contexts for aesthetic expression. Building upon the dialogism and ventriloquism concepts of Bakhtin (Reference Bakhtin1981), Hull and Katz (Reference Hull and Katz2006) also determined that youths developed agentive senses of self and became adept at engaging the audience by composing, sharing, and responding to digital products. Multimodal meaning makers as agents can use and combine various semiotic modes to design products or events (Kress & van Leeuwen, Reference Kress and van Leeuwen2001), to adjust their voices (van Leeuwen, Reference van Leeuwen2014), and to reshape contextual resources (Hampel & Hauck, Reference Hampel and Hauck2006). By interfacing with technologies, making digital videos, and using gestures, L2 students could develop multimodal expressions and critical reflections (Goulah, Reference Goulah2007).
Storytelling through the multimodality of virtual reality differs from digital storytelling using resources from a library of images or other digital tools in that storytellers can control their characters’ actions and interact with environmental resources to create stories. Neville (Reference Neville2010, Reference Neville2015) has reported that students’ navigation and immersion in narrative simulations helped them construct story maps. In Second Life, although L2 students mainly interact with others through verbal communication (Deutschmann, Panichi & Molka-Danielsen, Reference Deutschmann, Panichi and Molka-Danielsen2009; Liang, Reference Liang2012a, Reference Liang2012b, Reference Liang2013; Liou, Reference Liou2012; Peterson, Reference Peterson2012), players can also communicate through the nonverbal modes of avatar appearance, movement, and gestures (Wigham & Chanier, Reference Wigham and Chanier2013). By moving around in the multimodal landscapes of Second Life, L2 storytellers could interact with multisensory resources and enact particular narrative viewpoints and identities to construct narratives (Liang, Reference Liang2015). The author extended this work to investigate L2 students’ discursive processes and agentive experiences of engaging in multimodal storytelling practices using MNDA. The following method sections present a technology-mediated approach to multimodal narrative discourses with associated pedagogical and analytical procedures in order to highlight styles and modalities of L2 storytelling.
3. Methodology
3.1 Participants
The author was the instructor and researcher who conducted this study within an elective professional communication class at a major university in Taiwan. Seventeen third- and fourth-year English majors or minors between the ages of 20 and 21 were enrolled in this class. Their English competence varied, but all had achieved an upper-intermediate level. At the end of the semester, 12 students (S1–S12) voluntarily provided their story productions and 14 students (S1–S14) responded to questionnaires for this study. Of the 14 participants, three were male (S6, S11, S13) and the others were female. The instructor inserted particular interactional procedures (see section 3.2) into classroom storytelling practice to foster reflective teaching and learning (cf. Nassaji, Reference Nassaji2012). The author also served as a participant and observer in this particular context of constructing and analyzing participants’ storytelling.
3.2 Pedagogical procedures
The pedagogy included the following interactional procedures: (a) teacher demonstrations and instructions on transcriptions and analyses of storytelling; (b) students’ narrative simulations and story compositions in Second Life; (c) students’ PowerPoint presentations of the stories to classmates, and peer evaluations and class discussions; (d) students’ transcriptions of the video recordings of the narrative presentations; (e) participant self-evaluations; and (f) students’ digital video productions.
First, the instructor introduced the students to the affordances of Second Life and demonstrated the work of previous students. Second, individual students created avatars to interact with contextual resources (scenes, simulations, objects, other players, etc.) in Second Life and then composed stories through PowerPoint presentations based on snapshots or still images taken in Second Life. Third, in each 150-minute classroom storytelling session, the class was divided into two groups. Half of the students told stories in real-time PowerPoint presentations without reading scripts, and the other half provided feedback based on peer-evaluation questions (see Appendix A). Each storyteller was assigned five minutes to present the story to the whole class sequentially and received 15 minutes of feedback. Fourth, the storytellers transcribed storytelling after class based on transcription notations (see Appendix B). Fifth, individual students answered self-evaluation questions (see Appendix C). Sixth, after receiving F2F and written feedback, participants edited their PowerPoint presentations using resources from Second Life and other online materials. Participants then used their revised PowerPoint slides to record their digital stories through iMovie, Movie Maker, or other software without any direction from the instructor. This task was not part of the graded activities; however, the participants could get help from peers.
MNDA related storytelling practices to various pedagogical contexts. L2 students presented stories to a physically present audience acting as reviewers by using multimodal features and transcribed, edited, revised, recorded, and shared stories as more careful digital productions. Such storytelling practices provided L2 students with activity settings, social interactions, and digital environments, which afforded multiple modes of expression and layers of reflection. In particular, multimodal transcription and analysis transform a partial view of the ethnography of speaking into the situated and dynamic complexity of different communicative resources in sociocultural contexts (Flewitt et al., Reference Flewitt, Hampel, Hauck and Lancaster2014). The affordances of various contexts and activities also helped them develop awareness of multimodal narrative designs and storytelling styles supported with MNDA.
3.3 Data and analytical procedures
The technology-mediated discourse approach to L2 storytelling includes multiple data and analytical procedures. Major sources of data included (a) video recordings and transcripts of the F2F narration, (b) the revised digital storytelling videos and transcripts, (c) presented PowerPoint slides, (d) recordings and transcripts of peer feedback, and (e) written self-evaluation forms. Two research assistants (a) recorded students’ F2F storytelling and sent the video files to students; (b) collected students’ PowerPoint slides, digital storytelling videos, transcripts, and self-evaluation forms; and (c) edited students’ transcripts.
MNDA shares many of qualitative and interpretive methods with ethnolinguistics, sociolinguistics, multimodal discourse analysis, and narrative analysis, including close-participant observation and using video recordings with transcription and analysis of storytelling and narrative discourse to allow adequate presentations of discourse contexts influencing L2 storytelling. MNDA extends different forms of multimodal transcription and narrative analysis into a series of procedures using tools and technologies in the research process with respect to (a) narrative elements, (b) discourse structures, and (c) stylistic devices.
First, narrative elements as outlined in Ko (Reference Ko2001) and Hull and Nelson (Reference Hull and Nelson2005) were analyzed (Liang, Reference Liang2012c) and rephrased in the peer-evaluation questions (see Appendix A), including (a) the plot (characters, actions, or story structures), (b) elaboration (interesting dialogues or details), (c) coherence (connected details or actions), (d) evaluation (viewpoints and perspectives towards events), and (e) resolution (the ending of the story).
Second, the participants’ discourse structures of storytelling in terms of elocution, lexical and grammatical use (i.e. speech rates, accuracy, lexical variety, recycle rate, lexical sophistication), and the use of pictures were analyzed (Liang, Reference Liang2016). Trimmed speech rate excluded the times spent on long pauses, false starts, repeated syllables and words, and hesitations, which were measured by running Praat (http://www.fon.hum.uva.nl/praat/). Lexical diversity referred to the number of word types. Lexical sophistication referred to the presence of difficult words beyond the 2,000 most frequent words of English. The levels of lexical diversity and sophistication were all measured by running the trimmed speech transcripts through Lexical Tutor (http://www.lextutor.ca), and accuracy was determined by the ratio of error-free clauses per total number of clauses (cf. Thai & Boer, 2016).
Third, analyses of stylistic devices were based on an array of prosodic, paralinguistic, and physiological features (cf. Crystal & Davy, Reference Crystal and Davy1969; van Leeuwen, Reference van Leeuwen2014) and visual and bodily resources (cf. Debreslioska et al., Reference Debreslioska, Özyürek, Gullberg and Perniss2013; Liang, Reference Liang2015) in F2F narration, as well as audio and video resources in digital storytelling. Further investigations of the participants’ contextual styling (cf. Coupland, Reference Coupland2007) help contextualize changes in L2 storytelling.
The analytic method of MNDA has afforded priority to the L2 storytellers’ dialogical process of narrative inquiry. The interactive process of MNDA empowers social actors, which not only avoids inappropriate transfer of narrative methods across contexts (cf. De Fina & Georgakopoulou, Reference De Fina and Georgakopoulou2012) but also differs from other forms of multimodal discourse analysis that focus on the systemic grammars rather than the sign-makers (cf. Jewitt, Reference Jewitt2014). In addition to the researcher’s ethnographic observations and analyses of how multimodal and narrative resources were used across different technology-mediated contexts, analyses of participants’ responses to the peer- and self-evaluation questions further provided agentive storytellers’ narrative discourses and experiences circulated among learners to explore the possibilities and problems with MNDA and to assess the areas in which L2 storytelling may be refashioned.
The examples of MNDA are shown in the following sections, including the use of verbal, visual, and bodily resources of F2F presentations for rehearsals (section 4.1); the use of visual, digital, and video resources for enhancing L2 storytelling specific to the digital productions (section 4.2); and multimodal narrative designs and storytelling styles in social interactions (section 4.3).
4. Results and discussions
Participants’ discourse structures of storytelling are presented in Appendix D. Although the change of contexts from F2F to digital storytelling evoked comparable shifts, the author focused on qualitative analyses of participants’ L2 storytelling across contexts. The author first selected narrative segments, which were associated with PowerPoint slide sequences and time spent in storytelling. The author then analyzed more precisely how the stories were creatively styled and modified between the F2F presentations and the revised digital versions of the stories, and explored the affordances and constraints of digital resources for L2 storytelling. The author finally examined the L2 storytellers’ multimodal construction of narrative discourse.
4.1 Participants’ L2 storytelling styles
Six storytellers tended to maintain relatively constant L2 storytelling styles with respect to narrative structures, discourse structures, and stylistic mechanisms across contexts. In the following excerpts, the author analyzed their F2F storytelling using MNDA.
The male student S11 demonstrated relatively fast speech while generating speech repetitions and errors (e.g. something, been, set me up in Figure 1) in both F2F and digital storytelling contexts. His slightly exaggerated elocution and gesture animated the story A Scifi-Action Hotchpotch (Figure 1). To narrate a professional life as a secret agent, he used the present tense of the verb to position the audience as participants in here-and-now events. His hand gestures were synchronized with stressed expressions and actions (e.g. HEAVILY WALKING, WHAT IS GOING ON, lose my breath, hear his footstep, touch my daughter, break his neck, feed him to my dogs). The verbal and bodily performance also worked synergistically with the visual image (e.g. a T-Rex’s walking) to present the observer’s viewpoint (cf. Debreslioska et al., Reference Debreslioska, Özyürek, Gullberg and Perniss2013).
S11F2F. A Scifi-Action Hotchpotch
S8 also spoke quickly, although the within-clause pauses, hesitations, repetitions, and false starts give an impression of poor elocution. In contrast to S11’s observer viewpoint, in her story, A Hammock Dream, S8 used onomatopoeia and simpler sentences through a character’s point of view (cf. Debreslioska et al., Reference Debreslioska, Özyürek, Gullberg and Perniss2013) to imitate actions (hopped) and moving sounds (SHU::) as in Figure 2a and Figure 2c. The scenes of transformation in Second Life were materialized in gestures and movements. S8 expressed emotions and viewpoints and connected actions through a series of humorous graphic designs (e.g. emoticons in pic#8) and user-controlled camera movements and animations in Second Life (e.g. the avatar’s facial expressions and actions in pic#4, 8, 13, and 14).
S8F2F. A Hammock Dream
In contrast, S12 spoke more slowly with simpler and more accurate sentences. By drawing on the character of a personified Autobot in the style of Toy Story, she created a cartoon-like speech and plot through higher pitch, exaggerated stress (e.g. ↑WHAT I can’t even kill ↑myself), and hand gestures (Figure 3a), along with an unexpected sound act (e.g. bomb in Figure 3b).
S12F2F. Mr. Panda’s Bizarre Christmas
S2 also spoke slowly. In her story, Dream by Dream, S2 cast herself in the role of a pianist learning about her past lives through a semiotic resource eohippus in Second Life as in Figure 4. Her frequent pauses seemed to indicate elocutional shortcomings but actually showed aspirations of philosophical thinking about the long history of life. Her hand gestures went along with diverse lexical expressions (e.g. eohippus, sky, freedom, flight) to create natural rhythms of pausing and breathing for more accurate grammatical constructions.
S2F2F. Dream by Dream
S5’s speaking speed was the slowest in F2F storytelling. In her story, Little Mermaid Growing Up, she used relatively more diverse words to set up a dramatic scene. She mimicked the character’s voice in direct speech and laughing (i.e. HA HA my sweetie), created a character by interfacing with phantom technology in Second Life, and wrote the word Poseidon (Figure 5a) and drink me (Figure 5b) on the picture to make the painting information graphic. In addition to a charming speech style, her hand gestures (i.e. curving her index finger inward) added suspense and mystery to the plot.
S5F2F. Little Mermaid Growing Up
In Figure 6, S7 expressed the subjective feelings of the red mushroom with increased volume and high pitch accents, which seemed to be linked to the peer group’s exciting experiences of playing the video game Super Mario as children. Although S7 displayed a relatively higher level of accuracy and lexical diversity, she also used self-regulatory gestures (e.g. pushing her chin up to think) for retrieving dialogues and descriptions about the other scene. The involuntary gestures (e.g. touching hair and combing hair) revealed her nervousness.
S7F2F. The Island of Memory
The results of MNDA illustrated the significance of teaching and studying L2 storytelling using narrative simulations in Second Life and PowerPoint presentations in class. In Second Life, participants must intentionally use avatars’ gestures, facial expressions, and body motions (cf. Wigham & Chanier, Reference Wigham and Chanier2013). Using semiotic resources (e.g. T-Rex, Autobot, eohippus) or material objects (e.g. rabbit and mushroom) in the virtual world for story maps (cf. Neville, Reference Neville2010, Reference Neville2015) and narrative discourse (cf. Nelson, Reference Nelson2011), these L2 students could perform various figures of speech (e.g. metaphor, hyperbole, onomatopoeia, and personification) in different ways (e.g. swift, philosophical, humorous, or comedic). To develop the plot, add details and viewpoints, or connect actions, cultural scenes (Figures 1, 3, 4, and 6), gestural images (Figure 2), and information graphics (Figure 5) worked synergistically with contextual styling (cf. Pennycook, Reference Pennycook2007; Rampton, Reference Rampton2006). However, a few still images of characters’ embodied actions in the virtual world contained inadequate contextual clues, which caused low levels of elocution or narration in some students (e.g. Figures 2 and 6). Although the multimodality in Second Life may afford or constrain F2F storytelling, L2 students could potentially refashion storytelling during digital productions, as shown in section 4.2.
4.2 Participants’ refashioning of L2 storytelling
Compared to the six students mentioned in section 4.1, the following six students changed narrative elements, discourse structures, or stylistic mechanisms, and even incorporated visual and video resources specific to the digital productions. Although this section compares F2F and digital contexts, as participants only recorded their voices in the digital stories, these versions lacked the body language of the participants and a live audience. Therefore, some of the changes could be due to these differences in the multimodal landscape of each context.
S6 spoke slowly. His story was adapted from several science-fiction films, such as The Matrix, Inception, and In Time, but the wide shot of the scene made it hard to understand the multilayer plot. In digital storytelling (Figure 7b), S6 further reduced his speaking speed to enhance accuracy. He also made significant revisions to narrative elements, especially the ending of the story, for clear elocution. The changes made included deleting confusing expressions (e.g. living table and patient) and adding details (e.g. everything was listening to me), which led to a shocking discovery (i.e. that we were all fake).
S6. Alcoholic Fantasy
S4’s A Painter’s Story (Figure 8) included comic dialogue in speech bubbles between two avatars. Although she used high pitch and facial expressions in complicating actions (cf. Wennerstrom, Reference Wennerstrom2001) during F2F storytelling (Figure 8a), her digital version (Figure 8b) included expressive prosodic features (e.g. volume, speed, breaths, lengthened sounds).
S4. A Painter's Story
S10 was one of the two participants (S9 and S10) who had neither gestures nor eye contact with the audience during F2F storytelling. When S10 told the story, Memory Seeking, she spoke fluently while performing various character roles in a soft and clear voice within the context of the dialogue, as in Figure 9a. For digital storytelling (Figure 9b), she retained three fourths of the original text, while adding nine pictures and shortened sentences, as well as dialogues for pictures. She also reduced her speaking speed.
S10. Memory Seeking
S9 recycled about three fourths of the original text and added five pictures in the revised, digital stories. In the original version of Never Stop Learning, she included a lengthy pictorial description of a woman making a vase on the ground (Figure 10); in the digital version, this narrative sequence was presented in three pictures with the same scene shot at night (Figure 10b) and during the day (Figure 10c) through the media editing of pictorial brightness. She also made full use of voice as, for example, in voiced alveolar trills rrr and nasalization (Figure 10a).
S9Dig. Never Stop Learning
S1 retained approximately 80% of her original words in digital storytelling. She used few words but was one of the participants (S1, S2, S3, S5, S7, S9, and S10) who showed agency and interest in incorporating multimedia effects in digital videos. In her F2F version, false starts, errors, and pauses weakened the introduction of the story. In the digital version (Figure 11b), background drumbeats were added, the graphic was modified with a sepia tone, and the film age was set to the oldest in order to establish the plot.
S1. Do You Believe Second Life?
S3 substantially revised a drama cliché, which involved restyling or genre breaking (cf. Coupland, Reference Coupland2007). In digital storytelling, she reduced speaking speed. Her original short story featured the narrator’s voice with wide shots and a flat tone. By inserting environmental sounds (e.g. the murmuring stream and blowing wind in Figures 12b and 12c) and changing graphic frames (e.g. cropping to emphasize feet in Figure 12c), S3 highlighted a particular time and space of the events. The digital storytelling allowed her to play character roles by contrasting two stereotypical voices: the low and husky voice of Tomoe (the mother-in-law) versus the high and squeaky voice of Momoko (the daughter-in-law), which were delivered together with a change of pace (Figure 12a). Although the pronunciation was not fully clear because of the use of a high-pitched, anime-style shrill tone (e.g. Figure 12b), S3 experimented with distinct genres and prosodic features to add black humor to the Asian-style soap-opera trope.
S3Dig. The Wife and the Mother-in-Law
During digital productions, multimodal resources were contextualized in three significant ways. First, narrative elements and discourse structures were modified, and English stylistic devices (cf. Crystal & Davy, Reference Crystal and Davy1969; Pennycook, Reference Pennycook2007) were localized in digital productions for more accurate expression and clear elocution. Second, prosodic and vocal variations of human voice, background sounds, and music (Figures 9–12) were used for developing dramatic plots or adding interesting details (e.g. times, places, people, and objects) to digital productions. Third, video and visual techniques, such as adding speech bubbles (e.g. Figure 8), using contrast in color and lighting (e.g. Figure 10–11), and cropping and changing camera angles (e.g. Figure 12) helped connect and highlight details or points in digital productions. Although a combination of media and modalities (verbal, vocal, visual, and musical) enabled multimodal contextual styling, it could also disrupt enunciation (e.g. Figure 12). One could speculate that the participants have had broader multimodal and interactional experiences than verbal practices. In what follows, the author discusses how L2 students’ agentive uses of multiple modes and media in supportive social interactions could help reshape L2 storytelling.
4.3 Participants’ narrative discourse and transformative reflections
Let us now examine the participants’ responses to the self- and peer-evaluation questions (see Appendices A and C) with respect to different perspectives on multimodal narrative designs and storytelling styles.
4.3.1 Multimodal narrative designs
The participants showed alternative ways of using certain narrative elements in telling the stories.
∙ The plot. S12 commented on the effects of her narrative structure and design:
(13) There’re some plots for humorous conversations but my story’s lack of turning points can prevent the audience from knowing the end of the story while they were listening to the middle of it. (cf. S12 in Figure 3)
∙ Elaboration. The participants added contextual details by improving visual and sound effects to engage the audience (e.g. S1 in Figure 11; S9 in Figure 10). For instance, by envisioning how the storytelling performance might have been, S9 produced delightful characters:
(14) The speed is kind of ok but there’s no much variation of voice in my storytelling. I think I can use my voice to make it more interesting because there’re many cute animals in my story but I didn’t make them sound different. (cf. S9 in Figure 10)
∙ Coherence. The participants omitted unimportant details and rearranged graphics to better construct a coherent entity (e.g. S3 in Figure 12, S6 in Figure 7, S9 in Figure 10, and S10 in Figure 9 and 15). For instance, the following excerpt (15) of a peer session indicates that S10 deleted many words (as in Figure 9), particularly character dialogues during the digital deliveries:
(15)1. S8: I like to ask is there any coherence between those paintings and pigs and snow man?
2. S10: um
3. S8: I mean is there any connection between them?
4. S10: I make it the magic connection, just like eh in the story they don’t: talk. … But they know they have some some mysterious connection that they can go together and help Kana Rosa to find who she is.
5. S14: eh I think your your tone is very fluent and your story is also creative but there are too many elements and missions in the story that it’s hard for me to link them together and remember them. Well, so maybe you can de-delete some some eh some little character…
∙ Evaluation. Most of the students favored humorous narratives, whereas others emphasized moral meaning. Students’ different ways of judging the objective of the story showed evidence of a critical stance, with S2 noting:
(16) This is not a story about humor but it has much deep meaning. It’s mystery as dream is. Thus I think people will feel touched by its moral perspective. (S2 in Figure 4)
∙ Resolution. S6’s last slide did not provide a clear context that listeners could use to identify the word patient. The student’s peers had to address his unclear pronunciation in order to experience the intrigue of the resolution:
(17) 1. S1: so she he drink too much?
2. S6: uh maybe you can say that because uh at the end of of my story actually he found he he is eh he is stand on the table. And he is a sleep walk. He is a patient of sleep walk. And whenever his lives, he will to work. And after he wake up, he will find he at uh very weird or different place at last. So he is he’s an alcoholic when he sleep or is the patient cause him cause him like it.
3. S1: [、peʃən] ?
4. S6: yes, he is a patient.
5. S1: *[、peʃən]* ((turning her head to ask S11))
6. S11: he what?
7. S6: uh *bìng rén* ((providing Chinese translation for the word patient))
8. S1: oh patient.
4.3.2 Multimodal storytelling styles
The following critics of and reflections on narrative discourses showed participants’ awareness of social meanings built up around stylistic differences.
As shown in (18), S3 seemed to sacrifice enunciation for dramatic effects, which extended beyond the dominant elocution confines to recontextualize the storytelling performances in the broader context of pop culture:
(18) I tried hard to build up a very Japanese and semi-cliché scene, but some of the audience couldn’t catch. I’m not pretty satisfied with my ending part. It should be more … dramatic. (S3 in Figure 12)
S3 noted that the use of stereotypical images might not lead to a satisfying ending for the audience, which is a common problem with using popular cultural elements to over-accommodate the audience (cf. Nelson, 2006).
Excerpt (19) shows how S13, a music lover, championed a vibrant style of storytelling and criticized S2’s plain style:
(19) 1. S13: I think your story is very surreal and it’s good but your voice doesn’t help you tell to tell the story, always lifeless like ((waving his finger to imitate S2’s monotone)) yeah
2. Ss: ((laugh))
3. Instructor: so what would you do if you were S2?
4. S13: uh I will use some intonation and different.
5. Instructor: for example?
6. S13: I don’t know like more uh live lively sound.
7. Ss: ((laugh))
8. Instructor: animated sounds?
9. S13: yeah or you can yeah sing along with your violin. uh I mean there’re lots of ways. Yeah.
Although S2 added background music, she focused on the position of expressing a pianist’s sorrow in verbal modalities of pitch, timbre, and tempo, as in (16) and (19). She recycled 90% of the original text, and maintained a relatively slow and gloomy tone.
The storytellers’ expressions of diverse perspectives intensified their feelings of agency in representing experiences and identities within contexts of media and interaction. On the one hand, certain students’ (e.g. S2) resistance to alternative storytelling styles appeared to agree with Ko et al. (Reference Ko, Schallert and Walters2003) in that learner willingness plays a critical role despite scaffolding. On the other hand, most of the students as social actors reshaped L2 storytelling by jointly managing situational identities with others in interactions (cf. Barkhuizen, Reference Barkhuizen2011; De Fina & Georgakopoulou, Reference De Fina and Georgakopoulou2012; Nelson, Reference Nelson2011) and adding multiple media and modes in digital productions (cf. Hull & Katz, Reference Hull and Katz2006). In this sense, their agentive experiences could be modulated depending on contexts to afford greater flexibility for developing L2 storytelling styles, which were also reflectively transformed in social interactions.
5. Conclusions and implications
The author proposed MNDA to teach and study L2 storytelling. The discursive process of MNDA, including narrative simulations, observing and transcribing storytelling, making and sharing digital stories, as well as self- and peer-evaluations, helped L2 students construct the agentive experiences of storytelling. Although the results showed that the proper use of narrative elements, discourse structures, and stylistic devices, as well as bodily, visual, and video resources, assisted the students in developing multimodal designs and storytelling styles, MNDA also posed challenges for teaching and analyzing L2 storytelling. Therefore, attention should be devoted to three areas. First, although narrative simulations in the virtual world could assist students in formulating ideas and structuring stories (cf. Neville, Reference Neville2015), instructions on embodied and digital practices might be required to help certain L2 students fully explore the affordances of Second Life. Second, although digital storytelling could engage the communities of adults and youth (cf. Hull & Katz, Reference Hull and Katz2006), future projects can draw on constructs of discourse and agency to explore L2 storytellers’ learning trajectories in various interactional contexts and activities. Third, MNDA and other forms of multimodal analysis and transcription (e.g. Flewitt et al., Reference Flewitt, Hampel, Hauck and Lancaster2014; Jewitt, Reference Jewitt2014) can be further applied in different pedagogical contexts to inform the design and evaluation of multimodal narrative activities. In summary, this technology-mediated discourse approach to L2 storytelling suggests the importance of teaching and researching broader narrative contexts and activities other than simply elocution in multimodal communicative activities.
Ethical statement
The author has been personally involved in the research study and is held responsible for its content. The participants in this study have been informed of the funder and purposes of the research. The data collected are kept confidential and used for teaching and research purposes only.
Acknowledgements
This study received funding from the Ministry of Technology, Taiwan. I would like to thank Jun-Xian Wu and Wei-An Lai for their assistance in data processing. I am grateful to the editors and reviewers of this journal for their insightful suggestions.
Appendix A
Peer evaluation questions
1. Plot
a. What are some of the interesting episodes?
b. Which episode(s), character(s), or action(s) can be further developed?
2. Elaboration
a. What interesting dialogue or unique information helps you understand the story?
b. What contextual details can be added to the story?
3. Coherence
a. What are some of the irrelevant situations or redundant ideas that confuse you?
b. What transitional expressions or connecting devices can be added?
4. Evaluation
a. What are the speaker’s attitudes, emotions, or moral perspectives towards events?
b. What should the speaker focus on to stimulate your interest in the story?
5. Resolution
a. What resolved tensions or brought new lights to the story?
b. What information can be added to improve your satisfaction with the resolution?
6. Other comments (e.g. pronunciation, volume, speed, vocal variety, word choice, grammar, eye contact, gesture, visual aid, etc.)
Appendix B
Transcription notations
(.) the length of a pause in seconds
↑ ↓ pitch rise or fall
: or :: lengthened sound
WORD louder in value
>word< pacing faster
<word> pacing slower
* * (( )) author’s comments
Appendix C
Self-evaluation questions
1. What was the most interesting point of storytelling?
2. What was the most difficult part of storytelling?
3. How much planning did you do?
4. How satisfied are you with your storytelling?
5. How well do you think the audience understood your story?
6. Please describe your speech performance.
Appendix D
Participants’ discourse structure of storytelling
Story pictures, elocution, lexical and grammatical use
About the author
Mei-Ya Liang is a professor in the Department of English at National Central University Taiwan.
