New Dimensions in Literacy

We have moved from “telling the world to showing the world” (Kress, Reference Kress2003, p. 140). At a recent scholarly meeting, Lotherington (Reference Lotherington2010) proposed the idea of two-dimensional (2D) literacies to metaphorically capture the static, linear, paper-based reading and writing agendas of school language and literacy curricula and assessment. Digitally mediated, multimodal communication is dynamic, adding a third dimension of space, in that the reader can enter the text in new and exciting ways (e.g., as cowriter in collaborative texts, actor in augmented reality contexts, or avatar in virtual games); and it is interactive, adding the fourth dimension of time. In this metaphorical view, classroom literacies are flat: They are lifted off the page only in the mind of the reader—surely the aim of reading the statically encoded information therein. Literate engagement in the interactive, multimodal genres created in digital space engages the participant in dynamic, multidimensional communication, (potentially) involving

• • social interaction,

• • haptic activation,

• • physical coordination,

• • visual design,

• • modal complexity (e.g., multiple language engagement, musical accompaniment, and animation),

• • dynamic, collaborative text construction, and

• • alphabetic literacy.

These new possibilities reshape how we understand, teach, and test language and literacy in the classroom.

In L2 teaching contexts, whether characterized as second, foreign, or international language education, teachers have been hesitant to acknowledge and engage these new dimensions of literacy. Valdés (Reference Valdés, Ball and Freedman2004) pointed to L2 teachers’ tendency to conceptualize language in their teaching as an abstract linguistic system, detached from a broader socially constructed multimodal perspective:

Reducing L2 learning to the flat literacies of paper-based resources in the classroom raises questions of authenticity in L2 learning. If teachers are to meaningfully engage L2 learners in communication as it exists in the social world, these brave new dimensions of literacy must be woven into classroom learning.

Language, Culture, and Convergence

New media are built from old media (Bolter & Grusin, Reference Bolter and Grusin1999; McLuhan, Reference McLuhan1964). Proliferating digital technologies do not displace older communication technologies, but merge cultural forms and practices (Buckingham, Reference Buckingham2003). According to Henry Jenkins (Reference Jenkins2006), we are living in a convergence culture, where the media, including those who control, consume, and produce it, have converged. This is most evident in the semantic capabilities of Web 2.0, which allow greater production and user control than previous media.

For Jenkins (Reference Jenkins2006), convergence is not necessarily technologically driven; it is an artifact of new production and consumption practices. Jenkins offers a positive example in a Harry Potter fan fiction (or fanfic) community, where a young girl created her own online school newspaper for the mythical magic school, Hogwarts, attended by the students in the Harry Potter series. Children from around the world participated and were offered help with their contributions to the paper. Whereas the convergence piece appears to be driven largely by popular culture and by the consumption of the Harry Potter enterprise in this example, underscoring Lemke's (Reference Lemke, McKenna, Labbo, Kieffer and Reinking2006) observation that media convergence is blatant in commercial media where pop culture artifacts have tentacles in multiple media franchises, Jenkins argued that this need not be read simply as an artifact of mass market global capitalism. It is a positive example of children becoming apprenticed in creative expression, in this case, through writing. In sites such as this, culture and language are converging, whether the sites are connected with novels; are based mostly on alphabetic input; are game-based sites involving language, image, and movement (e.g., Club Penguin or Farmville); or are game-based sites with massively multiplayer online role-playing games (MMORPG), which immerse the player in image, written dialogue, sound, action, and animation (e.g., World of Warcraft).

Thorne, Black, and Sykes (Reference Thorne, Black and Sykes2009) commented that creative apprenticeship in digital activities can benefit L2 learners:

Black (Reference Black2005) described how fan fiction provided L2 learners unconfident in their written English with opportunities to meaningfully participate in the fanfic community, whose members reviewed fictions as “unofficial beta-readers” (p. 125) or proofreaders, providing holistic critiques and suggestions for rewording.

In the convergence culture of the 21st century, the individual has become simultaneous creator and consumer of mediated communication. The collaborative authorship and digitally connected knowledge communities created by participatory culture are new basics that educational infrastructures must accommodate. All teachers, especially those in the L2 classroom, must understand that single authorship is now an option, not a model in writing, and that the physical classroom extends beyond its brick walls, connected digitally to resources and learning partners.

Epistemological Shifts in Digital Literacies

Changes in the relative value of knowledge (i.e., what counts as knowledge and how one comes to know) is as Lyotard (Reference Lyotard1984) anticipated, a condition of postmodernity. Jewitt (Reference Jewitt and Jewitt2009a) ascribed the 21st-century turn to multimodalism to postmodern influences; the increasing democratization of knowledge in the networked society has challenged modern configurations of truth and authority. This is evident in the ways in which traditional notions of literacy are being reshaped by digital forms. An example of this is the wiki, which is a democratically produced text, accessed and edited collaboratively by multiple users (e.g., Wikipedia). Though these changes have been brought about through the proliferation of computer-based and screen-based technologies, these same technologies, in the not-so-distant future, could disentangle from the screen altogether and be broadcast without aid of a screen through new pixel technologies. What these technologies have brought about is a new understanding of what it means to know, including how it means to know multimodally.

Lankshear and Knobel (Reference Lankshear and Knobel2008) viewed the shift to digital literacies as one that is “shorthand for the myriad social practices and conceptions of engaging in meaning making mediated by texts that are produced, received, distributed, exchanged, etc. via digital codification” (p. 5). These texts (e.g., images, movies, podcasts, blogs, and online social networking sites) encode knowledge very differently, and both what is produced and how one knows and comes to know are different from these processes in traditional print-based literacies. What it is to know under these conditions, then, in multiple languages in L2 contexts is central to this review.

Multimodality and Literacy

The New London Group, in their 1996 manifesto introducing the concept of multiliteracies, theorized changing the “‘what’ of literacy pedagogy” (p. 65) to include six design elements in the meaning-making process: linguistic, visual, audio, gestural, and spatial meaning, and multimodal interplay. This formative theorizing of multiliteracies in the era of the static Web, or Web 1.0, however, lacked the interactive, participatory capabilities of Web 2.0. Theoretical and practical exploration of multimodal literacies has since mushroomed from multiple perspectives, including systemic functional linguistics (Kress, Reference Kress2003, Reference Kress2010; Unsworth & Cléirigh, Reference Unsworth, Cléirigh and Jewitt2009), digital epistemology (Jenson & de Castell, Reference Jenson and de Castell2004; Kellner, Reference Kellner2004; Lankshear & Knobel, Reference Lankshear and Knobel2003, Reference Lankshear and Knobel2006), new literacies (Street, Pahl, & Rowsell, Reference Street, Pahl, Rowsell and Jewitt2009), critical multimedia (Lemke, Reference Lemke, McKenna, Labbo, Kieffer and Reinking2006), visual literacy (Jewitt, Reference Jewitt2002, Reference Jewitt2008; Kress & Van Leeuwen, Reference Kress and Van Leeuwen2006; Unsworth, Reference Unsworth2006), videogaming (Beavis, Reference Beavis and Snyder2002; de Castell & Jenson, Reference de Castell and Jenson2009; Gee, Reference Gee2003; Peterson, Reference Peterson2010; Zheng, Wagner, Young, & Brewer, Reference Zheng, Wagner, Young and Brewer2009), and multilingual/multicultural inclusion in literacy education (Cummins, Reference Cummins, Garcia, Skutnabb-Kangas and Torres-Guzmán2006; Dagenais, Toohey, & Day, Reference Dagenais, Toohey and Day2006; Gutiérrez, Reference Gutiérrez2008; Kenner, Al-Azami, Gregory, & Ruby, Reference Kenner, Al-Azami, Gregory and Ruby2008; Kenner, Ruby, Jessel, Gregory, & Arju, Reference Kenner, Ruby, Jessel, Gregory and Arju2008; Lotherington, Reference Lotherington and Hansson2008, Reference Lotherington2009, Reference Lotherington2011; Lotherington, Sotoudeh, Holland, & Zentena, Reference Lotherington, Sotoudeh, Holland and Zentena2008; Martin-Jones & Saxena, Reference Martin-Jones and Saxena2003). Most, but not all, theorists—or practitioners—work with digital mediation.

Kress framed multimodality as a “domain of inquiry,” (Reference Kress and Jewitt2009b, p. 54), noting that particular media offer different modal possibilities; these capabilities include the combination of audio, visual, linguistic, gestural, and spatial modalities to convey rich meanings. Kress (Reference Kress2010) demonstrated how even the traditional textbook has undergone significant changes in both appearance and content, becoming increasingly image-centered, and moving away from the linear toward a more modular design framework. Cope and Kalantzis (Reference Cope and Kalantzis2009a) described changes in reading from page to screen, noting that Web sites are read more like images than linear text.

Researchers are showing the ways in which multimodal forms of knowing and coming to know are being pushed by the affordances of digital technologies, which are, in turn, changing the ways in which we think of curriculum and pedagogy (Hull & Nelson, Reference Hull and Nelson2005; Jewitt, Reference Jewitt2006; Kress, Reference Kress2003; Lotherington, Reference Lotherington2009; Mills, Reference Mills2010). The move toward multimodal literacies in the classroom, though, is a rocky one, and L2 learning contexts are notoriously resistant in adapting (Tan & McWilliam, Reference Tan and McWilliam2009; Valdés, Reference Valdés, Ball and Freedman2004; Warschauer, Reference Warschauer2008). Indeed, even where technological hardware is available, school contexts have been shown to be underutilizing the creative potential of such technologies for the purposes of L2 teaching and learning (Ware, Reference Ware2008).

From the 3 Rs to Multidimensional Worlds

The new literacy movement to refocus literacy as social practice began decades ago with the work of anthropological researchers, such as Shirley Brice Heath (Reference Heath1983), Ruth Finnegan (Reference Finnegan1988), and Brian Street (Reference Street1984, Reference Street1995), who framed textual practices within a wider sphere of social communication. The New London Group's (1996) collaboration to discuss “what was happening to meaning making and representation in the worlds of work, citizenship and personal life that might prompt a reconsideration of our approaches to literacy teaching and learning” (Cope & Kalantzis, Reference Cope and Kalantzis2009b, p. 166) propelled creative conversations about literacy pedagogy as multimodal. With increasing population mobility, the social practice orientation of new literacy scholars has converged with the concerns of applied linguists and sociolinguists working in L2 learning and language policy; and with rapidly evolving information and communications technologies, these concerns have increasingly involved those of contemporary media scholars working with digital multimedia.

New basics apply to new literacies. Modes, according to Kress (Reference Kress and Jewitt2009b), are realized through semiotic resources, which are culturally negotiated, so they are not static or universal. Cope and Kalantzis (Reference Cope and Kalantzis2009a) framed a grammar of multimodality around core modes of expression—linguistic, visual, spatial, gestural, and audio—based on questions of meaning, leaning conceptually on systemic functional grammar (Halliday, Reference Halliday1994). Cope and Kalantzis explained, “some of the differences in meaning potential afforded by the different modes are fundamental. . . . writing's intrinsic temporality orients it to causality; image to location” (p. 264). Their grammatical dimensions are the following:

• • Representational—What do the meanings refer to?

• • Social—How do the meanings connect the persons they involve?

• • Organizational—How do the meanings hang together?

• • Contextual—How do the meanings fit into the larger world of meaning?

• • Ideological—Whose interests are the meanings skewed to serve? (p. 365)

Sinclair (Reference Sinclair2010) visualized students as knowledge makers in contemporary education who need room for digital play. Sinclair revised the fundamental 3 Rs of modern education (viz., reading, writing, arithmetic) to 4 Rs for the digital era that are fundamentally ludic and collaborative: reuse (backup), revise (adapt), remix (combine), and redistribute (share). These practices are basic to digitally mediated multimodal text creation in which information is understood as recombinant (Sinclair, Reference Sinclair2010).

The significance of this shift to a multimodal approach to learning is that it presumes first and foremost that the primary mode of transmission and production is digital, which opens up possibilities beyond just one approach. Multimodal learning supports collaborative authorship, including the search for, creation, and layering of modes that parallel, extend, and expand textual production, bringing L2 learners together in pursuit of communicative objectives and supporting contextualized acquisition of coded target language forms.

A Ludic Perspective: Play as Work; Work as Play

As Homo Ludens author Johan Huizinga (Reference Huizinga1960) noted decades ago, play is an essential part of what it means to be human, not to mention that play is one of the key ways children learn (Piaget, Reference Piaget, Gattegno and Hodgson, Trans1951; Singer & Singer, Reference Singer and Singer2005). And yet, in schools, play is something that is consistently relegated to the playground. It is something that occurs outside of the hard work of learning and of classrooms, sometimes in the gym or the art room or the music class (though less and less as governments continue to cut back funding to the humanities and physical education). As an outsider to the serious business of learning and work, play is also something that is very much associated with childhood; freedom of movement; creativity and permission to construct new rules; and experimenting with status quo, inversion, and irony. Play is elemental to meaningful engagement with digital technologies—including schoolwork, language learning, and engaging in language play (Derrida, Reference Derrida and Bass1978). For example, when tasked with creating a comic-like piece, students in the Joyce Public School project in Toronto, Canada, described in this article literally played with text size, type, font, color, image placement, and word choice to produce the effects they wanted, including irony and word play. The hard work of producing the kinds of multimodal, multilingual digital objects we explore in this article are also ludic (viz., play in Latin) activities.

Extended Classroom Borders

What is an L2 teaching and learning context in the second decade of the 21st century? This article reviews a variety of contexts for L2 teaching and learning where multiple media offer a panorama of communicative possibilities, calling into question the applicability of familiar dichotomized constructs.

The L2 classroom has traditionally relied on polarized binaries, boxing language into pairs of productive-receptive skills (i.e., reading-writing, speaking-listening), and dividing learners into L1 (first language) and L2—native speakers and nonnative speakers. These concepts are epistemologically grounded in the social and linguistic worlds of speech communities and flat literacies. Many online communities of practice do not acknowledge the binary distinctions that mark insiders and outsiders in physical speech communities, where sociopolitical concepts such as second language and foreign language have explanatory power. Zheng et al. (Reference Zheng, Wagner, Young and Brewer2009), describing social interaction in a virtual world, explained: “In online interaction, learners do not have many opportunities to perceive social, cultural, and linguistic cues directly through embodiment” (p. 505). Though this degree of anonymity is not universal in digital space, it is certainly the case with MMORPG and virtual world environments where players interact through avatars (created online personae).

Though multimodality does not necessarily involve the digital, increasingly this is the case, and certainly it is the way of this century. However it is realized, multimodal learning approaches learners and learning differently. In this review, we group multimodal pedagogies as intergenerational, digitally mediated interactional, and ludic, though these categories are neither static nor exclusive, and some projects fit into all three.

Intergenerational Connections

As Dooley (Reference Dooley and Healey2008) pointed out, a multiliteracies approach guides diversity into rather than out of literacy education. As an example of this, the creation of dual language books and materials—multimodal resources utilizing multiple languages and modes oriented to supporting children's L1 in L2 learning environments—has blossomed in different social and cultural contexts. Cummins (Reference Cummins, Garcia, Skutnabb-Kangas and Torres-Guzmán2006), in discussing a dual language book project involving a host of immigrant languages in suburban Toronto (see Schecter & Cummins, Reference Schecter and Cummins2003), explained the concept of “identity text” (p. 59): stories written bilingually in home and school languages, as self-affirming student identity investments that maximize conditions for learning. McCarty and Bia (Reference McCarty and Bia2002), describing decades-long work in the evolution of an indigenous education program at Rough Rock Community School in Arizona, compared the relatively unstructured Navajo literacy materials created in-house to the regimented, skill-oriented English as a second language (ESL) curriculum. Kenner, Al-Azami, et al. (Reference Kenner, Al-Azami, Gregory and Ruby2008) described a bilingual poetry project in a Bangladeshi community in East London where, with intergenerational support, children learned a poetic range spanning generations, languages, and cultures. In addition, Naqvi (Reference Naqvi2008), working with Pakistani immigrants, reported on bicultural minority language resources for children in schools in western Canada.

Though these projects involved primarily alphabetic literacies (though not necessarily the same alphabets), they incorporated the languages of families split by acculturation into old and new countries in ways that were less rigid and more fun than timetabled L2 classes. Some of these projects used the Web as a publishing venue, though these were sites of display rather than interactivity (e.g., http://thornwood.peelschools.org/Dual).

Similar bilingual and multilingual resources for schoolchildren have been developed as dynamic digital objects. Khvtisiashvili (Reference Khvtisiashvili2010) described a University of Utah project to preserve indigenous languages through film animation of traditional stories. The stories were scripted as action sequences, with the characters’ lines voiced by native community members in vernacular translation. The animated stories utilized both indigenous and majority languages, facilitated intergenerational cultural and linguistic transmission, and offered opportunities to craft and animate characters, as well as to create and edit video. Though these multimodal activities were intended to support and archive indigenous languages, by linking English speakers with indigenous elders to make a collaborative intellectual product, the animated traditional legends project offered a splendid participatory pedagogy for teaching multimodal literacies using any languages.

Digitally Mediated Interactional Spaces

In an interesting genealogical take on digital literacies in a context understood as foreign language learning, Bo-Kristensen and Meyer (Reference Bo-Kristensen, Meyer and Hansson2008) examined the language laboratory. Long considered a pedagogical dinosaur, the language lab is the ancestor of technologically mediated L2 learning, an example of remediation (Bolter & Grusin, Reference Bolter and Grusin1999) in educational innovation. In the context of a military college in Denmark, Bo-Kristensen and Meyer looked at the relationship between L2 pedagogy and technologically mediated pathways for learning in the contexts of English as a foreign language (EFL), and Danish as a second language (DSL) in Denmark. They found traditional audio-lingual and behavioral language teaching methodology simply reinscribed in the remediated EFL digital lab. However, in the virtual language laboratory where the affordances of Web 2.0 mediated the experience of language learning, a sense of immediacy was effected by the temporal-spatial flexibility, fostering more informal EFL learning and answering more closely the students’ needs. Here, the teacher assumed much more agency in assembling dynamic resources in the target language, such as film and radio archives, and Web sites that the learners could access instantaneously. The latest incarnation of the language lab, the mobile language lab, linked learners (in the adult DSL context researched) through mobile devices such as cell telephones and facilitated authentic, relevant content, as did the virtual lab. However, the mobile lab gave more agency to learners who could produce their own authentic learning content by capturing and uploading communication scenarios and materials themselves.

This comparative study reveals that the more portable the digital technology used in the context of L2 teaching—moving from the large, fixed language lab to the virtual lab to linked mobile devices—the more agentive, participatory learning was enabled. More agency was given to the student for locating and contributing relevant and authentic language content, extending the walls of the classroom and the reach of teaching materials to include learner-selected language data in oral, written, and other visual forms (e.g., gestural) that were meaningful and authentic to the learning context.

Using the affordances of digital texts to implement French as a foreign language instruction at the college level in the United States, Williams (Reference Williams, Abraham and Williams2009) found novel ways of comparatively teaching the grammatical forms, tu and vous, which are socially differentiated forms of the second person pronoun (viz., you) in French that have no direct parallels in English, and as such present socially perplexing learning to Anglophones. Using the Coca-Cola France Web site (www.coca-cola-france.fr) to stimulate critical framing (following Kern, Reference Kern2000; New London Group, 1996), Williams described how even the nuisance pop-up window survey could be used to analyze when and where formal and informal pronominal use was preferred. Interactional patterns included typical class and small group discussions in addition to interaction with and on the site itself. Analytical framing in class also included semiotic resources affecting the perceived formality of pronominal use on the Web site, such as font color, size, and pronominal choice in embedded video clips.

Ware (Reference Ware2008) observed 20 ELL (English language learner) students over the course of a year in an intermediate ESL program at a technologically rich, urban American middle school to qualitatively capture their digital literacies during and after school. She found that the laptops students carried were used shockingly seldom for school activities, taking up about an hour a week in class time, and that those activities centered on PowerPoint presentations and word processing. Though she defends the complexities of navigating the Web for information retrieval and critique, we note that these activities do not reach the literacies of this century, which engage the interactive potential of the semantic Web. Nor are they in any sense ludic.

As a component of this research, Ware (Reference Ware2008) instituted an after school multimedia project with ELL students to explore digital storytelling, which, though time-consuming, was highly motivating to the students. The digital storytelling project engaged the students in a collaborative learning model for multimodal learning and presentation, but it did not incorporate their home languages. Ware suggested that multimodal activities can provide ELL students with visual and verbal alternatives in text creation and recommended positive potential for L2 learning in mode-switching activities, addressing what Jewitt (Reference Jewitt and Jewitt2009b) described as intersemiotic relationships in multimodality: “the interplay between modes” (p. 25). Ware (Reference Ware2008) offered examples of L2 learning opportunities embedded in mode-switching activities such as students translating textbook materials into comic strips or choosing endings in branched hyperlinked stories, but cautioned, “multimedia literacy practices certainly broaden the breadth of those experiences, but we still have little empirical evidence of the depth in which students develop their linguistic repertoire when moving across textual, visual, and aural modes” (p. 49).

Ludic Approaches

In the world of digital gaming, Zheng et al. (Reference Zheng, Wagner, Young and Brewer2009) studied artifacts from Chinese L2 learners’ engagement with American players in the virtual world, Quest Atlantis, to examine how their environmental interaction supported English language acquisition. The problem-solving nature of games coupled with the immersive environment of a virtual world creates the conditions for embodied interaction wherein players collaboratively problem solve. Though interaction in a virtual world is distinguished from real-world interactions in that avatars provide an anonymous shield, and their virtual behavior is ontologically regulated by the virtual world they inhabit, which might allow them to fly, for example, the game leaves a trace that learners can track and follow, read, critique, and learn from. Zheng et al. (Reference Zheng, Wagner, Young and Brewer2009) claimed that the embodied learning enabled by playing Quest Atlantis supported the L2 learners’ real-world English language development, by requiring them to “coordinate in-the-moment actions” (p. 489) using English.

Digital gaming encourages the learner by eradicating the fatality of the right-wrong, pass-fail assessment of static literacies (de Castell & Jenson, Reference de Castell and Jenson2003). The mechanical reboot-restart mechanism of digital games makes failing to make the grade or not finding the clue or, as an avatar, losing the battle or even getting killed, a certain kind of learning, motivating the player to retry, promoting critical evaluation. This could be viewed as a foundational building block of the New London Group's (1996) initial theorizing of the how of multiliteracies, as well as creative thinking and problem solving.

Virtual games provide a kind of identity building milieu. The player creates an avatar's superficial physical characteristics, and then learns to interact in the game ontology, acquiring and building the avatar's linguistic, sociocultural, and pragmatic competence. Cummins (Reference Cummins, Garcia, Skutnabb-Kangas and Torres-Guzmán2006) discussed the importance of supporting L2 learners’ positive, self-affirming identity building to facilitate learning a new language. In the Olifantsvlei fresh story project (Stein, Reference Stein, Jewitt and Kress2006, Reference Stein2008) that we review next, the aim was children's real-world identity building in postapartheid South Africa.

Stein's (Reference Stein, Jewitt and Kress2006, Reference Stein2008) Olifantsvlei fresh story project demonstrated the ludic principle in pedagogical development without the intervention of digital mediation. The storytelling project took place in the early part of the decade in a rural South African context of very high social and economic need and manifest multilingualism. This multimodal storytelling project welcomed young children's languages, which included both local and foreign African languages into a ludic space, despite the reservations of teachers in their English-medium school. Stories were created around homemade dolls that were constructed in a “semiotic chain of narrative” (Stein, Reference Stein2008, p. 119), as character representatives of the children's lifeworlds for the purpose of developing “a body of imaginative, local ‘fresh stories’ based on and arising from the children's lives and local experiences” (Stein, Reference Stein2008, p. 98).

Children developed homemade dolls that were repurposed for school after teachers’ attempts to create papier mâché dolls failed. In fact, the teachers’ artistic failure created an educational juncture for children's agency in creating dolls from local materials that, in themselves, told stories. The dolls were used in improvisational storytelling, and stories were invited in any language. The project extended children's semiotic resources for communicating in a playful, agency-building context that tapped their local knowledge, welcomed their input in multiple languages, and created opportunities for self-affirming identity building in postapartheid education.

One of the objectives of this project was to move from the controlled spaces of the classroom; rules for language use and ways of learning were rejected for more improvisational learning, inviting playful engagement with objects, and thereby with the meanings ascribed to those objects through language(s). Similarly, in the next example from our own work, we show how play can be harnessed to create provocative learning.

In Toronto, the teachers at Joyce Public School have been working with researchers at York University to codesign flexible, ludic (de Castell & Jenson, Reference de Castell and Jenson2003, Reference de Castell and Jenson2009), culturally sensitive projects in the primary and junior grades (K–5) that include children's home and community language networks and the digital literacies that are an ineluctable component of fundamental literacy in the 21st century. The ongoing collaborative community of practice has as its goal the creation of experimental multiliteracies pedagogies (Cope & Kalantzis, Reference Cope, Kalantzis, Cope and Kalantzis2000, Reference Cope and Kalantzis2009; New London Group, 1996) that create projects welcoming community languages, cross-curricular learning, multimodal expression, and digital play into the elementary language and literacy classroom (Lotherington, Reference Lotherington and Hansson2008, Reference Lotherington2009, Reference Lotherington2011). The agenda is approached from a grassroots perspective based on a supportive learning community comprising university researchers and elementary school teachers for the theoretical and practical advancement of project-based multimodal literacies education. Our orientation to work within a professional development platform circumvents criticisms researchers have made about multimodal literacies paradigms lacking adequate teacher preparation (Tan, Bopry, & Guo, Reference Tan, Bopry and Guo2010).

The Joyce project approached multimodal literacies in terms of

• • sharing responsibility for education beyond classroom walls to open pathways for language support and inclusion;

• • playful engagement with and development of digitally supported, multilingual projects that instantiate Courtney Cazden's (Reference Cazden1981) notion of “performance before competence” (p. 7);

• • supporting multilingualism in dynamic modal communication; and

• • ludic epistemology (de Castell & Jenson, Reference de Castell and Jenson2009).

In this work in particular, we sought not just the modalities of text and image, but the modality of play. As a modality, play can be activated and learned from and used to frame and develop ideas by playful engagement with digital media.

Imagine a World

This teacher-directed, student-supported project, called Imagine a World, was implemented in the fall of 2009. It was developed in support of a schoolwide initiative focused on developing student understandings of similarity and difference at all grade levels, and for the upper grades in the school, this generally fell under the theme of respect. For grade 4 and 5 students, this meant a yearlong, cross-subject inquiry that attempted to familiarize students with the range of differences present in their own schools and families as well as the world more generally. Topics covered included family structure, race, ethnicity, class, religion, immigration stories, languages, sexual orientation, gender (masculinities and femininities), and individual difference within those categories. To begin this yearlong inquiry, the four grade 4 and 5 teachers involved (including one special education teacher) decided that the best way to begin to talk about difference was to look at similarities; they used, across all their classes, a beautifully illustrated book by Mem Fox (Reference Fox1997), entitled Whoever You Are, that specifically addressed the question of what makes people similar.

After reading the book, the teachers asked students to fill in the phrase, “We are all. . . .” Students individually contributed up to three written responses. The second part of the activity involved the students refining the phrases and redeploying them to create a collagraph print, which uses glue and other materials (much like a collage) to create a three-dimensional (3D) image in a mirrored state, which is then transferred from plate to paper for printing. The next activity was to focus on difference in language, in contrast to the similarities that were being displayed in the images, and students and their parents were invited to rewrite the English phrases in their home languages. Finally, a multimedia, multilingual talking book including eight languages and a printed artifact was produced from the collaborative student, teacher, and parent work.

What is significant here in terms of modalities is that multiple languages, play, sound, text, and 3D image making (collagraph print) are intertwined. The hybrid intersection of these pieces created a multimodal, multilingual digital artifact that showed not only the powerful similarities of humanity (see Figure 1, which depicts the commonalities of human fertilization and the perceived opportunities in life) but also the richness of linguistic differences (both spoken and written) present in one small community and accessed by all 70 students, incorporating L2 students’ abilities as assets to the process, in print as well as narrative voice-over. The artifact represents a deep engagement not only with the phrases invoked in multiple languages but also in representing those complex ideas through sound and image, showing the complexity of abstract thought in creatively attempting to find similarities that cross cultures, languages, and generations.

Fig. 1.

A page from multimodal project, Imagine a World, by grade 4 and 5 students at Joyce Public School.

Hurdles in Multimodal Literacy Education

In 2002, Venezky and Davis, reporting on the transformation of schooling in the networked world, gave a generally failing grade:

We are now so socially enmeshed in digital literacy practices that the concept of optional extrication from the digital world is not realistic, yet language and literacy instruction continues to resist digitized multimedia and multimodal literacy practices as optional or secondary to flat textual practices. Pockets of resistance reside in the complexities of the educational system, in teachers’ socialization and professional expectations, and in assessment paradigms. Warschauer (Reference Warschauer2008) queried whether the educational establishment understands the point of language and literacy learning to prepare students for civic engagement, higher education, and employment possibilities—or to improve test scores.

Street et al. (Reference Street, Pahl, Rowsell and Jewitt2009) signaled that not including multimodal resources in the classroom undermines what children bring to the learning task by ignoring the modes children already use to make meaning. However, Dooley (Reference Dooley and Healey2008) pointed out that there are deep epistemological differences in ways of knowing English (as an L2) in digitally mediated popular culture worlds and in historically entrenched school literacies, noting that the former facilitate procedural literacies, invented identities, and performance epistemologies marked by spontaneity and hybridity, whereas the latter focus on propositional knowledge, archival identities, and reproducing social texts. Dooley suggested that migrating pop culture into the classroom is a colonizing act, if youth cultures are inserted into epistemologies of modern print. This is precisely what Lotherington, Neville-Verardi, and Sinitskaya Ronda (Reference Lotherington, Neville-Verardi, Sinitskaya Ronda, Abraham and Williams2009) showed with high school students, who, though scathing toward the mandatory provincial literacy tests they had written, were resistant to a revised version incorporating digital pop culture content.

In L2 education, there has been a tendency for researchers to focus on the digital or the multilingual, sidestepping the complexities of intersecting contemporary literacies in the classroom. Much research on multimodal literacies in the classroom still focuses on the dual modalities of text and image to the exclusion of the myriad modalities that construct learning and understanding in this digital age. Though good teaching and learning have resulted from multimodal pedagogies that do not incorporate digital mediation, avoiding the digital world is another thing altogether. One reason for avoidance of digital technologies in multimodal learning is evident in Tan and McWilliam's (Reference Tan and McWilliam2009) comparative description of the relative success of two educational initiatives to implement a digital literacies program in Queensland, Australia. One of these contexts was a public reception school specializing in English language instruction for newly arrived migrants and refugees of diverse cultural and linguistic origins and educational backgrounds. Tan and McWilliam described how well-meaning ESL teachers deferred to traditionally held basics in print literacy, adhering to an understanding of print as central to L2 learning and expressing a concern that students might be overloaded by digital demands. Meanwhile these students were observed to be actively using their personal iPods during school lessons to download and share music files. In fact, it was observed to be the teachers who were struggling to understand how they might incorporate technologies that were new to them—not to the students. This is unsurprising if one surveys core courses in teacher education and L2 certification programs, which, like educational institutions, have been slow in providing opportunities to learn about and explore new technologies for language teaching.

A brick wall facing many educators is the standardized testing culture that functions as a watchdog over flat literacy practices. Even in cases where there is political will to transform ELL education, the teachers themselves may manifest resistance. Tan et al. (Reference Tan, Bopry and Guo2010) ascribed the reluctance of a teacher in a Singaporean secondary school to move on from traditional ELL expectations to tensions between a transformative critical multiliteracies approach and the power of traditional assessment practices:

As Lankshear and Knobel (Reference Lankshear and Knobel2006) aptly put it, the point here is not that these are new forms of letteracy, nor is the central importance the construction of written texts, but in the everyday lives of many, the production and consumption of multimodal, digital artifacts. That means very different kinds of knowing—the difficulty, as McLuhan (1967/Reference McLuhan2001) foresaw, is figuring out what the nature of those differences are. The challenge in education policy and practice is that testing still typically tests the former, under conditions that are no longer simply print-based, monocultural, or first and foremost local.