2.1 Online teaching competencies

Synchronous online language learning is a semio-pedagogical activity that requires the teacher’s use of appropriate socio-affective, pedagogical, semiotic, and technological competencies to create and maintain optimal learning conditions to favour language skill development. Guichon (Reference Guichon2009) outlines these skills, describing socio-affective regulation as the ability to establish and maintain a relationship with learners. Pedagogical regulation requires expert knowledge of the target language, the ability to design learning scenarios adapted to online learning, and propose “clear and concise instructions, providing positive and negative feedback and deploying an array of strategies to facilitate second-language learning” (p. 169). Finally, semiotic and technological competencies relate to the online teacher adopting the different communication tools available in the technological environment that are appropriate to the learning scenario and his or her ability to manage the ensuing interactions using adequate semiotic resources in order to optimize learning potential and maintain learners’ attention (Ricci Bitti & Garotti, Reference Ricci Bitti, Garotti, Kappas and Krämer2011). This latter competency is crucial, as teaching via videoconferencing allows teachers to communicate through an array of modes, including spoken language, print, posture, gesture, head movement, gaze, and proxemics, and thus demands that teachers develop critical semiotic awareness (Guichon, Reference Guichon2013; Kern, Reference Kern2015) considered as “a high level of consciousness regarding all the information they are conveying when they interact with their learners” (Guichon & Wigham, Reference Guichon and Wigham2016: 67).

2.2 Online teaching and the use of print mode

One affordance of online platforms utilised for teaching synchronously is the availability of text chat. Text chat affords the potential to enhance teaching using the spoken mode with written language and visuals (Meskill & Anthony, Reference Meskill and Anthony2010). This potential enables the language teacher to capitalise on the multimodality of the teaching medium, for example, by offering feedback without interrupting the learners, encouraging learners who are unwilling to participate orally to contribute in writing, checking and confirming vocabulary upon communication breakdown, and to compensate for the absence of a blackboard (Hampel & Stickler, Reference Hampel and Stickler2005, Reference Hampel and Stickler2012; Wigham & Chanier, Reference Wigham and Chanier2015).

Investigating synchronous language teachers’ areas of attention on the screen in an audio-conferencing platform using heat maps and gaze plots recorded in an eye-tracking study, Shi, Stickler and Lloyd (Reference Shi, Stickler and Lloyd2017) categorised text chat as part of the technical areas of interest “where participants activate communication modes” (p. 217), and demonstrated teachers’ monitoring of the text chat box during all lesson stages (i.e. opening, teaching, and closing). Concerning videoconferencing (i.e. Skype-based English lessons), Kozar (Reference Kozar2016) studied how experienced online teachers employed text chat to especially serve pedagogical and organisational functions. Outlining both benefits and challenges, Kozar concluded that although text chat serves predominantly pedagogical and organisation functions, learners incorporated “a text message if: (a) it introduced new vocabulary, and (b) if it was produced bimodally (speaking and typing)” (p. 231).

The ability to employ the print mode is, thus, crucial for online teachers who should be “comfortable in using computer-mediated communication (CMC) technologies which include text, audio, and video conferencing” (Compton, Reference Compton2009: 83). This brief overview suggesting online teachers should skilfully utilise the print mode, particularly text chat, in synchronous teaching attests to the need to further explore modal configuration and use in interaction.

2.3 Delivering task instructions

The context for this study is task-based language teaching (TBLT) in multimodal synchronous online language teaching. TBLT encourages meaning-focused language use to achieve an outcome, and a task involves, first, some input for learners to process and use in their interactions, and, second, instructions explaining the outcome to be achieved through interaction, as well as how to achieve it (Ellis, Reference Ellis2000). Drawing on Breen (Reference Breen1987), although teachers can plan task instructions as part of their designed task (instructions-as-workplan), the actual instructions they deliver in interaction with the learners might differ (instructions-as-process). Investigating this perspective in which teachers give instructions verbally during face-to-face lessons, Markee (Reference Markee, Jenks and Seedhouse2015) identified six key instruction-giving fragments, including explanation of what resources learners will need during task accomplishment. In Satar and Wigham (Reference Satar and Wigham2020), we analysed experienced language teachers’ pedagogical instructions to build on Markee’s (Reference Markee, Jenks and Seedhouse2015) fragments. However, given our analytical lens, multimodal (inter)action analysis (Norris, Reference Norris2004), which scrutinises mediated higher- and lower-level actions, in our work Markee’s (Reference Markee, Jenks and Seedhouse2015) fragments constitute various higher-level actions the teachers engage in as part of their instruction-giving practices, while the modes they employ in doing so form the lower-level actions. Satar and Wigham (Reference Satar and Wigham2020) described that explanation of resources can present challenges in the online teaching context due to a lack of shared physical interactional space and artefacts (e.g. task handouts). Rather, teachers need to navigate, for example, sharing URL links to resource websites or shared online collaborative word processors via text chat, and the distribution of text documents to learners via email or text chat. Satar and Wigham (Reference Satar and Wigham2020) reported that the higher-level action labelled managing resources took about a quarter of the instruction-giving sequence in two of the three teachers’ lessons examined, and observed the following sub-categories: sending the resource, allocating the resource, receiving the resource, opening the resource, confirming access to the correct resource, describing the content of the resource, and reading the resource. This paper focuses on this higher-level action of managing resources.

Within the online language teaching literature, two studies to date have addressed instruction-giving practices, both focusing on trainee teachers. Comparing written instructions in an asynchronous teaching platform with verbal instructions in multimodal synchronous teaching, Cappellini and Combe (Reference Cappellini and Combe2017) reported that trainee teachers experienced unexpected challenges concerning managing the digital task resources, such as having to instruct the learners on how to download a document, especially in the synchronous platform. Satar and Wigham (Reference Satar and Wigham2017) addressed the multimodal resources trainee teachers employed while delivering task instructions, and demonstrated the use of text chat in introducing key vocabulary items in both spoken and written language (dual modality).

Given this background in online language teaching, we now explain the main concepts from multimodal (inter)action analysis that underpin our study.

2.4 Multimodal (inter)action analysis

Multimodal (inter)action analysis (Norris, Reference Norris2004, Reference Norris2019, Reference Norris2020) is “a holistic analytical framework that understands the multiple modes in (inter)action as all together building one system of communication” (Norris & Pirini, Reference Norris and Pirini2016: 24). Within this framework, all actions are considered interactions between social actors and other social actors, objects, or the environment. It offers a broad sociocultural perspective that is compatible with detailed microanalysis of interaction, allowing the researcher to address the macro-, intermediate, and microanalyses as required (Norris & Pirini, Reference Norris and Pirini2016). We now explain the key analytical tools of multimodal (inter)action analysis.

4.1 RQ1: How does the site of engagement demonstrate any intersection of different social practices?

A screenshot (Figure 1,Footnote Left) allows us to analyse the different social practices constructed through the higher-level actions at play within the site of engagement (see section 2.4.2).

Figure 1.

Left: site of engagement; Right: higher-level actionsFootnote ³

The site of engagement involves five social actors. Three actors (a volunteer teacher and two learners) responded to an advert to enlist participants for a research study (conducted by the other two actors: researchers) that comprises one large-scale action: partaking in a research project. This is evident in the print mode by the Skype avatar of one of the two principal investigators in the research project [a] and by a software icon in the toolbar for a screen-recording programme to capture the interaction data [b]. Although other social practices regarding data analysis are not visible (e.g. watching the videos, transcribing the mediated actions, writing up findings, etc.), Figure 1, Left, helps make sense of our part as researchers in relation to the data set as regards data collection.

The participants are engaged in a second large-scale action comprising participating in online language lessons via videoconferencing. This social practice is shown in the sidebar – the videoconferencing call is named “Online lessons with Craig” [c] – and through the presence of the teacher’s and learners’ webcam images [d].

The higher-level actions of completing language-learning tasks and also the higher-level actions of giving instructions for these tasks and of task accomplishment are evident in frozen actions entailed in the resources visible in the text chat window (i.e. in the disembodied print mode). They include task resource/instruction sheets [e] for both subtasks and lexical items offered by the teacher to learners as linguistic feedback during the accomplishment of the information exchange [f].

The site of engagement shows an intersection between the social practices of engaging in a research study and partaking in a teaching–learning activity constructed through the higher-level actions of participating in the second lesson, and particularly completing a convergent task (Figure 1, Right).

4.2 RQ2: What is the role of the print mode in achieving the higher-level action of instruction-giving?

We now examine the role of the print mode (text chat) in achieving the higher-level action of instruction-giving, and specifically managing resources, by examining Extract 1 (Figures 2 & 3) produced during the information-exchange activity (see section 3.1).

Figure 2.

Left: Extract 1, lines 1–3; Right: Extract 1, lines 4–7

In line 1 (Figure 2), Craig begins announcing the next task stage with a gaze shift downwards (#1 ^{Footnote 4} ). This is followed by Craig declaring what he will do next in the spoken language mode with emphasis on the word send (#2), with his gaze directed towards the learners, displaying an iconic gesture, and raised eyebrows acting as a beat gesture (for gesture types, see McNeill, Reference McNeill1992). As Craig tells the learners that he will send them some information, his gaze shifts towards the web browser, which displays an online newspaper in the current tab (#3).

In line 2, Craig begins to engage with two higher-level actions: sending the resource and allocating the resource to a specific learner. To allocate the resource, he first uses the learners’ name, a vocative in the spoken language mode. Yet Craig’s screen no longer portrays the webcam images of both learners. Rather, his focal point of attention, illustrated by his downward gaze (#4), is turned to a disembodied electronic resource in the print mode (an online document now displayed on the web browser, which includes the task information for student A, Eda).

In line 3, Craig highlights a URL in the resource sheet (#5), which he had generated and pasted into the document in preparation for the lesson. This document entails two frozen actions that are salient in this context, and which are the closest in time, place, and relevance to the current (inter)action: the researchers creating the resource sheet, and the teacher adding a URL to appropriate it for his own teaching practices. These frozen actions reinforce the site of engagement we described in section 4.1.

In line 4, Craig uses hesitation markers okay u:hm and downward gaze (#7) to signal an upcoming shift to the next instruction-giving fragment, confirming access to the correct resource while still sending the resource, as Craig places his mouse cursor on Didem’s webcam image (#6) to activate Skype and access text chat (#7). In line 5, the print mode continues to be embodied while Craig pastes the URL previously copied (line 3) from the resource sheet into the text chat (#8). In lines 5–6, Craig reinforces allocation of the resource to one learner, Eda, by stating that the other learner, Didem, should not look at the URL he has sent. In doing so, his attention/awareness is on the Skype window, his gaze shifts upwards to focus on Didem’s webcam image, his gestures become visible in his webcam frame, and he raises his eyebrows (#9). This information is repeated (line 6), and Craig’s gaze moves to focus on his webcam as he uses the pronoun “you” to address Didem (#10) before the URL is sent on text chat (#11). In line 7, we see Didem’s reaction: she covers her face with her hand and turns away from her screen (#12), demonstrating the success of this instruction-giving fragment.

In lines 8–12 (Figure 3, Left), Craig begins the higher-level action of sending the resource for the second learner, Didem. This requires a similar set of lower-level actions by embodying and disembodying the print mode (#13, #14, #16, and #17). Consequently, the simultaneous, multiple higher-level actions become frozen and available throughout the lesson.

Figure 3.

Left: Extract 1, lines 8–12; Right: Extract 1, lines 13–15

An interesting point to note in lines 3–9 (Figures 2 and 3) is Craig’s use of the demonstrative adjectives to refer to the URLs. In lines 3, 5, and 6 (#5, 6, 7, 8, 11), while selecting, copying, and pasting the first URL (embodied print mode), Craig refers to the link with the singular demonstrative adjective this. While embodying the print mode for the second URL, he again uses this (line 8) and adverb here (line 9); both signify closeness to the speaker in time and space. However, once the URL is sent, and thus disembodied (Figure 2, line 7, #12), Craig refers to it using the singular demonstrative adjective that, indicating distance from Craig in time and space.

Lines 13–15 (Figure 3, Right) demonstrate Craig’s critical semiotic awareness as he engages with the higher-level action of confirming access to the resource in the spoken language mode by checking whether learners can see the resources, and announcing his awareness of a potential semiotic lag (line 15). Semiotic lag concerns a time delay or lag between the multimodal communication of one social actor (e.g. a gesture unit, a gaze shift, and electronic document) and its reception by another whereby it no longer forms an aggregate due to delays in online transmission or technical issues (e.g. microphone not working). For instance, one actor may send a text chat message in the print mode while simultaneously referring to it in the spoken language mode. However, due to semiotic lag, the receiving actor may receive the written message at a different moment in interaction than its temporal position in the sending actor’s interactional space.

In line 14, Craig’s confirmation check receives an affirmative response from one learner, before Craig announces verbally a shift into the next higher-level action (here we go then), which concludes the higher-level action of sending the resource.

In Extract 1, Craig achieves the higher-level actions of sending and allocating an electronic resource, and confirming access to and describing the contents of it by actively employing the print mode (text chat) alongside other available modes: gaze, gestures, facial expressions, and spoken language. Gaze shifts appear to be the most salient modal change signalling to learners when Craig is engaged with the technical aspects of sending the resource by embodying the print mode (downward gaze direction in #1–5, 7, 8, 15, 16, 18), and when he is not while directly addressing the learners (upward gaze direction in #9, 10, 17, 19, 20).

We now examine use of another communication technology (online document/print mode) to create content during instructions for the second microtask (see Appendix in the supplementary material), illustrating how the higher-level actions are created and become frozen within the document.

Extract 2 (Figures 4 & 5) begins with Craig summarising the decision made by the learners to choose the cheaper package in the first microtask. Craig types the price of this package in text chat and, similar to Extract 1, his embodiment of the print mode is signalled to the learners via downward gaze shifts (#1 and #2). Summarising previous task stage becomes a frozen action in the text chat as key information for the next stage. Craig then proceeds to verbally communicate the instructions (line 2), contextualising the task for the learners, stating the task outcome (an email), giving key task information, and informing them how they will be working (together), before initiating the higher-level action of sending the resource fragment (line 3). In line 3, Craig’s gaze moves towards the web browser icon in the task bar at the bottom of his screen (#3), and once the pre-prepared online document is visible (#4), he proceeds to embody the resource in the print mode.

Figure 4.

Left: Extract 2, lines 1–3; Right: Extract 2, lines 4–6

In line 4, Craig opens the menu in the resource to share the online document (#5), selects and copies the URL (#6), pastes it in text chat (#7), sends the URL (#8), and as it becomes visible to the learners (#9), the URL and the online document becomes disembodied again. During this time, his utterances show his critical semiotic awareness regarding semiotic misalignment. We define semiotic misalignment as a difference in the semiotic meaning-making resources available to the social actors involved in the interaction that distorts the shared interactional space. It can be due to two factors. First, a difference in how each social actor views the interactional space where both/all actors are present and, potentially, differences in the semiotic resources available to them due to variations in hardware and software specifications and screen layout (e.g. the appearance and affordances of Skype on an Android phone as opposed to a Mac laptop or when Skype is open in full-screen mode as opposed to a minimised window). Second, semiotic misalignment can occur when a social actor uses or has access to a semiotic resource in his online or physical personal interaction space that cannot be seen by the other participant(s). For example, a participant may use or have access to a dictionary app/website on his or her screen or to paper notes that are not available to other participants. To bring these resources into the interaction, the social actor can verbalise/describe the resource or share it visually (e.g. by screensharing or bringing the resource within the webcam frame).

In line 4, Craig announces here we go but shows there is some semiotic misalignment between the semiotic resources to which the teacher and learners have access for the task. Indeed, he states I’ve got the e:r thing here all ready to go, demonstrating he is accessing a previously prepared disembodied resource to which learners do not have access in their interactional spaces. Similarly, in line 5, his choice of the future tense (all you’ve gotta do is click on it and you’ll see the page) shows his awareness of the fact that due to semiotic lag (see supra), the learners do not yet have access to the URL. In line 6, as the URL becomes visible in the shared text chat space, Craig’s gaze moves from the text chat towards the learners’ webcam images. This gaze shift allows him to check that the link has appeared in the text chat: the learners now have the possibility to embody the link and access the online collaborative writing space. The resource in the print mode thus becomes an integral part of the pedagogical interaction, as it is embodied and disembodied for the different participants. The next higher-level action of describing the content of the resource (line 6) allows the learners to check that, by clicking on the URL thus embodying it, they have accessed the correct resource and ensures coordinated action.

The last sequence of Extract 2 (Figure 5), begins with a shift to the next higher-level action marked by a posture change towards the webcam and a gaze shift downwards (line 7, #10) and towards the web browser icon (#11). As Craig clicks on this icon, the disembodied online document becomes, for him, embodied (#12). In line 9, Craig announces the next task stage (let’s have a little bit of a discussion about what we have to do) before moving to the higher-level action of summarising previous task stage. As he does so, he begins to type in the online document (#13). However, we notice that the learners have not yet embodied this resource. Only as Craig finishes the turn in line 9 (#14) does the avatar of one learner, Didem, appear in the top right-hand corner indicating that she is connected to the space. In line 10, Craig continues to summarise the previous task stage before stating the task outcome (write an email) and giving key task information. Throughout his utterance, he types these instructions into the online document. In doing so, he creates a record of the instructions to which learners can refer during the task, thereby creating frozen actions in which his instruction-giving sequence is entailed. As Craig completes this action (#15), there is joint attention on the written instructions by the learner, Didem, and Craig: the learner’s cursor is now placed at the beginning of the text Craig just wrote.

Figure 5.

Extract 2, lines 7–10

Having presented a turn-by-turn microanalysis of Extracts 1 and 2, we now analyse Craig’s modal configuration and density in selected representative lines from these extracts to illustrate simultaneous management of multiple higher-level actions with reference to the print mode.

4.3 RQ3: How does the teacher’s modal configuration and density vary when the print mode is utilised?

Variation in Craig’s modal configurations and density of his interactional space indicates his level of attention/awareness for various higher-level actions as he utilises the print mode. In this section, we examine how Craig simultaneously engages in multiple higher-level actions related to instruction-giving and evidenced in the actions performed in Extract 1, line 1 (Figure 6); Extract 1, lines 3–5 (Figure 7, Left); and Extract 2, line 10 (Figure 7, Right).

Figure 6.

Extract 1, line 1. Modal configurations

Figure 7.

Left: Extract 1, lines 3–5. Modal configurations; Right: Extract 2, line 10. Modal configurations

First, in Extract 1, line 1 (Figure 6), the higher-level action of announcing next task stage is foregrounded in Craig’s attention/awareness. This is mainly achieved through the spoken language mode, which is absolutely necessary to perform this action, and thus has high modal intensity. Therefore, spoken language appears to have the highest hierarchical prominence to perform this instruction-giving fragment. Second, Craig’s modal configuration varies throughout this spoken utterance. We observe highest modal density, achieved through modal complexity, in #2 as Craig produces the keyword send for this instruction-giving fragment, accompanied by the modal aggregate that comprises word stress in spoken language, iconic gesture, upwards gaze shift towards the screen (learners), posture shift towards the screen, and raised eyebrows, all of which carry medium modal intensity as they would impact the higher-level action were they discontinued. Finally, Craig’s other lower-level actions, object handling (moving the cursor over the browser icon) and attempt to access the print mode (the electronic resource), carry low modal intensity for the higher-level action of announcing next task stage: they are not visible to learners via the webcam (due to semiotic misalignment) and would not change the higher-level action if discontinued.

However, Craig uses three lower-level actions (gaze, object handling, print) in line 1 to perform another simultaneous higher-level action in the background of his attention/awareness: preparing for the next instruction-giving fragment (i.e. sending the resource). This is achieved using different modal aggregates throughout #1–3. While #1 shows Craig’s gaze and object handling, #2 involves object handling and print mode. However, #3 demonstrates the highest modal complexity for this higher-level action achieved through the modal aggregate gaze (towards the browser), object handling (cursor over the browser), print (webpage), and spoken language (information) as the higher-level action sending the resource moves to the foreground of his attention/awareness.

During Extract 1, lines 3–5 (Figure 7, Left), Craig embodies the print mode as he selects, copies, and pastes a URL for the learners to access the resource, thus performing the higher-level action of sending the resource (see section 4.2). For this action, although only signalled to the learners through gaze shifts, object handling and the print mode gain high modal density because the action cannot be achieved without these modes. On the other hand, spoken language (line 4) has low modal intensity as it functions as a filler, and would not change the action of sending the resource. In lines 3 and 4, the higher-level action sending the resource is thus foregrounded in Craig’s attention/awareness.

Spoken language in lines 3 and 5, however, carries high modal intensity for two other higher-level actions. While Craig sends the resource, he performs the actions of allocating the resource and confirming access to the correct resource, both which cannot be accomplished without spoken language. In line 3, his modal configuration involves object handling (moving the mouse/using the keyboard to select and copy the URL), embodied print mode (the electronic resource), and downward gaze. This modal aggregate, produced alongside spoken language, has low modal density to achieve the action of allocating the resource. In line 5, however, the modal aggregate that accompanies the spoken language has medium modal density: its removal might impact the higher-level action of confirming access to the correct resource. Image #8 shows Craig’s downward gaze and embodied print mode as he pastes the URL into text chat; the referent of the word this in the spoken language mode. Likewise, Craig’s upward gaze direction towards the learner, Didem, his gesture, and his raised eyebrows (facial expression, #9) reinforce the negative imperative in the spoken language (don’t look).

Finally, we examine Craig’s modal configuration in Extract 2, line 10 (Figure 7, Right), as he embodies the print mode by typing his instructions on the shared online document while simultaneously verbalising them. During this sequence, Craig performs three higher-level actions: summarising previous task stage, stating task outcome, and communicating key task information. Two modes carry high modal intensity here: print and spoken language. Once Craig types the instructions, the print mode becomes disembodied and continues to entail these three higher-level actions during task completion as frozen actions. Thus, by its temporal virtue of being available for a sustained period, the print mode carries high modal intensity in performing the higher-level actions in line 10. Spoken language also has high modal intensity because although it is a think-aloud sequence that ensures bimodal production of the typed instructions (indicated by Craig’s verbalisation of his spelling mistakes: for Anne e:r that wasn’t Anne fo:r Anne); it involves repetitions and vocabulary explanations (i.e. contribute), discontinuation of which would significantly change the action. Consequently, modal density achieved through high modal intensity of both the print mode and spoken language foregrounds these higher-level actions in all participants’ attention/awareness and becomes a joint focal point of attention.