2.1 Interaction-based research

SLA research is approached from many disciplinary directions. One area that has been central to an overall understanding of how second and foreign languages are learned is interaction-based research, finding its beginnings in L2 circles in the mid-1970s (see Ferguson Reference Ferguson and Hymes1971; Wagner-Gough & Hatch Reference Wagner–Gough and Hatch1975; Long Reference Long1980 and overviews by Gass & Mackey Reference Gass, Mackey, Bardovi-Harlig and Dörnyei2006, Reference Gass, Mackey, VanPatten and Williams2007; Mackey Reference Mackey2012; Mackey, Abbuhl & Gass Reference Mackey, Abbuhl, Gass, Gass and Mackey2012). Interactionist research is without a doubt at the forefront of L2 research, with numerous empirical studies (see Mackey Reference Mackey2007) investigating the role of interaction in language learning, books supporting (Gass Reference Gass1997; Mackey Reference Mackey2012) and critiquing (Block Reference Block2003) the interactionist approach, meta-analyses on various parts of the approach (Nicholas, Lightbown & Spada Reference Nicholas, Lightbown and Spada2001; Keck et al. Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbalaka, Norris and Ortega2006; Russell & Spada Reference Russell, Spada, Norris and Ortega2006; Mackey & Goo Reference Mackey, Goo and Mackey2007; Li Reference Li2010; Lyster & Saito Reference Lyster and Saito2010; Goo & Mackey Reference Goo and Mackey2013; Lyster, Saito & Sato Reference Lyster, Saito and Sato2013), and entries in prominent handbooks and encyclopedias.

Early research in this area was primarily descriptive and included detailed analyses of conversational structures involving native speakers (NS) and non-native speakers (NNS) with a focus on input to learners and interactions in which they engaged. The emergence of two constructs – output (e.g., Swain Reference Swain, Gass and Madden1985, Reference Swain, Cook and Seidlhofer1995, Reference Swain and Hinkel2005) and feedback – was central to a fuller understanding of the role of interaction in acquisition. Corrective feedback (see Loewen Reference Loewen, Gass and Mackey2012; Lyster et al. Reference Lyster, Saito and Sato2013 for an overview) is central to the two articles discussed in this paper. There are many ways that corrective feedback can be delivered, ranging from explicit feedback with or without metalinguistic information, as in example 1 taken from Goo (Reference Goo2012), to implicit feedback such as recasts, as in example 2 taken from Révész (Reference Révész2012).

1. 1. Example of metalinguistic feedback

1. NNS: Who does John think that is reading a newspaper?

2. NS: Don't use the conjunction ‘that’ when you ask about the subject of the subordinate clause. By the way, it's Bill. Okay, let's move to the next item.

1. 2. Example of recast

1. NNS: And I saw a boy next to the bar. I think he was with his girlfriend. They talking to each other.

2. NS: They were talking to each other.

The rephrasing by the NS in (2) with the correct form while at the same time maintaining the original meaning is a recast and is hypothesized to serve a corrective function, assuming that the NNS recognizes the difference between his/her original utterance (they talking to each other) and the subsequent utterance (they were talking to each other).

It is claimed that interactions themselves provide opportunities for learners to gain information about the language being learned, with corrective moves being an important part of that process and representing the point at which learners engage cognitive mechanisms that allow them to focus attention on language form and make crucial connections between form and meaning. In other words, corrective feedback draws attention to form and provides opportunities for learners to notice the differences between what they have uttered and what a NS might say.

A question arises as to what might enable some learners to make the crucial comparison between their own utterances and their interlocutors’ utterances. Why are some individuals better than others? One such difference is individual working memory capacity.

2.2 Working memory capacity

Working memory, it is argued, has two parts: it refers to the structures and processes that humans use to store and manipulate information (for an overview see Williams Reference Williams, Gass and Mackey2012). What is important for the purposes of this paper is that working memory capacity varies from individual to individual and if there is individual variability in this capacity, there might also be individual capacity to benefit from interactional feedback.

There is no single way of measuring working memory capacity. Rather, a number of elicitation measures have been used, the majority favouring a dual-task format combining some form of memory measure with a processing measure. The following gives a brief description of each of the four measures referred to in the two papers discussed in section 3.

1. 1. Phonological Short Term Memory (PSTM)

1. a. Digit span (DS). Participants hear a series of random numbers and are asked to repeat these numbers in the order in which they were presented.

2. b. Non-word span (NWS). Participants hear non-words (with a range of syllables – in the Révész study) and are asked to repeat each non-word.

1. 2. Reading span. Participants read sentences one at a time and are told to remember the last word of each sentence (or a letter at the end of the sentence). They are asked to determine whether the sentence is plausible or not (to ascertain that they have actually processed the sentence). At the end of a set of sentences (usually 2–6), they are asked to write down (in order) the last word of each sentence (or the letters that appeared after each sentence).

2. 3. Operation span. This is similar to the reading span, but instead of sentences, mathematical operations are used ([2 × 3) + 1 = ?]). After each operation, participants are shown a possible response and they must indicate whether the answer is correct. This is then followed by a letter. At the end of each set, participants are asked to repeat the letters of that set in correct order.

Recently, within the SLA literature, studies have emerged that link working memory capacity and interaction (Mackey et al. Reference Mackey, Philp, Egi, Fujii, Tatsumi and Robinson2002; Mackey & Sachs Reference Mackey and Sachs2012; Gass, Behney & Uzum Reference Gass, Behney, Uzum, Droździał-Szelest and Pawlak2013). It is the combined significance of these two topics that led us to consider the importance of replication of studies in this area.

In the following section we suggest two recent studies that represent current thinking within the interactionist paradigm. They both consider interaction as a form of corrective feedback and examine an individual trait, that of working memory capacity. With new directions highlighting individual differences and particularly working memory capacity, it is time for replication before we move too far down the path that one-off studies take us.

3.1 Study 1: Révész (Reference Révész2012)

This study, published in Language Learning, investigated how the effectiveness of recasts is influenced by working memory and specifically by two types of working memory measures (phonological short-term memory and complex working memory tasks [span tasks]). A large number of studies have supported the role of recasts in facilitating L2 development (see Long Reference Long2007 and Mackey & Goo Reference Mackey, Goo and Mackey2007 for a review; Goo & Mackey Reference Goo and Mackey2013), but the nature of the types of L2 learning processes they promote (inter alia declarative or procedural) is still controversial (e.g., Ellis, Loewen & Erlam Reference Ellis, Loewen and Erlam2006; Révész & Han Reference Révész and Han2006; Ellis Reference Ellis and Mackey2007; Loewen & Nabei Reference Loewen, Nabei and Mackey2007) and depend upon the different types of outcome measures employed. Moreover, even though a body of research suggests that ‘individual differences in working memory [. . .] are likely to be important determinants of how much is learned from recasts’ (Révész Reference Révész2012: 99), results from empirical studies are not conclusive. Révész’ study aimed to (1) provide empirical evidence about the effect of the outcome measures on the effectiveness of recasts and (2) to demonstrate the mediating role of working memory in the observed results on various outcome measures.

Participants in this study were 90 English as a foreign language (EFL) Hungarian first language (L1) high-school students in beginner-level language classes. Participants were selected based on their developmental stage on the target structure (English past progressive) assessed by a pretest, and were randomly assigned to one of three groups: recast (n = 36), non-recast (n = 36), or control (n = 18). The design consisted of a pretest, treatment, a posttest and a delayed posttest. The pretest was made up of an untimed grammaticality judgment task (GJT), a written description task, and an oral description task. For the GJT, participants were asked to judge the grammaticality of 36 sentences (target and fillers) and correct those sentences they believed to be ungrammatical. For the written description task, participants were asked to describe what each of eight people was doing in a picture with the time contextualized as being in the past. Finally, there was an oral description task in which participants had to describe photographs.

Treatment consisted of three 15-minute recast (or non-recast) sessions. Participants were told they were taking photographs at a time when a bank robbery took place. The task was to describe the photographs telling the researcher (whose role was that of a police officer) the activities in the photograph so that the researcher would know what everyone in the photo was doing at the time of the robbery. In the recast group, recasts were provided when there was a past progressive error (and randomly for other errors). In the non-recast group, no feedback was given.

PSTM was assessed by means of a digit span (DS) and a non-word span (NWS) test and complex working memory capacity was measured with a reading span (RS) test. Measurement of PSTM was by means of repetition of Hungarian numbers. To determine an individual's NWS, non-words with Hungarian phonotactics were presented to participants following which they had to repeat the sequences. An RS score was obtained via a test of Hungarian consisting of 60 sentences, presented one by one on index cards. Participants read each sentence aloud in sets and subsets after which they were asked to recall the last word of each sentence. This was followed by a question about one of the sentences, but not all (sentences were randomly selected for questioning) so as to make sure participants were focusing on meaning and not just remembering the last words.

Data were analyzed through three many-facet Rasch measurement analyses: pretest-posttest and pretest-delayed posttest gain scores on the (1) GJT, (2) written task, and (3) oral description task. Correlations were run separately for recast and non-recast groups. The correlations were between each of the WM measures and the pretest-posttest gain scores from the three tasks (GJT, written and oral description). Many-facet Rasch measurement and correlation analyses showed that oral recasts had the greatest gains on oral production tests, lesser gains on written production tests, and the least on untimed grammaticality judgment tests. According to these findings, the researcher suggested that recasts are ‘more likely to foster development in procedural knowledge.’ (p. 124). On the basis of the strong correlation between high PSTM scores and oral test scores on the one hand and high complex working memory capacity scores and writing test scores on the other, the researcher suggested that ‘learners engaged in different types of learning processes as a function of their complex WM and PSTM capacities.’ (p.124). Additionally, she argued that the types of outcome measures differently favored the deployment of L2 knowledge depending on learners’ complex WM or PSTM capacities.

3.2 Approaches to replication

A first question to be raised through replication is a methodological one. One would want to know, for example, if other tests of working memory or other means of delivery of the input sentences (in this study sentences were written on a card) would yield different or similar results. An important issue when assessing working memory is the need to ensure processing along with storage. When only random sentences were measured for processing rather than all sentences, one is left with an incomplete picture. The issue of scoring is also relevant. In some working memory studies, all sentences with incorrect plausibility judgments following the presentation of the sentence are eliminated. In the current study, that was not possible because not all sentences received plausibility judgments. Thus, a replication of this study (with computer delivery) would include all sentences tested and would allow one to ensure both processing and storage.

A second area of replication has to do with different results based on differing working memory tests. A replication with extended data analysis is one way to sort out these differences. One of the explanations Révész provides for differences in results depends on the working memory measure used, namely, high RS individuals gained more on written tasks, whereas those with high PSTM improved to a greater extent on oral tasks. She suggests that these differences have to do with learner engagement. To further support this notion, one could include measures of responses to recasts as a way of determining engagement with the recasted form. Given the role of feedback and responses to feedback, this is an important area for further investigation, as Révész herself points out. To do this, a researcher would create a coding method that would provide information on the responses to each recast, for example: (1) no opportunity, (2) opportunity, but did not repeat, (3) repeated the recasted form, (4) negotiated the response, or (5) used the recasted form later in the discourse, with (1) being low engagement and (5) greatest engagement. This would allow an examination of engagement through the type of response to recast and in relation to different working memory tests.

Another method to determine the degree of engagement with the recast could result from a (partial) conceptual replication adding stimulated recall to the experimental design. Stimulated recall is an important technique (Gass & Mackey Reference Gass and Mackey2000) used to elicit ‘data about thought processes involved in carrying out a task or activity’ (p. 1). As Bloom (Reference Bloom and French1954: 25) notes: ‘A subject may be enabled to relive an original situation with great vividness and accuracy if he is presented with a large number of the cues or stimuli which occurred during the original situation.’ Because some of Révész’ arguments rely on the notion of engagement and transfer-appropriate processing which states that individuals can better ‘remember what [they] have learned if the cognitive processes that are active during learning are similar to those that are active during rehearsal’ (Lightbown Reference Lightbown and Han2007: 27), probing individuals’ thought processes would be an important way of accessing that information. To do this, the original interaction would need to be taped (video is best) with the tape serving as the stimulus upon which the participant comments (adhering to the recommendations in Gass & Mackey regarding time lapse and other stimulus preparation). In this case, engagement would be operationalized as comments made about the recast. We make this suggestion in full recognition that absence of mention may not signify lack of noticing or engagement. Nonetheless, it could add information that goes beyond speculation and which could be triangulated with other data, such as the response categories mentioned above.

Generalizability of the results in this study depends on the extent to which the results can be extended to different types of individuals (e.g., age groups), different grammatical constructions, different proficiency levels, different L1-L2 relationships, and different contexts (classroom versus non-classroom). It is appropriate and important to consider learners other than high school age participants in order to determine if the results are unique to a particular age group or if they are generalizable to a wider range of ages. How do working memory measures relate to languages that are closely related (e.g., Italian and Spanish)? As Révész discussed in the article, it is important to bring in the construct of grammatical complexity and/or transparency. The past progressive is not complex and may be particularly susceptible to learning from recasts; complex structures, as is the case with that-trace, may not be. Therefore, one could conduct an approximate replication keeping the methodology the same, but varying the grammatical structures. This would allow a researcher to determine the potential generalizability of the original study through an understanding of the role of grammatical complexity.

We turn next to a study by Goo (Reference Goo2012), who dealt with some of the same variables, namely, working memory capacity and feedback.

3.3 Study 2: Goo (Reference Goo2012)

The topic of this paper is similar to the one by Révész discussed above. Goo's article, published in Studies in Second Language Acquisition, compares the benefits of two types of corrective feedback, recasts and metalinguistic feedback, on the acquisition of that-trace filter (*Who does John think that is reading a newspaper? versus Who does John think is reading a newspaper?). Goo specifically considered the mediating role of working memory capacity on these feedback types. Empirical evidence has been provided on the facilitative role of recasts in L2 development, but findings also suggest that other types of feedback, such as metalinguistic feedback, are more effective than recasts. Recasts and metalinguistic feedback differ, inter alia, in the level of implicitness or explicitness of the given feedback and therefore on the attention-control processes involved. Individual differences in working memory capacity can play a crucial role in the effectiveness of corrective feedback in interaction, but research (Mackey et al. Reference Mackey, Philp, Egi, Fujii, Tatsumi and Robinson2002, Reference Mackey, Adams, Stafford and Winke2010; Sagarra Reference Sagarra and Han2007; Trofimovich, Ammar & Gatbonton Reference Trofimovich, Ammar, Gatbonton and Mackey2007; Gass et al. Reference Gass, Behney, Uzum, Droździał-Szelest and Pawlak2013) yields mixed findings. Goo's study aimed to further the understanding of the relative effects of recasts and metalinguistic feedback and the potential role of individual differences in working memory capacity on their effectiveness.

Fifty-four EFL Korean L1 university students at intermediate proficiency level were randomly assigned to a recast (n = 14), metalinguistic feedback (n = 32) or control group (n = 8; this group participated in pretest-posttest working memory tests only). In a pretest-posttest design experiment, participants attended two 20-minute sessions during which the target construction was elicited with one-way information-gap activities and corrected respectively with full recasts or metalinguistic explanations. This was done as an in-class activity. A picture was projected onto a screen and each participant was given a list of 15 sentences (ten of which were the target) with some blanks. Students had to ask the teacher a question about the missing item. To avoid issues of modified output, the teacher did not allow participants the opportunity to respond to his/her comments. There was a second treatment session with the researcher three days after the first treatment; this was apparently done as a whole-group activity. Working memory capacity was assessed in the L1 with two span tasks: reading span (RS) task and operation span (OS). The pretest and posttest comprised a GJT and a written production test. In the GJT (computer administered), there were 24 sentence pairs for each of the two tests (eight of which were fillers). Each pair consisted of one declarative sentence and a question asking about an underlined part of that sentence (e.g., They think that Jeff loved Jennifer. Who do they think that loved Jennifer?); the task was to judge the question as grammatical or not. For the written production test, there were 15 critical (declarative) sentences (with one word underlined) for the pretest and for the posttest. The task was to write a question about the underlined word (e.g., They believe that John is working as a waiter at a Korean restaurant; They think that Mike designed Jane's jacket).

Findings showed similar beneficial effects of recasts and metalinguistic feedback, in a blocked modified output situation, in facilitating the acquisition of the target construction. With regard to the effect of working memory capacity, the significant correlation between working memory capacity and recasts suggested that working memory capacity mediates and predicts the effectiveness of recasts, but no such significant correlation existed for metalinguistic feedback, suggesting the lack of impact of working memory capacity on metalinguistic feedback. The researcher argued that the two types of feedback ‘reflect two different mechanisms for how noticing occurs’ (p. 465), as recasts necessitate attention-control processes (essential elements of working memory capacity) which are not particularly required for the noticing of metalinguistic feedback given their highly overt nature.

3.4 Approaches to replication

We turn now to possibilities for replication of the Goo study. The finding that individual differences in working memory capacity influence the effectiveness of recasts, but not of metalinguistic feedback, is particularly relevant with regard to its implications for the cognitive processes involved in the two corrective feedback procedures and how they can be beneficial in different situations and for different students. Yet, we are left with unanswered questions, some of which relate to a consideration of both of these studies together.

Because of the differences in population (age and L1) and context in the two studies reported on here, it is difficult to assess the impact of working memory as a general conclusion. This is made more difficult because the size of the recast group in Goo's study was relatively small (n = 14). Other modifications to Goo's study would be important. For example, the target structure used is a complex one and undoubtedly is not one that these learners use in everyday conversation. Further, it is not a structure that one would be likely to receive feedback on given its comprehensibility with or without the use of that. One of the criteria for inclusion in Goo's study was the rarity of this construction in natural discourse. This structure ‘. . .was assumed, to some extent, to minimize impacts of potentially confounding factors (e.g., prior learning or exposure)’ (p. 453). A replication study would be particularly useful to see if this confined usefulness of feedback can be generalized. Thus, a replication study could include a wider range of structures, or, at least structures that are more likely to receive feedback in a context that extends beyond the laboratory.

As with the Révész study, one would want to understand better what learners were thinking about as they received feedback. In the recast group, for example, did they notice the recast? Replicating the study with that component would yield a more nuanced understanding of the role of working memory. This could be accomplished through a stimulated recall procedure in which each student would be given a stimulus (the video of the task) and asked questions about what they were thinking about with each instance of feedback. What could be further explored in this way is the role of direct feedback (to the individual asking the wh- question) versus observed feedback (to a classmate).

In Goo's study, there was no delayed posttest, leaving one to wonder what the true impact is. Such an inclusion in a replication study would allow a more comfortable generalization regarding claims about learning rather than to just the immediate impact of the treatment. Finally, as we understand the treatment, it was done as a class and not individually. Because it is important for learners to notice feedback in order for that feedback to have the greatest impact and because a face-to-face interaction is more likely to lead to noticing (see recent overview by Mackey et al. Reference Mackey, Abbuhl, Gass, Gass and Mackey2012), a replication of the same study in a one-to-one setting might provide information that would help the field determine the extent of impact. A second possibility was mentioned above, namely, a replication in which introspective comments through a stimulated recall could potentially separate feedback to an individual and feedback to another individual.

Finally, since we know that scoring procedures (see Leeser & Sunderman, ms) result in different conclusions, one would want to analyze a set of data with different scoring procedures to determine the stability of the results. The interested reader is referred to Conway et al. Reference Conway, Kane, Bunting, Hambrick, Wilhelm and Engle2005 for alternative scoring methods in a possible replication study.