The interaction approach to second language (L2) acquisition suggests that conversational adjustments occurring during communication benefit L2 development by providing learners with opportunities to receive modified comprehensible input and interactional feedback; notice gaps between their interlanguage and the target language features; and produce output (Gass & Mackey, Reference Gass and Mackey2006; Long, Reference Long, Ritchie and Bhatia1996). Although interaction in face-to-face (FTF) contexts is largely accepted as beneficial to L2 development (Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Mackey & Goo, Reference Mackey, Goo and Mackey2007), the relationship between interaction and learner outcomes within computer-mediated environments is less clear. To determine whether interaction is similarly effective at promoting L2 development in computer-assisted language learning (CALL) and FTF contexts, research has sought to establish the ability of computer-mediated communication (CMC) to direct learners’ attention to specific target language features and to provide learners with opportunities to negotiate for meaning, receive comprehensible input and corrective feedback, and produce modified output (Smith, Reference Smith2004).
The growing body of research examining CMC and interaction suggests positive benefits within technology-supported environments (e.g. Blake, Reference Blake2000; Lai & Li, Reference Lai and Li2011; Sauro, Reference Sauro2011; Smith, Reference Smith2005), with some studies indicating advantages within computer-mediated contexts over FTF contexts. For instance, research has demonstrated that text-based synchronous computer-mediated communication (SCMC), which refers to real-time interaction such as a live chat, may result in increased and more equitable learner participation (Beauvois, Reference Beauvois1992; Kelm, Reference Kelm1992; Warschauer, Reference Warschauer1996a), as well as greater quality of language production (Chun, Reference Chun1994; Kern, Reference Kern1995) than in FTF interaction. In addition, SCMC may result in greater amounts of learner output (Chun, Reference Chun1994; Kelm, Reference Kelm1992) and may also provide learners with a less stressful environment than FTF interaction (Chun, Reference Chun1998), potentially leading to reduced anxiety and improved willingness to communicate. SCMC may also promote increased participation by less active students when compared with more active, dominating students (Chun, Reference Chun1994; Kern, Reference Kern1995; Warschauer, Reference Warschauer and Warschauer1996b) or encourage students’ risk-taking and hypothesis testing regarding language (Meunier, Reference Meunier and Muyskens1998).
Although research has indicated promising results regarding the effectiveness of interaction in facilitating L2 development in computer-mediated contexts, studies vary along a variety of dimensions, including oral and written modalities, settings, and learner and interlocutor characteristics. In addition, as the field of CALL continues to grow and become more theoretically and methodologically sophisticated, there is an increasing number of studies examining computer-supported contexts from an interactionist perspective, and with the increasing use of technology in the foreign-language and L2 classroom, there is a growing need to better understand the pedagogical implications of interaction within a computer-mediated context. A comprehensive synthesis and meta-analysis will allow researchers and educators to gain important insights into the effects of interaction on learners’ L2 development in computer-assisted environments.
Since Norris and Ortega’s (Reference Norris and Ortega2000) influential meta-analysis on the effectiveness of L2 instruction, the use of synthetic research has grown, demonstrated by the increasing number of analyses synthesizing findings over a wide range of topics (e.g., Goldschnieder & DeKeyser, Reference Goldschneider and DeKeyser2005; Masgoret & Gardner, Reference Masgoret and Gardner2003; Shintani, Li, & Ellis, Reference Shintani, Li and Ellis2013). Although a number of quantitative (Li, Reference Li2010; Lin, Huang, & Liou, Reference Lin, Huang and Liou2013; Mackey & Goo, Reference Mackey, Goo and Mackey2007) and qualitative (Lai & Li, Reference Lai and Li2011; Sauro, Reference Sauro2011) syntheses have directly and indirectly examined the relationship between interaction, technology, and L2 acquisition, no study to date has provided a direct and comprehensive comparison of interaction in SCMC and FTF contexts on L2 learning outcomes. In response to the growing body of work investigating L2 development in SCMC from an interactionist perspective as well as the increasing use of technology in educational contexts, the current study aims to provide a more thorough and comprehensive meta-analytic assessment of the relative effectiveness of interaction in SCMC and FTF contexts on L2 learning outcomes.
BACKGROUND
The Interaction Approach to L2 Acquisition
Grounded in the notion that communication is the driving force behind language learning, the interaction approach posits that exposure to linguistic information (as input), opportunities for language production (as output), and positive and negative feedback on learners’ production (through interaction) are constructs critical to understanding L2 learning processes (Gass & Mackey, Reference Gass and Mackey2006). As summarized by Mackey (Reference Mackey2012), it is the ways in which learners interact with input and their interlocutors through the interactional processes of negotiating for meaning, giving and receiving corrective feedback, and producing modified output that are likely to lead to L2 development.
Early research suggested that negotiation of meaning may lead to increased opportunities for input (Ellis, Tanaka, & Yamazaki, Reference Ellis, Tanaka and Yamazaki1994) as well as positive effects on learners’ development, with findings indicating that modified input was significantly related to task performance, immediate gains in comprehension, and delayed effects on production (Gass & Varonis, Reference Gass and Varonis1994). Since these seminal studies first demonstrated the potential benefits of interaction for L2 learning, the interaction approach has inspired a vast body of research, with studies examining the overall impact of interactional feedback on the development of adult (e.g., Mackey & Sachs, Reference Mackey and Sachs2012; Mackey, Oliver, & Leeman, Reference Mackey, Oliver and Leeman2003; McDonough, Reference McDonough2005) and child learners (e.g., Oliver & Mackey, Reference Oliver and Mackey2003; Mackey & Silver, Reference Mackey and Silver2005), the effects of recasts on L2 learning (e.g., Loewen & Philp, Reference Loewen and Philp2006; Philp, Reference Philp2003), the differential efficacy of different types of feedback (e.g., Ellis, Reference Ellis and Mackey2007; Sheen, Reference Sheen and Mackey2007; Yilmaz, Reference Yilmaz2012), negotiation (e.g., de la Fuente, Reference de la Fuente2003; Iwashita, Reference Iwashita2003; Peterson, Reference Peterson2006), and learners’ perceptions of interaction (e.g., Carpenter, Jeon, MacGregor, & Mackey, Reference Carpenter, Jeon, MacGregor and Mackey2006; Mackey, Reference Mackey2002; Mackey, Gass, & McDonough, Reference Mackey, Gass and McDonough2000). In general, numerous empirical studies and meta-analyses (Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Li, Reference Li2010; Lyster & Saito, Reference Lyster and Saito2010; Mackey & Goo, Reference Mackey, Goo and Mackey2007; Russell & Spada, Reference Russell, Spada, Norris and Ortega2006) have indicated positive benefits for interaction and interactional features on learners’ development. During the past few decades, scholars have continued to extend the boundaries of the interaction approach by examining the effects of interaction in diverse contexts, with a growing body of empirical research conducted in computer-assisted environments.
Interaction and Computer-Mediated Communication
During the past few decades, CALL research has sought to situate itself more firmly within the theoretical foundations of a variety of SLA approaches, with the interactionist perspective serving as the predominant approach in current SLA-oriented CALL research (Chapelle, Reference Chapelle, Egbert and Petrie2005). Although there is a large body of research examining the role of interactional features in computer-mediated contexts (e.g., Smith, Reference Smith2004, Reference Smith2005), investigations have also sought to make direct comparisons between interaction in FTF and SCMC, demonstrating the existence of interactional similarities between the two modes (Fernández-Garcia & Martínez-Arbelaiz, Reference Fernández-Garcia and Martínez-Arbelaiz2002; Lai & Zhao, Reference Lai and Zhao2006). Comparative studies between SCMC and FTF environments have provided encouraging evidence for the use of technology in the classroom, with studies suggesting that text-based SCMC provides increased saliency (Smith, Reference Smith2004), extended opportunities for learners to notice target language features (Smith, Reference Smith2003; Toyoda & Harrison, Reference Toyoda and Harrison2002), and the opportunity to reread and review previous text, thereby allowing learners to more closely focus on target forms while not impacting the communicative success of the interaction (Chapelle, Reference Chapelle2001). In addition, the added time available to learners may provide more processing time, potentially leading to improved comprehension, more accurate production, and a higher quality of interaction than might have occurred in a FTF context (Pellettieri, Reference Pellettieri1999).
This growing body of research demonstrating that interactional features beneficial to L2 learning can and do occur in SCMC (e.g., Blake, Reference Blake2000; Petersen, Reference Petersen2010; Smith, Reference Smith2003; Toyoda & Harrison, Reference Toyoda and Harrison2002; Tudini, Reference Tudini2003) has inspired three recent syntheses (Lai & Li, Reference Lai and Li2011; Lin et al., Reference Lin, Huang and Liou2013; Sauro, Reference Sauro2011) examining the relationship between interaction and L2 acquisition. Sauro (Reference Sauro2011) examined the relationship between SCMC and learners’ grammatical, sociolinguistic, strategic, and discourse competences, finding that fewer than half of the included studies examining grammatical competence demonstrated developmental benefits. Although Sauro discusses primary research focused on interaction—specifically negotiation and feedback—the role of these interactional features is not examined in a quantitative, systematic manner, providing only limited evidence for the efficacy of interaction in SCMC across multiple empirical studies.
Taking a similarly qualitative approach to the primary literature, Lai and Li’s (Reference Lai and Li2011) review of research at the intersection of task-based learning and teaching (TBLT) and technology examined the benefits and challenges to integrating technology into the task-based classroom, finding that although a number of empirical studies provide encouragement for the use of computer-mediated learning, there are substantial differences between TBLT in FTF and technology-mediated environments, with some findings demonstrating greater performance in FTF interaction over computer-mediated environments (e.g., Baralt, Reference Baralt2010; de la Fuente, Reference de la Fuente2003), suggesting that more research is needed to understand the relative efficacy of interaction in these two communicative contexts. Considering the lack of conclusive, generalizable findings, there is a clear need for a statistically sound, quantitative review of the effects of interaction on learners’ L2 development within computer-mediated environments.
Moderating Factors in Interaction Research
Type of Target Feature
Research in interaction has demonstrated that interactional feedback may be more or less effective depending on the type of target feature, with findings suggesting that feedback on lexical or phonological errors may be more noticeable than feedback on errors in morphosyntax (Carpenter et al., Reference Carpenter, Jeon, MacGregor and Mackey2006; Ellis, Basturkmen, & Loewen, Reference Ellis, Basturkmen and Loewen2001; Long, Inagaki, & Ortega, Reference Long, Inagaki and Ortega1998; Mackey et al., Reference Mackey, Gass and McDonough2000; Yang & Lyster, Reference Yang and Lyster2010). In their meta-analysis, Mackey and Goo (Reference Mackey, Goo and Mackey2007) found that interactional benefits were greater for lexical items when compared to grammatical items. Similarly, research in SCMC has demonstrated that the type of target feature may play a role in the efficacy of the interaction, with findings demonstrating that lexical items seem to lead to more instances of negotiation than grammatical items (Blake, Reference Blake2000; Fernández-Garcia & Martínez-Arbelaiz, Reference Fernández-Garcia and Martínez-Arbelaiz2002; Smith, Reference Smith2004; Toyoda & Harrison, Reference Toyoda and Harrison2002).
Type of Interlocutor
Previous research has shown that nonnative speaker-nonnative speaker dyads may use more interactional features (Gass & Varonis, Reference Gass and Varonis1994; Long & Porter, Reference Long and Porter1985; Pica, Lincoln-Porter, Paninos, & Linnell, Reference Pica, Lincoln-Porter, Paninos and Linnell1996; Varonis & Gass, Reference Varonis and Gass1985) and provide more modified output opportunities (Sato & Lyster, Reference Sato, Lyster and Mackey2007) when compared to native speaker-nonnative speaker dyads. However, studies have also demonstrated that native speaker interlocutors may provide more feedback (Mackey et al., Reference Mackey, Oliver and Leeman2003; Oliver, Reference Oliver2002), suggesting that there are differences in learners’ production and development depending on interlocutor type, highlighting the need for an empirical comparison across type of interlocutor.
Research Setting
Although some researchers have proposed that the quantity and quality of interactional feedback in classrooms may be highly variable and may be task or context-dependent (Foster, Reference Foster1998), recent meta-analyses (Mackey & Goo, Reference Mackey, Goo and Mackey2007; Russell & Spada, Reference Russell, Spada, Norris and Ortega2006) have indicated that interaction is facilitative of L2 development in both classroom and lab settings. Although much of the research on SCMC has been conducted within a laboratory setting, a growing number of studies within CALL have been conducted in a classroom context (Felix, Reference Felix2008), highlighting the need for investigation into the overall effects of interaction on L2 development in different SCMC research settings.
Educational Context
Potential constraints to learner-learner interactions have been pointed out specifically with respect to English as a foreign-language contexts (Mayo & Pica, Reference Mayo and Pica2000; McDonough, Reference McDonough2004; Philp & Tognini, Reference Philp and Tognini2009), with previous research suggesting that learners in a variety of contexts may produce low rates of interactional feedback, due to cultural preferences (Burrows, Reference Burrows2008; Fujii & Mackey, Reference Fujii and Mackey2009) or other affective factors, such as perceptions of proficiency differences or learner attitude (Philp, Walter, & Basturkmen, Reference Philp, Walter and Basturkmen2010; Watanabe & Swain, Reference Watanabe and Swain2007; Yoshida, Reference Yoshida2008). Few studies in CALL research have directly compared L2 and foreign-language settings, highlighting the need for further investigations into the effects of interaction in diverse educational settings.
Meta-Analyses Related to the Effectiveness of Interaction in L2 Acquisition
Research syntheses, through the systematic review of previous studies, offer researchers the opportunity to compare and combine results across numerous primary studies, to conclusively answer specific research questions and identify gaps in research agendas and methodologies (Norris & Ortega, Reference Norris and Ortega2006). As Lipsey and Wilson (Reference Lipsey and Wilson2001) point out, meta-analysis provides a more sophisticated assessment of key study findings than other synthetic approaches, such as vote counting based on statistically significant results or qualitative syntheses, because it is able to account for findings of different strength across studies of varying sample sizes. These advantages have established meta-analysis as a preferred synthetic approach in the fields of SLA and applied linguistics, with the influential meta-analysis by Norris and Ortega (Reference Norris and Ortega2000) firmly establishing the relevance of this technique. Since then, numerous meta-analyses have examined interaction. For example, Russell and Spada (Reference Russell, Spada, Norris and Ortega2006), Li (Reference Li2010), and Lyster and Saito (Reference Lyster and Saito2010) examined corrective feedback, Keck et al. (Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006) and Mackey and Goo (Reference Mackey, Goo and Mackey2007) focused on task-based and conversational interaction, and Plonsky and Gass (Reference Plonsky and Gass2011) examined the methodological quality of interactionist research.
Extending the results of Russell and Spada (Reference Russell, Spada, Norris and Ortega2006) and Lyster and Saito (Reference Lyster and Saito2010), which found medium effects for corrective feedback on L2 development, Li’s (Reference Li2010) analysis found a medium overall effect of .64 for corrective feedback. In addition, Li’s results indicated no significant differences between feedback in computer-mediated contexts—provided both by the computer and an interlocutor during text chat— and feedback in FTF contexts, highlighting the efficacy of corrective feedback in both communication modes. Focused more on the overall effectiveness of task-based interaction on L2 development, the results of Keck et al. (Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006) indicated a large combined effect size for interaction (d = .92), whereas Mackey and Goo (Reference Mackey, Goo and Mackey2007) found positive effects for interaction on immediate (d = .75), short-term delayed (d = 1.20), and long-term delayed (d = .99) measures, providing robust evidence for the positive benefits of interaction on learners’ development.
More recently, Lin et al. (Reference Lin, Huang and Liou2013) sought to provide a quantitative assessment of the effects of SCMC on L2 acquisition by meta-analyzing 10 published articles and dissertations. The researchers compared text-based chat to FTF, audio and video chat, and asynchronous computer-mediated contexts (ACMC), with findings indicating a small overall effect (d = .33) for text chat over other communication modes on overall learning outcomes. Although Lin and colleague’s study provides a much needed quantitative assessment of the impact of communication mode on learners’ L2 development, there are a number of issues that remain unaddressed. For example, Lin and colleagues restricted their sample to include only text-based chat, providing a limited understanding of the overall comparative effectiveness between traditional FTF and SCMC modes on facilitating L2 development. In addition, because their analysis is focused on the comparison between text chat and all other modes of communication, including FTF as well as ACMC and voice chat contexts, results are limited in their specificity and generalizability. By excluding audio and voice chat from their operationalization of SCMC, the results are not interpretable in relation to the broader understanding of the construct, thereby limiting any direct comparisons of the overall efficacy of computer-mediated contexts with traditional FTF contexts. In addition, rather than including only those studies conducted from one theoretical paradigm, such as that of the interaction approach, the researchers focused on comparing any text-based chat activities with those in FTF or ACMC contexts. Although these findings provide important information regarding the effects of text-based chat, the theoretical implications are limited due to the inclusion of studies from multiple perspectives. Furthermore, although a number of the primary studies are conducted from an interactionist perspective, the sample includes ACMC and FTF groups where learners did not participate in any interactional activities (e.g., Satar & Ozdener, Reference Satar and Ozdener2008) as comparison and control groups as well as one study in which it is not clear if the results are due to interactional activities or another type of asynchronous noninteractional task (Blake, Reference Blake2009). The inclusion of these studies, although appropriate for the research questions of Lin et al. (Reference Lin, Huang and Liou2013), prevents the drawing of firm conclusions regarding the potential comparative benefits of interaction in SCMC and FTF communication modes.
Although these recent synthetic studies provide important information regarding the efficacy of computer-mediated activities in L2 acquisition, a number of issues remain unresolved. For instance, the foci of each of the meta-analyses and syntheses has varied, leaving questions regarding the comparative efficacy of text- and voice-chat SCMC and FTF unanswered. Although Mackey and Goo (Reference Mackey, Goo and Mackey2007) include computer-mediated studies, no direct comparison is made between technology-mediated and traditional contexts. In addition, though Li (Reference Li2010) makes a distinction between mode of feedback delivery, comparing teacher, learner, and computer-delivered corrective feedback, studies using interlocutor-provided feedback through computer programs and preprogrammed computer-delivered feedback are included in a single group, thereby preventing any meaningful conclusion regarding the effects of real-time synchronous interaction on learners’ development. In addition, it is not clear which model of meta-analysis was used by Lin et al. (Reference Lin, Huang and Liou2013), a potentially problematic omission. The fixed and random effects models are based on different assumptions and may yield different findings, highlighting the need for more methodological transparency and further investigations.
Overall, none of the previous research has comprehensively investigated the relationship between interaction, SCMC, and L2 development. In an effort to fill this gap, the current study seeks to provide a more thorough and all-inclusive meta-analysis of the relative efficacy of SCMC and FTF contexts on L2 acquisition by investigating whether well-established effects of interaction on L2 learning outcomes can be found in computer-mediated contexts. Specifically, the current research focuses on SCMC research from 1990 to 2012 and addresses the following research questions:
-
1. Compared to FTF interaction, how effective is synchronous computer-mediated interaction at promoting L2 learning outcomes?
-
2. To what extent do the following factors impact the degree to which interaction facilitates L2 development in computer-mediated environments?
-
a. Type of learning outcome (oral, written, receptive, productive)
-
b. Context (foreign language and L2)
-
c. Setting (laboratory and classroom)
-
d. Interlocutor characteristics (nonnative and native speaker)
-
e. Type of target feature (syntactic, morphological, phonological, lexical, pragmatic, productive)
-
METHODOLOGY
Study Identification and Retrieval
The initial stage of data retrieval involved a comprehensive and exhaustive search using the online research databases Linguistics and Language Behavior Abstracts (LLBA), Education Resources Information Clearinghouse (ERIC), and Education Full Text to identify potentially eligible empirical studies and associated literature, such as previous reviews or syntheses. In addition, the online databases ProQuest and Dissertation Abstracts International were searched to retrieve relevant unpublished research reports.
Previous syntheses and meta-analyses (Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Mackey & Goo, Reference Mackey, Goo and Mackey2007; Norris & Ortega, Reference Norris and Ortega2000) have elected to exclude unpublished research for a variety of reasons, including convenience and to represent a higher quality of research. However, many meta-analysts have encouraged the inclusion of unpublished work to have a sample more representative of a particular research domain and to avoid introducing any publication bias into the size of effects that may be found (Hunter & Schmidt, Reference Hunter and Schmidt2004; Konstantopoulos & Hedges, Reference Konstantopoulos, Hedges and Kaplan2004; Norris & Ortega, Reference Norris and Ortega2006). To obtain a larger sample of relevant research, dissertations were selected for inclusion in the current study as they are likely to include detailed information regarding methodology and statistical analyses, providing the necessary information for the calculation of effect sizes. Because dissertations have been subjected to review by experts in the field and, though unpublished, are carefully designed and piloted, it is necessary to acknowledge that their inclusion may only minimally impact the possible effects of publication bias.
The next phase of retrieval was conducted using the online search function for targeted L2, applied linguistics, and CALL publications. Five CALL-specific journals (CALICO Journal, CALL-EJ Online, Computer Assisted Language Learning, ReCALL, Language Learning and Technology) and 14 applied linguistics journals (Applied Linguistics, Journal of Applied Linguistics, Canadian Modern Language Review, Foreign Language Annals, International Journal of Educational Research, Language Learning, Language Testing, International Review of Applied Linguistics, Language Teaching Research, Modern Language Journal, Second Language Research, Studies in Second Language Acquisition, System, TESOL Quarterly) were targeted for relevant studies and literature.
A keyword search was used to locate potentially relevant research in the mentioned sources with the following words and terms: computer-mediated communication, computer-assisted language learning, synchronous chat, asynchronous chat, multimedia, interaction, input, output, feedback, corrective feedback, recast, uptake, negotiation, tasks, task-based language learning, task-based instruction, and language learning technology. Forward searches and citation chaining through the online search engine Google Scholar were also used to identify relevant research, and experts in the field were consulted regarding potentially eligible studies and reports.
Inclusion Criteria
More than 500 studies relating to interaction and SCMC were identified through the retrieval process and were reviewed using the following inclusion and exclusion criteria to determine relevance to the proposed research questions.
-
1. The study was published in a peer-reviewed journal or was a dissertation completed between 1990 and 2012. This temporal range was selected to capture the initial investigations as well as the most recent research on interaction in SCMC.
-
2. The study examined a target language that was either an L2 or foreign language for the participants.
-
3. The study compared interaction in SCMC (e.g., text chat, video conferencing, audio conferencing, or bimodal chat) to interaction in FTF, with both groups participating in interactional activities. In other words, because the current research was concerned with the comparative efficacy of interaction, studies in which the FTF group only completed the testing measures were excluded.
-
4. The study was of an experimental or quasi-experimental design. Because the present study was concerned with the comparative efficacy of interaction in two groups, FTF and SCMC, only studies of an empirical nature were included.
-
5. The study incorporated one or more open or closed interactive tasks (one-way or two-way information gap, information exchange, problem solving, and decision-making tasks) or an interactive setting, such as conversation groups, that facilitated interaction through open group discussion.
-
6. The study examined the effects of SCMC on L2 learning outcomes, including measures of production, performance, and development.
-
7. The study measured L2 learning outcomes using a pretest-posttest or posttest only design.
-
8. The study included the necessary information for the calculation of effect sizes, including mean, standard deviation or variance, and sample size, for each group.
-
9. The study was published in English.
Coding Procedures
Fourteen unique sample studies satisfying the inclusion criteria were identified for the current study. All of the studies were coded according to the features listed in Table 1, with a second and third rater coding 75% of the sample to ensure reliability. Interrater reliability was .96 and was calculated using simple percentage agreement. Because most of the items were considered to be low inference, this was considered to be an acceptable amount of agreement across raters.
Coding characteristics
ANALYSIS
Statistical Analysis
All analyses were conducted using the Biostat meta-analysis software program Comprehensive Meta-Analysis (CMA; Version 2.0), developed by experts in the field of meta-analysis (Borenstein, Hedges, Higgins, & Rothstein, Reference Borenstein, Hedges, Higgins and Rothstein2005).
Fixed-Effects and Random-Effects Model
Meta-analyses are based on either the fixed-effects or random-effects model depending on the assumptions of the data. Because the participants and treatments used in the current sample studies varied in ways that may have influenced the outcomes, for example treatment length and intensity, it is logical to assume that there is not a common effect size across the studies in the primary literature. In addition, the goal of the current analysis is to make generalizations regarding interaction in SCMC across a range of populations and scenarios, rather than drawing conclusions for identical, narrowly defined participants, providing additional support for the selection of the random-effects model over the fixed-effects model.
Calculation of Effect Sizes
Following previous meta-analyses on L2 interaction and corrective feedback research (Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Li, Reference Li2010; Lyster & Saito, Reference Lyster and Saito2010; Mackey & Goo, Reference Mackey, Goo and Mackey2007; Russell & Spada, Reference Russell, Spada, Norris and Ortega2006), Cohen’s d was used as the effect size for each study in the sample, as the descriptive statistics necessary for computing this statistic—namely, means, standard deviations, and sample size—are commonly reported in interaction research. However, not all studies used in the current analyses reported the descriptive statistics needed for the calculation of Cohen’s d. For instance, one study (Sykes, Reference Sykes2005) reported only the means and p value of the post hoc comparisons. To include the effect size in the analysis, STATA (Release 13) was used to estimate the standard deviation based on the reported p value.
Effect sizes were then calculated for each separate group on immediate posttests. Although previous research has included effect sizes based on delayed posttests (Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Mackey & Goo, Reference Mackey, Goo and Mackey2007), only three studies in the current sample reported delayed posttest scores (Baralt, Reference Baralt2010; de la Fuente, Reference de la Fuente2003; Yanguas, Reference Yanguas2012). Due to the insufficient amount of data, delayed posttests were not included in the current analysis. Because effect sizes were calculated for each of the possible contrasts in each study according to the inclusion criteria, multiple effect sizes were obtained from a single study. In other words, for studies that examined the acquisition of different features, separate effect sizes were calculated for each feature. Including an effect size for each outcome measure used within individual studies can lead to an underestimation of the standard error, which may result in unreliable statistical results (Lipsey & Wilson, Reference Lipsey and Wilson2001). To correct the variance to account for the relationships between data points and therefore increase the robustness of the calculations, effect sizes were combined across similar outcomes measures, following the principle of one study, one effect size for the main analysis. For the analyses of individual skills, multiple effect sizes were averaged for each category of outcome measures within individual studies. This approach allowed for a robust examination of multiple outcomes, providing summary effects for the impact of mode on overall L2 learning outcomes, as well as the difference in effect sizes across various skills. Effect sizes were then weighted according to individual sample sizes, providing a reliable consideration of the impact of larger sample sizes.
Because the overall sample size of this meta-analysis was small, with 14 unique studies and 37 total effect sizes, there was a possibility that outliers would skew the data, potentially producing misleading results. The CMA software program provides for the examination of change in the mean effect size when one study is removed from the sample. Using this approach, outliers were defined as any study resulting in a change in the mean effect size of more than .05 standard deviation units. Analyses indicated that none of the identified studies were extreme outliers, therefore, all remained in the final sample. Confidence intervals of 95% were also calculated to provide information regarding the reliability of the mean effect sizes and the differences across groups. In addition, a power analysis was conducted to reduce the risk of Type I error. Using G*Power software (Faul, Erdfelder, Lang, & Buchner, Reference Faul, Erdfelder, Lang and Buchner2007), statistical power for the current analysis was found to be sufficient at .95.
RESULTS
Methodological Features
Research Distribution
Fourteen studies comparing the effectiveness of interaction in FTF and SCMC contexts met the inclusion criteria for the current meta-analysis. Of these, eight (57%) were published in the following refereed journals: CALICO, Computer-Assisted Language Learning, Modern Language Journal, and System. The remaining six (42%) were unpublished doctoral dissertations.
Publication Bias
To explore for the possibility of publication bias, a funnel plot analysis was used to analyze the current sample. A biased sample would be more likely to yield an overestimated effect size due to the possibility that published studies are more likely to have significant effects. If publication bias were present, the bottom of the funnel plot would show a higher concentration of studies on one side of the mean than the other. This type of distribution would reflect the tendency for smaller studies with larger than average effect sizes (making them more likely to achieve statistical significance) to be published (Borenstein, Hedges, Higgins, & Rothstein, Reference Borenstein, Hedges, Higgins and Rothstein2009). Figure 1 demonstrates that the majority of effect sizes were equally distributed around the mean, indicating the absence of publication bias in the current sample. In addition, because three of the four journals included in the sample have a specific focus on technology, the possibility for substantive bias due to editorial preferences was also examined. Results of a funnel plot analysis did not indicate any evidence of bias as a result of journal preferences.
Publication bias: Funnel plot by effect sizes.
Research Setting
The 14 studies included in the current study represented a variety of instructional settings, research contexts, and learner characteristics. The majority of studies occurred in a classroom-based environment (k = 10; 71%), with only four of the studies (29%) in the sample taking place in a laboratory context. Seventy-nine percent (k = 11) of the studies were conducted in a foreign-language context, with the remaining three studies (21%) in a L2 context. Table 2 summarizes the research characteristics of the included studies.
Included studies by research characteristics
a In some studies, more than one outcome measure or mode was used, so the sum of these categories is not the same as the total number of studies.
Learner Characteristics
Most of the studies were conducted in university contexts (79%), whereas the remaining 21% were in high school contexts. Participants’ ages ranged from 14 to 25 years, although 57% of the studies failed to report an age range or mean age for their participants, highlighting the need for more rigorous reporting of participant characteristics. Eleven studies (79%) reported participants’ proficiency level as intermediate, and three reported beginning proficiency levels (21%); however, only one study (Petersen, Reference Petersen2010) considered the developmental readiness of participants. Table 3 illustrates learner characteristics across the studies included in the sample.
Included studies by learner characteristics
Statistical Reporting
Many studies identified in the initial retrieval phase were eliminated due to incomplete reporting of statistical data (e.g., Beauvois, Reference Beauvois1998; Salaberry, Reference Salaberry2000; Warschauer, Reference Warschauer1996a). Of the 14 studies that met the inclusion criteria, all but one used inferential statistics for the analyses. The most commonly used statistical test to examine the differences in learning outcomes between groups was ANOVA (64%), followed by t tests (29%) and ANCOVA (21%). This finding matches those of recent studies that have found ANOVA to be the most frequently used inferential test (Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Mackey & Goo, Reference Mackey, Goo and Mackey2007; Plonsky & Gass, Reference Plonsky and Gass2011). However, despite the widespread use of inferential statistics, Yanguas (Reference Yanguas2012) was the only study published in a peer reviewed journal to report testing the assumptions necessary for the selected statistical procedure, such as tests of normality. Four dissertations in the sample stated that assumptions had been checked before running the statistical analyses (Baralt, Reference Baralt2010; Hirotani, Reference Hirotani2005; Lin, Reference Lin2009; Petersen, Reference Petersen2010), but 71% of the sample failed to mention an a priori alpha level. Similar to the results of Keck et al. (Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006), fewer than half of the studies (43%) reported an exact p value, and only Baralt (Reference Baralt2010) reported conducting a power analysis. Only five studies reported effect sizes (36%), four of which were dissertations. Recent research has highlighted the need to report effect size (e.g., Plonsky & Gass, Reference Plonsky and Gass2011), a feature critical to understanding the amount of variance observed in the dependent variable due to the influence of the independent variable. On the basis of the data in the current sample, there is still ample room for improvement in the reporting of this important feature. In addition, reliability measures for interrater reliability or internal validity and reliability of testing materials were not reported by more than half of the studies in the current sample (57%). Furthermore, only one of the studies reported confidence intervals. Although these issues in reporting may impact readers’ ability to thoroughly understand the results, nearly all of the studies (except Conaim & Wong, Reference Conaim and Wong2004) within the current sample relied on statistical significance to interpret the findings.
Meta-Analytic Results
As previously stated, the current meta-analysis combined effect sizes from 14 unique study reports to compare the relative effectiveness of interaction in FTF and SCMC contexts on L2 learning outcomes as measured on immediate posttest scores. In addition, effect sizes were calculated to assess the comparative impact of interactional mode on the development of learners’ oral and written skills.
Overall Relative Effectiveness of Interaction in SCMC Compared to FTF
Table 4 provides the effect size statistics for the first comparison, where k represents the number of unique studies and N represents the total number of individual participants in the selected studies.
Descriptive effect size statistics for overall L2 learning outcomes
Note. SE SM = Standard error of standardized mean effect size.
Analyses indicated that interaction in both FTF and SCMC contexts resulted in significant improvements (p < .001) in L2 learner outcomes. Although large effects were found for both modes, with interaction in SCMC (d = 1.13) resulting in higher scores on immediate posttests than FTF interaction (d = .84), the comparative results indicated only a relative effect size of d = .13 for SCMC, p = .284. According to Plonsky and Oswald’s (Reference Plonsky and Oswald2014) assessment of L2 specific effect sizes, mean differences between groups should be considered small when Cohen’s d is around .40. Interpreted against these domain-specific reference values, the relative advantage for SCMC is small, with results indicating learning outcomes in SCMC were .13 standard deviation units larger than for FTF contexts. A relative effect size of .13 is associated with a U 3 value of 55.17%, where U 3 represents the degree of nonoverlap between two groups (Cohen, Reference Cohen1988; Lipsey et al., Reference Lipsey, Puzio, Yun, Hebert, Steinka-Fry, Cole, Roberts, Anthoney and Busick2012). In other words, 55.17% of learners in SCMC contexts had learning outcome scores above the mean of those in FTF environments, suggesting that interaction in SCMC contexts provides a slight advantage over FTF contexts in facilitating L2 learning outcomes. The 95% confidence interval [–.11, .38] provides a realistic estimate of the limits within which the true difference between contexts may be found. It should be noted here that the range is relatively narrow, suggesting that the estimated effect sizes are sufficiently precise.
Relative Effects of Individual Moderating Variables on Learning Outcomes
Because previous research in interaction has demonstrated that a number of factors may affect learning outcomes (e.g., Li, Reference Li2010; Mackey & Goo, Reference Mackey, Goo and Mackey2007), analyses examining the following moderating variables were conducted to provide a more nuanced understanding of the possible impact of mode on L2 learning outcomes: (a) type of learning outcome, (b) type of target feature, (c) research setting, (d) research context, and (e) interlocutor characteristics.
Type of Learning Outcome
Treatments focusing on productive learner outcomes resulted in a small relative effect size for SCMC, d = .15, [–.09, .39], p = .230. The associated U 3 value of 55.96% suggests that SCMC has a small relative effect on learners’ productive skills when compared to learners participating in FTF interaction. In contrast to these findings, results suggest a small relative effect size for the development of receptive skills in favor of FTF interaction, d = –.12, [–.45, .20] p = .460, with the associated U 3 value indicating that 54.78% of learners in FTF contexts had receptive scores above the mean of those in SCMC environments. Findings also indicated a small relative effect size on the development of oral skills for FTF over SCMC contexts, d = .04, [–.30, .34], p = .890. For this comparison, the small d value and associated U 3 value (51.60%) indicate that the relative advantage for performance on oral measurements in FTF contexts over SCMC contexts was minimal. A small effect size in favor of SCMC for written learning outcomes was also obtained, d = .16, [–.04, .37], p = .123, with the associated U 3 value indicating that 56.36% of learners in SCMC contexts had greater performance on measurements of writing above the mean of those in FTF contexts. However, for written and receptive measures, the extremely small sample size of individual effect sizes (k = 8 and k = 7, respectively) included in this comparison highlights the need to interpret the results of this particular contrast with caution as this comparison may be lacking the precision of the other contrasts. Rather than draw definitive conclusions based on these findings, additional studies examining the impact of mode on learners’ written and receptive language skills should be conducted to form a more robust sample size, thus strengthening the conclusive validity of the findings of this comparison.
Type of Target Feature
The current analysis coded for type of target feature, and although findings reveal a positive effect for interaction in SCMC on overall learning outcomes, there were not enough individual studies focusing on similar target features to warrant a more finely grained statistical analysis. Twenty-nine percent (k = 4) assessed the acquisition of grammatical targets (29%), 21% investigated the acquisition of lexical features (k = 3), and 7% (k = 1) explored the development of pragmatic knowledge. The remaining studies examined fluency (14%), complexity (7%), or general oral or written proficiency (29%). Analyses were conducted with a subset of the current sample to examine whether effectiveness was affected by grammatical or lexical target items. Findings revealed a small relative effect size (d = .07, [–.19, .34], U 3 value = 52.79%) for interactional treatments in SCMC focused on grammar and a small relative effect size (d = .34, [–.72, .04], U 3 value = 63.31%) for interactional treatments in FTF focused on lexical items. In other words, interaction conducted in a SCMC environment was on average .07 standard deviation units greater than interaction in FTF contexts with 52.79% of learners in SCMC performing better on grammatical measures that than those in FTF, whereas interaction occurring in a FTF context resulted in performance on lexical items that was on average .34 standard deviation units greater than those in SCMC contexts, with 63.31% of learners in FTF obtaining greater levels of performance than those in SCMC. However, the results for lexical features should be interpreted with caution, as the confidence intervals are relatively wide, suggesting that the small sample size may have yielded an imprecise estimate of the true effect size. Although more research is needed to examine the effect of mode of interaction on specific learning outcomes, the wide variety of target items featured in the current sample, coupled with the positive overall average effect size on L2 learning outcomes, demonstrates the potential of SCMC interactional treatments to improve performance and development on a diverse set of learner measures.
Research Context (L2 or Foreign Language)
One of the goals of the current analysis is to examine whether the research context had any impact on the observed mean effects. The comparison between the two communication modes revealed a small relative effect for SCMC foreign-language contexts, d = .08, [–.21, .34], U 3 value = 53.19%, and a small relative effect in favor of SCMC in L2 contexts, d = .33, [–.08, .74], U 3 value = 62.93%. Although neither comparison was statistically significant, the relative effect size in L2 environments indicates that performance differences across mode were on average .33 standard deviation units larger for learners participating in SCMC interaction than those in FTF, with 62.93% of learners performing above the mean outcome scores of those in FTF contexts. Overall, these findings suggesting that mode of communication may have a substantial impact on learners’ performance in a L2 context.
Research Setting (Laboratory or Classroom)
Although much of the body of research investigating interaction has been conducted in laboratory settings (e.g., Ellis, Reference Ellis and Mackey2007; Ellis et al., Reference Ellis, Tanaka and Yamazaki1994; Mackey, Reference Mackey1999; McDonough, Reference McDonough2005; Philp, Reference Philp2003), previous meta-analyses have demonstrated that interaction is also beneficial for L2 development in the classroom (Mackey & Goo, Reference Mackey, Goo and Mackey2007). Contrasts of interaction in FTF and SCMC environments revealed a very small relative effect size for SCMC in laboratory settings, d = .06, [–.17, .29], U 3 value = 52.39%, although it was not significant. In classroom-based environments, results indicated that there was a small relative effect found in favor of SCMC, d = .18, [–.09, 045], U 3 value = 57.14%. Although additional research directly comparing the efficacy of interaction in SCMC lab and classroom environments would further our understanding of the influence of mode on L2 learning outcomes, these findings provide encouraging evidence that interactional treatments in computer-mediated contexts have a wide range of pedagogical applications.
Interlocutor
This meta-analysis also investigated whether the effects of mode of interaction would be influenced by the type of interlocutor participating in the interaction. For this comparison, studies were categorized according to native speaker status (nonnative vs. native speaker) rather than by proficiency. In the current study, a small relative effect size was found in favor of SCMC for interactions with native speaker (NS) interlocutors, d = .09, [–.20, .39], U 3 value = 53.59%, and a small relative effect was found for nonnative speaker (NNS) interlocutors in SCMC interactions, d = .11, [–.16, .37], U 3 value = 54.38%. Due to the small number of effect sizes for NS interlocutors (k = 5), results should be interpreted with caution as the between groups variance may be in error due to the small sample size. More data are clearly needed to gain a better understanding of the role that type of interlocutor plays in the treatment effects of interaction in SCMC contexts. Table 5 provides a summary of the results examining moderator variables.
Summary of results: Comparative effects of moderator variables
Note. Comparisons marked with * were calculated using multiple outcome measures due to the small sample size.
DISCUSSION
The first research question sought to determine the relative effectiveness of interaction in SCMC and FTF contexts. Results indicate a small relative effect size in favor of SCMC for overall L2 learning outcomes. Furthermore, results revealed that interaction in both contexts plays a beneficial role in promoting L2 learning outcomes, providing additional support for the positive developmental benefits of interaction. However, an important caveat to these findings is the fact that the average relative effect sizes were calculated for immediate posttests only. Only three studies in the current sample reported delayed posttest scores (Baralt, Reference Baralt2010; de la Fuente, Reference de la Fuente2003; Yanguas, Reference Yanguas2012), making any comparisons statistically unreliable. The lack of a delayed posttest contrast limits the generalizability of the findings, as research has empirically demonstrated that interactional features may have delayed effects (e.g., Mackey, Reference Mackey1999; Muranoi, Reference Muranoi2000; Sheen, Reference Sheen and Mackey2007). In addition, research suggests that delayed effects may be larger than those on immediate posttests (Mackey & Goo, Reference Mackey, Goo and Mackey2007), particularly on measures of L2 grammatical development. Although results for the immediate effects on interaction in SCMC are promising, more long-term research is needed to enhance our knowledge of the role of interaction on learners’ sustained development.
The second research question aimed to identify the moderator variables that may influence the relative effectiveness of interaction in SCMC and FTF contexts. Small advantages were found for SCMC for productive measures and for FTF for receptive measures. The trends in favor of SCMC on productive measures might be explained by several factors. For instance, production may require greater levels of processing than comprehension (Izumi, Reference Izumi2003; Swain, Reference Swain, Cook and Seidlhofer1995). Because computer-mediated contexts have been argued to provide additional time for processing and planning (Beauvois, Reference Beauvois1992; Pellettieri, Reference Pellettieri, Warschauer and Kern2000; Smith, Reference Smith2004; Smith & Gorsuch, Reference Smith and Gorsuch2004), the small effect size in favor of SCMC may be due to this potential advantage in computer-mediated interaction. This is particularly true of text-chat exchanges, as learners are presented with a written record of the interaction, which may provide added opportunities for learners to attend more closely to the form and content of the input while still maintaining the real-time feel of conversation (Smith, Reference Smith2003; Toyoda & Harrison, Reference Toyoda and Harrison2002).
These additional opportunities for noticing provide learners with more developmental support, particularly as the computational model of L2 processing (Ellis, 1997) requires that learners must first attend to, or notice, the input. Schmidt’s noticing hypothesis (Schmidt, Reference Schmidt and Robinson2001) also emphasizes the importance of noticing, and specifically the noticing of the formal features in the target language, as well as noticing gaps between a learner’s interlanguage and the target language. As noted earlier, researchers have suggested that SCMC may increase learners’ opportunities to notice target items in the input as well as notice gaps between their interlanguage and the target language, potentially also leading to increased instances of intake. Overall, these findings suggest that SCMC may be more facilitative of certain target forms than FTF interaction. In other words, interaction in SCMC may offer a small advantage over FTF interaction in promoting L2 learning, particularly on productive measures, by providing learners additional time to notice and process the target forms.
A small relative advantage was found for FTF contexts over SCMC for the receptive language skills of listening and reading. This result may be due to the features of FTF interaction that provide learners with multiple opportunities to attend to the input. For instance, FTF interaction provides visual, verbal, and gestural cues, offering learners a variety of complementary input sources to aid in the processing of spoken language. Learners in FTF interaction are exposed to a range of input, thereby potentially supporting the later recognition of the target forms.
However, because the current analysis only includes immediate posttest measures, it is unclear how durable these effects for receptive and productive learning may be. For example, learners may have simply attended to the target forms, supporting their aural or visual recognition on immediate posttests, but failed to process the forms at a deeper level. In their recent meta-analysis, Shintani et al. (Reference Shintani, Li and Ellis2013) found that production resulted in more durable effects than comprehension, suggesting that shallow processing may result in lower scores on receptive measures, whereas the more integrative processing associated with production may manifest in increased scores on delayed posttest measures. These findings highlight the need for more long-term measures to better understand the duration of effects on receptive and productive skills.
Additional analyses sought to examine the role of communication mode on learners’ written and oral outcomes. Comparing the two contexts, SCMC resulted in a small relative effect size for written outcomes, and a small relative effect was found for oral outcomes in FTF contexts. Because the majority of studies in the current sample favored oral production tasks for assessment, learners participating in FTF interaction may have had a slight advantage over those in SCMC contexts, particularly written SCMC environments, due to the similarity between the treatment and testing tasks. Likewise, the slight advantage for interaction in SCMC for the development of learners’ written skills suggests that learners may benefit when the modality of treatment and assessment are the same.
Another possible explanation for the advantage of FTF contexts for the development of learners’ oral skills is gesture. Available only in FTF or video SCMC, of which there were only two studies included in the current analysis, gestures provide learners with important communicative information. For instance, previous work reveals that gestures are not restricted to a supplementary role but instead complement speech and interaction (Gullberg, Reference Gullberg1998). Research has also shown that gesture directly impacts L2 development, with studies demonstrating improved listening comprehension when learners are able to view video showing speakers’ facial gestures in addition to hearing an audio track (Kellerman, Reference Kellerman1992; Sueyoshi & Hardison, Reference Sueyoshi and Hardison2005). These promising findings suggest that gesture plays an important role in interaction and may help to explain the trend in favor of FTF contexts for the development of oral skills. The role of gesture in L2 development also highlights the potential learning benefits of video SCMC, which has been shown to produce similar results to FTF interaction in both receptive and productive measures (Yanguas, Reference Yanguas2010, Reference Yanguas2012). On the basis of the data presented in the current sample, the meta-analytic findings of the current study underscore the need for more research into this relatively underexplored domain.
The current trends indicating an advantage for SCMC for written skills and FTF for oral skills seem to also be consistent with claims that practice and learning may be skill specific (DeKeyser, Reference DeKeyser1997, Reference DeKeyser, VanPatten and Williams2007), as learners seem to exhibit better performance on posttest measures when the mode of treatment and assessment are the same. However, it should be noted that not all research has yielded similar results. For example, some researchers have found that the developmental benefits of interaction in SCMC are not confined to skill-specific learning, with findings showing that written SCMC may gradually (Chun, Reference Chun1994) or directly (Beauvois, Reference Beauvois1992; Kern, Reference Kern1995; Payne & Whitney, Reference Payne and Whitney2002) transfer from written to oral modalities. Although both modes are interactive in nature, these conflicting results seem to suggest that there are certain features unique to each mode that make direct comparisons difficult.
Turning to the comparisons examining laboratory and classroom settings, relative effect sizes were larger for SCMC in laboratory and classroom contexts. These minor advantages in favor of SCMC may be due to a number of factors. For example, interaction in laboratory settings may be more controlled, providing increased opportunities for consistent feedback and fewer distractions from target tasks than in classroom-based contexts. The heightened control and consistency of lab-based treatments may offer learners greater opportunities to attend to input and focus on form, potentially leading to greater performance and scores on measures of learning outcomes. Although some scholars have suggested that the quantity and quality of interaction may be impacted by settings, with previous meta-analytic research indicating significantly larger effects for interaction and corrective feedback in laboratory over classroom settings (Li, Reference Li2010; Mackey & Goo, Reference Mackey, Goo and Mackey2007), other studies have found no differences in outcomes across settings (Plonsky & Gass, Reference Plonsky and Gass2011). These results highlight the importance of setting as a moderating variable (Oswald & Plonsky, Reference Oswald and Plonsky2010), and, as the body of SCMC research conducted in classroom settings continues to grow, future analyses should reconsider this comparison to further our understanding of the role of communication mode in controlled versus more natural, educational settings.
Regarding the comparison between foreign-language (FL) and L2 settings, a small relative effect was found for SCMC in FL settings (d = .08) and in L2 settings (d = .33), possibly reflecting some of the advantages researchers have attributed to computer-mediated contexts, such as its ability to reduce anxiety. Within L2 settings, anxiety may have a greater effect on learner-learner interaction, as L2 programs are more likely to consist of a student body with diverse first languages (L1) and cultural backgrounds. For example, research has demonstrated that learners from different cultural backgrounds may be hesitant to provide feedback because they feel their interlocutors may find it socially inappropriate (Mackey et al., Reference Mackey, Oliver and Leeman2003). For these learners, a SCMC environment, particularly text chat, may be more facilitative of interaction due to the potential reduction in face-threatening acts. In FL environments, where learners may be more likely to be from a shared L1 and cultural backgrounds, cultural factors may play less of a role.
The influence of type of interlocutor on L2 learning outcomes was also investigated. Small relative effects for SCMC were found for native speaker and nonnative speaker interlocutors, a finding that might be explained by several factors. Previous research has indicated that native speaker-nonnative speaker dyads may experience more anxiety (on the part of the learner; Lee, Reference Lee2004; Satar & Ozdener, Reference Satar and Ozdener2008) as well as fewer instances of negotiation, pushed output, or correction (Gass & Varonis, Reference Gass and Varonis1985; Iwashita, Reference Iwashita2003). For example, Lee (Reference Lee2004) found that NNSs with low proficiency levels reported that they did not feel comfortable interacting with NSs due to levels of anxiety and discomfort. The small relative effect for SCMC found in the current research may be due to the potentially reduced anxiety attributed to SCMC interactions, particularly during text chat (Satar & Ozdener, Reference Satar and Ozdener2008). According to Beauvois (Reference Beauvois and Terry1997), computer-mediated contexts provide “an anonymous, less pressured environment that tends to lower the affective filter” (p. 171), potentially providing anxious or introverted students with a more comfortable and safer environment in which they can feel free to make mistakes and test their hypotheses about the target language. However, research examining the effects of NS-NSS dyad composition on learners’ production and performance remains limited. Researchers should consider conducting further experiments examining NS-NSS pairs, in conjunction with measures of anxiety and willingness to communicate, to gain a better understanding of the role the interlocutor plays in L2 development. Furthermore, research has demonstrated a variety of results when learners interact with an interlocutor with a different proficiency (e.g., Leeser, Reference Leeser2004; Ohta, Reference Ohta2001; Swain & Lapkin, Reference Swain and Lapkin1998; Watanabe & Swain, Reference Watanabe and Swain2007), highlighting the need to further investigate pair composition and its relationship to individual and affective factors in computer-mediated contexts.
Previous research in interaction has found that the type of target form influences the effectiveness of the interactional treatment (Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006; Mackey & Goo, Reference Mackey, Goo and Mackey2007; Mackey et al., Reference Mackey, Gass and McDonough2000). Current findings revealed a small relative effect size (d = .06) for interactional treatments in SCMC focused on grammar and a small relative effect size (d = .11) for interactional treatments in FTF focused on lexical items. Research in SCMC has demonstrated that learners may have greater opportunities to self-repair grammatical features due to the increased monitoring and planning time provided by SCMC. These self-repair opportunities may have subsequently contributed to improved performance on posttest measures, resulting in the small relative effect for grammatical items in SCMC when compared to FTF.
Although the current research indicates promising potential for interaction in SCMC for the development of a variety of linguistic features, including lexical and grammatical items as well as pronunciation and pragmatic development, more research examining specific types of features is needed. For instance, there are no studies examining the effects of interaction and corrective feedback in SCMC on the development of simple and complex grammatical features, an area that remains somewhat underexplored in FTF contexts as well (although see de Graaff, Reference de Graaff1997; DeKeyser, Reference DeKeyser1995; Ellis, Reference Ellis and Mackey2007; Robinson, Reference Robinson1996; Spada & Tomita, Reference Spada and Tomita2010). In general, findings indicate that different types of feedback affect the learning of simple and complex target structures in unique ways. Researchers should explore this area further, as additional investigations will provide important information regarding the impact of mode of interaction on the development of simple and complex L2 structures.
Overall, according to the data included in the current study, findings have identified underresearched areas in need of further exploration. For instance, pursuing a research agenda that examines more specific target features as well as the impact of structural salience and complexity, will allow more finely grained analyses regarding the potential of SCMC to support learners’ development across a range of L2 items and skills. The increased inclusion of delayed testing measures is also needed to gain a better understanding of the long-term effects of mode of communication on L2 development. Previous research has demonstrated that certain interactional features may have a greater effect on delayed tests (e.g., Li, Reference Li2010; Mackey & Goo, Reference Mackey, Goo and Mackey2007), suggesting the need for more longitudinal research to enhance our knowledge of the role of interaction in SCMC on long-term learning. Results also highlight the need for further investigation into oral SCMC, including direct comparisons of audio and video forms of CMC and their relative effectiveness compared to FTF contexts, as well as the need for more systematic investigation into the effects of context, setting, and interlocutor characteristics. Such investigations would provide the field with a deeper understanding of the relationship between interaction, mode, and L2 development. However, until more research is conducted in these areas, firm synthetic conclusions regarding their impact on L2 learning cannot be made.
METHODOLOGICAL IMPLICATIONS
This study highlighted the need for improved reporting of statistical measures in SCMC research. Reporting practices varied widely across studies, with some early studies failing to report basic descriptive statistics such as means and standard deviations (e.g., Warschauer, Reference Warschauer1996a). In addition, most studies in the current sample failed to report a number of critical features, such as effect sizes or confidence intervals, and all but one (Baralt, Reference Baralt2010) lacked power analysis. Future researchers should consider power when preparing to collect data to ensure that any differences between groups will, in fact, be discovered. Furthermore, consumers of research should be skeptical of studies reporting only relative p values, as any difference between groups will attain statistical significance if the sample size is large enough. To provide better interpretations of results, exact p values should be provided in addition to confidence intervals and effect sizes. The reporting of effect sizes is crucial to understanding the true impact of the independent variable on the dependent variable, and effect sizes should be included in the reporting of all statistical analyses. Finally, reliability measures should be conducted and reported for materials, as this will provide readers with the knowledge necessary to accurately interpret any relevant findings.
LIMITATIONS
Although the current meta-analysis makes valuable contributions to the field, there are some important limitations that must be noted. Due to the application of rigorous eligibility criteria, the sample of individual effect sizes (ES = 37) calculated from the 14 primary studies is relatively small. Although there are ample examples of meta-analyses published with similar samples sizes in L2 research (e.g., Keck et al., Reference Keck, Iberri-Shea, Tracy-Ventura, Wa-Mbaleka, Norris and Ortega2006, k = 14; Lin et al., Reference Lin, Huang and Liou2013, k = 10; Yun, Reference Yun2011, k = 10), and the power analysis was adequate, the reliability and external validity of the results would be improved with a larger sample size. In addition, a number of comparisons were negatively impacted by the lack of sufficient data. Important constructs, such as the differential efficacy of individual types of interactional feedback, type of target feature, modified output opportunities, and type of CMC, could not be analyzed because there were not enough studies examining these features.
Another important limitation of this research is that some key studies in the field have not been included in the final sample. The inclusion criteria for this study were rigorous, causing a number of studies to be eliminated for failing to meet the standards set forth in the methods section. Although the eligibility criteria were driven by the research questions, it is important to acknowledge that this limited the current sample to quasi-experimental and experimental studies using a comparison groups design. To deepen our understanding of the differential efficacy of interaction on the rate of development over time, future meta-analytic work may wish to expand the inclusion criteria to focus on a broader range of research.
Although the current study notes a number of areas for improvement in research methods, specifically regarding the reporting procedures, the suggestions regarding quality are somewhat limited in scope. To better understand the methodological progress and limitations, a more complete methodological review of SCMC research should be conducted with a larger sample. As mentioned in the previous limitation, a number of studies were excluded because they failed to meet the eligibility criteria for the substantive features that were the target of the current research. By focusing more closely on methodology, a systematic review encompassing both experimental and descriptive designs may provide a more global perspective on the design aspects of SCMC research over the past few decades. An empirical comparison across as well as within a wider range of SCMC research in CALL and SLA journals would provide important information regarding the methodological quality of published studies in the field.
Although careful steps were taken to control for the possibility of publication bias in the current research, the results may have been impacted by bias nonetheless. Studies were not retrieved from conference proceedings, collections of working papers, or master’s theses, which may have contributed to the likelihood of bias. The impact of these missing studies may have resulted in an upward bias caused by unreported nonstatistically significant findings. Furthermore, the studies in the current sample were limited to publications in English, a substantial limitation considering the worldwide interest in CALL. Future research should consider expanding the source materials as well as including primary studies published in other languages.
Despite these limitations, the current study has deepened our understanding of the relationship between interaction and SCMC and has provided a structured framework, which, as the body of relevant work continues to grow, can be used for future meta-analytic investigations. The eligibility criteria, while strict, provide a foundation on which later studies can elaborate or expand to address new research questions.
