1. Introduction
This article argues for replications of two recent studies that focused on explaining L2 pragmatics learning in a study abroad context. Learning context research has been an important area of inquiry in the field of L2 pragmatics (for a review, see Taguchi Reference Taguchi2016). Compared with other learning contexts such as a virtual environment or an immersion setting, the study abroad context has by far generated the largest amount of empirical evidence. The bulk of literature on the study abroad context, however, has a predominant descriptive focus, investigating whether and to what extent specific pragmatic features are acquired over time or are better learnt than in a different context. The findings of these descriptive studies, however, have often been inconsistent and not comparable. More recently, researchers have been motivated to seek explanations underlying the varied learning outcomes in the study abroad context (for reviews, see Q. Li Reference Li2016 and Taguchi Reference Taguchi2016). Available findings in this area have revealed a complex interplay between learner characteristics (e.g., intercultural competence, proficiency) and contextual affordances (i.e., quality and quantity of L2 contact) in shaping different aspects of pragmatic competence (e.g., Matsumura Reference Matsumura2003; Shively Reference Shively2011; Taguchi Reference Taguchi2012; Taguchi, Li & Xiao Reference Taguchi, Li and Xiao2013; Hassall Reference Hassall2015). Since there is a wide range of learner characteristics and contextual factors that may potentially influence pragmatics learning while abroad, it is meaningful to test the robustness and generalizability of available empirical findings through replication research.
The two studies (Bardovi-Harlig & Bastos Reference Bardovi-Harlig and Bastos2011; Taguchi, Xiao & Li Reference Taguchi, Xiao and Li2016) that we propose for replication are good examples of most recent development (i.e., within the last six years) of this research agenda. These studies appeared in major refereed second language acquisition (SLA)/Applied Linguistics journals (i.e., Intercultural Pragmatics and The Modern Language Journal). Both studies adopted a quantitative research paradigm and provided a thorough reporting of methods to enable replication. In addition, these studies included both contextual and individual learner factors in the research design, thereby allowing comparisons of the effects of these factors on pragmatics learning. On the other hand, these studies complement each other because of their differences in research design (i.e., cross-sectional vs. longitudinal), focal contextual and individual factors (e.g., intercultural competence and proficiency; length of residence (LoR), intensity of interaction, and amount of social contact), targeted pragmatic features (i.e., conventional expressions vs. speech acts), and target languages (L2 English vs. L2 Chinese). The similarities and differences between these studies thus constitute an ideal basis for discussing options of replication.
In the following, we first provide a brief overview of the development of L2 pragmatics as a field, highlighting its gradual shift from a descriptive to an explanatory focus in research orientation. We then present the rationale for proposing replications of research on the study abroad context. The two focal studies will then be described in detail, followed by discussions of possible approximate and conceptual replications. According to Porte (Reference Porte2012), an approximate replication study follows the original study's experimental procedures closely but modifies, for example, non-target variables to allow for comparison of findings between the original study and its replication. Conceptual replication, on the other hand, might aim to test previous results using a different research methodology (e.g., different data collection methods and instruments). This article ends with a conclusion highlighting the significance of replication studies for L2 pragmatics research in a study abroad context.
2. Background
2.1 Explanation of pragmatic development: Individual and contextual influences
Since its inception in the 1980s, research in L2 pragmatics has shown continuing growth in its scope and in the number of empirical investigations. One major progress in the last three decades is the field's increasing emphasis on the explanation over description of pragmatic development (Bardovi-Harlig Reference Bardovi-Harlig1999, Reference Bardovi-Harlig, Gass and Mackey2012). Early cross-linguistic studies focused on describing linguistic strategies in speech acts, and variation in the use of linguistic forms according to contextual parameters (e.g., speakers’ power relationship and social distance). Subsequent studies adopted a cross-sectional design, describing differences in speech act strategies between L2 learners and native speakers, as well as among L2 groups of different proficiency and length of formal study. In the same period, longitudinal studies have contributed to our understanding of pragmatic development by tracing learners’ changes over time. However, longitudinal investigations, as well as cross-sectional studies, have been largely limited to describing patterns of changes, not accounting for the influences on the changes.
Only recently has cross-sectional and longitudinal research started to explore explanations of pragmatic performance and development in conjunction with individual and contextual factors. For example, Taguchi (Reference Taguchi2011) examined the influence of proficiency and study abroad experience (i.e., no study abroad experience vs. one year minimum of study abroad) on accuracy and speed involved in implicature comprehension (i.e., indirect refusals, routines, and non-conventional implicatures) among Japanese learners of English. The results showed that proficiency, but not study abroad experience, significantly influenced comprehension speed. In terms of comprehension accuracy, proficiency continued to exert a positive effect regardless of implicature type; however, learners with study abroad experience outperformed those without such experience in terms of non-conventional implicatures and routines, but not in terms of indirect refusals. In another study, Xu, Case & Wang (Reference Xu, Case and Wang2011) investigated the effects of proficiency and LoR on learners’ ability to identify pragmatic infelicities in L2 English in a study abroad context. The results showed that both proficiency (i.e., graduate vs. undergraduate students) and LoR (i.e., less than one year vs. more than one year) significantly affected pragmatic performance, with proficiency showing a slightly larger effect size than LoR. As we can see from these two studies, recent cross-sectional research seeks to explain learners’ pragmatic performance from multiple contingent factors including individual (i.e., proficiency) and contextual influences (e.g., LoR).
On the other hand, recent longitudinal studies examining pragmatic development during study abroad have been informed by the socially oriented SLA theories that view L2 learning as being situated in a social context, affected by the individual's traits, persona, and agency. For example, adopting the language socialization framework (Schieffelin & Ochs Reference Schieffelin and Ochs1986; Duff Reference Duff2007), Shively (Reference Shively2011) analyzed service encounter exchanges self-recorded by L2 Spanish learners in Spain. Several learners acquired the imperative forms of requests in service exchanges by observing other customers and adopting their forms, while others learnt the forms through feedback from their host families. However, one participant never used imperatives because she perceived them as impolite based on her first language (L1) cultural values. Exercising her subjectivity, she consciously opted not to use imperatives in L2.
In another study, Taguchi (Reference Taguchi2012) adapted the complex, dynamic systems theory (de Bot Reference de Bot2008; Larsen-Freeman & Cameron Reference Larsen-Freeman and Cameron2008) to reveal speech act development among Japanese ESL (English as a second language) students in an English-medium university in Japan. Data from a spoken discourse completion task revealed students’ limited progress with high-imposition speech acts (e.g., expressing a negative opinion to a teacher). This was explained by the observation that ESL teachers and students co-adapted their behaviors and expectations in the immersion context. In real-life situations where students expressed disagreement with their teachers, they often used strong modals (e.g., ‘should’) and direct expressions of dislike. Because teachers were keen on getting students’ feedback, they did not care about the impoliteness in speech, either neglecting to correct students’ inappropriate forms or feeling no need to correct them.
These cross-sectional and longitudinal studies attempted to capture changes in pragmatic competence, and to explain the changes by closely examining influences – both contextual and individual – that may be related to the changes. Such an attempt can contribute to the field's theoretical advancement by addressing the central question: What mechanisms drive learners from their current stage to a higher stage of pragmatic competence, and can we theorize those mechanisms? Individual and contextual influences found in these studies are part of such mechanisms that characterize development. The research agenda of contextual and individual influences in L2 pragmatics merits a replication to expand the body of findings contributing to this central question. This agenda will help us move from the mere amassing of empirical findings on pragmatic development, to the construction of knowledge as to why and how such development happens within certain individuals in certain contexts.
While this agenda is promising, as shown in the studies cited above, the contextual and individual account of pragmatic development has been mostly drawn from qualitative data involving observations, naturalistic recordings, and interviews. Although Porte & Richards (Reference Porte and Richards2012) contend that both quantitative and qualitative replications are feasible and important, they also point out the challenges involved in a qualitative replication. This is because specifications of data collection and analysis of an original study need to meet rigorous standards to allow replication, but attempts to improve those standards for qualitative research are relatively recent and ‘less obvious’ (p. 289). The authors cite Atkins et al.’s (Reference Atkins, Lewin, Smith, Engel, Fretheim and Volmink2008) meta-analysis of 44 ethnographic studies, which shows that the majority of studies did not provide sufficient descriptions of sampling, the researcher's role, or data analysis process, therefore making comparisons across findings difficult. Acknowledging this challenge, this paper focuses on a quantitative paradigm for replication. Specifically, we will present an argument for replication of studies that attempt to identify individual and contextual factors as explanations for pragmatic development. We will turn to the body of study abroad research to locate original studies for replication.
2.2 Replication in pragmatic development in a study abroad context
There are several reasons why we propose replications in the study abroad research. First, among various learning contexts, the study abroad context has accumulated the most empirical findings by far in L2 pragmatics and thus provides a rich empirical ground for replication. Because a critical mass of findings has been reached, researchers are well-situated to formulate a replication research agenda. Given the sizable pool of studies available, researchers can apply rigorous criteria to assess the quality of candidate studies and select only those that have met methodological standards for a replication.
Second, contextual affordance and individual variation have been the two primary topics of investigation in the study abroad research, but studies exploring these two topics together in a quantitative design are still rare and thus merit a replication to confirm (or disconfirm) existing findings. Findings from qualitative studies have reached a generalization that it is not the study abroad context per se that impacts pragmatics learning: it is the interaction between resources in the context and individual learners’ characteristics that shapes development (e.g., Barron Reference Barron2003; Iwasaki Reference Iwasaki2011; Shively Reference Shively2011; Hassall Reference Hassall2015; Taguchi Reference Taguchi2015). For instance, two learners of Indonesian in Hassall's (Reference Hassall2015) study showed contrasting development with Indonesian address terms. One participant acquired address terms relatively quickly because his positive stance to the local environment led to an extensive social network for practice. Another participant did not improve because the negative attitudes she developed towards the community, along with her sustained connection to her home country, isolated her from the local context and restricted her pragmatic practice. These findings indicate that individual characteristics often mediate learners’ access to social practice in the target language context. As a result, not all learners benefit equally well from the social practice available in the context or show uniform gains in their pragmatic abilities.
What is limited in the current literature is quantitative research that can contribute to such generalizations. Only a few quantitative studies have investigated how context and individual factors jointly affect pragmatic development. In Matsumura's (Reference Matsumura2003) study, L2 English learners’ choice of advice-giving expressions improved over a year abroad, but this gain was mediated by self-reported exposure to English and general proficiency. In Taguchi's (Reference Taguchi2008) study, L2 English learners’ comprehension of implicature improved over time while abroad. Comprehension speed (assessed by response times) significantly correlated with the self-reported language contact and a cognitive ability (i.e., lexical access skill).
Replication research can add to these findings to confirm (or otherwise) contextual and individual effects in pragmatic competence. Rather than simply asking the long-standing question of whether study abroad is effective for pragmatics learning, a replication study with explicit focus on context and individual factors will help us theorize mechanisms behind pragmatic development while abroad. Such a replication will generate useful knowledge for instructors and coordinators involved in a study abroad program. Replication of the pertinent studies can reveal what resources are available in a study abroad context, whether learners’ individual characteristics promote efficient use of those resources, and how learners’ pragmatic competence develops as a byproduct of this context–individual intersect in a study abroad program.
3. The original studies and suggested approaches to replication
3.1 Study 1: Taguchi, Xiao & Li (Reference Taguchi, Xiao and Li2016)
This study, published in The Modern Language Journal, investigated the effects of a contextual factor (i.e., language contact) and an individual factor (i.e., intercultural competence) on the development of speech act production in L2 Chinese in a study abroad program. A line of studies revealed that exposure while abroad, assessed quantitatively via perceived amount of language contact or frequency of participation in social activities, was related to pragmatic gains (e.g., Matsumura Reference Matsumura2003; Taguchi Reference Taguchi2008; Bardovi-Harlig & Bastos Reference Bardovi-Harlig and Bastos2011; Taguchi, Li & Xiao Reference Taguchi, Li and Xiao2013). These studies typically used a survey as a time-on-task measure by asking participants to report the amount of time spent using L2 over a range of social activities. Although these studies found that availability of language practice in a study abroad program has impact on pragmatics learning, a question remains as to whether availability automatically leads to accessibility to practice. In other words, are learners able to take advantage of practice opportunities available in a study abroad context to make progress in pragmatics learning? To answer this question, individual characteristics, which are likely to promote or hinder learners’ access to practice, are a critical source of influence to consider. Taguchi et al.’s study took a quantitative approach to investigate this three-way relationship among context, individual characteristics, and pragmatics learning.
The study examined intercultural competence as an individual characteristic. Intercultural competence is defined as abilities for ‘perform[ing] effectively and appropriately when interacting with others who are linguistically and culturally different from oneself’ (Fantini Reference Fantini2006: 12). Intercultural competence is a critical personal quality because opportunities for cross-cultural communication and cultural learning are abundant in a study abroad context. Sufficient intercultural competence is likely to lead to a successful cultural adjustment, which results in a great amount of social contact in the local community. Pragmatic knowledge may develop as a byproduct of this cultural adjustment process, because exposure to diverse social situations and communication styles, promoted by a large amount of social contact, serves as a resource for pragmatics learning. Indeed, Taguchi et al.’s study found this path from intercultural competence to social contact, leading to pragmatic gains.
Participants were 109 American college students studying Chinese in a semester-long study abroad program in Beijing. Their Chinese proficiency was at the intermediate level as assessed by Level 4 of the HSK Test (a standardized Chinese proficiency test). Based on Kelley & Meyers’ (Reference Kelley and Meyers1995) framework, intercultural competence was operationalized as cross-cultural adaptability, referring to one's potential to succeed in cultural adjustment. It was assessed with a 50-item Likert scale survey consisting of four dimensions: flexibility/openness, emotional resilience (ability to react positively to new experiences), personal autonomy (sense of self as a unique entity), and perceptual acuity (ability to pay attention to verbal and nonverbal cues during communication). Social contact was assessed with a 14-item survey in which participants reported the estimated amount of time per week spent on various social activities in Chinese (e.g., communicating with Chinese friends). Pragmatic knowledge was measured with a spoken task, which assessed participants’ ability to produce speech acts (e.g., requests, refusals). Speech acts were recorded, transcribed, and rated by two native Chinese speakers on six-point rating scales, which reflected three aspects: appropriateness of expressions, clarity of communicative function, and grammaticality. The three measures (i.e., cross-cultural adaptability survey, social contact survey, and speech act task) were administered twice at the beginning and end of a semester study abroad period to examine the effects of changes in cross-cultural adaptability and social contact on changes in speech act production. Latent Growth Curve Modeling (LGM) (Bollen & Curran Reference Bollen and Curran2006) was used for data analysis. LGM automatically takes time as an independent variable to analyze relationships among variables across different time points.
LGM results revealed that cross-cultural adaptability and social contact, when combined, explained 26% of gains in speech act production. The goodness-of-fit analyses showed that social contact had a direct effect on speech act changes, while cross-cultural adaptability had an indirect effect through social contact. In other words, social contact mediated the effects of cross-cultural adaptability to gains in speech act knowledge. A larger gain in cross-cultural adaptability (individual factor) led to increased social contact (contextual factor), resulting in improved speech acts performance (pragmatic knowledge).
3.2 Approaches to replication of Study 1
The finding about the differential impact of social contact and intercultural competence on pragmatic gains (i.e., direct effect of social contact and indirect effect of intercultural competence via social contact) is important as we conceptualize the sequence and interaction of contextual and individual effects on pragmatic development. The finding indicates that availability of contextual resources does not guarantee accessibility to those resources. Learners can benefit from social experiences while abroad, but such benefit depends on their individual characteristics. Although qualitative studies revealed the interaction between context and individual characteristics (e.g., Shively Reference Shively2011; Hassall Reference Hassall2015), these studies did not reveal causality relationship or directionality between these two factors. Hence, the findings of this study have clear implications for researchers and practitioners in a study abroad program: amount of social contact has a decisive effect for pragmatics learning, but individual characteristics need to be moderated to bring out the best of the effect. The indirect effect of intercultural competence suggests that intercultural training could serve as a valuable pre-departure program for students studying abroad. Teachers can emphasize that preparation on intercultural skills can result in many opportunities for language contact, which leads to language development while abroad (for a sample curriculum, see Jackson Reference Jackson2015 and Garrett-Rucks Reference Garrett-Rucks2016).
There are several directions for a replication study. First, given that this is probably the only study to date that identified the causality link among social contact, intercultural competence, and pragmatic development, approximate replication that closely follows this study's procedures will help us assess the robustness and generalizability of the findings, while maintaining our focus on the original findings. Such a study needs to commit to the longitudinal design by assessing variables via multiple data points. It is also important to use the same measures of social contact and intercultural competence, since a number of similar instruments exist in the literature (Fantini Reference Fantini and Jackson2012). A close approximation of the original study will help us discern the stability of the knowledge produced in the original study. Because approximate replication allows changes on non-target variables, participant population can be altered, involving students of a different target language and/or in a different study abroad program. However, their proficiency level should be kept at the intermediate level as defined in the original study because proficiency can serve as another factor that affects learners’ access to social contact and interaction (e.g., Matsumura Reference Matsumura2003).
A further possibility is a conceptual replication that aims to test the original study's findings using different target variables. The original study revealed that the amount of social contact – a contextual factor – had direct impact on pragmatic development, but intercultural competence – an individual factor – had indirect impact on pragmatic gains through social contact. This causality link can be the focus of replication by using different contextual and individual variables. The original study operationalized the contextual factor as time-on-task by using a survey to document reported amount of weekly hours spent on different social activities. Because this study administered the survey at two time points (beginning and end of the study abroad period), more frequent documentation of language contact via journals and logs can usefully generate more reliable time-on-task data. Alternatively, learning context can be operationalized and documented using different indicators. Previous literature identified a range of indicators for successful cultural integraton, including number of intercultural friendships that people develop while abroad (Hammer Reference Hammer2005), amount of intercultural activities involved (Van Oudenhoven & Van der Zee Reference Van Oudenhoven and Van der Zee2002), and degree of intercultural cooperation (Mor, Morris & Joh Reference Mor, Morris and Joh2013). These factors can present alternative ways to assess learners’ experiences with the study abroad context.
Similarly, a variety of individual difference factors can be incorporated into the original study design. Other than intercultural competence, a number of individual characteristics have been examined as factors that may explain the variation among learners in their process and outcome of pragmatic development. Those factors include cognitive variables (e.g., aptitude, working memory, and lexical access skill), general proficiency, motivation, willingness-to-communicate, and personality, with valid and reliable measures available for assessment. Existing findings have revealed both descriptive and predictive relationships between these variables and pragmatic competence in a study abroad context (see Taguchi & Röever Reference Taguchi and Röever2017 for a review of the findings). A question for replication is whether these individual characteristics exhibit similar explanatory power as intercultural competence, and produce a significant indirect effect to pragmatic development via a contextual factor. By incorporating other individual variables in replication, we can test the general proposition emerging from the original study: individual learner characteristics, be they intercultural competence or other personal traits, can facilitate learners’ access to opportunistic resources available in a study abroad context (e.g., contact and interaction with local members), and such access to L2 practice eventually leads to pragmatic development. If such a causality relationship is confirmed using different variables (but in the same design using LGM), the finding will strengthen the generalizability of the original study's claim. On the other hand, if such a causality claim is not confirmed, we will know that not all individual characteristics can explain pragmatic development equally. Some variables play a greater role than others in promoting access to practice in a study abroad context.
3.3 Study 2: Bardovi-Harlig & Bastos (Reference Bardovi-Harlig and Bastos2011)
Published in Intercultural Pragmatics, this study investigated the effects of general proficiency, LoR (length of residence), and intensity of interaction on recognition and production of conventional expressions in L2 English in a study abroad context. Among these three variables, proficiency represents an individual learner factor, while LoR and intensity of interaction are directly related to the affordances of the study abroad context because they respectively index the potential and actual amount of contact with the target language. By and large, the influences of these three variables on L2 pragmatic competence have been investigated separately, resulting in three lines of research. Proficiency, for example, generally has a positive impact on pragmatic competence regardless of learning contexts, although its effects are also contingent upon factors such as specific pragmatic features, modality of performance, and sociopragmatic variables (see Xiao Reference Xiao2015 for a recent review). On the other hand, mixed findings have been reported on whether LoR influences various aspects of pragmatic competence (e.g., Félix-Brasdefer Reference Félix-Brasdefer2004; Warga & Schölmberger Reference Warga and Schölmberger2007). Finally, the role of amount of interaction in affecting pragmatic competence has only recently been explored quantitatively (e.g., Taguchi Reference Taguchi2008; Taguchi, Li & Xiao Reference Taguchi, Li and Xiao2013). Hence, the positive impact of this variable, as reported in a few existing studies, awaits future research to confirm the generalizability.
Although proficiency, LoR, and amount of interaction have been shown to independently affect L2 pragmatic competence, it remains unclear which factor(s) plays a more important role in shaping specific aspects of pragmatic competence. This topic is particularly meaningful to explore in a study abroad context because the learning opportunities in such a context may or may not override the influence of proficiency on pragmatic competence. Moreover, conventional expressions constitute an ideal component of pragmatic competence to be investigated because of their frequent occurrence, community-wide use, and close ties to specific communicative contexts, all of which are affordances of the study abroad context.
In Bardovi-Harlig & Bastos's study, participants were 120 learners of L2 English recruited from four course levels (from low-intermediate to low-advanced) of an intensive English program in a U.S. university. In addition, 49 native speakers of American English (35 undergraduates and 14 ESL instructors) were recruited from the same university to ensure community-wide use of the targeted conventional expressions. The researchers developed three data collection instruments. The first was a computerized recognition task including 35 target conventional expressions (e.g., ‘no problem’) and 25 modified expressions (that differed from their conventional counterparts with one lexical/grammatical modification) as distracters (e.g., ‘no problems’). The learners listened to each expression twice and indicated their perceived frequency of encountering by choosing one of the three options: ‘often’, ‘sometimes’, and ‘never’. The second instrument was a computerized production task with 32 scenarios, among which 20 led to the production of 22 conventional expressions by at least 50% of the native speakers. These 22 expressions became the targets in analyzing the learner data. When responding to the production task, participants simultaneously heard and read scenario descriptions on computer screens and were prompted to provide oral responses. The third instrument was a background survey used to collect information about LoR (i.e., months of stay in the U.S.) and amount of target language contact through speaking and listening activities (i.e., talking to native speakers, talking to international students, and watching American TV).
Two separate repeated logistic regressions were performed for recognition and production scores of conventional expressions. Because the dependent variable of the logistic regression needs to be binary scored, the three levels of perceived frequency of encountering conventional expressions were converted into two categories: a score of 1 (incorporating ‘often’ and ‘sometimes’) and a score of 0 (‘never’). Responses to six of the 35 targeted items of the recognition task were removed from statistical analysis due to a ceiling effect (i.e., all learners received a score of 1). The independent variables of the logistic regression model were proficiency, LoR, intensity of interaction, and item. The results showed that, besides the significant effect of item, intensity of interaction was the only significant predictor of performance on the recognition task. In analyzing the production data, the learner responses were binary scored for 1 (target expressions produced) and 0 (no target expression produced). The same four independent variables were entered into a repeated logistic regression procedure, which revealed three significant predictors: item, proficiency, and intensity of interaction on production.
3.4 Approaches to replication of Study 2
The results showing varied contributions of proficiency, LoR, and intensity of interaction to recognition and production of conventional expressions indicate the relative importance of contextual and individual influences on different aspects of pragmatic competence in a study abroad context. The results suggest that actual access to learning opportunities (indicated by intensity of interaction), rather than mere availability of such opportunities (indicated by LoR), is a meaningful predictor of pragmatic competence. This finding adds to the limited empirical evidence showing a positive impact of interaction amount on pragmatic development (e.g., Matsumura Reference Matsumura2003; Taguchi Reference Taguchi2008; Taguchi et al. Reference Taguchi, Li and Xiao2013; Taguchi et al. Reference Taguchi, Xiao and Li2016), and reinforces Kasper & Rose's (Reference Kasper and Rose2002) conclusion that LoR is not a reliable predictor of pragmatic competence.
Because Bardovi-Harlig & Bastos's study is probably the only one that directly compares the effects of both contextual variables, the findings should be further tested through approximate replication that can overcome some of the limitations of the original study. For example, as Bardovi-Harlig & Bastos noted, the range of LoR in their study was relatively narrow, with the majority of the learners (116 out of 120) having had 0 to 8 months of stay in the U.S.; meanwhile, existing studies (e.g., Bouton Reference Bouton1994; Kecskes Reference Kecskes2000; Félix-Brasdefer Reference Félix-Brasdefer2004) have shown that LoR is more likely to impact pragmatic competence with a longer period. Hence, expanding the range of LoR in participant sampling, while keeping other variables constant, will allow a better evaluation of the contribution of LoR, compared to intensity of interaction, to pragmatic competence.
Another interesting finding in Bardovi-Harlig & Bastos's study was the differential impact of proficiency (an individual factor) and intensity of interaction (a contextual factor) on different aspects of pragmatic competence: whereas recognition of conventional expressions was affected by intensity of interaction but not by proficiency, production of conventional expressions was influenced by both proficiency and intensity of interaction. These results suggest a potential modality effect. Intensity of interaction can override proficiency in predicting the receptive aspect of pragmatic competence, but when it comes to the productive aspect, both intensity of interaction and proficiency are key predictors.
One option to test the generalizability of this modality effect is through approximate replication. This involves expanding the range of learners’ proficiency levels to see whether proficiency can be a significant predictor of recognition (in addition to production) of conventional expressions. In Bardovi-Harlig & Bastos's study, the lowest proficiency level was low-intermediate. Learners at this level were probably already familiar with many of the targeted expressions in the recognition task (e.g., ‘You too’, ‘Thank you’, ‘I'm sorry’, ‘no problem’), leaving little room for improvement. The fact that the responses to six items of the recognition task were removed from the statistical analysis due to a ceiling effect is a good indication of the simplicity of (some of) the targeted expressions for the learners. Hence, including elementary-level learners would enable a better examination of the role of proficiency, relative to the role of intensity of interaction, in influencing recognition (and production) of conventional expressions.
A further option to check the modality effect is through conceptual replication that taps the processing dimension, in addition to the knowledge dimension, of pragmatic competence. Bardovi-Harlig & Bastos focused solely on learners’ knowledge of conventional expressions, and they did not examine learners’ processing capacity of such knowledge (i.e., how efficiently learners are able to access relevant pragmatic knowledge). Knowledge and processing are distinct dimensions of pragmatic competence (Bialystok Reference Bialystok, Kasper and Blum-Kulka1993; Taguchi Reference Taguchi2012), which are differentially affected by various contextual and learner factors including amount of L2 contact and proficiency (e.g., Taguchi Reference Taguchi2012). Hence, including both dimensions of pragmatic competence in one research design would allow us to gain a fuller picture regarding the effects of proficiency and intensity of interaction on recognition and production of conventional expressions. In carrying out such a conceptual replication, researchers can keep the measures of pragmatic knowledge adopted in Bardovi-Harlig & Bastos's study, but use additional measures of processing capacity by drawing on existing studies (e.g., Taguchi et al. Reference Taguchi, Li and Xiao2013; Edmonds Reference Edmonds2014). In these studies, reaction times and planning times were used as indicators of processing capacity involved in recognizing and producing conventional expressions, respectively.
Finally, Bardovi-Harlig & Bastos only investigated direct effects of proficiency, LoR, and intensity of interaction on recognition and production of conventional expressions. This approach might have overlooked meaningful indirect effects of these contextual and learner factors. Such indirect effects are highly plausible and worth empirical effort given the existing research findings. Matsumura (Reference Matsumura2003), for example, reported that exposure to the target language directly contributed to pragmatic development (assessed as developing knowledge of appropriate advice-giving expressions), and that proficiency had an indirect effect on pragmatic gains via exposure. Hence, both direct and indirect effects need to be considered in evaluating the relative importance of contextual and individual factors in shaping pragmatic competence. To this end, a conceptual replication using Structural Equation Modeling (Byrne Reference Byrne2009) can help discern the possible direct and indirect causal relationships among proficiency, LoR, intensity of interaction, and ability to recognize and produce conventional expressions. In this replication, the target pragmatic features can be expanded (e.g., to include various speech acts), and the measure of intensity of interaction can be refined to incorporate more detailed descriptions of interactive and non-interactive activities (e.g., Moyer Reference Moyer2005). Moreover, as mentioned in the discussion of Taguchi et al.’s (Reference Taguchi, Xiao and Li2016) study, researchers can introduce alternative measures to document learner's quantity and quality of contact with the target language during study abroad.
4. Conclusion
Although L2 pragmatics is a relatively young field of study, several replication studies already exist, including three highly influential ones that used the same instrument to examine the effects of the study abroad context on pragmatic knowledge (i.e., Bardovi-Harlig & Dörnyei Reference Bardovi-Harlig and Dörnyei1998; Niezgoda & Röever Reference Niezgoda, Röever, Rose and Kasper2001; Schauer Reference Schauer2006). These three studies, however, treated the study abroad context as a ‘black box’ (Taguchi Reference Taguchi2016), equating availability of learning opportunities with actual access to such opportunities. Since we have seen inconsistent findings regarding the role of the study abroad context in affecting pragmatic development, we need to focus on whether and how individual learners can take advantage of the affordances of the study abroad environment. Future replication studies, such as those discussed in this paper, can shed light on what kind of contextual and individual factors are at play, as well as how such factors exert their influences on L2 pragmatic development while abroad.
Naoko Taguchi is Professor in the Modern Languages Department at Carnegie Mellon University. Her research interests include pragmatics, second language acquisition, English-medium education, and intercultural competence. Her recent books include Developing interactional competence in a Japanese study abroad context (2015/Multilingual Matters) and Second language pragmatics (2017/Oxford University Press).
Shuai Li is Associate Professor of Chinese and Chinese Program Coordinator at Georgia State University. His research interests include interlanguage pragmatics and second language acquisition of Chinese. He has published in journals such as Language Learning and The Modern Language Journal.