INTRODUCTION
Models of communicative competence (e.g., Bachman & Palmer, Reference Bachman and Palmer2010) consider that pragmatic competence (i.e., the ability to use language appropriately in context) is equally important as other areas of language competence such as grammar and vocabulary. This conceptualization of communicative language ability has been adopted in the descriptions of proficiency levels in language assessment. The relationship between proficiency and pragmatic competence has been a major research interest in the field of second language (L2) pragmatics. Several researchers consider that a threshold level of proficiency is necessary to perform a pragmatic function (for a review, see Bardovi-Harlig, Reference Bardovi-Harlig2013; Kasper & Rose, Reference Kasper and Rose2002; Taguchi & Roever, Reference Taguchi and Roever2017; for an opposite view, see Bardovi-Harlig & Stringer, Reference Bardovi-Harlig and Stringer2017). To test this assumption, many studies used a cross-sectional design and investigated pragmatic competence across different proficiency levels (e.g., Dalmu & Gotor, Reference Dalmau and Gotor2007; Félix-Brasdefer, Reference Félix-Brasdefer2007; Taguchi, Reference Taguchi2007). However, in a longitudinal design (i.e., collecting data from one participant group repeatedly over time), the relationship between proficiency and pragmatic competence may be more complex because these two variables may not develop in parallel, meaning that the proficiency effect on pragmatic competence may change over time, as evidenced in a few longitudinal studies (Li, Reference Li2014; Matsumura, Reference Matsumura2003; Taguchi, Reference Taguchi2013a). Hence, investigating a changing relationship between proficiency and pragmatic competence can help us understand pragmatic development as an adaptive process within the L2 systems.
Adopting the longitudinal approach, this study models the relationship between proficiency and pragmatic competence across two time points. Unlike other longitudinal studies, this study focuses on the causal relationships (probabilistic causation based on statistical inference) between proficiency subskills (i.e., listening, speaking, reading, and writing) and pragmatic development because proficiency subskills represent the ability to process linguistic knowledge in different modalities. Speaking and listening are involved in oral language processing, while writing and reading are associated with written language processing and thus represent different aspects of language processing in cognition (see Hulstijn, Reference Hulstijn2015, for basic and higher language cognition). Therefore, proficiency subskills may differentially contribute to changes of pragmatic competence measured in different modalities (e.g., a written task vs. an online spoken task). This study addresses this hypothesis in a Chinese study-abroad context.
BACKGROUND
Bachman and Palmer (Reference Bachman and Palmer2010) contend that language knowledge consists of organizational knowledge and pragmatic knowledge. The former refers to the knowledge for constructing utterances and texts (e.g., grammar, vocabulary, and discourse), while the latter consists of functional and sociolinguistic knowledge on how to use linguistic forms in context. This conceptualization implies that learners with more linguistic knowledge are generally more advantaged because they have a larger repertoire of linguistic forms to choose from when they perform pragmatic functions. In the following section, we will review existing studies on proficiency effects on pragmatic competence, focusing on two areas: perception (ability to recognize pragmatic functions) and production (ability to convey pragmatic functions).
POSITIVE EFFECTS OF PROFICIENCY ON PRAGMATIC COMPETENCE
Many cross-sectional studies revealed positive effects of proficiency on pragmatic perception and production. Among those studies that examined pragmatic perception, Bardovi-Harlig and Dörnyei’s (Reference Bardovi-Harlig and Dörnyei1998) study found that high-proficiency learners of L2 English (proficiency determined by learners’ self-assessment) recognized more pragmatic than grammatical errors embedded in speech acts regardless of learning context (ESL vs. EFL). Garcia (Reference Garcia2004) found that high-proficiency learners (proficiency measured by TOEFL) recognized speech acts more accurately than their low-proficiency counterparts.
Studies that examined pragmatic production also found positive proficiency effects. Using a discourse completion task (DCT), Dalmu and Gotor (Reference Dalmau and Gotor2007) revealed that the high-proficiency learners (proficiency determined by a placement test) produced targetlike English apology strategies more frequently than low-proficiency learners. Taguchi (Reference Taguchi, Salazar and Marti2013b), however, used a role-play task to compare production of English refusals between high- and low-proficiency learners (proficiency measured by TOEFL), and found positive proficiency effects on appropriateness and fluency of speech act production.
LIMITED OR MIXED EFFECTS OF PROFICIENCY ON PRAGMATIC COMPETENCE
A few studies revealed limited or mixed effects of proficiency on pragmatic competence. Niezgoda and Roever (Reference Niezgoda, Roever, Rose and Kasper2001) replicated Bardovi-Harlig and Dörnyei’s (Reference Bardovi-Harlig and Dörnyei1998) study and found that, contrary to Bardovi-Harlig and Dörnyei’s results, high-proficiency learners (as determined by in-house tests) recognized fewer pragmatic errors than their low-proficiency counterparts. These contradicting findings suggest that proficiency may not always lead to better pragmatic performance. Several studies on pragmatic production also supported this claim. Using a DCT, Allami and Naeimi (Reference Allami and Naeimi2011) reported that Iranian learners with higher English proficiency (measured by TOEFL) transferred more L1 social norms to L2 requests. Taguchi (Reference Taguchi2007), by contrast, used a role-play task to examine production of English requests in two types of situations: PDR-low (equal power status, small social distance, and low imposition) and PDR-high (higher power status, large social distance, and high imposition). She found that higher proficiency (determined by TOFEL scores and instructor ratings) led to higher scores on appropriateness of speech acts, but the proficiency effect was much smaller for PDR-high situations, indicating that proficiency effect is mediated by different task items. Similarly, using a role-play task, Félix-Brasdefer (Reference Félix-Brasdefer2007) examined L2 Spanish learners’ requests and found that high-proficiency learners (proficiency determined by course levels) used fewer direct requests and instead used more conventional indirect requests than low-proficiency learners. However, there was no between-group difference in their use of unconventional indirect requests (e.g., hinting).
EFFECTS OF PROFICIENCY AND TARGET LANGUAGE EXPERIENCE ON PRAGMATIC COMPETENCE
Several cross-sectional studies examined the relationships among proficiency, target language experience, and pragmatic competence. These studies examined length of stay in the target community (LOS) and intensity of language use (ILU, measured by weekly hours using the target language) as factors that can affect pragmatic competence. Among the studies that examined pragmatic perception, Xu, Case, and Wang (Reference Xu, Case and Wang2009) replicated Bardovi-Harlig and Dörnyei’s (Reference Bardovi-Harlig and Dörnyei1998) study and found that, compared to LOS, proficiency (measured by TOEFL) had a larger effect on learners’ abilities to recognize pragmatic and grammatical errors. In terms of pragmatic production studies, Taguchi (Reference Taguchi2011) found a positive effect of proficiency on speech act production, overriding the effect of study abroad experience. Similarly, Roever, Wang, and Brophy (Reference Roever, Wang and Brophy2014) found a positive effect of proficiency on speech act production, comprehension of implicatures, and recognition of formulaic expressions, overriding the effects of LOS, gender, and multilingual background. Bardovid-Harlig and Bastos (Reference Bardovi-Harlig and Bastos2011), however, examined the effects of proficiency, LOS, and ILU on recognition and production of English conventional expressions. They found that ILU affected both recognition and production of conventional expressions. However, proficiency only affected production but not recognition, and LOS had no effect. These findings suggest that proficiency has a larger impact on pragmatic production than perception. However, not many studies have examined how task modality (e.g., productive vs. receptive skills) mediates the impact of proficiency on pragmatic competence.
LONGITUDINAL STUDIES ON THE EFFECTS OF PROFICIENCY ON PRAGMATIC DEVELOPMENT
Compared with cross-sectional studies, longitudinal studies examining proficiency impact on pragmatic development are scarce. To our knowledge, there are only three such studies (Li, Reference Li2014; Matsumura, Reference Matsumura2003; Taguchi, Reference Taguchi2013a). Li (Reference Li2014) used a computerized oral DCT to compare changes of request production in L2 Chinese learners with intermediate and advanced proficiency (proficiency determined by course levels) over one semester abroad. Both groups gained on appropriateness ratings over time, but the advanced group outperformed the intermediate group at the end of the study abroad. Moreover, only the advanced group gained on fluency of speech act production (assessed by speech rate), and neither group gained on planning speed. These findings suggest varied proficiency effects on the development of pragmatic knowledge (appropriateness) and processing (speech rate).
Matsumura (Reference Matsumura2003), by contrast, measured proficiency (using TOEFL) and L2 English learners’ ability to select an appropriate advice-giving expression three times over one year abroad. He found no direct effect of proficiency on learners’ selection of advice-giving expressions but found an indirect effect of proficiency through exposure to English (assessed on a self-report survey). These findings suggest that higher proficiency learners can achieve greater pragmatic gains if they seek more opportunities to use target language while abroad.
Finally, Taguchi (Reference Taguchi2013a) investigated whether changes in L2 English speech acts were affected by changes in proficiency, lexical access speed, and orientation toward English study over one academic year. She found positive effects of lexical access speed and orientation toward English study on changes in appropriate and fluent speech act production, but no effects of proficiency (measured by TOEFL). Hence, when proficiency and pragmatic competence are treated as time-varying variables, the impact of proficiency may not appear as strong as found in cross-sectional studies.
Positive proficiency effects on pragmatic development found in these studies should be interpreted with caution for two reasons. First, while the instruments used to assess pragmatic competence require different proficiency subskills (e.g., written DCTs requiring reading and writing skills; role-plays requiring speaking skills), almost no studies operationalized proficiency in different subskills. Instead existing studies used general proficiency measures to assess composite proficiency (e.g., TOEFL scores, course levels). This is a serious neglect because proficiency subskills may contribute to L2 pragmatics differently because specific pragmatic tasks require activation and execution of language knowledge in different modalities. For example, the impact of speaking and listening subskills on pragmatic performance may be larger with tasks that require aural comprehension and oral production (e.g., a role-play) than with tasks that require text comprehension and written production (e.g., a DCT). However, no study has investigated the effects of different proficiency subskills on L2 pragmatics.
The second limitation in the existing practice is that most studies treated proficiency and pragmatic competence as static variables in a cross-sectional design. Consequently, findings of these studies cannot reveal whether changes in proficiency affect changes in pragmatic competence. A few longitudinal studies explored this issue, but the findings are inconclusive: Li (Reference Li2014) found a significant proficiency effect on appropriateness of request production, while Taguchi (Reference Taguchi2013a) showed no significant proficiency effect on appropriateness of speech act production. Given the paucity of existing findings, the longitudinal relationship between proficiency and pragmatic competence remains unclear and awaits more empirical efforts.
To fill these gaps, this study treats proficiency as a time-varying latent variable (a hypothetical construct) with four subskills (indicators): listening, speaking, reading, and writing. Likewise, this study treats pragmatic competence as a time-varying latent variable operationalized as production of speech acts measured by an online spoken task. Admittedly, speech acts cannot represent pragmatics at the discourse level, but they clearly involve the functional aspect of linguistic forms that distinguish pragmatics from semantics (Levinson, Reference Levinson and Huang2017). Hence, we consider speech acts as an essential component of L2 pragmatic knowledge.
This study applies the latent growth curve modeling (LGM) to generate the group-level pattern between changes of proficiency and pragmatic competence over two time points in three months because compared to repeated-measures ANOVA and regression analysis, LGM does not assume linearity or an equal covariance structure but it allows all variables to change and accepts measurement error. We address the following research question: What are the contributions of proficiency subskills to pragmatic development among L2 learners of Chinese over three months abroad in China?
METHODS
This study was part of a larger study that examined the relationship among intercultural competence, social contact, and pragmatic development (Taguchi, Xiao, & Li, Reference Taguchi, Xiao and Li2016). The two studies shared participants and the measure of pragmatic competence.
PARTICIPANTS
Participants were 109 American learners of Chinese enrolled in a study abroad program in Beijing. They were 49 females and 60 males (age range: 19–23). The program curriculum included a comprehensive Chinese class, a conversation class, and a one-on-one tutorial session. There was no explicit pragmatics instruction. Participants lived in on-campus dormitories with their peers.
MEASURE OF PROFICIENCY
Participants’ proficiency was measured with Hanyu Shuiping Kaoshi (HSK), a standardized L2 Chinese proficiency test. In this study, the reliability of HSK was high as indicated by the Cronbach’s α coefficient of .93. HSK has a written test (from level 1 to 6) including sections assessing listening, reading, and writing subskills and a separate speaking section (elementary, intermediate, and advanced levels) (score range for each section: 0–100, score range for total HSK: 0–400). The participants took the level 4 written test and the intermediate-level speaking test because they had on average two years of formal Chinese study (range: 1–5 years). In the level 4 written test, the listening section included three parts: comprehension of single utterances (k = 10), two-turn dialogs (k = 15), and four-turn dialogs (k = 20); the reading section had three parts: completing sentences with proper words (k = 10), putting sentences into the right order (k = 10), and choosing correct answers to comprehension questions (k = 20); and the writing section included two parts: putting words into the correct order (k = 10) and using words to make sentences according to pictures (k = 10). The intermediate-level speaking test had three parts: repeating sentences after hearing them (k = 10), describing pictures (k = 2), and answering prescribed written questions (k = 2). None of the HSK items targeted on pragmatics. The participants’ initial total HSK scores ranged from 122.5 to 337.5 with an average of 228.7 (SD = 51.2), suggesting that their initial level of proficiency was from intermediate-low to advanced-low.
MEASURE OF PRAGMATIC COMPETENCE
A computerized oral DCT (CODCT) was used to measure pragmatic competence. The CODCT included 24 speech act items (see Taguchi et al., Reference Taguchi, Xiao and Li2016 for all items). The targeted pragmalinguistic forms in the CODCT did not appear as target linguistic forms in the textbooks used by the participants. Among the 24 items, 12 were more conventional items (routines) where realization of speech acts required production of fixed expressions. For example, (要/想/得) 寄包裹 (want to mail a parcel) is typically used, hence linguistically conventionalized, in the situation of mailing a parcel at a post office. These items were adapted from Taguchi, Li, and Xiao (Reference Taguchi, Li and Xiao2013). See a sample item in the following text.
EXAMPLE 1:
At a post office, you want to send a parcel. What would you say to this clerk?
The other 12 items were less conventional items where speech act realization is characterized with more variability in the choice of linguistic forms and semantic moves. Items in this category thus do not rely on single fixed expressions. There were three speech act types in this category: requests (k = 4), refusals (k = 4), and compliment responses (k = 4). The scenarios used to elicit these speech acts included two types of relationships: friend–friend (equal-power-status) and student–professor (unequal-power-status). These items were developed based on existing literature (e.g., Li, Reference Li2012). See a sample item in the following text.
EXAMPLE 2:
You meet your friend Xiao Li after class. Xiao Li invites you to a dinner with his friend, but you don’t want to go. What would you say to Xiao Li?
A written version of the CODCT was piloted among 20 Chinese native speakers. They were asked to judge whether candidate scenarios were common in their daily lives (yes/no) and to write down what they would say in each situation. Situations that more than 50% of the native speakers considered common and that elicited intended speech acts were included in the finalized CODCT. Similarly, formulaic expressions produced by more than 50% of the native speakers were included in more conventional items. A picture depicting each scenario was provided to facilitate visualization. Participants read and listened to a scenario description in English with a picture on the computer screen. Following a beeping sound emitted three seconds after the audio recording, the participants started to record their oral responses when the picture was still on (but not the written scenario). When finished, they clicked on the “finished” button to proceed. To minimize practice effect, an alternate version of the CODCT was created by changing the original order of scenarios, proper names, and voice of the narrator. Items were also counterbalanced between the two versions.
DATA COLLECTION AND ANALYSIS
Participants completed the HSK and CODCT at the beginning and the end of their stay in China. Each data collection lasted for two days with day one for the HSK and day two for the CODCT.
A six-point rating scale was developed to evaluate speech act responses (score range: 0–144 for the entire CODCT). Each band on the scale focused on appropriateness of expression, clarity of communicative function, and grammaticality (see Taguchi et al., Reference Taguchi, Xiao and Li2016 for the scale). Two Chinese native speakers familiar with pragmatics rated the responses. Discrepancies in ratings by two points or more were resolved through discussion.
Validation of the CODCT (the items and rating scale) followed Cronbach’s (Reference Cronbach and Linn1989) strong version of construct validity (p. 162). The representation of pragmatic competence was established based on the latest model of communicative competence (Bachman & Palmer, Reference Bachman and Palmer2010). The relationships between pragmatic competence and its indicators (i.e., less and more conventional speech acts) were tested at the measurement level of LGM (see Figure 1), where path coefficients between them were all significant. The relationships between the indicators and the rating scale (score interpretation) were validated in two aspects: scale functionality and interrater reliability. Scale functionality was checked by using the Rasch Model. The infit mean square statistics for both raters (1.20 and 0.81) were within the M ± (2 × SD) range (1.00 ± 2 × 0.27), indicating acceptable rater behavior. The interrater reliability of both CODCT versions were high as indicated by the Cronbach’s α coefficients of .90 and .83, respectively. With the aforementioned positive empirical evidence, the CODCT can be considered as a validated low-stakes test of pragmatic competence (operationalized as speech act production) for L2 Chinese learners in the context of this study.
Path model of Proficiency and Pragmatic Competence.
Note: LC= Less conventional speech arts. MC=More conventional speech arts.
LGM was used to generate the developmental pattern of proficiency subskills and pragmatic competence because it is a latent variable modeling technique specialized in analyzing longitudinal data (Beaujean, Reference Beaujean2014). LGM can generate group-level developmental patterns and individual trajectories. If individual trajectories are not of research interest (which is the case in this study), two data points are considered sufficient (Voelkle, Reference Voelkle2007). All data were modeled in R using the Lavaan package (Rosseel, Reference Rosseel2012).
RESULTS
DESCRIPTIVE STATISTICS AND PRE-POST COMPARISONS
Table 1 shows that the learners’ post-CODCT scores were all higher than their pre-CODCT scores.
Descriptive statistics of CODCT scores
Note: Pre-LC = prescore of less conventional speech act (range: 0–72); pre-MC = prescore of more conventional speech acts (range: 0–72); pre-CODCT = prescore of total CODCT (range: 0–144); post-LC = postscore of less conventional speech act (range: 0–72); post-MC = postscore of more conventional speech acts (range: 0–72); and post-CODCT = postscore of total CODCT (range: 0–144).
Paired t-test revealed that all score changes from pre- to posttest were significant: t = -12.6 for less conventional speech acts; t = -18.1 for more conventional speech acts, and t = -12.5 for both (df = 108, p < .01). The post-pre effect sizes were large as indicated by Cohen’s d: d = .67 for less conventional speech acts; d = .83 for more conventional speech acts; and d = .77 for both, indicating that pragmatic competence changed significantly over three months.
Moreover, as Table 2 shows, scores of less conventional speech acts, more conventional speech acts, and both speech acts combined were significantly correlated with each other (both pre- and posttest).
Correlations among less conventional, more conventional, and total speech act of pre- and post-CODCT data
Note: Pre-LC = prescore of less conventional speech act (range: 0-72); pre-MC = prescore of more conventional speech acts (range: 0–72); pre-CODCT = prescore of total CODCT (range: 0–144); post-LC = postscore of less conventional speech act (range: 0–72); post-MC = postscore of more conventional speech acts (range: 0–72); post-CODCT = postscore of total CODCT (range: 0–144).
** p < .01.
Table 3 shows that the learners’ post-HSK scores were all higher than their pre-HSK scores.
Descriptive statistics of HSK (proficiency test) scores
Note: Pre-LI = prescore of listening (range: 0–100); pre-SP = prescore of speaking (range: 0–100); pre-RE = prescore of reading (range: 0–100); pre-WR = prescore of writing (range: 0–100); pre-HSK = prescore of total HSK (range: 0–400); post-LI = postscore of listening (range: 0–100); post-SP = postscore of speaking (range: 0–100); post-RE = postscore of reading (range: 0–100); post-WR = postscore of writing (range: 0–100); and post-HSK = postscore of total HSK (range: 0–400).
Paired t-test revealed that all score changes from pre- to posttest were significant: t = -19.9 for the total HSK, t = -13.4 for the listening score, t = -14.9 for the speaking score, t = -14.1 for the reading score, and t = -17.5 for the writing score (df = 108, p < .01). The post-pre effect sizes were large as indicated by Cohen’s d: d = .84 for listening; d = .87 for speaking; d = .74 for reading; d = .69 for writing; and d = .82 for the total HSK, indicating that proficiency subskills changed significantly over three months. Moreover, as Table 4 shows, proficiency total scores and subscores were significantly correlated (both pre- and posttest).
Correlations among proficiency subskills and total scores of pre-HSK and post-HSK data
Note: LI = listening, SP = speaking, RE = reading, and WR = writing.
** p < .01.
RESULTS OF LATENT GROWTH CURVE MODELING
In our analysis, pragmatic competence (total CODCT score) had two indicators (scores of less and more conventional speech acts). Proficiency (total HSK score) had four indicators (scores of listening, speaking, reading, and writing sections). Relationships among these variables are illustrated in paths across two time points (see Figure 1).
Figure 1 includes all significant correlations and path coefficients (p < .05) found in the analysis. The total contribution of each proficiency subskill to pragmatic competence across time points was calculated to show how changes of an indicator variable affect changes of pragmatic competence (see Table 5).
Path coefficients of proficiency and its indicators
Note: *p < .05; **p < .01.
As Table 5 shows, proficiency explained 54.1% of the variance in pragmatic changes. Among the proficiency subskills, speaking was the strongest predictor of pragmatic changes, which had a path coefficient of .187, followed by listening (.132), reading (.121), and writing (.101). In other words, all proficiency subskills contributed to pragmatic development, while speaking and listening had more contributions than reading and writing. The goodness-of-fit of this path model was examined by using four criteria: χ2, root mean square error of approximation (RMSEA), expected cross-validation index (ECVI), and comparative fit index (CFI). Table 6 shows that the χ2 value was not significant, the RMSEA value was small (< .08), the ECVI value was small (< .20), and the CFI was close to 1, indicating that the path model was acceptable.
Results of goodness-of-fit of the path model of proficiency and pragmatic changes
Note: RMESA = root mean square error of approximation (range: 0–1); ECVI = expected cross-validation index (range: 0–1); CFI = comparative fit index (range: 0–1); and CI = confidence interval. *p < .05.
Responding to the research question, findings showed different contributions of proficiency subskills to pragmatic development (i.e., oral speech acts production): speaking (18.7%), listening (13.2%), reading (12.1%), and writing (10.1%). In other words, speaking contributed most to pragmatic development. The following examples can show the differences in speech act production between participants with high and low levels of speaking subskill. The difference between high (HS) and low speaking (LS) subskills was defined by a two-step cluster analysis. The HS group (N = 68) had a mean score of 85.32 on the speaking section of post-HSK, whereas the LS group (N = 41) had a mean score of 68.51. The silhouette measure of cohesion and separation was larger than .5 (out of 1), indicating a good cluster quality. The two groups were significantly different in their posttest CODCT score. Examples 3 and 4 show the differences between the HS and LS groups in the situation of mailing a parcel at a post office.
EXAMPLE 3 (HS)
你好, 我想寄东西。(Hello, I want to mail things.) (Rating: 5 out of 6)
EXAMPLE 4 (LS)
我想给东西给朋友。(I want to give things to friends.) (Rating: 3 out of 6)
In Example 3, the pragmatic function (to mail a parcel) was clearly achieved. Moreover, the use of 你好 (hello) was appropriate as a greeting at a service encounter. However, this learner used 寄东西 (to mail things) instead of the target routine 寄包裹 (to mail a parcel), indicating that his choice of pragmalinguistic forms (e.g., lexical items) remained nonnativelike, although he was able to convey the illocutionary force.
In contrast, in Example 4 from the LS group, the pragmatic function was judged as only somewhat achieved because 给东西给朋友 (to give things to friends) does not necessarily mean to mail a parcel. This can indicate an intention to offer a gift. Moreover, the word order of 给东西给朋友 (to give things to friends) shows L1 English transfer (verb + indirect object + preposition + direct object) because the word order in Chinese should be the direct object (朋友, friends) prior to the indirect object (东西, things).
DISCUSSION
Consistent with existing cross-sectional studies (e.g., Bardovi-Harlig & Bastos, Reference Bardovi-Harlig and Bastos2011; Taguchi, Reference Taguchi2011, Roever et al., Reference Roever, Wang and Brophy2014), our findings confirmed a positive effect of proficiency on pragmatic development. However, different from cross-sectional studies that treated proficiency as a categorical variable, this study treated both proficiency and pragmatic competence as continuous, time-varying latent variables. LGM allowed us to trace changes of these two variables in the same participant group with no assumption of linear development. As such, we were able to overcome the limitation of cross-sectional studies that compared pragmatic performance across proficiency groups (based on t-test, ANOVA and ANCOVA). Going beyond the predominant cross-sectional practice, our study revealed pragmatic development in relation to changes of proficiency subskills.
Notably, our findings are different from existing longitudinal studies that examined proficiency effects on pragmatic development. We found significant proficiency effects on pragmatic gains, but Matsumura (Reference Matsumura2003) and Taguchi (Reference Taguchi2013a) found no such effects. In Matsumura’s (Reference Matsumura2003) study, L2 English learners’ proficiency (measured by TOEFL) had no direct effect on changes in their recognition of appropriate advice-giving expressions. Instead, proficiency had an indirect effect through learners’ self-reported exposure to English. These different findings may be attributed to different modalities involved in the measures of pragmatic competence. Our study examined oral production of speech acts using an online spoken task (CODCT), whereas Matsumura’s study assessed perception of advice-giving expressions using a written multiple-choice questionnaire (MCQ). The MCQ measured learners’ perception by having them choose the most appropriate advice-giving form in a situation without having them produce any linguistic forms. Therefore, the MCQ was less demanding, linguistically and cognitively, because it only required offline language processing (i.e., reading sentences and judging their appropriateness). Hence, learners did not have to draw on a great amount of linguistic and cognitive resources to complete the task. Consequently, proficiency did not matter so much as long as learners had threshold abilities to comprehend simple sentences. In contrast, the CODCT used in this study was more demanding because it required the online language processing (i.e., producing utterances after activating knowledge and monitoring appropriateness of the utterances). When coping with these linguistic and cognitive demands, proficiency played a decisive role: increased proficiency led to better speech act production. In other words, different skill-specific cognitive demands involved in pragmatic tasks can affect the relationship between proficiency and pragmatic competence differently.
Another difference between Matsumura’s and our study is the treatment of proficiency. Although Matsumura’s study examined the effects of general proficiency (indicated by a composite TOEFL score), our study examined the effects of proficiency subskills (indicated by separate scores for all subskills) and showed their unique contributions to pragmatic development. In an online spoken task such as the CODCT, production of pragmatic knowledge is likely to draw on the processing of oral language to a large extent. Indeed, we found that speaking and listening subskills both had a large impact on changes of pragmatic competence, with speaking (18.7%) having a larger contribution than listening (13.2%). These findings make sense because the CODCT required learners to use listening skill (listening to situational descriptions) and speaking skill (oral production of speech acts). Because of these shared modalities, speaking and listening subskills had a greater impact on pragmatic development than reading and writing subskills in this study.
However, it is important to note that Taguchi (Reference Taguchi2013a) also used an online spoken task to measure pragmatic competence but found no proficiency effect. Rather, lexical access speed (speedy retrieval of word meaning) had a significant effect on the gains of speech act production. These contradictory findings should be interpreted carefully because the TOEFL used in Taguchi’s study as a proficiency measure did not include a speaking section. Hence, proficiency probably did not play a role in speech acts assessed with an online spoken task in her study. However, lexical access speed can be considered to underlie listening and speaking skills because it involves the speed of meaning activation at the word level. Theoretically, lexical access skill is part of basic language cognition (BLC) (Hulstijn, Reference Hulstijn2015), which is restricted to speaking and listening skills that represent the automatic processing of oral language in high-frequency lexical and grammatical items. Because faster lexical assess speed requires less working memory capacity to decode meaning at the word level, high-order processing such as oral pragmatic production can be optimized in an online spoken task, as shown in Taguchi’s study. Therefore, the significant effect of lexical assess speed can be interpreted as evidence supporting positive effects of speaking and listening subskills on changes of pragmatic competence, and this interpretation is consistent with the findings of this study.
In addition to the relatively large impact of speaking and listening subskills, reading and writing subskills also significantly contributed to the changes of pragmatic competence albeit with smaller effect sizes (12.1% for reading and 10.1% for writing). Reading and writing are part of higher language cognition (HLC) (ibid.). Different from BLC that represents the basic speaking and listening skills, HLC processes information encoded in low-frequency lexical and grammatical items. Therefore, in an online spoken task (e.g., CODCT) requiring the basic processing of colloquial expressions, contributions of reading and writing skills (measured by a written test) may be overtaken by speaking and listening skills, suggesting that contributions of proficiency subskills on pragmatic development are partially mediated by task modality.
LIMITATIONS AND FUTURE DIRECTIONS
This study has several limitations. First, the short study duration with two data points did not allow us to examine individual trajectories of pragmatic development for a long time span. Future studies should use multiple data points across a longer study period to trace the developmental patterns of proficiency and pragmatic competence at both group and individual levels. Such studies will help us evaluate the generalizability of the findings gleaned from the present study.
Another limitation is that pragmatic competence was examined at the utterance-level production of a few speech acts. Therefore, our findings may not be extrapolated to other types of speech acts or pragmatic competence assessed at the discourse level (e.g., sequential organization of communicative acts). Future studies should explore whether effects of proficiency subskills differ between the utterance and discourse level of pragmatic competence.
In addition, this study used a written DCT (rather than an oral DCT) to collect native speaker data at the pilot stage. The purpose of the pilot study was to make sure that the items were able to elicit specific speech act utterances from competent users of the target language. The data for the more conventional speech act items were also analyzed to see whether native speakers produced uniformed utterances. In the future, an oral DCT should be used at the pilot stage to be consistent with the instrument used in the main study.
Finally, although the present study revealed causal relationships between proficiency subskills and pragmatic development, we used rating scores of overall appropriateness of speech acts and did not focus on linguistic analysis of speech acts. We made this decision because frequency data for linguistic analysis are not suitable for statistical models assessing causality (e.g., LGM). Future studies can cultivate a creative method to quantify linguistics analysis in a way that the data can be used for LGM.
