Research-informed approaches toward second language (L2) pronunciation development in classroom and research contexts can be described according to two competing perspectives, the nativeness and intelligibility principles (Levis, Reference Levis2005). The nativeness principle targets nonaccented L2 speech, or speech free from linguistic features that might mark the speaker as nonnative. By contrast, the intelligibility principle focuses on L2 speech that is understandable to an interlocutor, despite the presence of a detectable accent. Because accented speech in adults is viewed as normal and unavoidable and the attainment of nativelike speech is uncommon (e.g., Abrahamsson & Hyltenstam, Reference Abrahamsson and Hyltenstam2009), there is a strong scholarly emphasis on the promotion of intelligible speech as the primary goal of L2 pronunciation development (e.g., Derwing & Munro, Reference Derwing and Munro2015; Levis, Reference Levis2005).
In support of this goal, several studies have examined which linguistic dimensions of L2 speech are associated with accentedness versus comprehensibility—key constructs in L2 speech research (Derwing & Munro, Reference Derwing and Munro2015). Accentedness, aligned with the nativeness principle, captures listeners’ perception of how strongly L2 speech is influenced by the speaker’s native language or is colored by other nonnative features. Comprehensibility is aligned with the intelligibility principle, as this construct encompasses listeners’ perception of ease or difficulty of understanding L2 speech. Although comprehensibility is not a measure of listeners’ actual understanding of L2 speech (i.e., intelligibility), typically operationalized through listeners’ orthographic transcriptions or retells of a speaker’s utterance, comprehensibility offers an appropriate measure of understanding in a broad sense, particularly in real-life contexts (Isaacs & Trofimovich, Reference Isaacs and Trofimovich2012). For example, many rating scales for standardized proficiency tests use the term intelligibility (e.g., TOEFL, IELTS) when what is, in fact, being measured is comprehensibility (Harding, Reference Harding, Isaacs and Trofimovich2017). Furthermore, for many speakers, it is their subjective perceptions of how easy or difficult it is to process linguistic input, more so than actual performance measures, that predict various cognitive and linguistic behaviors (Oppenheimer, Reference Oppenheimer2008). Comprehensibility is therefore a construct common to many rating scales and is also reflective of people’s general experience with speech.
Accentedness and comprehensibility appear to be partially independent in two key respects. First, L2 speakers can possess heavy accents and still be comprehensible (Derwing & Munro, Reference Derwing and Munro2015). Second, the linguistic measures of L2 speech that feed into comprehensibility are more diverse than those that underlie accentedness (Varonis & Gass, Reference Varonis and Gass1982), such that accentedness is primarily associated with segmental accuracy while comprehensibility is additionally linked to suprasegmental (e.g., word stress, intonation, rhythm), fluency, and lexis/grammar considerations (e.g., Crowther, Trofimovich, Isaacs, & Saito, 2015a; Isaacs & Trofimovich, Reference Isaacs and Trofimovich2012; Saito, Webb, Trofimovich, & Isaacs, 2016). However, most existing evidence for the linguistic independence of accentedness and comprehensibility, the focus of this study, comes from research utilizing a single task (picture narrative). Because L2 speakers draw on different linguistic resources (pronunciation, fluency, vocabulary, grammar) to complete different tasks (Robinson, Reference Robinson2005; Skehan, Reference Skehan2009), the linguistic underpinnings of accentedness and comprehensibility might crucially depend on the demands of the speaking task. The goal of this report was to investigate this possibility.
LINGUISTIC INDEPENDENCE OF ACCENTEDNESS AND COMPREHENSIBILITY: A POSSIBLE TASK EFFECT?
In early research on linguistic correlates of accentedness and comprehensibility, many measures of L2 speech were considered in isolation, often across separate publications (see Crowther, Trofimovich, Saito, & Isaacs, Reference Crowther, Trofimovich, Saito and Isaacs2015b). To address this limitation, recent studies examined the combined contribution of 19 linguistic measures to L2 accentedness and comprehensibility (Isaacs & Trofimovich, Reference Isaacs and Trofimovich2012; Trofimovich & Isaacs, Reference Trofimovich and Isaacs2012), and then expanded this work to investigate the roles of speakers’ first language (L1) background (Crowther et al., Reference Crowther, Trofimovich, Saito and Isaacs2015b), listeners’ L1 status (Crowther, Trofimovich, & Isaacs, Reference Crowther, Trofimovich and Isaacs2016), and rating scale type (Saito, Trofimovich, & Isaacs, 2015) in determining which linguistic dimensions of L2 speech pattern with accentedness and which with comprehensibility. Across these studies, the wider range of linguistic dimensions linked to comprehensibility—compared to accentedness, which is associated predominantly with segmental accuracy—was taken as support for the partial linguistic independence of the two constructs.
The most common task used to elicit L2 speech for analysis of accentedness and comprehensibility has been a picture narrative, where speakers describe a series of sequenced pictures. However, this methodological choice means that nearly all evidence for the linguistic independence of accentedness from comprehensibility comes from one task. In fact, to our knowledge, there is no research comparing linguistic dimensions of both accentedness and comprehensibility, within a single report, across different task types. Yet previous task-based research focusing on L2 oral production suggests that task effects should not be ignored. All theoretical frameworks that center on the effects of task on L2 learners’ linguistic performance indicate that differences in task demands impact L2 spoken output in terms of segmental and prosodic content (Tarone, Reference Tarone1983) and lexical and grammatical features (Robinson, Reference Robinson2005; Skehan, Reference Skehan2009). With respect to pronunciation accuracy, for example, segmental accuracy is greater in read-aloud than in spontaneous tasks (Rau, Chang, & Tarone, Reference Rau, Chang and Tarone2009). There also appears to be a hierarchy of perceived fluency for listeners, with L2 output in dialogue-based tasks rated as more fluent than speech elicited through picture narratives (Derwing, Rossiter, Munro, & Thomson, Reference Derwing, Rossiter, Munro and Thomson2004). In terms of lexis and grammar, differences across tasks in planning and completion time (Yuan & Ellis, Reference Yuan and Ellis2003), completion objectives (Robinson, Reference Robinson2005), and topic familiarity (Foster & Skehan, Reference Foster and Skehan1996) elicit varying levels of lexical and grammatical accuracy and complexity. Clearly, a consideration of task demands is crucial if viable claims are to be made about which linguistic dimensions of L2 speech are relevant to accentedness and which pattern with comprehensibility.
THE CURRENT STUDY
While previous research has focused on which linguistic dimensions of speech promote comprehensible L2 speech over nativelike performance, the bulk of evidence for linguistic correlates of comprehensibility is based on a limited task repertoire. If increased demands of a speaking task place greater strain on speakers’ production processes, encouraging them to resort to all available linguistic resources (Robinson, Reference Robinson2005; Skehan, Reference Skehan2009), it becomes necessary to understand if the linguistic measures that differentiate comprehensibility from accentedness also vary across task type. It could be that the linguistic distinction between comprehensibility and accentedness (in terms of the linguistic dimensions that pattern with each construct), which has been robust in previous work with picture narratives, might attenuate or disappear if L2 speakers are tested in other tasks, especially those that vary in difficulty. Put differently, complex speaking tasks, requiring speakers to deploy all available linguistic resources, might lead to multiple, overlapping linguistic dimensions feeding into listeners’ perceptions of L2 accentedness and comprehensibility, reducing the extent to which the linguistic signatures of accentedness and comprehensibility are distinct. One reason for this might be that greater task demands would require speakers to draw upon a wider range of linguistic dimensions, which may reduce the extent to which these dimensions differentiate accentedness from comprehensibility. If this were to be the case, beyond an empirically based understanding of what contributes to listeners’ perceptions of speech, alignment between the two constructs during more demanding task performance may impact both pedagogical and assessment considerations.
Therefore, for this report, we revisited the large-scale dataset previously featured in two conceptually distinct publications (Crowther et al., Reference Crowther, Trofimovich, Isaacs and Saito2015a, Reference Crowther, Trofimovich, Saito and Isaacs2015b). In Crowther et al. (Reference Crowther, Trofimovich, Saito and Isaacs2015b), the focus was on the role of speakers’ L1 in determining which linguistic dimensions are relevant to listeners’ perception of L2 accentedness versus comprehensibility in a picture narrative. In Crowther et al. (Reference Crowther, Trofimovich, Isaacs and Saito2015a), the focus was on task differences (using two nonpicture narrative tasks), but only in how they related to comprehensibility (and associated linguistic dimensions), which is in line with a scholarly emphasis on intelligibility over nativeness (e.g., Derwing & Munro, Reference Derwing and Munro2015; Levis, Reference Levis2005). Neither report examined how task impacted both accentedness and comprehensibility, specifically in regard to the linguistic dimensions associated with each. Subsequent analyses of the dataset in full revealed more nuanced findings in this regard, findings necessary to refine our understanding. Therefore, the goal of this report was to extend previous studies by (a) analyzing the data across three speaking tasks that were previously targeted in two separate publications and (b) comparing the data for accentedness and comprehensibility across three speaking tasks, which has not been done previously. As in prior work, the focus here was on 10 rated linguistic dimensions of L2 speech—namely, segmental accuracy, word stress accuracy, intonation, rhythm, speech rate, grammatical accuracy and complexity, lexical richness and complexity, and discourse richness—to determine whether the linguistic correlates of comprehensibility differ from those associated with accentedness across tasks. By consolidating all analyzed data for a single report, it was possible to address the following question, which previous studies were unable to examine: Can L2 accentedness and comprehensibility be differentiated at the level of linguistic dimensions across speech elicitation tasks varying in task demands?
METHOD
PARTICIPANTS
Participants included 60 L2 speakers of English (22 female, 38 male) who provided audio recordings of speech and 10 native English-speaking listeners (8 female, 2 male) who evaluated the recordings. The 60 speakers were drawn from an unpublished corpus of 143 L2 university students representing 19 linguistic backgrounds. All speakers were in the first semester of undergraduate (n = 29) or graduate (n = 31) study at an English medium university in Montreal, Canada, and represented four distinct L1 backgrounds: Farsi, Hindi/Urdu, Mandarin, and Romance. Because the goal of this report was to determine relationships between linguistic dimensions of L2 speech and listeners’ ratings of comprehensibility and accentedness for speakers from multiple L1s, the 60 speakers were considered as a single group (for L1 differences, see Crowther et al., Reference Crowther, Trofimovich, Saito and Isaacs2015b). Speakers’ biographical information is summarized in Table 1. The listeners included 10 native speakers of English (M age = 32.7 years, range = 25–56), raised in English-speaking homes with at least one native English-speaking parent, all current or recent graduates in applied linguistics. The listeners reported using English 89% of the time daily (range = 80–100%). All listeners indicated high familiarity with accented English and had on average 6.6 years of language teaching experience. Experienced listeners were chosen over naïve listeners as they tend to demonstrate more consistency in their linguistic ratings (Saito et al., 2015).
L2 Speakers’ background characteristics
a 1 = extremely poor, 9 = extremely fluent.
SPEAKING TASKS
Each speaker completed three tasks. The first was a picture narrative (hereafter, picture task, available at http://www.iris-database.org), used in Derwing, Munro, Thomson, & Rossiter (Reference Derwing, Munro, Thomson and Rossiter2009) and other studies by these authors. Speakers described an eight-frame picture narrative, in which two strangers bumped into each other, accidentally exchanged their identical suitcases, and realized their mistake upon returning home. The second task was the IELTS long-turn task (hereafter, the IELTS task), in which speakers received a card with one of two assigned topics (describe a sports event you enjoyed watching, describe a job you would like to do in the future) and two suggested discussion points (IELTS, 2009). Following IELTS procedures, after 1 minute of preparation time, speakers were given 1–2 minutes to respond, with 1–2-minute optional follow-up questions by the interviewer. Though the interviewer did not have IELTS examiner qualifications, they followed the IELTS interviewing procedures consistently across all individual interviews. For the third task, speakers completed a TOEFL iBT integrated task (hereafter, the TOEFL task), with stimuli presented through a computer interface (Educational Testing Service, 2006). Speakers were allotted 45 seconds to read a 93–105-word passage and then listened to a 80–90-second audio lecture on a related topic. Drawing from both, speakers responded to a question related to the content of both the listening and reading stimuli. They had 30 seconds to prepare their response before speaking for up to 1 minute. Speakers responded to one of two topics (audience effects in psychology, behavioral explanations in sociology), with approximately half of the speakers assigned to each. Independent-samples t tests indicated no differences between the two IELTS and TOEFL task versions (p > .05), so data across task versions were combined.
To determine task type differences, the three task types were analyzed by the first two authors following Robinson’s (Reference Robinson2005) framework for task classification, with respect to resource-directing variables (see Table 2). The TOEFL task appeared to draw on different cognitive resources, compared to the picture and IELTS tasks, specifically requiring reasoning and perspective taking. Neither was necessary to complete the picture and IELTS tasks. However, the picture and the IELTS tasks differed in the degree of topic familiarity (and associated linguistic freedom) they afforded the speaker. Whereas speakers needed to draw on their linguistic resources to address a familiar and personal topic in the IELTS task, the picture task constrained the range of lexical items required for completing the narrative in a sequence without much choice in content and organization (see Foster & Skehan, Reference Foster and Skehan1996). For a more in-depth description of how task complexity was determined, see Crowther et al. (Reference Crowther, Trofimovich, Isaacs and Saito2015a).
Summary of task complexity variables for the three speaking tasks
To further assess the hierarchy of task differences, participants were asked to rate the difficulty of each task on a 9-point Likert Scale (1 = very easy, 9 = very hard). While no significant difference was found between the picture and IELTS tasks (M diff = .07, p = .82, 95% CI [–0.56, 0.71]), the TOEFL task was perceived as more difficult than both the picture (M diff = .71, p = .009, 95% CI [–1.24, –0.19]) and IELTS (M diff = .80, p = .005, 95% CI [–1.34, –0.26]) tasks. Speakers’ perception of the TOEFL task as the most difficult aligned with our view that it would be the most cognitively challenging, similar to the findings in Révész, Michel, and Gilabert (Reference Révész, Michel and Gilabert2015), who studied learner versus expert judgments of task complexity. Thus, the three tasks likely represented different task demands, with the picture and IELTS tasks being less demanding than the TOEFL task.
RATING PROCEDURE
All audio recordings were edited to include the initial 30 seconds of speech, minus initial fillers and disfluencies, and were normalized for peak amplitude. The recordings, transcribed and verified by two transcribers, served as the materials for listener-based assessments. The 10 listeners evaluated the 60 speakers in all tasks using 1,000-point scales in Z-Lab (Yao, Saito, Trofimovich, & Isaacs, Reference Yao, Saito, Trofimovich and Isaacs2013), a custom-designed MATLAB program. Listeners first familiarized themselves with the task materials (e.g., images from the picture task), then perceptually evaluated 12 categories in total, including two global measures (accentedness, comprehensibility) and 10 specific linguistic variables, described in Table 3. To rate the two global dimensions and five pronunciation and fluency variables, listeners used audio recordings. To rate the five remaining variables, they used transcripts of each audio file, which minimized the influence of pronunciation and fluency variables on ratings of lexis, grammar, and discourse (Crossley, Salsbury, & McNamara, Reference Crossley, Salsbury and McNamara2015). Each of the three sets of measures were rated simultaneously within their associated sets (global; pronunciation and fluency; lexis, grammar, and discourse) because there are few differences between measures rated simultaneously or consecutively (O’Brien, Reference O’Brien2016). Listeners were trained by the first author on all measures (using three practice recordings) prior to evaluating each speech sample. Listener ratings were obtained in four individual 2-hour sessions within a 3-week span. Samples were blocked and counterbalanced by task and presented in unique randomizations. Subjective measures were chosen, as Saito et al. (2015)—using the dataset featured in Isaacs and Trofimovich (Reference Isaacs and Trofimovich2012)—found that listener ratings of these 10 linguistic measures aligned closely with the measures derived through coding by trained coders in the original study. Further details about the rating categories and assessment procedure can be found in Crowther et al. (Reference Crowther, Trofimovich, Isaacs and Saito2015a, Reference Crowther, Trofimovich, Saito and Isaacs2015b).
Rated categories with scalar endpoint descriptors (0–1,000) and category summary
RELIABILITY
Listeners were consistent in their judgements of all rated measures across tasks (Table 4), exceeding the benchmark value of Cronbach’s α > .70–.80 (Larson-Hall, Reference Larson-Hall2009). Therefore, single mean scores per speaker were derived by averaging across all listener ratings for each rated measure.
Interrater reliability across tasks (Cronbach’s α)
RESULTS
GLOBAL RATINGS
The first analysis considered the relationship between accentedness and comprehensibility (Table 5). The two constructs were strongly correlated in the picture (r = .80), IELTS (r = .79), and TOEFL (r = .74) tasks. A two-way repeated measures ANOVA with global rating (accentedness, comprehensibility) and task (picture, IELTS, TOEFL) as within-groups factors yielded a significant main effect of global rating, F(1, 59) = 231.02, p < .001, η ρ2 = .80, and task, F(2, 59) = 3.21, p = .044, η ρ2 = .05, along with a significant interaction, F(1, 59) = 31.46, p < .001, η ρ2 = .35.
Means (standard deviations) for accentedness and comprehensibility across tasks
Note: 0–1,000 scale (0 = heavily accented, 1,000 = no accent at all; 0 = hard to understand, 1,000 = easy to understand).
Post hoc (Bonferroni adjusted) analyses revealed that comprehensibility was rated significantly higher than accentedness in all tasks: picture (M diff = 112.45, p < .001, d = 0.69, 95% CI [85.15, 139.75]), IELTS (M diff = 227.58, p < .001, d = 1.46, 95% CI [200.93, 254.24]), and TOEFL (M diff = 175.73, p < .001, d = 1.09, 95% CI [145.28, 206.18]). Additionally, speakers were perceived to be less accented in the picture, compared to the IELTS (M diff = 31.50, p = .032, d = 0.19, 95% CI [2.04, 60.96]) or TOEFL (M diff = 30.43, p = .017, d = 0.18, 95% CI [4.28, 56.58]) tasks, with no difference between the IELTS and TOEFL tasks (M diff = 1.07, p = 1.00, d = 0.01, 95% CI [–30.93, 28.79]). For comprehensibility, speakers were rated higher in the IELTS, compared to either the picture (M diff = 83.63, p < .001, d = 0.55, 95% CI [45.01, 122.26]) or TOEFL (M diff = 50.78, p = .008, d = 0.33, 95% CI [10.92, 90.65]) tasks, with no difference between the picture and TOEFL tasks (M diff = 32.85, p = .087, d = 0.21, 95% CI [–3.32, 69.02]).
PRONUNCIATION AND LEXICOGRAMMAR AS PREDICTORS OF GLOBAL RATINGS
The next analysis targeted the relationship between accentedness and comprehensibility and the 10 rated linguistic measures. For each task, an exploratory principal component analysis (PCA) with Oblimin rotation was conducted to determine if the 10 linguistic measures showed any patterns based on their clustering. Kaiser-Meyer-Olkin values (picture = .85; IELTS = .91; TOEFL = .86) and Bartlett’s tests of sphericity (picture: χ 2(60) = 692.09, p < .001; IELTS: χ 2(60) = 822.95, p < .001; TOEFL: χ 2(60) = 700.57, p < .001) suggested excellent factorability of the correlation matrix, despite the relatively small sample size (N = 60). As shown in Table 6, the PCA yielded a clear two-factor solution for each task, accounting for a substantial amount of variance per task (picture = 82%; IELTS = 82%; TOEFL = 87%). In all tasks, Factor 1 was labeled pronunciation and Factor 2 lexicogrammar. Aside from speech rate loading on both factors in the picture task, the factors were identical and distinct across tasks.
Factor loadings (> .40) for PCA of linguistic measures across tasks
Pearson correlations, which were computed between accentedness and comprehensibility and the two PCA factor scores (pronunciation, lexicogrammar), derived through the Anderson-Rubin method of obtaining noncorrelated factor estimates (see Table 7), revealed that comprehensibility was strongly associated with both factors in all tasks, featuring strong associations (r > .60), following Plonsky and Oswald’s (Reference Plonsky and Oswald2014) guidelines. Accentedness was strongly correlated with the pronunciation factor across tasks. While the relationships between accentedness and lexicogrammar were overall weaker, they increased in strength from the picture and IELTS tasks, where associations were weak (r > .25), to the TOEFL task, where the association was medium in strength (r > .40).
Correlations between accentedness, comprehensibility, and two PCA factors across tasks
Note: * p < .05, ** p < .005.
To investigate the relative contribution of the pronunciation and lexicogrammar factors to explaining the variance in accentedness and comprehensibility ratings across tasks, hierarchical multiple regression analyses were carried out, separately per task, with accentedness or comprehensibility as criterion variables. Considering the strength of factor associations (see Table 7), pronunciation was entered as a predictor first, followed by lexicogrammar, which allowed for determining if lexicogrammar could explain any additional variance in accentedness or comprehensibility beyond that accounted for by the pronunciation factor. As shown in Table 8, both pronunciation and lexicogrammar were significant predictors of comprehensibility in the picture (78% of total variance explained), IELTS (74%), and TOEFL (87%) tasks. However, only pronunciation predicted accentedness scores in the picture (77%), IELTS (74%), and TOEFL (76%) tasks.
Summary of regression analyses for accentedness and comprehensibility across tasks
INDIVIDUAL LINGUISTIC MEASURES AND GLOBAL RATINGS
Though the lexicogrammar factor was not found to be a significant predictor of accentedness, as task complexity increased, there was a gradual increase in the likelihood of lexicogrammar predicting accentedness across tasks: picture (p = .988), IELTS (p = .250), and TOEFL (p = .053). Considering this trend toward lexicogrammar being more relevant to explaining accentedness ratings with increasing task difficulty, the final analysis explored associations between accentedness and comprehensibility and the full set of 10 linguistic measures. Following Plonsky and Oswald’s (Reference Plonsky and Oswald2014) guidelines, a series of Pearson correlations (r) were conducted (Table 9).
Correlations between accentedness, comprehensibility, and 10 linguistic measures across tasks
Note: * p < .05, ** p < .01.
Individual correlations confirmed that comprehensibility in all tasks was strongly linked to all pronunciation and lexicogrammar measures, with strong associations (r > .60), and that accentedness across tasks was associated with the five pronunciation measures (r > .40). However, there was a gradual increase in the relevance of lexicogrammar measures to accentedness. Whereas only two lexicogrammar measures (lexical appropriateness, grammatical accuracy) featured associations with accentedness in the picture task (small-to-medium association strength), all five lexicogrammar measures were linked to accentedness in the IELTS and TOEFL tasks. These latter two tasks differed in the strength of these correlations. In the IELTS task, four associations were weak (r < .40). However, in the TOEFL task, two of the five (lexical appropriateness, grammatical accuracy) featured medium-strength associations.
DISCUSSION
The goal of this report was to investigate whether the partial independence between accentedness and comprehensibility—established in prior research about linguistic measures of L2 speech associated with each construct (e.g., Trofimovich & Isaacs, Reference Trofimovich and Isaacs2012)—would hold when analyzed across speaking task. Three speech production tasks (picture, IELTS, TOEFL) were considered, with task complexity determined through an analysis of resource-directing variables (Robinson, Reference Robinson2005). Comprehensibility across all tasks was strongly associated with measures of L2 phonology and lexis/grammar, suggesting (in line with prior research) that listeners draw on multiple linguistic dimensions of L2 speech when evaluating comprehensibility (e.g., Crowther et al., Reference Crowther, Trofimovich, Isaacs and Saito2015a; Isaacs & Trofimovich, Reference Isaacs and Trofimovich2012). While linguistic correlates of comprehensibility remained consistent across task, linguistic correlates of accentedness did not. For the picture task, listener perception of accentedness was associated almost exclusively with phonology measures, replicating prior findings for picture narratives, including for participants from a different dataset (e.g., Trofimovich & Isaacs, Reference Trofimovich and Isaacs2012). With increased task complexity, however, associations between accentedness and measures of L2 lexis/grammar strengthened (see Table 9), implying that aspects of lexis and grammar could serve as predictors of accentedness. Linguistic distinctions between accentedness and comprehensibility were thus clearest in the picture task but were most blurred in the more complex TOEFL task. These findings extend, but do not contradict, prior results: Because previous reports either did not include a focus on both accentedness and comprehensibility (Crowther et al., Reference Crowther, Trofimovich, Isaacs and Saito2015a) or targeted only one task (Crowther et al., Reference Crowther, Trofimovich, Saito and Isaacs2015b), a task-specific pattern was not established.
As predicted by task complexity frameworks, increased task difficulty likely elicits more elaborate language as L2 speakers strive to meet the increased demands of the task (Robinson, Reference Robinson2005). At least for the TOEFL task, one reason for these greater demands is the greater flexibility provided in how to respond. In the picture task, speakers had a predetermined structure to follow (describe pictures in order), and in the IELTS task, they dealt with (usually) a familiar topic. They thus likely felt little pressure to engage all available linguistic resources, apart from the need to retrieve and use vocabulary drawn from a restricted lexical set (pictures objects, actions) or a familiar subject (future job, favorite sports), which might have been their primary source of difficulty (Hilton, Reference Hilton2008). By contrast, the TOEFL task offered more flexibility in how speakers chose to respond. With this flexibility of having multiple ways to make themselves understood—for example, when integrating the content of the listening and reading passages—speakers likely relied on all linguistic resources in their possession, making it likely that issues of lexis and grammar (in addition to being relevant to comprehensibility) also contributed to accentedness. Put simply, increased task demands bring both constructs—which are already highly interrelated (r = .74–.80)—into greater alignment with respect to linguistic dimensions associated with each, such that similar linguistic dimensions contribute to the perceptual signatures of accentedness and comprehensibility.
Whereas increased task demands ostensibly minimized distinctions between accentedness and comprehensibility (in terms of the linguistic dimensions associated with each), task complexity appeared to produce the opposite effect on each rated construct. As task difficulty increased, L2 speakers were generally rated as more accented (less nativelike) but also more comprehensible (see Table 5). However, this interpretation must be considered in light of the small effect sizes associated with differences in accentedness and comprehensibility ratings between tasks. For accentedness, even though speakers’ performances were rated as being more heavily accented in the IELTS and TOEFL tasks, compared to the picture task, the actual effect was minimal (d = .01–.17). For comprehensibility, while the effect for a difference in ratings between the picture and IELTS tasks was notable (d = .55), it was still relatively small (Plonsky & Oswald, Reference Plonsky and Oswald2014). One way to interpret these general patterns is to suggest that increased task difficulty was linked to a broader range of linguistic dimensions feeding into listeners’ perception of accent. Yet greater accentedness, in more complex tasks, was not associated with speech that was also harder to understand. This result likely reflects similar effects of the greater flexibility provided by more complex tasks—this time, for listeners. As part of the training procedure, listeners had the opportunity to view the picture task prompt, an approach used to minimize task familiarity effects for the few initial samples rated. Listeners thus likely developed clear expectations of what they were about to hear, such that ease or difficulty of understanding was related to the extent to which each story conformed to these expectations. In contrast, with no clear path for task completion in the IELTS and especially the TOEFL task, there were many more avenues for speakers to make themselves understood. Paradoxically, for listeners, this may make more complex tasks easier in regard to understanding the message. Listeners could, for example, rely on multiple linguistic cues (i.e., not just a handful of lexical items called for by the task) to help them process the message. Yet with increased linguistic freedom comes greater opportunity for producing language which might diverge from what listeners would consider nativelike. Essentially, for the IELTS and TOEFL tasks, while speakers had more ways to get it right in terms of comprehensibility, they also had more ways to get it wrong in regard to how accented (nativelike) they sounded.
CONCLUSION
Theoretically speaking, these findings question the strength of one specific previously reported distinction between accentedness and comprehensibility, namely, with respect to the linguistic dimensions of speech associated with each construct. Because the two constructs were largely overlapping in the TOEFL task, this distinction appears to be task specific, such that the linguistic variables relevant to each construct vary with task demands. This finding implies a multifaceted relationship between linguistic correlates of accentedness and comprehensibility, one that must be situated within task differences and likely also listener expectations. In terms of pedagogical implications, the finding that linguistic distinctions between accentedness and comprehensibility are blurred in a complex task might be encouraging for both language learners and teachers. Using complex tasks, L2 speakers might be able to practice a range of linguistic targets (phonology, fluency, lexis, and grammar), with most having a bearing on both accentedness and comprehensibility. From an assessment perspective, as pointed out by an anonymous reviewer, pronunciation as a specific criterion of a speaking test may be best isolated in a less complex task, where nonphonological linguistic measures are less likely to influence a rater’s assessment.
Of course, the preceding interpretations imply that further research is necessary to strengthen our understanding of the role of task in listener perceptions of L2 speech. The findings are based on the ratings of 10 highly educated and experienced, native-speaking L1 English listeners who may not be representative of listeners that L2 speakers are likely to encounter in everyday interactions. The perceptions of listeners with less formal linguistic training, different levels of L2 speech exposure, and L1 backgrounds would help elaborate on the findings presented here. Additionally, there might have been effects of both pretask and online planning time on the accuracy and complexity of language produced across the tasks. While planning time was not controlled in the picture task (such that speakers could start narrating their story as soon as they had familiarized themselves with the images), both the IELTS and TOEFL tasks differed in the amount of time allotted before speaking and in the time pressure imposed on speakers during production. Amount of planning time and availability of task materials (access to prompt images or texts) need to be carefully considered in future research exploring task effects on judgments of accentedness and comprehensibility. Finally, while the targeted tasks were representative of both existing literature (picture narrative) and common L2 assessment tools (IELTS, TOEFL), they do not encompass the full scope of task complexity. For instance, what happens when task goals are no longer individual, but require a speaker to take into consideration the views and actions of another? If we learn L2s for communication, then explorations of linguistic correlates of accentedness and comprehensibility in authentic communication would be a logical, and necessary, direction of future work. Clearly, a consideration of task complexity has led to a more complex understanding of the linguistic measures that define L2 pronunciation.
