Introduction
A number of studies have provided evidence that in certain perceptual tasks, non-native speakers may outperform native speakers when their first language directs attention to cues that are particularly helpful for perception of the relevant L2 distinction (Chang, Reference Chang2018; Chang & Mishler, Reference Chang and Mishler2012; Choi, Reference Choi2021; Choi, Tong & Samuel, Reference Choi, Tong and Samuel2019; Kim & Tremblay, Reference Kim and Tremblay2021; Lin, Wang, Idsardi & Xu, Reference Lin, Wang, Idsardi and Xu2014; Tong, Lee, Lee & Burnham, Reference Tong, Lee, Lee and Burnham2015; Wiener & Goss, Reference Wiener and Goss2019). A non-native perceptual advantage is especially likely with stimuli that have been manipulated to remove some or all information other than the L1-relevant cues (Choi, Reference Choi2021; Kim & Tremblay, Reference Kim and Tremblay2021; Yu & Andruski, Reference Yu and Andruski2009), and the role of a cue in the listener's L1 appears to be a better predictor of performance than the listener's level of L2 proficiency and/or experience (Chrabaszcz, Winn, Lin & Idsardi, Reference Chrabaszcz, Winn, Lin and Idsardi2014; Kim & Tremblay, Reference Kim and Tremblay2021; Wiener & Goss, Reference Wiener and Goss2019).
The majority of studies that have shown an L2 advantage involves the perception of segmental or pitch cues at the level of single words. This paper investigates whether Mandarin–English sequential bilinguals, whose first language directs attention to pitch cues, show an advantage over L1 English speakers in making use of contrastive intonation within discourse, both with natural stimuli and with stimuli in which segmental information is masked, leaving prosody as the only cue to the intended referent.
While a number of studies have demonstrated the ability of native speakers of English to use contrastive prosody to anticipate contrastive referents (e.g., Dahan, Tanenhaus & Chambers, Reference Dahan, Tanenhaus and Chambers2002; Ito & Speer, Reference Ito and Speer2008; Sedivy, Tanenhaus, Chambers & Carlson, Reference Sedivy, Tanenhaus, Chambers and Carlson1999), considerably less attention has been paid to the use of contrastive prosody by listeners whose first language is not English, and no studies have investigated whether contrastive prosody alone, in the absence of segmental information, is sufficient to allow listeners to choose an appropriate contrastive referent. The comparison of English and Mandarin speakers is of particular interest because English and Mandarin differ substantially in their use of suprasegmental information. In Mandarin – a tone language – the primary function of pitch is to encode lexical contrast, with roughly 70% of lexical items distinguished only by tonal differences (Wiener & Goss, Reference Wiener and Goss2019, citing Shibata & Shibata, 1990). In English – an intonation language – pitch functions largely to convey aspects of information structure such as contrastive focus as well as speakers’ beliefs, attitudes, and intentions. While prosodic cues may also be used in Mandarin to signal information structure (Ouyang & Kaiser, Reference Ouyang and Kaiser2015; Xu, Reference Xu1999), the phonetic realization of focus cues is constrained by the need to preserve lexical tone contrasts (Chen, Reference Chen2010; Chen & Gussenhoven, Reference Chen and Gussenhoven2008), and the discourse function of pitch is secondary to the signaling of lexical information.
Because pitch cues play such a central role in languages like Mandarin, the hypothesis that tone language speakers enjoy an advantage in the use of pitch cues predicts that Mandarin–English bilinguals should perform as well as or better than English native speakers in the interpretation of contrastive prosody, particularly in a task in which cues other than pitch are removed or attenuated, and that the appearance of this advantage need not correlate with their level of English proficiency or experience with English. It is these predictions that this study is designed to test.
Evidence of L1 perceptual advantage
Evidence of a perceptual advantage for listeners whose first language directs attention to specific L2 cues has emerged in two domains: (i) in the use of segmental cues to phonemic contrasts (by listeners whose L1 uses a wider range of cues to phoneme identity than the L2), and (ii) in the use of pitch-related cues to L2 lexical pitch accent and stress contrasts (by listeners whose L1 uses pitch to signal lexical contrasts).
In the area of segmentally-signaled lexical contrasts, Chang (Reference Chang2016) found that Korean heritage speakers outperformed L1 English speakers in discriminating and identifying final voiceless unreleased stop-zero contrasts both in English words (e.g., beet/bee) and in English-like nonce forms. Relatively late learners of English who had grown up in South Korea also outperformed L1 English speakers, though they showed slightly lower perceptual sensitivity to stop-zero contrasts than the heritage Korean speakers (Chang, Reference Chang2016; Chang & Mishler, Reference Chang and Mishler2012). The L1 Korean advantage is attributed to the fact that final stops, which are variably released in English, are never released in Korean. L1 Korean listeners are therefore presumably more sensitive than L1 English listeners, who rely heavily on release cues, to other cues signaling the presence and type of a final stop, such as transitional cues in the preceding vowel. That the Korean speakers’ better performance was a function of an L1 that encourages attention to specific cues, rather than a function of multilingualism in general, was established by comparison with L1 speakers of Mandarin and Japanese. Despite comparable levels of English experience and proficiency among the L1 Korean, L1 Mandarin, and L1 Japanese listeners, the L1 Mandarin and L1 Japanese groups were significantly less able than either the L1 English or the L1 Korean listeners to discriminate the English pairs (Chang, Reference Chang2018), presumably because neither Mandarin nor Japanese contains words ending in stops.
In the area of pitch-signaled lexical contrasts, several studies have provided evidence of a perceptual advantage for speakers of pitch accent or tone languages, even when the L1 and L2 employ typologically distinct uses of pitch. In tone languages such as Mandarin or Cantonese, each syllable is typically associated with a lexical pitch; in pitch accent languages such as Japanese, each word is associated with a distinct pitch pattern based on the location of the accented mora in the word; and in stress languages such as English, pitch excursions associated with intonational melodies are one cue to the position of lexical stress. Several studies have indicated a perceptual advantage for speakers of a language that uses pitch to signal lexical contrasts, even across differing pitch systems.
Evidence of a tone language advantage in the perception of lexical pitch accent patterns emerged in a study by Wiener and Goss in which L1 Mandarin students of Japanese showed higher accuracy than native Japanese speakers in discriminating Japanese forms with different pitch accent patterns (Wiener & Goss, Reference Wiener and Goss2019). Furthermore, L1 Mandarin speakers with no Japanese experience at all showed discrimination performance comparable to that of Japanese native speakers, even for Japanese pitch accent patterns that were distinct from Mandarin tonal patterns. L2 study of Mandarin was also advantageous: English speakers who had studied Mandarin as an L2 discriminated pitch accents as accurately as Japanese native speakers, and considerably more accurately than English speakers with no study of Mandarin.
Evidence of an advantage in the perception of English stress contrasts for speakers of a pitch accent language appeared in a study comparing the ability of speakers of two dialects of Korean to identify and recall English stress patterns (Kim & Tremblay, Reference Kim and Tremblay2021). In a sequence-recall task of disyllabic forms differing either in stress (IMport/imPORT) or in a single segment (table/cable), speakers of Gyeongsang Korean (a pitch accent language) outperformed native English speakers in the recall of stress differences but not segmental differences. The stress stimuli were of two types: natural, containing cues of pitch, duration, and intensity, and f0-only, with duration and intensity cues suppressed. The L1 English listeners’ performance was better with the natural stimuli, while the Gyeongsang Korean speakers’ performance was not enhanced by the presence of duration and intensity cues. The Gyeongsang Korean speakers also outperformed native speakers of Seoul Korean on recall of stress differences, consistent with the fact that Seoul Korean does not employ lexical pitch accent. For both groups of Korean speakers, there was no correlation between recall accuracy and any measure of English proficiency or experience (self-rated English proficiency, self-rated English accent, LexTALE score, length of residence in an English-speaking country, and age of first exposure to English), suggesting that the stronger performance of the Gyeongsang speakers was solely a function of the presence of lexically-determined pitch in their dialect.
Finally, evidence that an L1 tone language confers sensitivity to pitch cues to English stress comes from several studies in which L1 tone speakers (L1 Cantonese and L1 Mandarin) did as well as or better than English speakers in stress-based tasks, but only when pitch cues were available. In an AXB discrimination task with English pseudowords differing in stress position, Tong et al. (Reference Tong, Lee, Lee and Burnham2015) found that Cantonese–English bilingual adults were faster and marginally more accurate than English monolinguals. In an AX discrimination study involving both English real words and pseudowords differing in stress, Choi et al. (Reference Choi, Tong and Samuel2019) found that L1 Cantonese speakers outperformed L1 English speakers in both response accuracy and response latency on stimuli that contained cues of pitch, duration, and intensity, though the L1 Cantonese advantage disappeared with stimuli that were manipulated to contain only duration and intensity cues. Choi et al. argue (consistent with the claims of Chang, Reference Chang2018; Kim & Tremblay, Reference Kim and Tremblay2021; Wiener & Goss, Reference Wiener and Goss2019) that the Cantonese speakers’ advantage was due not to their multilingualism per se but rather to the specific (tonal) nature of their L1, contrasting their participants’ results with those of a study on the perception of Spanish stress (Dupoux, Peperkamp & Sebastián-Gallés, Reference Dupoux, Peperkamp and Sebastián-Gallés2010) in which French–Spanish bilinguals, even those with the highest Spanish proficiency, performed no better than Spanish monolinguals.
Several additional studies revealed particular sensitivity to pitch cues to English stress by speakers of Mandarin. Lin et al. (Reference Lin, Wang, Idsardi and Xu2014) found that in a sequence-recall task involving English stress differences, native speakers of Mandarin outperformed not only native speakers of Seoul Korean (which has no lexical pitch accents) but also native speakers of English. Qin, Chien, and Tremblay (Reference Qin, Chien and Tremblay2017) found that L1 speakers of both Standard Mandarin and Taiwanese Mandarin performed similarly to L1 English speakers in a sequence-recall task involving stimuli that differed only in stress, though the performance of the two Mandarin groups fell below that of the native English speakers for manipulated stimuli in which f0 cues were removed as well as for stimuli differing in a single phoneme. And in an investigation of the role of L1 in the weighting of different cues to English stress, Chrabaszcz et al. (Reference Chrabaszcz, Winn, Lin and Idsardi2014) found that both L1 English and L1 Mandarin participants relied on pitch over duration and intensity, while L1 Russian participants prioritized duration and intensity over pitch. Notably, the L1 Mandarin speakers were more native-like than the L1 Russian speakers in the interpretation of English stress cues even though the Russian speakers scored significantly higher than the Mandarin speakers on a cloze test designed to measure English proficiency.
Factors affecting the likelihood of L1 perceptual advantage
Evidence of a perceptual advantage for tone language speakers in processing English prosody has failed to emerge under certain experimental conditions. Choi (Reference Choi2021) exposed Cantonese–English bilinguals to a series of segmentally different disyllabic forms, the majority of which (the standard) had trochaic (initial syllable) stress. Participants were directed to press a button when they heard a form with deviant (final syllable) stress. Stimuli were of three types: (i) segmental, containing both segmental and prosodic cues (f0, duration, and intensity); (ii) segmental-f0, with duration and intensity cues suppressed; and (iii) nonsegmental, containing only prosodic cues. The behavioral results revealed that the Cantonese–English listeners outperformed the L1 English listeners, but only on the nonsegmental stimuli; on stimuli containing segmental information, their accuracy rates were lower than those of L1 English speakers, who in contrast performed similarly across the three conditions. Comparing these results with those of the Choi et al. (Reference Choi, Tong and Samuel2019) study, in which listeners were required to discriminate segmentally identical forms differing only in stress position, Choi ascribes the loss of the Cantonese speakers’ perceptual advantage in the presence of segmental variation to “the diversion of cognitive resources [to processing of segmental information] that would have otherwise supported temporary storage or other operations” (Choi, Reference Choi2021, p.10). The claim that the processing of stimuli that included both segmental and prosodic information imposed greater cognitive demands for the non-native listeners is confirmed by ERP investigation of the participants' neurophysiological responses in the Choi Reference Choi2021 study. One difference between the two groups emerged in the P3b (a positive amplitude typically occurring at around 300 ms following the relevant stimulus), which has been claimed to decrease as the cognitive demands of a task increase (Kok, Reference Kok2001). The Cantonese listeners showed a smaller P3b with segmental than non-segmental stimuli, unlike the English speakers, who exhibited similar P3b amplitudes under all three conditions. The Cantonese listeners also showed smaller P3b amplitudes than the English listeners in the segmental conditions, consistent with the claim that the processing of English stimuli containing both segmental and non-segmental information was especially demanding for the Cantonese listeners. Drawing on a study of simultaneous processing of speech and music by Cantonese–Dutch bilinguals and L1 Dutch speakers (Asaridou, Hagoort & McQueen, Reference Asaridou, Hagoort and McQueen2015), Choi suggests that “the Cantonese listeners processed stress and segmental information more integrally than the English listeners did” (Choi, Reference Choi2021, p.10), increasing the cognitive demand of the task. This holistic processing of segmental and prosodic information is confirmed by the evidence presented by Choi, Tong, Gu, Tong, and Wong (Reference Choi, Tong, Gu, Tong and Wong2017) that native speakers of Cantonese integrate vowel and tone information in processing their native language.
Taken together, the studies discussed above point to two factors that may affect the performance of L1 tone speakers in tasks involving English stress: the nature of the stimuli (where performance is stronger with stimuli that suppress information distracting from pitch cues) and the cognitive load (where performance is weaker with increased task complexity). The positive impact of impoverished stimuli is confirmed by the findings of Yu and Andruski (Reference Yu and Andruski2009), who compared the ability of L1 Mandarin and L1 English participants to identify and discriminate the position of lexical stress in English real words, pseudowords, and hummed words. While the two groups performed comparably in identification, the L1 Mandarin speakers did show an advantage over the English speakers in the discrimination task, but only on the hummed words. Furthermore, the two language groups differed in their relative accuracy on different types of stimuli: the L1 Mandarin speakers showed higher accuracy on hummed words than on real words or pseudowords, while the English speakers were least accurate on the hummed words. Thus, as in the Choi (Reference Choi2021) study, the L1 tone speakers were clearly advantaged by hearing stimuli that contained only prosodic information.
The impact of cognitive load on performance is consistent with the results of Ou, Reference Ou2016, which introduced variation in sentence-level intonation into the comparison of L1 Mandarin and L1 English speakers’ perception of English stress. Stimuli consisting of five noun-verb pairs differing only in stress (e.g., IMpact/imPACT) were presented in two different carrier sentences: I said ___ (with declarative intonation in which the stressed syllable carried a high pitch accent – the more common, unmarked English pattern) and Did you say ___? (with interrogative intonation in which the stressed syllable carried a low pitch accent – the less common pattern). Listeners carried out two discrimination tasks (ABX and AX) as well as an identification task in which they were asked to indicate whether the target was a noun or a verb. In the ABX task (which Ou argues taps into phonological levels of processing), the L1 Mandarin speakers exhibited native-like accuracy for stimuli in which the stressed syllable carried a high pitch accent. On stimuli in which the stressed syllable carried a low pitch accent, however, both learner groups were less accurate than L1 English speakers, although the experienced learners (those with at least 10 years of English instruction) showed higher performance than inexperienced learners (less than 3 years of English instruction). Ou suggests that the more experienced learners had greater awareness of the interaction between sentential intonation and word-level stress cues in English, but were still less successful than native English speakers in integrating information from lexical and sentential domains.Footnote 1 These results are consistent with the claim of Choi (Reference Choi2021), that the perceptual advantage of attention to pitch cues fostered by an L1 may be outweighed by increased cognitive demands of the task.
In contrast to the studies above which support the hypothesis that knowledge of a tonal language provides an advantage in processing pitch cues to English lexical stress differences, at least under certain conditions, the few studies that have investigated L1 Mandarin speakers’ online processing of English contrastive prosody at a discourse level have found no evidence of an L1 tone language advantage. Perdomo and Kaan (Reference Perdomo and Kaan2021) used a visual world paradigm to compare L1-English and L1-Mandarin speakers’ gazes to pictures of new objects vs. objects already referred to, where the contextual prosody was appropriate either for given information or for new, contrastive information. The results were somewhat inconclusive, since both groups showed an unexpected bias toward the new objects regardless of prosody (though this bias was reduced under certain conditions for the English speakers, suggesting a possible L1 English advantage). Perdomo and Kaan conclude that although their learners were able to use prosodic cues, “we, however, do not have strong evidence that learners [use contrastive prosody] predictively, or that this is affected by proficiency or working memory” (Perdomo & Kaan, Reference Perdomo and Kaan2021, p.23). In an ERP study using the same stimuli, Lee, Perdomo, and Kaan (Reference Lee, Perdomo and Kaan2020) did find differences in the L1 English and L1 Mandarin listeners’ neural responses to inappropriate prosody which suggested that L1 Mandarin speakers were actually less sensitive than the L1 English speakers to the English prosodic information. Lee et al. interpret their results as support for the view that “L2 speakers have difficulty integrating information from various domains (in this case, prosody and information structure), as proposed by the Interface Hypothesis (Sorace, Reference Sorace2011)” (Lee et al., Reference Lee, Perdomo and Kaan2020, p.524).
While the studies discussed above involve a wide range of methodologies, language groups, and L2 proficiency levels, they do establish that at least in some areas – L2 segmental contrasts, pitch accent contrasts, and lexical stress contrasts – bilingual participants whose L1 directs attention to cues relevant for the L2 phenomenon in question may show a perceptual advantage over native speakers. Tasks that require integration of information from different domains are, however, less likely to yield evidence of a non-native advantage, though such an advantage is more likely to appear with stimuli that are manipulated to suppress some or all of the non-relevant cues. Furthermore, studies that examined the relationship between perceptual accuracy and individual factors (Chrabaszcz et al., Reference Chrabaszcz, Winn, Lin and Idsardi2014; Kim & Tremblay, Reference Kim and Tremblay2021; Ou, Reference Ou2016; Wiener & Goss, Reference Wiener and Goss2019) suggest that the informativity of the relevant cue in the L1 is a better predictor of a perceptual advantage for that cue than degree of L2 proficiency and/or experience.
Present study
The current study was designed to investigate (1) whether the pitch sensitivity required by their L1 allows Mandarin speakers to perform with nativelike or even better than native performance in their use of English contrastive focus prosody at a discourse level; (2) whether the suppression of segmental cues increases the likelihood of a Mandarin speaker advantage by reducing the need to process simultaneous segmental and prosodic information; and (3) whether performance correlates with level of English proficiency and/or experience. Although earlier studies (Lee et al., Reference Lee, Perdomo and Kaan2020; Perdomo & Kaan, Reference Perdomo and Kaan2021) failed to find an advantage for L1 Mandarin speakers in interpreting English contrastive prosody with naturalistic stimuli presenting both segmental and prosodic information, it is possible that any L1 Mandarin advantage was obscured by the cognitive demands of interpreting stimuli that varied in both segmental and prosodic structure.
In order to determine whether L1 Mandarin speakers were better able to select a contrastive referent even in the absence of segmental information, this study presented half the instructions in a visual search task using normal speech, while the other half masked the segmental information identifying the target object, so that the only cue to the intended referent was the contrastive prosody. This task required listeners to interpret the contrastive prosody as indicating a target object that contrasts with a previously named object in the discourse.Footnote 2
If contrastive prosody does indeed prompt listeners to anticipate alternative referents, we expect that both groups of speakers should be able to use prosody at better than chance level to select a prosodically appropriate referent, even when segmental information is masked. The presentation of prosody-only stimuli should benefit the Mandarin speakers in two ways: the task relies on sensitivity to pitch, and it removes the necessity to simultaneously process segmental and prosodic information. And if tone language experience confers an advantage in interpreting pitch cues – as suggested by the results of Choi et al., Reference Choi, Tong and Samuel2019; Qin et al., Reference Qin, Chien and Tremblay2017; Tong et al., Reference Tong, Lee, Lee and Burnham2015; Wiener & Goss, Reference Wiener and Goss2019 – we expect that when prosodic information is the only information available, the L1 Mandarin speakers should perform as well as if not better than the L1 English speakers on prosody-only stimuli, and that this advantage should emerge regardless of the level of English proficiency and experience. If, however, the integration of phonological and semantic/pragmatic cues at the level of discourse imposes an undue cognitive load for non-native speakers, as suggested by Sorace (Reference Sorace2011); Perdomo and Kaan (Reference Perdomo and Kaan2021); Lee et al. (Reference Lee, Perdomo and Kaan2020), then we should not see an advantage emerge for the L1 Mandarin listeners, even with prosody-only stimuli which require listeners to rely heavily on prosodic information. And if the Mandarin speakers’ performance is affected by interference from the different ways in which prosody is used to signal contrast in their native language and in English, we expect to find some correlation between accuracy and English proficiency and/or experience.
Participants
All participants were students at a single US university. The twenty-three L1-Mandarin speakers (9 female, 14 male) considered Mandarin their first language but were living and studying in an English-speaking environment. Their mean age was 20.3; their mean length of residence in the US was 5.6 years, ranging from .25 to 15; and their mean age of initial exposure to English was 9.2, ranging from 3 to 15. As part of the study, all Mandarin-speaking participants completed the Versant English Test (Pearson Education, 2019). Versant English Test scores “represent the ability to understand spoken English and speak it intelligibly at a native-like conversational pace on everyday topics” (Pearson Education, 2019) and range from 20 to 80. The overall score reported by Pearson Education is based on a weighted combination of four diagnostic subscores – Sentence Mastery, Vocabulary, Fluency, and Pronunciation – with which the overall score is highly correlated (with Sentence Mastery: r = .8, with Vocabulary: r = .8, with Fluency: r = .97, with Pronunciation: r = .95). The mean overall score of the participants was 58.9, with a range from 43 to 78. Nineteen L1-English speakers (14 female, 5 male, mean age 22) also participated in the experiment. One female L1-English speaker was excluded due to experience with a tone language (Cantonese), bringing the total number of L1-English participants to 18.
Design and procedures
In each experimental trial, listeners responded to a pair of context-target instructions, as illustrated in Appendix S1 (see Supplementary Material). Each trial began with a fixation point (+) at the center of the screen. Once the participants fixated on the point, they saw three objects on the screen, and heard an instruction of the form Click on the ADJECTIVE + NOUN (e.g., purple mittens). When the participants pressed the key corresponding to the named item, that item then moved to the center of the screen and remained for 1000 msec, establishing a context for contrast, and participants were directed to focus on that item. Next, four new items appeared on the screen surrounding the previously named item, which then disappeared after 800 msec. Participants then heard the second instruction, Now click on the ADJECTIVE + NOUN (e.g., scarlet mittens). The onset of this auditory instruction was set at 100 msec after the onset of the visual stimuli.
In half the trials (n = 48), the second (target) instruction was produced with contrastive prosody on the adjective (ADJ-focus prosody), while in the other half (n = 48), contrastive focus prosody fell on the noun (NOUN-focus prosody). Each target noun phrase consisted of a color adjective and an object noun. All adjectives and nouns were disyllabic trochees (e.g., scarlet, ivory, purple and mittens, sweater, necklace).Footnote 3
In half of the main trials, both segmental and prosodic information was provided. For these trials, the same audio instruction was presented in two conditions. In the Helpful Prosody condition, a contrastive pitch accent appeared on a contrasting adjective or noun, as in Click on the purple mittens. Now click on the SCARLET mittens. In the Unhelpful Prosody condition, the contrastive pitch accent fell, inappropriately, on repeated information, as in Click on the scarlet necklace. Now click on the SCARLET mittens.
In the other half of the main trials, segmental information for the ADJECTIVE + NOUN in the target instruction was masked (Now click on the xx xx). Of these 48 target instructions with masked ADJECTIVE + NOUN, 24 were produced with a contrastive pitch accent on the masked portion corresponding to the adjective, and the other 24 with contrastive pitch accent on the noun portion. Each audio instruction was presented in two conditions. In the Helpful Prosody condition, the objects on the screen included exactly one choice that was consistent with contrastive prosody, given the context instruction. For example, when the context instruction referred to mittens (Click on the purple mittens), the screen accompanying the target instruction Now click on the XX xx contained a picture of mittens in a contrasting color (scarlet mittens; Table 1, in red). In the Unhelpful Prosody condition, the screen contained no object consistent with the contrastive prosody of the instruction – that is, no object contrasting with the one named in the context instruction. Table 1 illustrates each experimental condition (n = 12), with choices labeled in terms of whether each adjective and noun was given (G) information (the same as the adjective or noun in the context instruction) or new (N) information (different from the context adjective or noun).
Experimental conditions with unmasked (top) and masked (bottom) stimuli.
In addition to the 96 target trials, 24 filler trials were included to ensure that the same color pictures were not always placed opposite to each other, bringing the total number of trials to 120. The trials were divided into four sets. The first two sets contained stimuli with masked adjective-noun, and the last two sets contained unmasked stimuli.
At the beginning of each session, possible effects of lexical familiarity were minimized, by showing participants pictures of colors and items and having them repeat a recorded native English speaker's production of each color and object (as single words spoken in isolation). Practice trials were also given for participants to familiarize themselves with the response keys. The experiment was designed and conducted on Eyelink 1000 Plus platform (SR Research Ltd., Ontario, Canada), and behavioral responses to the second instruction were recorded as well as reaction times (RT),Footnote 4 measured as the elapsed time between the onset of the adjective in the audio stimulus to the response. Each experimental session, along with a short questionnaire on language background, took approximately 30 minutes in total.
Stimulus construction
The stimuli used in the unmasked trials were natural speech recorded by a linguistically trained male native speaker of English. In order to confirm that the contrastive part of the noun phrase (either adjective or noun) had a L + H* accent, a phonetician unrelated to this project ToBI transcribed a random set of the stimuli. An example stimulus for the ADJ-focus condition is shown in Appendix S2 (Supplementary Material).
Figure 1 shows the average pitch contours of the speaker's productions of ADJECTIVE + NOUN in the two focus conditions. The contours were time-normalized to show the general pitch contour pattern of the stimuli. Each unit on the x axis indicates 1/10 of a word, and the dotted line marks the boundary between the adjective and the noun. Figure 1 shows that stimuli for the ADJ-focus condition had a distinctive pitch rise and fall pattern, as did the NOUN-focus stimuli. In the ADJ-focus prosody, there was a rise in pitch up to the right edge of the stressed syllable in the trochaic adjective, followed by a sharp decrease. In the NOUN-focus prosody, the pitch stayed low for the adjective and the peak appeared in the stressed syllable of the focused noun.
Time normalized pitch contour of stimuli. Error bars represent the 95% confidence interval of the mean. The dashed vertical line indicates the boundary between adjective and noun.
Rather than using hummed speech, which might have introduced a bias toward items containing labial segments, we constructed the segmentally masked stimuli by manipulating the naturally produced stimuli. To create the 48 masked stimuli, we first attempted to mask the adjective and noun in Praat (Boersma & Weenink, Reference Boersma and Weenink2019) by low-pass filtering the adjective + noun portion of the sound file at 300 Hz with smoothing at 50 Hz, to remove all segmental cues for the NP above 300 Hz while leaving prosodic cues such as F0, intensity, and duration intact. To determine whether listeners could retrieve any segmental information from the masked portions, we invited eight native English speakers who were not linguistically trained and who were naive regarding the nature of the study to listen to instructions in which the adjective and noun were masked, and to choose an object from a screen containing three choices, one of which corresponded to the masked portion. Instructions were presented with no prior context. Because listeners were able to choose the original item at higher than chance level (64.6% for ADJ-focus stimuli and 63.5% for NOUN-focus stimuli), we introduced a more aggressive masking technique. We extracted the ADJECTIVE + NOUN portions of the audio files from two instructions recorded by the same speaker, where the two NPs shared no segmental information but had the same contrastive focus (e.g., the NPs from Now click on the FLOWERED towel and Now click on the DOTTED blanket). The two NP waveforms were aligned at the midpoint of the voiced portion of the focused syllable, with one file edited in length so that the voiced portions of the two files matched in duration. The two files were low-pass filtered and combined into a single audio file which was then concatenated with the instruction portion (Now click on the…) of a third file. None of the objects named in any of the three files appeared on the screen. An example of a resulting file is available in Appendix S3 (Supplementary Material). Three phonetically trained native speakers confirmed independently that no useful segmental information was available from the signal although the prosodic cues for contrastive focus were still present on the four syllables they perceived.
Analysis
Reaction times (RT) of correct responses for unmasked stimuli were calculated from the onset of the adjective. RTs were positively skewed (skewness: 1.78) and therefore were log10-transformed to meet the normality assumption underlying the general linear model. After the initial data screening that excluded 2 extreme outliers, data points with residuals larger than 2.5 sds were removed for model refinement, which improved the model in terms of R2 (Baayen & Milin, Reference Baayen and Milin2010).
The Log RT data were analyzed using linear mixed-effects regression models within the lme4 package (Bates, Mächler, Bolker & Walker, Reference Bates, Mächler, Bolker and Walker2015) of R (Baayen, Davidson & Bates, Reference Baayen, Davidson and Bates2008). Response data were analyzed using logistic mixed-effects regression modeling. In all analyses, fixed effects included prosody helpfulness (Helpful vs. Unhelpful Prosody), focus type (ADJ-focus vs. NOUN-focus), and language group (L1-English speakers vs. L1-Mandarin speakers). As random effects, we included random intercepts for participants and items, as well as by-participant random slopes for the effect of prosody helpfulness and focus type and by-item random slopes for the effect of prosody helpfulness and language group. However, when maximal models failed to converge, the random slope that captured the least variance was removed until the model converged (Barr, Levy, Scheepers & Tily, Reference Barr, Levy, Scheepers and Tily2013). The p-values were obtained from the lmerTest package (Kuznetsova, Brockhoff & Christensen, Reference Kuznetsova, Brockhoff and Christensen2017). Fixed factors were deviation coded with values −.5 and .5 prior to model fitting. In the response analysis, for the unmasked stimuli, response to the segmentally consistent target was coded as correct and the other three choices as incorrect. For the masked stimuli, NN1 (the first New-New object in which both the color and item were different from the context item) was arbitrarily chosen as the correct response in the Unhelpful prosody condition for comparison with prosodically appropriate responses in the Helpful Prosody condition. The choice was arbitrary, since the data showed that each choice in the Unhelpful Prosody condition was at chance level. Because the masked stimuli in the Unhelpful Prosody condition had no appropriate responses, we did not analyze RTs for masked stimuli.
Results
Prosody plus segments (unmasked stimuli)
As shown in Appendix S4 (Supplementary Material), the response proportion of the target object for both ADJ-focus and NOUN-focus prosody was above 96% for both language groups, regardless of the prosody (Helpful vs. Unhelpful). There was no main effect and no interactions, as can be seen in Table 2. This indicates that when segmental and prosodic information were contradictory, participants of both language groups relied on the segmental information.
Result of the logistic mixed effects model on response accuracy, unmasked stimuli
There was a main effect of language group (β = .07, SE = .02, p < .005) on Log RT, indicating that the L1-English speakers were overall generally faster in responding (see Figure 2). There was no main effect of prosody helpfulness (Helpful vs. Unhelpful prosody), suggesting that when segmental cues were available, Helpful Prosody was not advantageous for RT. There was a significant interaction between language group and focus type, indicating that the L1-English speakers responded more quickly in the ADJ-focus condition than in the NOUN-focus condition while the L1-Mandarin speakers did not.
English and Mandarin speakers’ mean Log RT (Log [ms]) for trials containing unmasked stimuli for Helpful Prosody (Left) and Unhelpful Prosody (Right). The length of the violins represents the range of y values; the width of the violins represents the density at a given y value; and the black lines represent the mean.
Result of the linear mixed effects model on Log RT, unmasked stimuli
Prosody without segments (masked stimuli)
Figure 3 shows the responses for the masked stimulus (prosody-only) trials. Dark grey violins indicate clicks on the prosodically appropriate choice (e.g., a picture of scarlet mittens in response to ADJ-focus prosody following the earlier instruction Click on the purple mittens). In order to test the responses against chance, we first computed a 95% multinomial confidence interval, indicated with dotted lines on Figure 3, assuming an even distribution across the 4 choices (25% each, given 4 available choices) for each condition and group. If responses were at chance level, the proportion of each choice should fall within that interval. Mean response proportions of the 4 choices in trials with unhelpful prosody all fell within the 95% confidence interval, confirming that there was no preferred choice. However, for ADJ-focus trials, when a prosodically appropriate object was present, both L1-English speakers and L1-Mandarin speakers were more likely to choose that object with a frequency above chance, as indicated by the fact that the response proportion of the prosodically appropriate item exceeded the upper limit of the confidence interval. With NOUN-focus trials, the L1-English speakers still showed a preference for the prosodically appropriate object when it was present, whereas the L1-Mandarin speakers no longer showed a preference for any specific choice, even when a prosodically appropriate object was available.
English and Mandarin speakers’ proportions of responses to each choice for ADJ-focus prosody and for NOUN-focus prosody in Helpful Prosody (Top) and Unhelpful Prosody (Bottom), masked speech. The length of the violins represents the range of y values; the width of the violins represents the density at a given y value; and the black dots represent the mean. The dark grey violins indicate clicks on the prosodically appropriate choice. N stands for ‘New’ and G for ‘Given’.
As shown in Table 4, we found two significant interactions in the logistic mixed-effects regression. First, the interaction between language group and prosody helpfulness (β = .72, SE = .30, p < .05) indicates that the two groups differed in their ability to use prosody, with the L1 English speakers generally better able to use contrastive prosody than the L1 Mandarin speakers when there was a prosodically appropriate choice. Second, the significant interaction between prosody helpfulness and focus type (β = .57, SE = .26, p < .01) shows that the preference for the prosodically appropriate choice was stronger for both groups when the contrastive accent fell on the adjective than when the contrastive accent fell on the noun.
Result of the logistic mixed effects model on response accuracy, masked stimuli
Effects of English language proficiency, length of residence, and age of acquisition
To determine whether the ability of the Mandarin–English bilinguals to use contrastive prosody was affected by individual factors, we examined the relationship between the L1-Mandarin speakers’ choice of prosodically appropriate items in masked speech and their level of English proficiency, as measured by their overall Versant scores, their Versant Fluency scores, their age of acquisition of English (AoA), and their length of residence (LoR) in an English-speaking environment. As the Mandarin–English bilinguals were not able to choose prosodically appropriate items in the NOUN-focus condition at a level above chance, we focused on the correlation between the individual background factors and the ability to choose a prosodically appropriate item with masked stimuli in the ADJ-focus condition, where the Mandarin speakers performed above chance. Although the L1 Mandarin speakers had a fairly wide range in background factors, we found no significant correlation (Pearson's r) between the Mandarin speakers’ prosodically appropriate response rate and their overall Versant scores (r = .20, p = .37), their Versant Fluency scores (r = .11, p = .62), their AoA (r = −.07, p = .77), or their LoR (r = .30, p = .20) (see Appendix S5 for correlation plots, Supplementary Material).
Discussion
This study was designed to determine whether bilinguals whose first language is tonal show a perceptual advantage over L1 English speakers in processing English contrastive prosody, for which pitch contour is a major cue. Previous studies (Choi et al., Reference Choi, Tong and Samuel2019; Qin et al., Reference Qin, Chien and Tremblay2017; Tong et al., Reference Tong, Lee, Lee and Burnham2015) have yielded results in which L1 tone language speakers did as well as or better than L1 English speakers in recognizing and discriminating English lexical stress differences so long as pitch cues to stress were available, though in some cases, this advantage emerged only with stimuli in which segmental information was suppressed (Choi, Reference Choi2021; Yu & Andruski, Reference Yu and Andruski2009). In contrast, previous studies investigating Mandarin speakers’ processing of English contrastive prosody – another area in which pitch cues are crucial – failed to find an L1 Mandarin perceptual advantage (Lee et al., Reference Lee, Perdomo and Kaan2020; Perdomo & Kaan, Reference Perdomo and Kaan2021). Because the lack of a tone language advantage in processing contrastive prosody might be attributable to the high cognitive load imposed when listeners are required to integrate prosodic, segmental, and pragmatic information, we tested the hypothesis that a perceptual advantage for contrastive prosody might emerge for Mandarin speakers when we removed the need to process simultaneously presented segmental and prosodic information identifying the target object.
For stimuli that contained both segmental and prosodic information, both language groups had similarly high accuracy rates, and neither group showed any obvious effect of appropriate vs. inappropriate prosody on reaction times, presumably because the small number of choice items on the screen made the segmental information sufficient for relatively rapid decisions independent of prosody. The Mandarin–English bilinguals did, however, show somewhat slower reaction times overall than the L1-English listeners, suggesting an overall disadvantage in processing. For stimuli in which the only cues to the intended referent were prosodic – which might be expected to be particularly advantageous for listeners with the heightened sensitivity to pitch cues conferred by a tone language – the Mandarin–English bilinguals were in fact less successful than the L1-English listeners in choosing a prosodically appropriate referent. We consider several possible explanations for why the perceptual advantage found in various earlier studies did not emerge in studies involving processing of contrastive prosody: that the perceptual advantage is task-specific; that performance was affected by differences between the Mandarin and English systems of contrastive prosody; and that performance was affected by the variability in English native speakers’ prosodic realization of contrast.
The tasks in which L2 speakers clearly outperformed L1 English speakers in earlier studies involved the identification, discrimination, or sequence recall of different word-level patterns of pitch accent or lexical stress (Choi et al., Reference Choi, Tong and Samuel2019; Kim & Tremblay, Reference Kim and Tremblay2021; Tong et al., Reference Tong, Lee, Lee and Burnham2015; Wiener & Goss, Reference Wiener and Goss2019) or of segmental contrasts (Chang, Reference Chang2016, Reference Chang2018; Chang & Mishler, Reference Chang and Mishler2012). These are tasks that could be carried out largely by focusing on the acoustic properties of single words, independent of semantic or pragmatic considerations. In contrast, the interpretation of contrastive prosody requires sensitivity not only to acoustic cues but to information at all levels from discourse down to individual phonemes. Taken together, these various studies suggest that a non-native advantage in perceiving L2 acoustic cues that are important in their L1 at the lexical level may not extend to tasks involving the interpretation of discourse-level prosody. The lack of a Mandarin advantage and the slower reaction times of Mandarin speakers in the current study are consistent with the hypothesis that integrating information from different grammatical domains is an area of particular difficulty for L2 listeners (Lee et al., Reference Lee, Perdomo and Kaan2020; Sorace, Reference Sorace2011), even when segmental information is removed from stimuli identifying the target object. This L2 disadvantage in integrating discourse-level and lexical information appears to be independent of the processing of contrastive prosody; for example, Kohlstedt and Mani (Reference Kohlstedt and Mani2018) found that L2-German speakers from various language backgrounds were less successful than native German speakers in using words earlier in the discourse (e.g., grandfather) to anticipate semantically related words (walking stick). These findings are also consistent with the hypothesis that a perceptual advantage is less likely to appear in tasks carrying a demanding cognitive load (Choi, Reference Choi2021), since in our task listeners were required to connect the contrastive prosody in the target instruction with the object named in the context instruction in order to generate an appropriate contrastive referent.
A second factor which might play a role in the lack of a perceptual advantage for the Mandarin–English speakers is the difference between the ways focus prosody is realized in Mandarin and in English. Whereas English contrastive focus is most often characterized by a specific pattern of pitch movement (generally analyzed as association of a pitch accent with the focused element), in Mandarin focus is signaled by a constellation of cues that may vary widely according to the lexical tone of the focused element and the surrounding tonal context. For example, in a word with high level Tone 1, contrastive focus is manifested as a raised pitch, while the same contrastive focus for a word with falling-rising Tone 3 is realized as a lowered pitch (Ouyang & Kaiser, Reference Ouyang and Kaiser2015; Wang, Liu, Lee & Lee, Reference Wang, Liu, Lee and Lee2020; Xu, Reference Xu1999). The use of different prosodic cues to focus is also highly tone-dependent. In a study of Mandarin corrective focus, Wang et al. (Reference Wang, Liu, Lee and Lee2020) investigated duration, intensity, and four pitch-related parameters (maximum F0, minimum F0, slope, and range) in the realization of focus and found considerable differences across lexical tones: for example, the primary cue to focus for Tones 1 and 4 was maximum F0, while the primary cue for Tones 2 and 3 was duration; intensity ranked third in importance for Tone 3 and fourth for Tone 1, but was not among the top four for Tones 2 and 4. Thus, the need to maintain lexical tone contrasts means that Mandarin has no single recognizable pitch contour associated with contrastive focus; rather, the specific prosodic cues used to realize focus vary with lexical tone. Post-focus compression of pitch, duration, and intensity is also dependent on tonal context, and may be lacking entirely in certain combinations of focus and post-focus tones (Chen, Reference Chen2010). The differences in the realization of focus prosody in English and Mandarin may account for differences between L1 English and L1 Mandarin speakers which have emerged in a number of production studies (Chen, Reference Chen2015; Kao, Hwang, Baek, Takahashi & Broselow, Reference Kao, Hwang, Baek, Takahashi and Broselow2016; Wennerstrom, Reference Wennerstrom1998) as well as studies suggesting that post-focus compression does not in general transfer across languages (Chen, Xu & Guion-Anderson, Reference Chen, Xu and Guion-Anderson2014; Wu & Chung, Reference Wu and Chung2011; Xu, Reference Xu2011).
If the performance of the Mandarin listeners was affected by interference from their L1, we might expect to see a correlation between accuracy and English proficiency and/or experience. Yet we found no correlation between any of the individual factors examined (age of acquisition of English, length of residence in the US, overall Versant score, or Versant fluency score), and even the two participants with very high Versant scores (77/80, 78/80) did not consistently choose a prosodically appropriate object. Similarly, Perdomo and Kaan (Reference Perdomo and Kaan2021) found that the ability of the L1 Mandarin speakers in their study to use prosody to predict an upcoming contrastive referent did not correlate with either English proficiency or working memory.Footnote 5
Finally, one factor that might account for the Mandarin–English speakers’ lower performance is the high degree of variability in the realization of contrastive prosody by native speakers of English. Indeed, some of our L1-English speakers were unable to reliably select an appropriate contrastive referent when the only information available was the prosody: the L1-English speakers’ prosodically appropriate choices with helpful adjective-focus prosody ranged from 8.3–91.6% (cf. the L1-Mandarin speakers 8.3–66.6%). While it might seem surprising that the performance of the L1-English speakers was not higher, this finding is consistent with studies demonstrating that the relationship between information structure and specific prosodic cues in English is probabilistic rather than categorical, with a great deal of both intra- and inter-speaker variation in the use of specific intonation contours (e.g., Chodroff & Cole, Reference Chodroff, Cole, Calhoun, Escudero, Tabain and Warren2019).
The hypothesis that listeners were affected by the variability of focus prosody in English is consistent with the fact that both language groups displayed greater sensitivity to adjective-focus prosody than to noun-focus prosody, as revealed by the significant interaction between prosody helpfulness and focus type on response accuracy to masked stimuli (Table 4). This dovetails with other studies suggesting that native speakers of English are more consistent in producing pitch peaks on contrasting adjectives than on contrasting nouns (Ito, Speer & Beckman, Reference Ito, Speer and Beckman2004; Yeung, Baek, Takahashi, Duncan, Benedett, Hwang & Broselow, Reference Yeung, Baek, Takahashi, Duncan, Benedett, Hwang and Broselow2019), which means that listeners are more likely to hear focused adjectives realized with a contrastive pitch accent in English than to hear focused nouns with contrastive accent. English speakers also seem better able to perceive focus prosody on a prenominal adjective than on the following noun: Ito and Speer (Reference Ito and Speer2008) found that felicitous contrastive prosody on a focused adjective, but not on a focused noun, was associated with anticipatory fixations to the target for English speakers, while Wayland, Guerra, Chen, and Zhu (Reference Wayland, Guerra, Chen and Zhu2019) found that both L1-English and L1-Mandarin speakers tended to confuse focus on a phrase-final noun phrase with broad focus prosody, where prominence falls on the final lexical category word of the phrase.
These various explanations – the difficulty of integrating information across different domains; the decreased advantage to cue attention in the presence of demanding cognitive load; interference from L1 prosodic realization patterns; and the variability in the prosodic signaling of contrast in English – are of course not mutually exclusive, and all these factors may combine to account for the lack of a Mandarin perceptual advantage found in this study. Taken together, the studies in this area suggest that while tone language speakers may enjoy an advantage in tasks that require the ability to recognize and discriminate sequences differing in pitch cues, this advantage is not sufficient to outweigh the difficulty of interpreting the meaning of prosody within the context of a discourse, even when listeners were forced to rely on prosodic cues alone.
The hypothesis that a tone language advantage is most likely in tasks that do not require the integration of prosody with semantic or discourse-level information could be tested in various ways. First, in a discrimination experiment in which listeners are simply required to determine whether prosodic contours (such as adjective-focus and noun-focus instructions) are the same or different, without the necessity to interpret what those differences indicate, we would expect that L1-Mandarin speakers should show accuracy rates as high as or higher than those of L1 English speakers. Second, if the task in this study were extended to include additional language groups, we would expect that L1-Mandarin speakers should outperform L1 speakers of non-tonal languages such as Italian and Spanish which differ from English in the prosodic signaling of contrast (Ladd, Reference Ladd2008). These predictions remain to be explored.Footnote 6
Supplementary Material
For supplementary material accompanying this paper, visit https://doi.org/10.1017/S1366728921001176
Acknowledgements
This study is based upon work supported by NSF under Grant # IBSS-1519908. Special thanks to Sharon Benedett and Kevin Henderson for stimulus construction.
