Introduction
Speech perception often occurs in suboptimal listening conditions with background noise or multiple talkers. In these conditions, speech perception is more effortful (Pichora-Fuller, Kramer, Eckert, Edwards, Hornsby, Humes, Lemke, Lunner, Matthen, Mackersie, Naylor, Phillips, Richter, Rudner, Sommers, Tremblay & Wingfield, Reference Pichora-Fuller, Kramer, Eckert, Edwards, Hornsby, Humes, Lemke, Lunner, Matthen, Mackersie, Naylor, Phillips, Richter, Rudner, Sommers, Tremblay and Wingfield2016; Zekveld, Heslenfeld, Johnsrude, Versfeld & Kramer, Reference Zekveld, Heslenfeld, Johnsrude, Versfeld and Kramer2014) and relies on the successful use of bottom-up and top-down processes (Bradlow & Alexander, Reference Bradlow and Alexander2007). Cognitive demands are even greater when listening in one's second language (L2) compared to one's first (L1) (e.g., Borghini & Hazan, Reference Borghini and Hazan2018; Mayo, Florentine & Buus, Reference Mayo, Florentine and Buus1997), likely due to L2 listeners having imperfect language knowledge (Garcia Lecumberri, Cooke & Cutler, Reference Lecumberri, Cooke and Cutler2010). Moreover, higher-order processes (e.g., prosodic and syntactic processing) may be less efficient in one's non-native language (e.g., Akker & Cutler, Reference Akker and Cutler2003; Clahsen & Felser, Reference Clahsen and Felser2006). Thus, the extent to which L2 listeners effectively use top-down processes and higher-order cues (e.g., semantic context) while perceiving speech-in-noise is unclear. The present study examined the use of semantic context during L1 and L2 speech perception in noise in bilinguals through the recording of electroencephalography (EEG) measures. Furthermore, we examined whether semantic context use in L2 speech perception is influenced by L2 knowledge and experience, as well as working memory.
Under optimal listening conditions, auditory word recognition is theorized to be driven by bottom-up perceptual processes and informed by higher-order cognitive processes (e.g., Gaskell & Marslen-Wilson, Reference Gaskell and Marslen-Wilson1997; Marslen-Wilson & Tyler, Reference Marslen-Wilson and Tyler1980). However, background noise distorts the neural representations of speech and impedes the listener's ability to extract the signal from competing background noise (e.g., Parbery-Clark, Marmel, Bair & Kraus, Reference Parbery-Clark, Marmel, Bair and Kraus2011). Speech signals become less intelligible with decreasing signal-to-noise ratios (SNRs) and increasing spectral overlap between the speech signal and the masking noise (e.g., Rogers, Lister, Febo, Besing & Abrams, Reference Rogers, Lister, Febo, Besing and Abrams2006). In order to disambiguate the target signal from background noise, it is argued that listeners employ cognitive resources to a greater extent compared to listening in quiet (e.g., Mattys, Davis, Bradlow & Scott, Reference Mattys, Davis, Bradlow and Scott2012; Peelle, Reference Peelle2018; Pichora-Fuller et al., Reference Pichora-Fuller, Kramer, Eckert, Edwards, Hornsby, Humes, Lemke, Lunner, Matthen, Mackersie, Naylor, Phillips, Richter, Rudner, Sommers, Tremblay and Wingfield2016). In line with this, perceiving degraded speech is associated with increased pupil dilation (a correlate of cognitive effort; e.g., Zekveld et al., Reference Zekveld, Heslenfeld, Johnsrude, Versfeld and Kramer2014). Increased listening effort is thought to shift the balance between bottom-up and top-down processes, such that listeners rely more on higher-order processes and cues while perceiving speech-in-noise (Mattys et al., Reference Mattys, Davis, Bradlow and Scott2012; Peelle, Reference Peelle2018). Consistent with this, working memory capacity may be related to the ability to successfully perceive speech-in-noise in a native language (e.g., Ingvalson, Dhar, Wong & Liu, Reference Ingvalson, Dhar, Wong and Liu2015; Millman & Mattys, Reference Millman and Mattys2017). Moreover, researchers have repeatedly shown that listeners benefit from semantic context while perceiving speech-in-noise in their native language (e.g., Boothroyd & Nittrouer, Reference Boothroyd and Nittrouer1988; Cohen & Faulkner, Reference Cohen and Faulkner1983; Kalikow, Stevens & Elliott, Reference Kalikow, Stevens and Elliott1977a; Miller, Heise & Lichten, Reference Miller, Heise and Lichten1951).
Most research on speech perception in noise has either been conducted with native listeners or without mention of participants’ language backgrounds. However, the number of bilingual individuals is increasing globally, with bi-/multilingualism often being more common than monolingualism (e.g., Statistics Canada, 2017; Ryan, 2013; Eurostat, 2015). It is therefore important to understand the processes involved in speech perception in noise in both native and non-native languages.
Bilinguals are hypothesized to experience speech perception challenges in L2 compared to L1 due to their increased difficulty with aspects of L2 language processing. For example, bilinguals have shown limits in vocabulary (e.g., Bialystok, Craik & Luk, Reference Bialystok, Craik and Luk2008) and less efficient syntactic (e.g., Hwang, Shin & Hartsuiker, Reference Hwang, Shin and Hartsuiker2018) and phonological processing (e.g., Navarra, Sebastián-Gallés & Soto-Faraco, Reference Navarra, Sebastián-Gallés and Soto-Faraco2005). Research does suggest that speech perception is more challenging and effortful for L2 listeners. For example, Borghini and Hazan (Reference Borghini and Hazan2018) reported greater pupil dilation, suggesting greater cognitive effort, for L2 compared to L1 listeners. Notably, when listening in quiet, bilinguals are able to use semantic information provided by sentences in their L2 to the same extent as in their L1 (Dijkgraaf, Hartsuiker & Duyck, Reference Dijkgraaf, Hartsuiker and Duyck2017). Thus, compared to L1, L2 listeners may experience processing challenges that make speech perception more effortful but are able to use higher-order cues to successfully perceive speech in quiet.
In contrast, perceiving L2 speech-in-noise is thought to require more effort than in quiet and an even greater involvement of higher-order processes (e.g., processing syntactic and prosodic cues; Garcia Lecumberri et al., Reference Lecumberri, Cooke and Cutler2010). L2 listeners have been repeatedly shown to be more affected by noise (i.e., perform with lower accuracy) compared to L1 listeners (e.g., Cooke, Garcia Lecumberri & Barker, Reference Cooke, Garcia Lecumberri and Barker2008; Garcia Lecumberri & Cooke, Reference Lecumberri and Cooke2006; Mayo et al., Reference Mayo, Florentine and Buus1997; Rogers et al., Reference Rogers, Lister, Febo, Besing and Abrams2006; Shi, Reference Shi2009, Reference Shi2010). However, higher-order processes may be less efficient during L2 listening (e.g., Akker & Cutler, Reference Akker and Cutler2003; Clahsen & Felser, Reference Clahsen and Felser2006), further compounding the language processing difficulties mentioned above.
Findings are mixed as to whether bilinguals benefit from semantic context while perceiving L2 speech-in-noise. While some studies failed to observe a benefit of semantic context (Golestani, Rosen & Scott, Reference Golestani, Rosen and Scott2009; Hervais-Adelman, Pefkou & Golestani, Reference Hervais-Adelman, Pefkou and Golestani2014), others show a benefit although to a lesser extent than in a native language (Bradlow & Alexander, Reference Bradlow and Alexander2007; Shi, Reference Shi2014). Other studies highlight the modulating influence of L2 AoA and language proficiency on the ability to capitalize on semantic context while perceiving L2 speech-in-noise. For example, Mayo and colleagues (Reference Mayo, Florentine and Buus1997) and Shi (Reference Shi2010) found that simultaneous and early bilinguals benefited more from semantic context than late bilinguals. Similarly, a recent study by our group (Kousaie, Baum, Phillips, Gracco, Titone, Chen, Chai & Klein, Reference Kousaie, Baum, Phillips, Gracco, Titone, Chen, Chai and Klein2019) found that simultaneous and early bilinguals, but not late bilinguals, benefited from semantic context while perceiving L2 sentences in noise using an fMRI paradigm. Furthermore, Gor (Reference Gor2014) reported that high-proficiency heritage speakers benefited more from semantic context while perceiving sentences in noise, compared to both low-proficiency heritage speakers and late L2 learners with high or low L2 proficiency. Thus, to better understand semantic context use during L2 speech perception in noise, factors related to language knowledge and experience must be considered.
As noted above, most studies of speech perception in noise have been conducted with monolinguals or native listeners, with relatively few studies on bilinguals. The few studies that have examined bilingual semantic context use during speech perception in noise have mainly used behavioural measures of speech recognition (i.e., accuracy in reporting a target word or phrase) and word-pair stimuli. Few studies have examined the influence of factors relating to bilingual language experience (e.g., AoA), with previous studies often focusing on bilinguals with a specific language background (e.g., late bilinguals with limited L2 proficiency). Therefore, in this study, we examine semantic context use during L1 and L2 speech perception in noise in highly proficient bilinguals with varying ages of L2 acquisition by measuring the N400 event-related brain potential (ERP). We examined whether the amplitude and latency of the N400 is modulated by L2 knowledge and experience, as well as working memory capacity. This EEG study builds on work by Kousaie et al. (Reference Kousaie, Baum, Phillips, Gracco, Titone, Chen, Chai and Klein2019) who used an fMRI SPIN task with similar stimuli and similar participant groups, with some participants overlapping across both studies. The combined use of these two methods allows us to determine the brain regions implicated in perceiving speech-in-noise with fMRI, while using EEG enables us to further examine the cognitive processes as they unfold over time.
The N400, a negativity peaking approximately 400 milliseconds after a word stimulus (Kutas & Federmeier, Reference Kutas and Federmeier2011; Kutas & Hillyard, Reference Kutas and Hillyard1980), is thought to be related to semantic processing. The N400 may reflect semantic access during language comprehension (e.g., Kuperberg & Jaeger, Reference Kuperberg and Jaeger2016; Kutas & Federmeier, Reference Kutas and Federmeier2011), with smaller amplitudes reflecting facilitated processing due to pre-activation of upcoming language representations. The N400 may also reflect lexico-semantic integration (e.g., Brown & Hagoort, Reference Brown and Hagoort1993; Hagoort, Baggio & Willems, Reference Hagoort, Baggio and Willems2009), with larger amplitudes reflecting more effortful integration of a word into the preceding context. The N400 is modulated by sentence context, with larger amplitudes evoked by words that are semantically anomalous with respect to the preceding sentence context or occur in sentences low in contextual constraint (e.g., Connolly, Phillips, Stewart & Brake, Reference Connolly, Phillips, Stewart and Brake1992; Kutas & Hillyard, Reference Kutas and Hillyard1980). Additionally, longer N400 latencies have been observed in response to low-constraint compared to high-constraint sentences (e.g., León-Cabrera, Rodríguez-Fornells & Morís, Reference León-Cabrera, Rodríguez-Fornells and Morís2017). N400 studies in bilinguals typically focus on visually presented word stimuli (e.g., Jankowiak & Rataj, Reference Jankowiak and Rataj2017; Zirnstein, van Hell & Kroll, Reference Zirnstein, van Hell and Kroll2018). Nevertheless, researchers have found smaller N400 amplitudes and longer N400 latencies in response to auditorily presented semantically incorrect L2 sentences compared to L1 sentences (e.g., Hahne, Reference Hahne2001).
Based on the literature reviewed above, we expect bilinguals to benefit more from semantic context while perceiving L1 speech-in-noise compared to L2, reflected by an earlier and larger N400 effect (greater amplitude difference between high-constraint and low-constraint sentences) in L1 compared to L2. Furthermore, we expect the N400 effect during L2 speech perception to be modulated by L2 language experience. That is, we hypothesize that more years of L2 experience, higher L2 proficiency, and more time spent using L2 compared to L1, will be associated with a greater benefit of semantic context during L2 speech perception, as indicated by a larger and/or earlier N400 effect. Lastly, we anticipate better working memory to be associated with a larger and/or earlier N400 effect while perceiving L2 speech-in-noise.
Methods
Participants
Participants were recruited using advertisements at Concordia University, McGill University, and Université de Montréal and included 19 simultaneous bilinguals, 20 early bilinguals, and 18 late bilinguals. As shown in Table 1, participants were bilingual speakers of English and French, with either as their L1, and had no functional knowledge of a third language. Simultaneous bilinguals had learned both of their languages from birth. Early bilinguals started learning their L2 by age five and late bilinguals after age five. L2 AoA ranged from age 0 to 15 across all participants. Simultaneous bilinguals self-reported which language was their dominant language at the time of testing and this was used as their L1 for our analyses. Participants self-rated their L1 and L2 proficiency in speaking and listening on a scale from 1 (‘not at all proficient’) to 7 (‘native-like proficiency’). On average, participants rated themselves as being moderately to highly proficient in both of their languages with speaking and listening proficiencies ranging from 5 to 7 for L1 and from 4 to 7 for L2. Participants varied in the percentage of their total conversations in which they used each of their languages, with L2 use ranging from 5% to 95%. Importantly, AoA groups did not differ in chronological age (range: 18-36 years old), self-rated L1 and L2 speaking and listening proficiency, or percentage of L1 and L2 language use (see Table 1 for participant demographics). All AoA groups self-reported lower L2 speaking proficiency compared to L1 (all p values < .007) while only the early and late bilinguals self-reported lower L2 listening proficiency compared to L1 (p = .001 and p = .002, respectively). Participants also completed objective measures of L1 and L2 language performance (i.e., letter fluency, category fluency, and sentence repetition) as well as working memory (i.e., forward, backward, and sequencing digit span, and letter number sequencing; see Table 1). These measures are described in more detail below. Notably, simultaneous, early, and late bilinguals did not differ in L1 language performance, L2 language performance, or working memory performance. Performance on the category fluency task did not differ between L1 and L2 for any of the AoA groups (p values > .178). Performance on the letter fluency task was greater for L1 compared to L2 for the early (p = .043) and late bilinguals (p = .001) only. Performance on the sentence repetition task was greater for L1 compared to L2 for the simultaneous bilinguals only (p = .005). All participants were right-handed young adults with normal hearing as assessed by pure-tone average (PTA) thresholds. Participants gave informed consent and were given monetary compensation for participating.
Participant Demographics.
Note. Standard deviations are presented in parentheses. Simultaneous bilinguals self-reported which of their two languages they felt was their dominant language at the time of testing and this dominant language was used as their L1. Groups did not differ in age, language measures, or working memory performance (all p values > .08).
Materials
SPIN stimuli
The SPIN stimuli used here are the same as those used in the complementary fMRI study by Kousaie et al. (Reference Kousaie, Baum, Phillips, Gracco, Titone, Chen, Chai and Klein2019); however, the SNR in the noisy listening condition was much lower in the fMRI study (−6 dB) compared to the present ERP study (+1 dB). The SNR for the current study was determined by behavioural performance in the most difficult condition (i.e., low-constraint, L2 sentences in noise) during pilot testing. Our goal was to select an SNR that would be challenging in the easier listening conditions, but not so challenging that it would lead to floor effects in the most difficult condition. Pilot testing revealed that an SNR of +1 dB resulted in a 30% error rate in the most challenging listening condition. This choice is supported by the error rates of our study showing neither floor effects nor ceiling effects, reflected by an error rate of 27% in the most difficult listening condition (i.e., low-constraint, L2 sentences in noise) and 5% in the easiest noise condition (i.e., high-constraint, L1 sentences in noise). A total of 240 sentences were adapted from the Revised Speech Perception in Noise Test (SPIN-R; Kalikow, Stevens & Elliott, Reference Kalikow, Stevens and Elliott1977b). Semantic context is manipulated in the SPIN-R sentences to yield high- and low-constraint sentences. High-constraint sentences provide rich semantic context, leading to a highly predictable final target word (e.g., “The lion gave an angry roar.”). In contrast, low-constraint sentences do not provide sufficient semantic context and thus lead to an unpredictable final target word (e.g., “He is thinking about the roar.”). Sentences from the SPIN-R test are English sentences with five to eight words and six to eight syllables. All terminal words are monosyllabic nouns with mid-range word frequency counts (5 to 150 per million words; Lorge & Thorndike, Reference Lorge and Thorndike1952). Sixty high-constraint and 60 low-constraint sentences were selected from the original SPIN-R list and matched on number of words (high-constraint: M = 5.5, SD = .81; low-constraint: M = 4.9, SD = .79) and number of syllables (high-constraint: M = 6.5, SD = .70; low-constraint: M = 6.6, SD = .70).
An additional 120 SPIN-R sentences (60 high-constraint and 60 low-constraint) were selected from the original set and adapted to French. To match high- and low-constraint French sentences on sentence length, French sentences were not all direct translations of English SPIN-R sentences. For example, “The bread was made from whole wheat” was adapted to “Le pain brun est fait de blé”. High- and low-constraint French sentences were matched on number of words (high-constraint: M = 5.8, SD = 1.01; low-constraint: M = 5.0, SD = 1.15) and number of syllables (high-constraint: M = 7.7, SD = 1.04; low-constraint: M = 7.3, SD = 1.21). Target terminal French words were approximately 45% monosyllabic and 55% disyllabic. English and French terminal words were matched on spoken frequency (English: M = 20.5, SD = 27.50; French: M = 24.4, SD = 28.90), phonological neighbourhood density (English: M = 15.4, SD = 9.22; French: M = 16.4, SD = 7.38), imageability (English: M = 539.5, SD = 65.77; French: M = 563.0, SD = 48.44), and familiarity (English: M = 524.5, SD = 51.36; French: M = 517.4, SD = 55.09) using the MRC Psycholinguistic Database (Coltheart, Reference Coltheart1981), Lexique 3 (New, Reference New2006; New, Pallier, Ferrand & Matos, Reference New, Pallier, Ferrand and Matos2001), and the Corpus of Contemporary American English (Davies, Reference Davies2008).Footnote 1
All sentences were recorded by a female, simultaneous bilingual speaker of English and French in a sound-attenuated booth using an Olympus recorder with a 44.1 kHz sample-rate and 32-bit resolution. Sentence stimuli were presented in both a quiet and a noise condition. The background noise used for both English and French listening conditions consisted of an English multi-talker babble adapted from Bilger, Nuetzel, Rabinowitz, and Rzeczkowski (Reference Bilger, Nuetzel, Rabinowitz and Rzeczkowski1984). The original eight-talker babble, which was low-pass filtered at 7500 Hz, was overlaid three times with a slight temporal jitter to create a babble mask that was less variable in its intensity fluctuations. Although it is possible that using an English-only babble may have influenced our findings, an acoustic analysis revealed that it did not likely provide any informational content. In addition, the low-pass filtering and the overlay of multiple speakers made it difficult to subjectively tell which language the babble was drawn from.
Sentence stimuli were presented in eight experimental conditions (four experimental conditions per language). For example, the English conditions were: high-constraint sentences in quiet; low-constraint sentences in quiet; high-constraint sentences in noise; low-constraint sentences in noise. Each target word was heard in each condition within each language, but two lists were created so that each word was heard only twice in each list. Thus, within each list, each target word was heard once in a high- and once in low-constraint sentence, as well as once in noise and once in quiet; (e.g., the terminal word “spoon” was heard in the high-constraint quiet and the low-constraint noise conditions in List 1 and was heard in the low-constraint quiet and high-constraint noise conditions in List 2). Each list consisted of eight experimental blocks. Lists were blocked by listening condition (quiet and noise) and language (English and French), both of which were counterbalanced within each list. Low-constraint and high-constraint sentences were pseudo-randomly intermixed within each block such that they were presented equally often within each block and no more than three similarly constrained sentences appeared in sequence. Each participant heard only one list and lists were counterbalanced across participants.
Language proficiency measures
Participants completed letter and category verbal fluency tasks, and a sentence repetition task in each language. These tasks were selected because they are objective, commonly-used language measures that have been shown to distinguish between first- and second-language performance (e.g., Sandoval, Gollan, Ferreira & Salmon, Reference Sandoval, Gollan, Ferreira and Salmon2010).
In the fluency tasks, participants were asked to say as many words as possible in one minute (excluding proper nouns, numbers, and words differing only in suffix) that began with a given letter of the alphabet (F, A, S for English; P, F, L for French) or drawn from a semantic category (animals and fruit for English and French, respectively). The number of words produced for all three letters within each language were summed to give a single score for each language. The number of words generated for each category was counted to give a score for category fluency for each language.
In the sentence repetition subtest of the Clinical Evaluation of Language Fundamentals, Fourth Edition (CELF-4; Semel, Wiig & Secord, Reference Semel, Wiig and Secord2003), the experimenter read single sentences aloud and the participant was asked to repeat each sentence immediately after hearing it. Each trial received a score out of three, with a score of three indicating zero repetition errors and a score of zero representing four or more repetition errors. This task was performed in each language, with 24 sentences per language. Scores for each trial were summed to give a total score out of 72 for each language.
Working memory tests
The Digit Span and Letter Number Sequencing subtests of the Wechsler Adult Intelligence Scale Fourth Edition (Wechsler, Reference Wechsler2008) were used as measures of working memory and were administered in the participants’ native language.
For the Digit Span subtest, participants were asked to repeat a series of spoken numbers in order (Forward Digit Span), in reverse order (Backward Digit Span), or in sequential order from smallest to largest (Sequencing). The span length increased by one digit after every two trials and the task ended after two incorrect responses per level. One point was given for each correct trial and summed to give a total score for each variant of the task.
In the Letter Number Sequencing task, participants were read a series of letters and numbers and asked to repeat the numbers in sequential order, followed by the letters in alphabetical order. The span length increased by one unit after every three trials and the task ended after three incorrect responses in a level. One point was given for each correct trial and the points were summed to give a total score.
Procedure
All participants completed two testing sessions on two different days. In the first session, participants completed the language proficiency tests, working memory tests, a PTA hearing threshold test, and a language background questionnaire in which they self-reported detailed information regarding their L1 and L2 proficiency, AoA, and patterns of language use. In the second session, participants performed the SPIN task while EEG was continuously recorded. During this session, participants were seated in a sound attenuated booth in front of a computer monitor. Participants first completed a practice block of the SPIN task in both languages. Practice trials consisted of 41 sentences (22 English, 19 French; approximately half high-constraint and half low-constraint). Five sentences in each language were presented in quiet and the rest in noise. Participants then completed one list (i.e., 240 sentences; described above) of the experimental SPIN task. Sentences were binaurally presented through EARLINK tube ear inserts (Neuroscan, El Paso, TX, USA) using Inquisit 4.0 (Millisecond Software, Washington). Thus, each participant listened to high- and low-constraint sentences in English and French, in quiet and in noise (SNR +1 dB). During sentence presentation, a fixation cross was presented on the computer screen. Participants were prompted to repeat the final word of the preceding sentence 1000 ms after the end of the sentence (i.e., when “Final Word?” appeared on the computer screen). Responses were manually scored as correct or incorrect by the experimenter. Scoring was lenient in that responses were accepted as correct if the participant made a pluralization error that was semantically and syntactically correct within the context of the sentence. Responses were also accepted if, in addition to reporting the correct word, participants included the appropriate determiner in the French sentences. Only trials scored as correct were included in the EEG analyses. A total of 1410 trials (i.e., 10.3% of trials) were excluded from the EEG analyses due to incorrect responses.
EEG data acquisition and analysis
EEG was recorded using a 64-electrode nylon cap and an ActiveTwo system (Biosemi, Amsterdam, NL) with a sampling rate of 2048 Hz and a bandwidth of .01 to 100 Hz. Additional electrodes were placed above and below the left eye and on the left and right canthi to monitor for horizontal and vertical eye movements. All electrodes were referenced to electrodes placed on the left and right earlobes.
EEG data was processed using BrainVision Analyzer 2.0.3 (Brain Products, Gilching, DE). The data were screened manually to remove exceptionally large artifacts and sections of data between experimental blocks. A low-pass filter of 100 Hz and a high-pass filter of 0.01 Hz were applied, as well as a DC drift correction. Artifacts from vertical and horizontal eye movements were removed using the Ocular Correction Independent Components Analysis. The EEG was then segmented into 1100 ms segments beginning 100 ms before terminal word onset and ending 1000 ms after terminal word onset. These segmentations were performed for each of the eight experimental conditions separately. Semi-automatic artifact rejection was conducted for all segments within each condition by removing segments where: a) the absolute difference between two adjacent data points exceeded 50 microvolts, b) the difference between the maximum and minimum amplitude within a segment exceeded 200 microvolts, or c) the activity fell below 0.5 microvolts. Based on these criteria, a total of 412 segments were removed (3.5%). No systematic differences were found in the number of segments removed as a function of condition. Segments were averaged per condition for each participant. The N400 peak was identified and scored in the averaged waveform for each condition at four midline electrode sites: Fz, FCz, Cz and CPz. Based on previous studies examining the N400 using auditorily presented sentences (e.g., Holcomb & Neville, Reference Holcomb and Neville1991), the N400 was operationally defined as the most negative peak between 250 and 600 ms, that was temporally consistent across the four midline electrode sites, following terminal word onset. The topographical distribution of the N400 was characterized by examining the mean amplitudes within 300 to 600 ms following terminal word onset at four lateral site electrode regions of interest (ROIs; Figure 1).
Electrode clusters used for the topographical analysis of the N400 effect. Left anterior: AF3, F5, F3, F1, FC5, FC3, FC1. Right anterior: AF4, F2, F4, F6, FC2, FC4, FC6. Left posterior: CP5, CP3, CP1, P5, P3, P1, PO3. Right posterior: CP2, CP4, CP6, P2, P4, P6, PO4.
Statistical analyses
Statistical analyses were performed using IBM SPSS 23.0 (IBM Corp., 2015). The Greenhouse-Geisser non-sphericity correction was applied to all analyses and the Bonferroni correction was applied to adjust for multiple comparisons.
A mixed factorial ANOVA was conducted to examine potential differences in behavioural accuracy between groups on the SPIN task. Within-subjects factors included language (L1, L2), listening condition (quiet, noise), and context (high-constraint, low-constraint). The between-subjects factor was AoA group (simultaneous, early and late bilinguals).
Two families of ANOVAs were done to analyze the N400; one for midline electrodes (Fz, FCz, Cz, and CPz) and one for lateral site ROIs (Figure 1). First, to examine any differences in the N400 peak amplitude and latency between groups and conditions, a 3 (group) X 2 (language) X 2 (listening condition) X 2 (context) X 4 (electrode) mixed factorial ANOVA was conducted. For each participant, and within each language and listening condition, the high-constraint waveforms were then subtracted from the low-constraint waveforms to better examine the N400 effect (i.e., the context effect). The N400 effect was identified in these subtracted waveforms following the same operational definition mentioned above and analysed in a 3 (group) X 2 (language) X 2 (listening condition) X 4 (electrode) mixed factorial ANOVA.
Second, mean amplitudes within 300 to 600 ms of the subtracted waveforms were analyzed from four lateral site ROI electrode clusters (left anterior, right anterior, left posterior and right posterior; Figure 1) using a 3 (group) X 2 (language) X 2 (listening condition) X 4 (ROI) mixed factorial ANOVA to better characterize any topographical differences between the groups.
Lastly, a principal components analysis (PCA) and Pearson correlations were run to examine the association between L2 semantic context use (i.e., the N400 effect amplitude and behavioural accuracy) and individual differences in language experience and working memory. The PCA was run using the “prcomp” function in R version 3.5.1 (R Core Team, 2018) and included the variables: years of L2 experience, percent L2 use, L2 minus L1 (L2−L1) letter fluency, L2−L1 category fluency, L2−L1 sentence repetition, forward, backward and sequencing digit spans, and letter number sequencing. Number of L2 years of experience was calculated for each participant by subtracting their L2 AoA from their age at testing. L1 scores on the fluency and sentence repetition tasks were subtracted from L2 scores to give measures of relative proficiency balance between the languages. Scores close to zero reflect balanced L1 and L2 proficiency. Larger positive scores reflect greater L2 proficiency relative to L1 and larger negative scores reflect greater L1 proficiency relative to L2. Given that L2 experience can influence L1 proficiency, it is arguably more meaningful to assess a bilingual's language proficiency by taking into account both their L1 and L2 instead of studying either language in isolation. Thus, L1 scores were subtracted from L2 to account for both L1 and L2 proficiency. Variables were centered at zero and scaled to have unit variance using “center” and “scale” arguments in the “prcomp” function. The first three principal components (PCs) accounted for 66.9% of the total variance and were found to reflect language proficiency, working memory performance, and L2 years of experience, respectively (see Table 2 for contributions of variables to each principal component). Pearson correlations were then conducted (“cor.test” function) between each of the PCs and the N400 effect amplitude in L2 quiet and noise conditions, as well as between the three PCs and behavioural accuracy in L2 noise conditions.
Percentage of variance accounted for by each variable for each principal component.
Results
Behavioural accuracy
All groups were more accurate on high-constraint sentences (M = 95.2%, SE = .51) compared to low-constraint sentences (M = 84.2%, SE = 1.18; F(1, 54) = 150.06, p < .001, η 2 = .141). Similarly, all groups were more accurate in quiet compared to noise (quiet: M = 96.3%, SE = .45; noise: M = 83.1%, SE = 1.45; F(1, 54) = 81.18, p < .001, η 2 = .207), and this difference was exaggerated for low- compared to high-constraint sentences, as well as for L2 compared to L1 sentences (Figure 2; F(1, 54) = 67.55, p < .001, η 2 = .063; F(1, 54) = 5.10, p = .028, η 2 = .001; respectively). A main effect of language was observed (F(1, 54) = 13.33, p = .001, η 2 = .010), which interacted with group (F(1, 54) = 3.69, p = .031, η 2 = .005) such that performance was overall less accurate in L2 compared to L1 for the late bilinguals only. See Table 3 for means and standard errors.
Behavioural accuracy in repeating sentence terminal words for high and low constraint sentences in quiet and noise, in L1 and L2, for all three AoA groups. Accuracy is shown in percent.
Behavioural Accuracy.
Note. Mean accuracies are reported in percentages. Standard error is indicated in parentheses. HC: high constraint. LC: low constraint
N400
General observations
The waveforms depicting the amplitude and latency effects of the N400 are shown in Figure 3. An N400, larger for the low- compared to high-constraint conditions, is evident beginning around 200 ms and peaking between approximately 300 and 400 ms.
High and low constraint grand average waveforms in quiet and noise, L1 and L2 for all three AoA groups. Waveforms are displayed at electrode site Cz, which is broadly representative of the effects. Arrows indicate the approximate N400 peaks. An effect of context is displayed, with larger amplitudes and longer latencies for low-constraint compared to high-constraint sentences. Larger amplitudes and longer latencies were also observed in the quiet compared to noise conditions. Moreover, more negative amplitudes were observed in L2 compared to L1 for early bilinguals only.
Amplitude
A main effect of constraint was observed: amplitudes were more negative for low- compared to high-constraint sentences (LC: M = −6.7 μV, SE = .41; HC: M = −3.4 μV, SE = .31; F(1, 54) = 145.30, p < .001, η 2 = .140). Amplitudes were more negative in quiet compared to noise (quiet: M = −5.5 μV, SE = .40; noise: M = −4.6 μV, SE = .35; F(1, 54) = 6.61, p = .013, η 2 = .009). There was no effect of language (F(1, 54) = 3.37, p = .072, η 2 = .004). However, there was a language by group interaction such that amplitudes were more negative in L2 compared to L1 for the early bilinguals only (F(1, 54) = 4.40, p = .017, η 2 = .011). A main effect of electrode indicated that peak amplitudes on the unsubtracted waveforms became increasingly more negative from CPz to Cz to FCz and Fz, with no difference between Fz and FCz, nor between Fz and Cz (F(3, 162) = 18.40, p < .001, η 2 = .011). See Table 4 for means and standard errors.
Peak amplitudes of the unsubtracted waveforms.
Note. Mean amplitudes are reported in microvolts. Standard errors are reported in parentheses.
Latency
The N400 peak latency was longer in response to low- compared to high-constraint sentences (LC: M = 400.7 ms, SE = 8.44; HC: M = 332.1 ms, SE = 7.97; F(1, 54) = 77.13, p < .001, η 2 = .117), as well as in quiet compared to noise (quiet: M = 379.9 ms, SE = 8.58; noise: M = 352.8 ms, SE = 7.56; F(1, 54) = 13.85, p < .001, η 2 = .018; respectively). No difference was observed between L1 and L2 (F(1, 54) = .901, p = .347, η 2 = .002).
N400 effect
General observations
The N400 effect waveforms and topographical distributions are shown in Figure 4 with the N400 peaking at approximately 500 ms.
Topographical distributions and grand average waveforms of the N400 effect at Cz (difference waveforms = low constraint minus high-constraint waveforms). Arrows indicate the approximate N400 effect peaks. The scalp plots depict the topographical distributions of the N400 effect (i.e., of the difference waveforms) in L1 and L2, collapsed across listening conditions, for all three groups. The N400 effect is more frontally distributed in early bilinguals compared to simultaneous and late bilinguals.
Amplitude
The N400 effect was largest in amplitude at CPz and Cz and then decreased from FCz to Fz (F(3, 162) = 13.15, p < .001, η 2 = .008, ɛ = .715). There was no reliable amplitude difference between the groups (F(2, 54) = 2.94 p = .061, η 2 = .029). An electrode by group interaction indicated a group difference in the topographical distribution of the N400 effect (F(3, 162) = 4.52, p = .002, η 2 = .006, ɛ = .715; see Table 5 for means and standard errors). The N400 effect was larger (i.e., more negative) at anterior electrode sites (Fz, FCz) for early compared to late bilinguals and tended to be more negative (Fz) in early compared to simultaneous bilinguals (p = .058). The amplitude of the N400 effect was not modulated by listening condition or language (F(1, 54) = .03 p = .857, η 2 = .000; F(1, 54) = .45 p = .504, η 2 = .002; respectively).
N400 Effect Amplitudes.
Note. Amplitudes are reported in microvolts. Standard errors are presented in parentheses.
Latency
The N400 effect was later in quiet compared to noise (quiet: M = 443.5 ms, SE = 10.78; noise: M = 416.5 ms, SE = 10.11; F(1, 54) = 4.46, p = .039, η 2 = .020), but was not modulated by AoA group, nor whether participants were listening in L1 or L2 (F(2, 54) = .27, p = .764, η 2 = .004; F(1, 54) = .09, p = .766, η 2 = .000; respectively).
Topography by regions of interest
Mean amplitudes for the left and right posterior electrode clusters were more negative compared to anterior electrode clusters (F(3, 162) = 8.255, p < .001, η 2 = .008). A trend was observed for a difference between the groups in mean amplitudes across the four electrode clusters (F(6, 162) = 2.330, p = .051, η 2 = .004; see Table 5). Planned simple effects comparisons revealed that only the simultaneous bilinguals showed more negative mean amplitudes for posterior compared to anterior electrode sites.
Correlational analyses
No reliable association was found between any of the three principal components and the amplitude of the N400 effect during L2 speech perception (Table 6). Exploratory correlations were computed between the three principal components and the N400 effect during L1 speech perception. Following the False Discovery Rate (FDR) correction for multiple correlations, using the Benjamini-Hochberg procedure, no correlations were statistically significant. Another exploratory analysis was run to determine whether there was any association between the three principal components and the midline topographical differences reported above (i.e., greater anterior negativity for the early compared to the simultaneous and late bilinguals). For each participant, the amplitude of the N400 effect at CPz was subtracted from that at Fz (amplitudes were collapsed across listening and language conditions). Amplitude differences closer to zero reflect more evenly distributed topographical negativities along the midline (i.e., reflecting individuals with negative amplitudes at anterior electrode sites). In contrast, larger negative values would reflect more typical centro-parietal distributions, as seen in the simultaneous bilinguals, for example. No reliable associations were observed between any of the three principal components and the anterior−posterior topography of the N400 effect.
Pearson correlation coefficients.
Note. The p-values shown are uncorrected for multiple correlations.
Correlations were run between the three principal components and behavioural accuracy in the two L2 noise conditions (Table 6). Following FDR correction, a statistically significant correlation was found between PC1 (relative language proficiency) and behavioural accuracy for high-constraint, L2 sentences in noise, such that greater accuracy was associated with greater L2 proficiency relative to L1 proficiency. No correlations were found between behavioural accuracy in L2 noise conditions and PC2 or PC3.
Discussion
The aim of this study was to examine whether bilinguals benefit from semantic context while perceiving speech in their L2 to the same extent as in their L1 and whether the benefit from semantic context was related to L2 language experience and/or working memory. We studied English–French bilinguals who had varying ages of L2 acquisition, but were highly proficient, regular users of both of their languages. Our behavioural and electrophysiological evidence indicates that bilinguals can benefit from semantic context in both of their languages. Moreover, semantic context use did not vary with language experience or working memory. In the following subsections, we discuss the effects of semantic context, listening condition, language, and individual differences on bilingual speech perception in noise. Although these effects will be discussed in separate subsections, some factors interacted with each other and these interactions will also be discussed.
Before interpreting our findings, it is important to note that there is still no strong consensus concerning the functional significance of the N400. There are two commonly held views in the literature. The first is that the N400 amplitude reflects semantic access during language comprehension (e.g., Kuperberg & Jaeger, Reference Kuperberg and Jaeger2016; Kutas & Federmeier, Reference Kutas and Federmeier2011). This view hypothesizes that higher-level information and previously encountered content allows one to predict or pre-activate potential upcoming representations, facilitating the processing of new input. Smaller N400 amplitudes are then taken to reflect this facilitated processing. In contrast, the N400 has also been viewed as reflecting lexico-semantic integration (e.g., Brown & Hagoort, Reference Brown and Hagoort1993; Hagoort, Baggio & Willems, Reference Hagoort, Baggio and Willems2009). Within this view, larger N400 amplitudes are taken to reflect more effortful integration of a word into the preceding context. Lau, Phillips, and Poeppel (Reference Lau, Phillips and Poeppel2008) note that the functional significance of the N400 is difficult to untangle because factors that facilitate lexical/semantic access may also facilitate integration and argue that neuroanatomical models of semantic processing based on fMRI and MEG studies support the semantic access view of the N400. We also note that the literature is mostly based on studies using visually presented language stimuli. Thus, it is not clear how strongly either of these views are supported by experimental work with spoken language. Despite this, it seems that the literature currently shows stronger support for the semantic access view and we will interpret our findings accordingly.
Semantic context
All participants were more accurate in repeating terminal words for high- compared to low-constraint sentences, replicating the robust effect of semantic context during speech perception (e.g., Boothroyd & Nittrouer, Reference Boothroyd and Nittrouer1988; Bradlow & Alexander, Reference Bradlow and Alexander2007; Miller et al., Reference Miller, Heise and Lichten1951; Shi, Reference Shi2014). The context benefit in our study was greater when listening in noise compared to quiet. This suggests that participants relied more on semantic cues when the speech signal was degraded.
Our ERP results also revealed an effect of contextual constraint for all participant groups such that low-constraint sentences elicited larger (i.e., more negative) amplitudes compared to high-constraint sentences. This suggests that the terminal words of low-constraint sentences were more effortful to process compared to high-constraint sentences (e.g., Connolly et al., Reference Connolly, Phillips, Stewart and Brake1992; Hagoort & Brown, Reference Hagoort and Brown2000; Kutas & Hillyard, Reference Kutas and Hillyard1980). Thus, the semantic context of high-constraint sentences facilitated semantic access and processing of terminal words. The N400 was also delayed following low- compared to high-constraint sentences for all three groups, suggesting that semantic access took longer for low-constraint sentences.
Listening condition
Overall, participants were more accurate in quiet compared to noise, an effect that interacted with both semantic context and language. These interactions are discussed in the Semantic context and L1 vs L2 subsections.
Amplitudes of the unsubtracted waveforms were more negative in quiet compared to noise. Given that this difference was not observed on the subtracted waveforms (i.e., the difference between high-constraint and low-constraint sentence waveforms), it does not reflect a meaningful difference in the N400 effect per se or the effect of semantic context on processing speech in quiet and noise. This finding is, however, consistent with previous research showing larger N400 responses for more vs less intelligible speech (Obleser & Kotz, Reference Obleser and Kotz2011) and for sentences presented in isolation vs with competing speech (Carey, Mercure, Pizzioli & Aydelott, Reference Carey, Mercure, Pizzioli and Aydelott2014). Such studies suggest that poor signal quality can disrupt semantic processes during comprehension and is reflected in smaller average N400 amplitude in noise compared to quiet. It is also possible that the noise condition may elicit trial-by-trial N400 latency jitter due to variability in the masking effect of the babble across trials. This would lead to greater variability in the N400 latency in noise compared to quiet, resulting in overall reduced amplitudes in the averaged waveforms in the noise condition.
In the current study, the N400 peak was earlier in noise compared to quiet for both the subtracted and unsubtracted waveforms. This contrasts with previous research that has found shorter N400 latencies for quiet compared to noise (e.g., Aydelott, Dick & Mills, Reference Aydelott, Dick and Mills2006; Connolly et al., Reference Connolly, Phillips, Stewart and Brake1992). However, there are important methodological differences between these previous studies and our study that may explain this inconsistency. For example, Aydelott and colleagues (Reference Aydelott, Dick and Mills2006) auditorily presented congruent and incongruent sentences that were acoustically intact or degraded by low-pass filtering. In contrast, the current study used auditorily presented high- and low-constraint sentences that were masked by a multi-talker babble noise. Importantly, Connolly and colleagues (Reference Connolly, Phillips, Stewart and Brake1992) used similar stimuli to the current study (i.e., auditorily presented high- and low-constraint sentences and a multi-talker babble mask). However, as with the other studies, Connolly et al. examined monolingual participants. Given that bilinguals typically show slower lexical access than monolinguals (e.g., Ivanova & Costa, Reference Ivanova and Costa2008; Shook, Goldrick, Engstler & Marian, Reference Shook, Goldrick, Engstler and Marian2015), the difference between our findings and those of Connolly et al. may be due to a difference in the populations tested. Additionally, although the exact SNR used by Connolly and colleagues was unreported, it is possible that our SNR of +1 dB was too favourable to delay the N400 in the noise conditions. However, this would be surprising given that our SNR was sufficient to produce a behavioural effect. Visual inspection of Figure 3 suggests that our latency effect may be due to an earlier N400 peak for high-constraint sentences in noise compared to quiet. This would be consistent with the idea that semantic context facilitates speech perception in noise. However, post-hoc analyses revealed that the comparison between N400 peak latency for high-constraint sentences in noise compared to quiet was not statistically reliable. In the absence of a monolingual group, it is difficult to determine whether the difference between our latency effect and that previously seen in the literature is due to a difference in the populations tested or the nature of the stimuli used. Future studies could address this by attempting to replicate our latency effect.
Given that our experimental trials were blocked by listening condition, our latency effect in quiet compared to noise could be a result of the different task demands elicited by our listening conditions. Listening in noise is more difficult and cognitively taxing than in quiet. Participants may have been more actively engaged in the task during the noise blocks in order to compensate for the increased task demands. In contrast, participants may have perceived the quiet conditions as being easier, leading to more passive engagement during the quiet blocks. More active engagement during noisy blocks may have led to the shorter N400 latencies in noise compared to quiet.
L1 vs L2
Although a main effect revealed greater accuracy for L1 compared to L2, an interaction indicated that this was only the case for late bilinguals (see Age of acquisition subsection). The effect of context on behavioural accuracy did not interact with language, suggesting that participants benefited from semantic context to the same extent in both of their languages. Consistent with previous research (e.g., Mayo et al., Reference Mayo, Florentine and Buus1997; Rogers et al., Reference Rogers, Lister, Febo, Besing and Abrams2006), our participants were more affected by noise in their L2 compared to their L1. This was evidenced by a greater difference in error rate between quiet and noise listening conditions in L2 compared to L1.
The unsubtracted waveform amplitudes were overall more negative for L2 compared to L1 sentences. Although this finding does not reflect a difference in the N400 effect per se, it may reflect more effortful semantic access and processing in L2 overall, compared to L1.
The N400 context effect (i.e., the difference in amplitude between the waveforms elicited by the high- and low-constraint sentences) did not differ in amplitude or latency between L1 and L2 sentences. Consistent with our behavioural results, this suggests that all groups benefited from semantic context to the same extent in both of their languages.
Previous studies have found delayed N400 latencies in L2 compared to L1 (e.g., Phillips, Segalowitz, O'Brien & Yamasaki, Reference Phillips, Segalowitz, O'Brien and Yamasaki2004). However, no effect of language was observed in N400 latency of the subtracted or unsubtracted waveforms, suggesting that our participants did not differ in processing speed between L1 and L2. This could be due to the high L2 proficiency of our participants. It could also be due to a difference in task demands. For example, in the study by Phillips and colleagues (Reference Phillips, Segalowitz, O'Brien and Yamasaki2004), participants made an animacy judgment in response to visually presented word pairs in both of their languages. In contrast, the task in our current study involved perceiving L1 and L2 sentences in noise and is arguably more challenging than the animacy judgment task. This added effort may have resulted in a delayed N400 overall, washing out the latency effect previously reported in the literature. Consistent with this, the N400 latency in response to L1 sentences was about 60 ms later in the current study compared to that reported by Phillips et al. (Reference Phillips, Segalowitz, O'Brien and Yamasaki2004).
Age of acquisition
As mentioned above, simultaneous and early bilinguals performed with similar accuracy in both of their languages. In contrast, late bilinguals were overall less accurate in their L2 compared to their L1, although they were still highly accurate. This is to be expected given that late bilinguals have likely had less experience with their L2 compared to simultaneous and early bilinguals. However, this effect did not interact with contextual constraint, suggesting all groups benefited from having semantic context in both of their languages to the same extent. By contrast, Kousaie and colleagues (Reference Kousaie, Baum, Phillips, Gracco, Titone, Chen, Chai and Klein2019) observed that only simultaneous and early bilinguals benefited from semantic context in their L2, despite similar experimental methodology and some overlap in participants across our two studies. Notably, the paradigm used by Kousaie and colleagues involved a much lower SNR (−6 dB) compared to our study (+1 dB). Therefore, late bilinguals may be able to benefit from semantic context when listening in higher SNR conditions, but may not in more difficult listening conditions.
The topographical distribution of the N400 effect differed between groups. Examining left and right anterior and posterior regions revealed that the N400 effect is distributed more posteriorly for simultaneous bilinguals. This more posterior topographical distribution is typical of the N400 effect seen in native monolingual listeners (e.g., Connolly, Stewart & Phillips, Reference Connolly, Stewart and Phillips1990; Van den Brink, Brown & Hagoort, Reference Van den Brink, Brown and Hagoort2006). In contrast, the early and late bilinguals in our study showed a more evenly distributed N400 effect across the four lateralized electrode ROIs. These topographical differences between our groups have two implications. First, the more posterior distribution of the simultaneous bilinguals suggests that they may be processing semantic context similarly to monolinguals. Second, the more distributed topography seen in the early and late bilinguals indicates that these two groups may be recruiting additional neural resources to support successful task performance compared to the simultaneous bilinguals. Moreover, the early bilinguals showed a stronger negativity at anterior midline electrode sites compared to the simultaneous and late bilinguals. This further suggests that the early bilinguals may be recruiting additional neural resources while processing low-constraint sentences compared to both simultaneous and late bilinguals. This is consistent with our finding that early bilinguals performed similarly in both their languages whereas late bilinguals performed worse in L2 compared to L1. Thus, recruitment of additional neural resources may be successfully supporting the maintained performance by the early bilinguals in their L2.
Due to the low spatial resolution of ERP measurements, we cannot comment on the neural sources underlying these topographical differences. However, the more distributed topography of the early and late bilinguals, as well as the stronger anterior negativity of the early bilinguals, may be consistent with literature implicating the left inferior frontal cortex in speech processing and semantics (for reviews see Lau et al., Reference Lau, Phillips and Poeppel2008; Peelle, Reference Peelle2018, Reference Peelle2019). Increased activity in left inferior frontal cortex, as indexed by the BOLD response, has been associated with semantic processing of auditorily presented sentences (e.g., Cardillo, Aydelott, Matthews & Devlin, Reference Cardillo, Aydelott, Matthews and Devlin2004). Some researchers have proposed that the inferior frontal cortex mediates top-down, controlled semantic retrieval and selection of lexical representations (Lau et al., Reference Lau, Phillips and Poeppel2008). Others have proposed that the inferior frontal cortex is involved in lexico-semantic integration (e.g., Hagoort, Reference Hagoort2013). Therefore, it is possible that the topographical differences reported above reflect a greater recruitment of inferior frontal cortex in early and late bilinguals.
Individual differences
Language experience
Previous studies have found that L2 AoA (e.g., Mayo et al., Reference Mayo, Florentine and Buus1997; Shi, Reference Shi2009) and L2 proficiency (e.g., Bradlow & Alexander, Reference Bradlow and Alexander2007; Gor, Reference Gor2014) moderate the benefit of semantic context, measured using behavioural accuracy, while perceiving L2 speech-in-noise. Consistently, we found that speech perception of L2 sentences in noise was associated with participants’ relative balance of L1 and L2 proficiency. Specifically, greater accuracy in perceiving L2 sentence terminal words in noise was associated with greater L2 proficiency relative to L1. In contrast, we found that the amplitude of the N400 effect was not associated with individual differences in participants’ relative L1 and L2 proficiency. Notably, the N400 in our study is based only on trials successfully perceived, whereas behavioural accuracy necessarily reflects both success and failure of speech perception. Thus, individual differences in L2 language experience may be more strongly associated with the success (or failure) of perceiving words but may not reliably moderate semantic processing (as indexed by the N400).
Years of L2 experience were not associated with behavioural accuracy or the N400 effect in the current study; however, the simultaneous, early, and late bilinguals did differ in their behavioural accuracy of L1 and L2 sentences, with only the late bilinguals performing more poorly in L2 versus L1. Thus, our behavioural data are consistent with previous behavioural studies that have found a relationship between AoA and semantic context use while perceiving L2 speech-in-noise (e.g., Mayo et al., Reference Mayo, Florentine and Buus1997; Shi, Reference Shi2009). It therefore appears that L2 AoA modulates behavioural performance during speech perception in noise but not the N400 effect. It is possible that this effect with behaviour was not observed in our correlational analyses because we correlated behaviour with years of experience instead of AoA. Given that one-third of our sample consists of simultaneous bilinguals who have an L2 AoA of 0 years, raw AoA data would not have provided a sufficient data distribution for conducting meaningful correlational analyses. Thus, L2 years of experience was used as a rough proxy of AoA. However, the mapping between these two variables may not be perfect because years of experience is necessarily confounded by age. It is also important to note that we did not use a pure measure of years of experience. Instead, we used a principal component (i.e., PC3). Although L2 years of experience makes up the majority of the loading on PC3, this component still has contributions, albeit smaller, from other variables.
Working memory
The amplitude of the N400 effect in our study was not related to individual differences in working memory, despite previous studies showing an association between working memory performance and behavioural accuracy on L1 SPIN tasks (e.g., Ingvalson et al., Reference Ingvalson, Dhar, Wong and Liu2015; Millman & Mattys, Reference Millman and Mattys2017). However, our finding is consistent with some studies reporting no effect of working memory on behavioural performance during non-native SPIN tasks (e.g., Kilman, Zekveld, Hällgren & Rönnberg, Reference Kilman, Zekveld, Hällgren and Rönnberg2014; Schmidtke, Reference Schmidtke2016). Notably, working memory may play a more important role during speech perception in older adult populations and populations with hearing impairments compared to normal-hearing, young adults (Füllgrabe & Rosen, Reference Füllgrabe and Rosen2016).
Limitations
First, the correlations in the current study were run using composite variables from a PCA and not the raw individual difference variables. A PCA was done in this study for multiple reasons. First, it allowed us to more simply test our hypotheses and increase the statistical power of our tests by reducing both the number of variables examined and the number of tests conducted. Second, it is, at the present time, impossible to accurately capture the bilingual language experience with a single test due to its inherent complexity. The PCA allowed us to examine the language experience of our participants more efficiently by providing a smaller number of variables that reflect the underlying constructs common to the multiple facets of language experience assessed. Despite these benefits, some variance is necessarily lost in computing composite variables. The principal components used in our analyses accounted for about 67% of the variance in the raw individual difference variables. This must be considered when interpreting our correlational analyses.
Second, it is possible that differences between the English and French stimuli may have influenced our findings, including the previously noted difference in the number of syllables between English and French terminal words. As noted, this difference had a small and inconsistent effect on behavioural performance and may have contributed to some variability in our data.
Despite these limitations, it is clear from our findings that proficient bilinguals do benefit from semantic context while perceiving speech-in-noise in both of their languages. This contrasts with previous studies reporting that bilinguals do not benefit from semantic context in their L2 (Golestani et al., Reference Golestani, Rosen and Scott2009; Hervais-Adelman et al., Reference Hervais-Adelman, Pefkou and Golestani2014). Importantly, these previous studies used small sample sizes of moderately proficient, late bilinguals. Their SPIN task involved semantically-related and unrelated word pair stimuli. In contrast, our study examined a large sample of highly proficient bilinguals with a range of AoAs using a more ecologically valid SPIN task with sentence stimuli. Consequently, our findings more strongly suggest that bilinguals do use semantic context while perceiving speech-in-noise in both of their languages.
Conclusion
To our knowledge, this is one of the first studies to examine the electrophysiology of semantic context use at the sentence level during speech perception in noise in bilinguals. It is also the first study, to our knowledge, to examine the association between an electrophysiological measure of sentence context use during bilingual speech perception in noise and individual differences in bilingual language experience and working memory. Based on the behavioural and electrophysiological evidence presented above, proficient bilinguals benefit from semantic context while perceiving speech in both of their languages. However, although they do still benefit from semantic context in their L2, bilinguals who learn their L2 at a later age (after age 6 in this study) may be more limited in their use of semantic context to facilitate semantic processing while perceiving L2 speech compared to bilinguals who learn their L2 earlier in life (before age 5). The more effective use of semantic context by bilinguals who acquired their L2 early in life appears to be supported by recruiting additional neural resources as compared to late bilinguals. Our findings also suggest that bilinguals who learn their two languages from birth may process semantic context during speech perception similarly to monolinguals. Moreover, the most reliable individual difference variable influencing our findings was participants’ age of second language acquisition, which indicated differences between simultaneous, early, and late bilinguals in the recruitment of neural resources.
Acknowledgements
This work was supported by a Master's Award from the Fonds de Recherche du Québec – Santé and a Graduate Student Stipend from the Centre for Research on Brain, Language and Music to KC, a grant from The Natural Sciences and Engineering Research Council of Canada (NSERC; grant # 203751) to NP, a team grant from the Fonds de Recherche du Québec – Société et Culture (FRQSC, grant #171276), and the Blema and Arnold Steinberg Family Foundation.
