Theoretical Models for Bilingual Speech Processing

Several theoretical accounts have been proposed to explain language coactivation and phonotactic access during bilingual comprehension. A connectionist computational model, the Bilingual Language Interaction Network for Comprehension of Speech (BLINCS; Shook & Marian, Reference Shook and Marian2013), suggests that bilinguals access phonological neighbors during spoken word comprehension. For example, the Spanish word tenedor (“fork”) activates the Spanish word tiburón (“shark”) and the English words tunnel and tent through phonological overlap within and between languages. Support for phonological neighbor activation, including access to phonotactic constraints, also comes from the Activation Threshold Hypothesis (ATH; Paradis, Reference Paradis2004). Activation of a word and its neighbors occurs as a threshold is approached. Selection of a target word requires that its activation exceeds the threshold of its alternatives. Thus, bottom-up activation of neighbors within the ATH occurs during initial stages of auditory word comprehension.

In addition to bottom-up activation of cross-linguistic structures, top-down information may influence how and which words are accessed as auditory input unfolds. The Perceptual Assimilation Model (PAM; Best, Reference Best1994) explains that rules specific to each language influence how individuals process speech. Conflict occurs when a bilingual hears a nonnative (L2) speech sound that does not exist in the L1 inventory. The model suggests that if the phonetic characteristics of the sound resemble those of an existing phoneme in the L1, the sound will be assimilated to the L1 category. Thus, bilinguals may apply knowledge of L1 rules when processing L2 auditory input. This top-down manner of processing speech sounds provides a potential explanation as to why a rule, such as a phonotactic constraint, may impact bilingual speech processing.

The BLINCS, ATH, and PAM shed light on the potential mechanisms used by bilinguals when processing L2 words that conflict with L1 rules. In addition to the bottom-up influences on activation of within- and between-language neighbors and phonotactic constraints, bilinguals are also likely influenced by L1 phonotactic constraints when processing L2 input in a top-down manner (e.g., Carlson et al., Reference Carlson, Goldrick, Blasingame and Fink2016; Parlato-Oliveira et al., Reference Parlato-Oliveira, Christophe, Hirose and Dupoux2010). However, it is unclear how auditory and visual inputs affect bottom-up and top-down mechanisms. Based on these models, when no L2 auditory input is present, visual (orthographic) input may mediate cross-linguistic access to L1 phonological representations and phonotactic constraints.

Language Coactivation in Bilinguals

Theoretical models of language coactivation are supported by empirical evidence from unimodal tasks with bilinguals in the auditory (Fitzpatrick & Indefrey, Reference Fitzpatrick and Indefrey2010; Weber & Cutler, Reference Weber and Cutler2006), visual (Chabal & Marian, Reference Chabal and Marian2015; Finkbeiner et al., Reference Finkbeiner, Forster, Nicol and Nakamura2004; Kaushanskaya & Marian, Reference Kaushanskaya and Marian2007; Martín et al., Reference Martín, Macizo and Bajo2010; Schoonbaert et al., Reference Schoonbaert, Duyck, Brysbaert and Hartsuiker2009; Sunderman & Kroll, Reference Sunderman and Kroll2006; Thierry & Wu, Reference Thierry and Wu2007), and audio-visual modalities (Blumenfeld & Marian, Reference Blumenfeld and Marian2013; Giezen et al., Reference Giezen, Blumenfeld, Shook, Marian and Emmorey2015; Ju & Luce, Reference Ju and Luce2004; Marian & Spivey, Reference Marian and Spivey2003a, Reference Marian and Spivey2003b; Shook & Marian, Reference Shook and Marian2019), suggesting that phonological overlap between words across languages leads to parallel activation (e.g., Blumenfeld & Marian, Reference Blumenfeld and Marian2007, Reference Blumenfeld and Marian2013; Marian & Spivey, Reference Marian and Spivey2003a, Reference Marian and Spivey2003b; Shook & Marian, Reference Shook and Marian2013).

The Present Study

The present study examined whether bilinguals of varying L2 proficiency activated L1 phonotactic constraints during L2 visual processing with different amounts of speech input. Across Experiments 1 and 2, participants viewed words on a display while eye movements were tracked. Using eye tracking enabled inferences about participants’ behavior before the decision level was reached, and cross-linguistic activation of phonotactic constraints could be examined over time (time course).

The overarching prediction across both experiments was that if bilinguals accessed the L1 Spanish “e” phonotactic constraint during L2 English comprehension, then there would be more looks to an “e”-onset competitor than to fillers when an s+c target word was present. Furthermore, we predicted that the degree of L1 activation would be moderated by L2 proficiency and input modality. We examined how L2 proficiency influenced parallel activation in both Experiments 1 and 2, and investigated the role of input modality by manipulating whether the Spanish phonotactic violation was processed in the visual modality alone (Experiment 1) or audio-visually (Experiment 2). Across both experiments, the same participants read an L2 target word that conflicted with Spanish phonotactics (e.g., spa), an e-onset competitor (e.g., egg), and two fillers (e.g., work and can). The critical manipulation in Experiment 1 was that participants processed the L1 violation visually, as the visual target was identified by hearing only the word onset (e.g., “Click on /s/” for spa). In Experiment 2, the L1 violation was processed in both audio-visual modalities, as participants heard the entire target word presented as the auditory cue (e.g., “spa”) while viewing it on the visual display.

Experiment 1: Sound Recognition and Activation of L1 Phonotactic Constraints in Bilinguals

Experiment 1’s goal was to determine whether bilinguals of varying L2 proficiencies accessed the L1 constraint when viewing L2 words. Participants heard only the onset sound of the target (e.g., “Click on /s/”) in a visual display of four items (e.g., target: spa, competitor: egg, two fillers: work and can). It was hypothesized that activation of the Spanish phonotactic constraint could occur independently of auditory input, as parallel activation does not always rely on hearing words (Chabal & Marian, Reference Chabal and Marian2015; Kaushanskaya & Marian, Reference Kaushanskaya and Marian2007; Thierry & Wu, Reference Thierry and Wu2007). Specifically, it was predicted that while bilinguals read words (orthography), phonological encoding would occur, which would activate phonological neighbors, translation equivalents, and phonotactic constraints of the irrelevant language in a bottom-up manner (Freeman et al., Reference Freeman, Blumenfeld and Marian2016). Therefore, orthography would indirectly provide access to phonological, and eventually phonotactic, representations. In addition, top-down processes, such as L1 phonotactic knowledge, would influence the phonological neighbors that are accessed (Best, Reference Best1994). We also examined the role of proficiency as previous studies have demonstrated differences in the extent and time course of parallel language access across bilinguals with lower and higher L2 proficiency (Blumenfeld & Marian, Reference Blumenfeld and Marian2007, Reference Blumenfeld and Marian2013; Mishra & Singh, Reference Mishra and Singh2016; Veivo et al., Reference Veivo, Porretta, Hyönä and Järvikivi2018). Because bilinguals in the current study were tested in their L2, it was predicted that those with lower L2 proficiency would experience increased L1 phonotactic interference during L2 processing than bilinguals with higher L2 proficiency (e.g., Grainger et al., Reference Grainger, Midgley, Holcomb, Kail and Hickmann2010; van Hell & Tanner, Reference van Hell and Tanner2012). Monolinguals were not expected to demonstrate any differences in looks to the different words in the visual display because the stimuli were designed without any phonological overlap within English.

Experiment 1 Methods

Participants

Participants were 28 English monolingual (8 males) and 33 Spanish–English bilingual (7 males) adults, with normal or corrected-to-normal vision, and no history of a neurological impairment. Bilinguals were native Spanish speakers. Monolinguals were excluded if they had a reported Spanish or another foreign language speaking proficiency of or greater than 3 (1–10 scale) on the Language Experience and Proficiency Questionnaire (LEAP-Q; Marian et al., Reference Marian, Blumenfeld and Kaushanskaya2007). See Table 1 for additional participant information.

TABLE 1.

Linguistic and cognitive background of Spanish–English bilingual (n = 33) and English monolingual (n = 28) participants

Materials

Stimulus pairings of interest for Experiment 1 included (a) s+c onset target (e.g., spa), (b) “e”-onset competitor (e.g., egg), and (c) two filler words (e.g., work and can), with a total of four words displayed in the visual world paradigm (see Appendix A for stimulus pairings of interest).Footnote ¹ Other pairings included the s+c onset replaced with a filler (control) word, the “e” onset replaced with a filler, and both s+c and “e” onsets replaced with a filler. All words (s+c onset, “e” onset, and filler) and their Spanish translation equivalents were matched on lexical characteristics as shown in Table 2 (CLEARPOND; Marian et al., Reference Marian, Bartolotti, Chabal and Shook2012). The stimuli were recorded in a soundproof room (44,100 Hz, 16 bits) by a male native speaker of English. The audio recording was split into individual audio files. All files were normalized (using audio compression) in Praat (Boersma & Weenink, Reference Boersma and Weenink2013) and exported into MatLab.

TABLE 2.

Lexical characteristics for target, competitor, and filler (control) items and Spanish translations (CLEARPOND: Marian et al., Reference Marian, Bartolotti, Chabal and Shook2012). Means (Standard Deviations). All ps > 0.05.

Experiment 1 contained a total of 156 trials (12 practice, 144 experimental). The task consisted of 48 trials in which cross-linguistic phonotactic competitors were present (i.e., s+c spa and “e” onset egg) where 24 trials included the s+c onset word as the target and 24 trials with the “e” onset word as the target.Footnote ² The task also included 96 additional trials where no phonotactic competition was present to obscure the purpose of the experiment. Task order was pseudorandomized so that no more than two consecutive trials contained s+c onsets. Trial order was counterbalanced across participants by reversing the order of presentation.

Procedure

Participants were administered the following tasks in the order listed in Table 3.

TABLE 3.

Order of tasks

After consent was obtained and the LEAP-Q was completed, participants were seated in a quiet room, 80 cm from the visual display, and the eye-tracker was calibrated. The sound recognition task was controlled by an iMac 3.3 GHz Intel Core i5 running MatLab 2011a (Brainard, Reference Brainard1997; Kleiner et al., Reference Kleiner, Brainard and Pelli2007; Pelli, Reference Pelli1997), and stimuli were displayed on a 27-inch monitor with a screen resolution of 5120 × 2880. Eye movements were recorded using a desk-mounted eye-tracker (EyeLink 1000 Version 1.5.2, SR Research Ltd.) at a sampling rate of 1,000 Hz. Mouse clicks to identify the target served for collection of accuracy and reaction time data.

Participants were instructed to identify the visual target word by hearing its onset sound (e.g., “Click on /s/”). The four word stimuli were presented orthographically in four quadrants (top-left, bottom-left, top-right, bottom-right) in a 3 × 3 square grid (1,440 × 1,440 pixels), on a white screen in black font. After 1,000 ms of viewing the words, participants heard the carrier phrase, “Click on…” (830 ms duration), with the onset sound immediately following. Participants therefore processed the words on the visual display and then selected the target word that contained the auditory-onset sound they heard. During experimental trials of interest, an s+c target word was presented with an “e”-onset competitor, as well as two unrelated filler words. In the noncritical trials, the s+c and/or “e” onset words were replaced with fillers. Response times were measured at the start of the visual display, until the participant responded. See Figure 1 for task procedure.

FIGURE 1.

Example trial from the sound recognition task. Participants viewed four words on the screen while eye movements were tracked. The words included a target that conflicted with the Spanish “e” onset constraint (strong), a competitor that contained an “e” onset (ensure), and two filler words (cross and friend). Participants then heard, “Click on /s/,” where /s/ was the onset of the target word (strong).

Experiment 1 Results

Time Course Analyses: Monolinguals and Bilinguals

See Appendix C for the results within a table. To uncover if bilinguals activated L1 phonotactics during L2 processing, fixation proportions were analyzed to the competitors (e.g., egg) relative to fillers (e.g., work and can) when a conflicting target (e.g., spa) was present in the visual display. There was a main effect of language group on the intercept term (i.e., differences in average overall fixation proportions with monolinguals and bilinguals in the model curve), β = 0.71, SE = 0.19, t = 3.72, p < 0.01, with monolinguals producing a greater proportion of fixations than bilinguals during the initial 0–1,100 ms time window posttarget sound onset. No main effects or interactions of cognate status emerged (ps > 0.1).

Visual inspection of the time course data suggested that monolinguals produced more looks to competitors and fillers overall early on. To further uncover whether this trend held for monolinguals and bilinguals in a narrower time window when cross-linguistic effects are typically present (Blumenfeld & Marian, Reference Blumenfeld and Marian2013; Shook & Marian, Reference Shook and Marian2019), we selected 300–600 ms postsound onset. These follow-up analyses revealed no main effects or interactions within the narrower time window. Within the longer time window, monolinguals and bilinguals differed in their overall fixation proportions, but no phonotactic-constraint activation (greater proportion of fixations to competitors than fillers) was observed within the bilingual group. See Figure 2 for monolingual/bilingual differences in the time course of fixations to “e”-onset competitor words, relative to filler words.

FIGURE 2.

Time course analyses for (a) monolinguals and (b) bilinguals in Experiment 1. Y axis represents mean proportion of fixations to competitor (egg) versus filler words (work and can). X axis represents the time course starting at the onset sound of the target (spa). (C) Dots represent mean fixations and lines represent GCA model fits for fixation proportions (competitor to filler words) for monolinguals and bilinguals. Error bars represent 95% confidence interval of GCA model fits.

Time Course Analyses: L2 Proficiency

Our next step was to assess the extent to which L2 (English) proficiency influenced bilinguals’ L2 visual word recognition.Footnote ³ The model revealed main effects of proficiency on the intercept term, β = −0.69, SE = 0.37, t = –2.48, p = 0.01 and on the quadratic term (i.e., the rise and fall rate of fixation proportions in the model curve), β = 0.51, SE = 0.28, t = 2.53, p = 0.01. There were also significant interactions of word type by proficiency on the intercept, β = 0.32, SE = 0.12, t = 2.67, p < 0.0, and on the quadratic terms, β = −0.62, SE = 0.28, t = –2.51, p = 0.01. The model demonstrated that bilinguals with lower L2 (English) proficiency produced a greater proportion of fixations to the “e”-onset word relative to filler words than did higher L2 proficiency bilinguals. There were no cognate status main effects or interactions (ps > 0.1).

Using the same procedure of a narrower time window across monolinguals and bilinguals, also consistent with the significant interactions on the quadratic and cubic terms, we followed up to confirm the result 300–600 ms postsound onset. There was a main effect of proficiency on the quadratic term, β = 0.06, SE = 0.01, t = 5.42, p < 0.01. There were significant interactions of word type by proficiency on the intercept, β = 0.20, SE = 0.05, t = 3.75, p < 0.01, and quadratic terms, β = –0.14, SE = 0.06, t = –2.02, p = 0.04. No additional main effects or interactions emerged. The main effect and interactions suggest that decreased L2 proficiency resulted in more looks to competitors than fillers. See Figure 3 for differences among lower and higher proficiency bilinguals in the time course of fixations to “e”-onset competitor words, relative to filler words.Footnote ⁴

FIGURE 3.

Time course analyses for (a) lower proficiency in English bilinguals and (b) higher proficiency in English bilinguals. Y axis represents mean proportion of fixations to competitor (egg) versus filler words (work and can). X axis represents the time course starting at the onset sound of the target (spa). (C) Dots represent mean fixations and lines represent GCA model fits for fixation proportions for competitor and filler words for lower and higher proficiency bilinguals. Error bars represent 95% confidence interval of GCA model fits.

Experiment 1 Discussion

The goal of Experiment 1 was to examine whether bilinguals activated L1 phonotactic constraints during L2 processing with minimal auditory input of an L1-conflicting L2 word (i.e., “Click on /s/” for spa). Time course data suggest that monolinguals produced more fixations in the early time window (300–600 ms postsound onset); however, when collapsed across language groups, participants looked at “e”-onset competitor and filler items equally. Bilinguals with lower L2 (English) proficiency fixated on the competitor more than the filler words, while bilinguals with higher L2 proficiency did not. These findings suggest that decreased L2 (English) proficiency resulted in activation of the L1 (Spanish) v+s+c phonotactic constraint when viewing L1-conflicting L2 s+c words and only hearing their onset sound (e.g., /s/). Therefore, bilinguals with lower L2 proficiency are more likely to be influenced by the L1 phonotactic constraint when reading L2 words. At the same time, increased L2 proficiency yields more efficient processing of L1-conflicting L2 input, as evidenced by no differences in looks to competitors versus fillers in bilinguals with higher L2 proficiency, and as suggested by previous studies (e.g., Blumenfeld & Marian, Reference Blumenfeld and Marian2013).

Experiment 1 demonstrated for the first time that bilinguals accessed the L1 phonotactic constraint upon reading a conflicting L2 word with minimal auditory/phonological input, suggesting that orthography mediated access to phonotactic representations. The constraint-conflicting structure (e.g., s+c, spa) was not heard, just the onset of the word (e.g., /s/). In Experiment 2, we examined whether bilinguals would access the L1 constraint when viewing and hearing L2 words.

Experiment 2: Word Recognition and Activation of L1 Phonotactic Constraints in Bilinguals

The purpose of Experiment 2 was to identify if bilinguals of varying proficiencies accessed L1 constraints during L2 whole-word auditory and visual processing, similar to previous findings of language coactivation with audio-visual input (Blumenfeld & Marian, Reference Blumenfeld and Marian2013; Giezen et al., Reference Giezen, Blumenfeld, Shook, Marian and Emmorey2015; Ju & Luce, Reference Ju and Luce2004; Marian & Spivey, Reference Marian and Spivey2003a, Reference Marian and Spivey2003b; Shook & Marian, Reference Shook and Marian2019). Experiment 2 was also designed to replicate the results from Experiment 1. As in Experiment 1, participants viewed four words on the screen while eye movements were tracked. Participants identified the target by hearing the word in its entirety (e.g., “spa”), as opposed to hearing the onset only (e.g., “Click on /s/”) in Experiment 1. Importantly, new stimulus pairings were created to minimize cross-experimental effects. Activation of the L1 constraint during L2 processing was expected in bilinguals. However, given the findings from Experiment 1, it was also anticipated that proficiency would impact bilinguals’ access of the L1 constraint during L2 processing.

Moreover, it was predicted that auditory input would be processed in a bottom-up way because within- and between-language phonological neighbors would be accessed (e.g., Shook & Marian, Reference Shook and Marian2013), and eventually phonotactic constraints (Freeman et al., Reference Freeman, Blumenfeld and Marian2016). However, due to the auditory input of the whole word, the phonological representation of the L1-conflicting L2 word could be directly accessed instead of mediated through the orthographic representation, as in Experiment 1, which is the primary difference between Experiments 1 and 2. Top-down information, specifically the L1 (Spanish) phonotactic constraint, would also influence how the input was processed. Bilinguals might initially access knowledge of the L1 (Spanish) phonotactic constraint upon hearing the conflicting word (e.g., spa/ espa) (Best, Reference Best1994), demonstrating parallel language access. Activation may thus rely on a combination of bottom-up and top-down processes when whole-word auditory and visual input is present. If bilinguals accessed the L1 phonotactic constraint directly through the phonological representation of the conflicting s+c L2 word, then more looks were expected to the competitor than filler items.

Experiment 2 Methods

Procedure

The word recognition task examined cross-linguistic activation between Spanish and English by tracking eye-movements to English “e” onset words (e.g., egg) when English s+c onset words (e.g., spa) were present. The eye tracker was recalibrated for Experiment 2. Instructions were to select the target by clicking on the word heard. As in Experiment 1, four words appeared on the screen, then a 1,000 ms delay occurred before the target word (e.g., spa) was played. The rest of the procedure was the same as Experiment 1, including practice and experimental trials, presentation of the visual display, and the collection of accuracy, reaction time, and eye fixation data. See Figure 4 for task procedure.

FIGURE 4.

Example trial from the word recognition task. In this example, participants viewed four words on the screen while eye movements were tracked. The words included a target that conflicted with the Spanish “e” onset constraint (spread), a competitor that contained an “e” onset (encroach), and two filler words (behavior and lawyer). Participants then heard the target word (spread) and made their response.

Experiment 2 Results

Time Course Analyses: Monolinguals and Bilinguals

See Appendix E for the results within a table. The language group model revealed no main effects or interactions, (ps > 0.1). Monolinguals and bilinguals produced similar looking patterns throughout the 0–1,100 ms time window to competitors and filler items. See Figure 5 for the time course of fixations to “e”-onset competitor words relative to filler words.

FIGURE 5.

Time course analyses for (a) monolinguals and (b) bilinguals. Y axis represents mean proportion of fixations to competitor (egg) versus filler words (work and can). X axis represents the time course starting at the onset of the target (spa). (C) Dots represent mean fixations and lines represent GCA model fits for fixation proportions to competitor and filler words for monolinguals and bilinguals. Error bars represent 95% confidence interval of GCA model fits.

Time Course Analyses: L2 Proficiency

Within the bilingual group,Footnote ⁶ GCAs revealed a main effect of L2 proficiency on the intercept term, β = 0.60, SE = 0.21, t = 2.80, p < 0.01. There were also significant interactions of word type by proficiency on the intercept, β = 0.34, SE = 0.01, t = –7.98, p < 0.01, and quadratic terms, β = –0.33, SE = 0.04 t = –8.00, p < 0.01, indicating that bilinguals with lower L2 proficiency produced a greater proportion of fixations to the competitors than filler words than did the higher L2 proficiency bilinguals. There were no main effects or interactions for cognate status (ps > 0.1).

We followed up on the main effect and interactions within the shorter, 300–600 ms postword onset, time window. There was a main effect of proficiency on the intercept term, β = 0.09, SE = 0.04, t = 2.26, p = 0.04. The model also revealed interactions of word type by proficiency on the intercept term, β = –0.17, SE = 0.06, t = –2.71, p < 0.01. No additional main effects or interactions emerged. The findings within the narrower time window confirm the significant effects observed for lower L2 proficiency bilinguals, with more looks to competitors than fillers. See Figure 6 for monolingual/bilingual differences in the time course of fixations to “e”-onset competitor words, relative to filler words.Footnote ⁷

FIGURE 6.

Time course analyses for bilinguals with (a) lower proficiency and (b) higher proficiency in English. Y axis represents mean proportion of fixations to competitor (egg) versus filler words (work and can). X axis represents the time course starting at the onset of the target (spa). (C) Dots represent mean fixations and lines represent GCA model fits for fixation proportions to competitor and filler words for lower proficiency and higher proficiency bilinguals. Error bars represent 95% confidence interval of GCA model fits.

Time course results indicated no overall differences between monolinguals and bilinguals in looks to fillers versus competitors. However, and as in Experiment 1, bilinguals with lower L2 proficiency made more looks to competitor versus filler words, relative to bilinguals with higher L2 proficiency.

Experiment 2 Discussion

The focus of Experiment 2 was twofold: (a) to examine the extent to which audio-visual input and L2 proficiency influenced language coactivation of the L1 Spanish phonotactic constraint during L2 English comprehension and (b) to replicate the findings from Experiment 1. Results suggest that decreased L2 (English) proficiency resulted in an increase in looks to competitors, relative to fillers, early on (300–600 ms postword onset) during the time course. Bilinguals with lower L2 English proficiency activated the L1 Spanish phonotactic constraint when viewing and hearing L1-conflicting L2 words, highlighting interference from the L1 during L2 processing (Grainger et al., Reference Grainger, Midgley, Holcomb, Kail and Hickmann2010; van Hell & Tanner, Reference van Hell and Tanner2012). The findings from Experiment 2 are in line with previous studies that suggest L1 phonotactic constraints are activated during L2 visual and spoken word comprehension (Freeman et al., Reference Freeman, Blumenfeld and Marian2016, Reference Freeman, Blumenfeld and Marian2021; Lentz & Kager, Reference Lentz and Kager2015; Parlato-Oliveira et al., Reference Parlato-Oliveira, Christophe, Hirose and Dupoux2010; Weber & Cutler, Reference Weber and Cutler2006) and replicate the results from Experiment 1.

General Discussion

The goal of the current investigation was to identify the extent to which bilinguals of different L2 proficiency levels activated L1 phonotactic constraints (i.e., rules for combining speech sounds) during visual word processing. Results across both experiments demonstrated that bilinguals with decreased L2 proficiency produced more looks to the “e”-onset competitor (e.g., egg) than to filler words (e.g., work and can). Experiment 1 was unique in that, for the first time, evidence was found for phonotactic-constraint activation of the unintended language during visual word processing without auditory input of the constraint-conflicting structure (i.e., s+c; spa). Experiment 2 replicated those results and also showed cross-linguistic effects in the auditory and visual modalities. Combined, the results from Experiments 1 and 2 suggest that visual orthographic input is sufficient to access phonotactic constraints across languages.

Interestingly, there were no performance differences between bilinguals and monolinguals to suggest that bilinguals overall as a group were experiencing L1 interference during L2 processing. However, when examining the role of L2 proficiency within the bilingual group, bilinguals with lower L2 proficiency accessed the L1, while bilinguals with higher L2 proficiency did not. Additional analyses revealed that there were no differences in time courses across language groups or L2 proficiency preinitial phoneme (Experiment 1) or preword onset (Experiment 2). This suggests that lower L2 proficiency bilinguals’ attention needed to be drawn to the target word to engender cross-linguistic interference from the phonotactic constraint. While we initially expected that bilinguals might produce more fixations to competitors than fillers, we also predicted that L2 proficiency could modulate fixation patterns indicative of activation of L1 phonotactic constraints. Because we controlled for item and participant variability in all GCA models, it was unlikely that the stimuli or participant sample caused an absence of an overall group difference in looking patterns between monolinguals and bilinguals.

Language Coactivation and Proficiency

Across Experiments 1 and 2, we found evidence that bilinguals activated both languages simultaneously when in a single-language context. These results contribute to the body of work investigating between-language phonological processing in bilinguals of varying proficiencies (e.g., Blumenfeld & Marian, Reference Blumenfeld and Marian2007, Reference Blumenfeld and Marian2013; Ju & Luce, Reference Ju and Luce2004; Linck et al., Reference Linck, Hoshino and Kroll2008; Mercier et al., Reference Mercier, Pivneva and Titone2014) and confirms findings that bilinguals experience cross-linguistic activation at the lexical (Finkbeiner et al., Reference Finkbeiner, Forster, Nicol and Nakamura2004; Fitzpatrick & Indefrey, Reference Fitzpatrick and Indefrey2010; Hartsuiker et al., Reference Hartsuiker, Pickering and Veltkamp2004; Ju & Luce, Reference Ju and Luce2004; Kaushanskaya & Marian, Reference Kaushanskaya and Marian2007; Loebell & Bock, Reference Loebell and Bock2003; Marian & Spivey, Reference Marian and Spivey2003a, Reference Marian and Spivey2003b; Martín et al., Reference Martín, Macizo and Bajo2010; Schoonbaert et al., Reference Schoonbaert, Duyck, Brysbaert and Hartsuiker2009; Sunderman & Kroll, Reference Sunderman and Kroll2006; Thierry & Wu, Reference Thierry and Wu2007) and sublexical levels (Freeman et al., Reference Freeman, Blumenfeld and Marian2016, Reference Freeman, Blumenfeld and Marian2021; Lentz & Kager, Reference Lentz and Kager2015; Weber & Cutler, Reference Weber and Cutler2006). For example, in Blumenfeld and Marian (Reference Blumenfeld and Marian2013), participants were tested in their L1 and experienced greater L2 activation as L2 proficiency increased. In the current study, bilinguals were tested in an L2 environment, and those with lower L2 proficiency experienced increased L1 phonotactic interference (also see Grainger et al., Reference Grainger, Midgley, Holcomb, Kail and Hickmann2010; van Hell & Tanner, Reference van Hell and Tanner2012 for review). These results suggest that bilinguals with higher L2 proficiency may resolve cross-language competition at an earlier stage than bilinguals with lower L2 proficiency during L2 auditory and/or visual word recognition. Therefore, bilinguals with higher proficiency levels may process language input with greater automaticity and are able to inhibit activation of the nontarget language more efficiently (Blumenfeld & Marian, Reference Blumenfeld and Marian2013; Mercier et al., Reference Mercier, Pivneva and Titone2014; Mishra & Singh, Reference Mishra and Singh2016; van Hell & Tanner, Reference van Hell and Tanner2012; Veivo et al., Reference Veivo, Porretta, Hyönä and Järvikivi2018).

The Roles of Auditory Input and Metalinguistic Demands in Language Coactivation

In addition to L2 proficiency, target stimulus presentation (auditory vs. visual) and task difficulty (metalinguistic demands) may have influenced participants’ performance in the current study. Experiment 2’s results bolster previous research on cross-linguistic access through the combination of auditory (phonological) and visual (orthographic) stimulus presentation (Mishra & Singh, Reference Mishra and Singh2016; Veivo et al., Reference Veivo, Porretta, Hyönä and Järvikivi2018). However, the findings from Experiment 1, along with others (Chabal & Marian, Reference Chabal and Marian2015; Kaushanskaya & Marian, Reference Kaushanskaya and Marian2007; Thierry & Wu, Reference Thierry and Wu2007), suggest that bilinguals activate their languages in parallel when viewing words (orthography) and/or pictures with no to minimal auditory input.

In one such study, Chabal and Marian (Reference Chabal and Marian2015) investigated English monolinguals’ and Spanish–English bilinguals’ eye movements to pictures containing target and competitor items overlapping in phonology within or between languages. Critically, a target picture was visually presented in the center of the screen (no auditory input) and participants matched the target amongst four pictures. For example, when the target item was clock, English monolinguals and Spanish–English bilinguals also looked at a picture of cloud. However, even though the experiment was conducted in an L2 “English” mode (i.e., participants were instructed in English and tested in an English environment), bilinguals additionally looked at gift (Spanish: “regalo”) because it phonologically overlapped with the Spanish translation of clock (“reloj”). This finding highlights cross-language interactivity with no auditory input. In addition, Kaushanskaya and Marian (Reference Kaushanskaya and Marian2007), along with Thierry and Wu (Reference Thierry and Wu2007), found cross-linguistic activation with only orthographic input in bilinguals. In line with these studies, findings from Experiment 1 demonstrated for the first time that bilinguals with lower levels of L2 proficiency activated the L1 “e”-onset constraint during L2 processing without whole-word auditory input.

In addition to input modality, metalinguistic demands of a task may also influence the extent to which parallel language activation occurs in bilinguals. Previous studies demonstrate that as a task’s metalinguistic demands increase, so do effects of parallel processing in bilinguals, especially at the sublexical level (Freeman et al., Reference Freeman, Blumenfeld and Marian2021; Parlato-Oliveira et al., Reference Parlato-Oliveira, Christophe, Hirose and Dupoux2010). In the current investigation, participants simply identified a word in a visual display after hearing its onset sound or the word in its entirety. Therefore, the metalinguistic demands were lower compared to a lexical decision task in which participants search the lexicon for an entry, or a vowel detection task in which they attend to aspects of the stimulus (e.g., onset or rime, consonant or vowel) to detect if a vowel is present (e.g., Freeman et al., Reference Freeman, Blumenfeld and Marian2021). The sound and word recognition tasks might not have been sensitive enough to capture differences across monolinguals and bilinguals due to their relative facility. However, despite low metalinguistic demands, bilinguals with lower L2 proficiency were still susceptible to L1 interference during L2 comprehension, highlighting the potential relation between proficiency and metalinguistic demands. A task with increased metalinguistic demands might result in bilinguals with higher L2 proficiency demonstrating effects of language coactivation with phonotactic constraints.

Theoretical Implications for Language Activation

The results of Experiments 1 and 2 have implications for bilingual language processing models. Current findings directly support and potentially extend BLINCS (Shook & Marian, Reference Shook and Marian2013) to include phonotactic constraints. Moreover, empirical and theoretical evidence suggests that linguistic input is processed in a bottom-up way and supports activation of within-language (monolinguals and bilinguals) and between-language (bilinguals only) neighbors (e.g., Marian & Spivey, Reference Marian and Spivey2003a, Reference Marian and Spivey2003b; Shook & Marian, Reference Shook and Marian2013). Constraints may influence which words are accessed in the bilingual lexicon in a top-down manner, in line with PAM (Best, Reference Best1994). Across Experiments 1 and 2, bilinguals with lower L2 proficiency relied on an interplay between bottom-up and top-down mechanisms during visual and spoken word comprehension. Bottom-up processing began with visual or auditory input, which then activated phonological representations, lexical items, and constraints. Top-down processing also occurred, starting with knowledge of the L1 constraint, flowing down to phono-ortho representations.

Though similar trajectories of activation were expected, the key difference across studies was that the phonological representations were accessed indirectly (Experiment 1) or directly (Experiment 2). In contrast to Experiment 2, which provided whole-word auditory input, access to phonotactic constraints in Experiment 1 was mediated by orthographic representations. The combination of the two experiments therefore sheds light on the individual contributions of auditory (phonological) versus visual (orthographic) input during bilingual comprehension. We find that orthography alone is sufficient to access cross-linguistic phonological structures (also see Kaushanskaya & Marian, Reference Kaushanskaya and Marian2007; Thierry & Wu, Reference Thierry and Wu2007), and subsequently, to activate phonotactic (constraint-based) representations.

Limitations and Future Directions

A potential limitation of Experiments 1 and 2 is the assumption that bilinguals are not accessing stored representations for sound sequences. For example, when visually presented with an English word that conflicts with the Spanish v+s+c rule (e.g., spa), it is unclear whether the L1 Spanish speaker’s orthographic and phonological representation of English spa is spa or espa, which would then account for eye movements to phonologically/orthographically overlapping targets like egg. Hallé et al. (Reference Hallé, Dominguez, Cuetos and Segui2008) showed Spanish monolinguals Spanish-like s+c nonwords (orthography), which conflicted with the Spanish v+s+c rule. Monolinguals appeared to perceptually or phototactically repair the visual input to conform to the v+s+c rule by reporting a vowel was present. However, an alternative explanation for this pattern of results is that orthographic input permitted access to phonological representations, and then eventually activated the Spanish phonotactic constraint.

Our research sets the stage for future work examining more carefully the bottom-up and top-down mechanisms that come online as bilinguals process auditory input. Understanding how these mechanisms interact depending on the nature of the task is essential, for example, based on type of linguistic interference created (e.g., semantic vs. phonological/orthographic) or the metalinguistic demands. Perhaps in a more explicit and metalinguistically challenging task combined with word recognition, such as asking whether a vowel is present at the target word’s onset (vowel detection: Carlson et al., Reference Carlson, Goldrick, Blasingame and Fink2016; Freeman et al., Reference Freeman, Blumenfeld and Marian2021), bilinguals of varying proficiency levels would demonstrate enhanced access to L1 sublexical structures during L2 processing.

Future research should also further examine the role of cognates on L1 phonotactic access. In this investigation, we observed no effect of cognate status across monolinguals and bilinguals, as well as bilinguals with lower and higher L2 proficiency. We speculate that the absence of a significant cognate effect on eye gaze patterns, especially amongst the lower L2 proficiency bilinguals, could be due to the limited number of cognate and noncognate stimuli in the experiment (12 of each type; 24 total). In addition, we included the same stimulus set in a previous investigation (see Freeman et al., Reference Freeman, Blumenfeld and Marian2021) and did not observe any cognate effects across three tasks investigating L1 phonotactic-constraint access during L2 processing. Including more cognate and noncognate s+c stimuli in future investigations would yield more power to examine this critical manipulation.

The results of the current study suggest that bilinguals activate sublexical phonotactic constraints across both languages when reading words with minimal auditory cues. Language proficiency and task demands modulated the extent to which L1 influenced L2 processing at the sublexical level. We conclude that the interactivity between a bilingual’s two languages is not limited to words that share form across languages, but also extends to sublexical, rule-based structures and thus permeates the language system throughout.