Introduction
The linguistic relativity hypothesis (Whorf, Reference Whorf1940, Reference Whorf, Carroll, Levinson and Lee1956) proposed that language and thought interact in many significant ways; speaking one or more specific languages can influence the way we think about the world. In recent research investigating this claim, language has been shown to impact time conceptualization (Boroditsky, Reference Boroditsky2001; Friedman, Reference Friedman2004; Hannah, Reference Hannah2009; Li, Jones & Thierry, Reference Li, Jones and Thierry2018), spatial reasoning (Li, Abarbanell, Gleitman & Papafragou, Reference Li, Abarbanell, Gleitman and Papafragou2011; Li & Gleitman, Reference Li and Gleitman2002), colour perception (Athanasopoulos, Reference Athanasopoulos2009; Hu, Hanley, Zhang, Liu & Roberson, Reference Hu, Hanley, Zhang, Liu and Roberson2014; MacLaury, Reference MacLaury1997; Paramei, Reference Paramei, MacLaury, Paramei and Dedrick2007; Roberson & Davidoff, Reference Roberson and Davidoff2000; Thierry, Athanasopoulos, Wiggett, Dering & Kuipers, Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009), and object categorization (Casasola, Reference Casasola2005; Edmiston & Lupyan, Reference Edmiston and Lupyan2015; Lupyan, Rakison & McClelland, Reference Lupyan, Rakison and McClelland2007). An implication of the linguistic relativity hypothesis for bilinguals is that these cognitive processes have two potential sources of influence from language instead of just one (see Pavlenko, Reference Pavlenko2014). The current study investigated the impact of a particular feature of word structure in Chinese on the perception of objects, and how this influence of Chinese word structure changes with Chinese speakers’ increased exposure to English.
Although there is ample evidence supporting the linguistic relativity hypothesis, a theoretical mechanism that explains language-thought interaction has been lacking. Here we make use of a relatively new view, the label-feedback hypothesis (Lupyan, Reference Lupyan2008, Reference Lupyan and Ross2012), as an explanatory framework to understand how word labels could influence perception (for an alternative see Fang, Zinszer, Malt & Li, Reference Fang, Zinszer, Malt and Li2016). We first describe this hypothesis, and then offer some supporting evidence for this view from research with monolinguals focusing on research using the visual-oddball paradigm, the paradigm used here. We then report on one such study with bilinguals before presenting our own study.
The label-feedback hypothesis
Lupyan (Reference Lupyan2008, Reference Lupyan and Ross2012) proposed a hypothesis that attempts to explain the cognitive mechanisms underpinning language-thought interactions. His label-feedback hypothesis applies insights from interactive models (e.g., McClelland & Rumelhart, Reference McClelland and Rumelhart1981; Seidenberg & McClelland, Reference Seidenberg and McClelland1989) to the issue of linguistic relativity. It proposes that language is highly interconnected with other cognitive processes, and influences other functional networks in a top-down fashion. Labels can provide feedback to the level of conceptual representations and perception: thus, named concepts should be activated differently under the on-line influence of the label compared to the same concepts activated by nonverbal means, or when the labels are prevented from affecting the concept (see Figure 1). According to this hypothesis, the activation of an object's verbal label results in the activation of the most typical features of the category (e.g., the label “car” activates the feature “wheels” more strongly than the feature “black”). This top-down activation from verbal labels to features produces a transient “perceptual warping” in which category members that share those features are drawn closer together and non-members are pushed away.
The Label-feedback Hypothesis. (A): A schematic view of the standard account in which a word label is simply a means of accessing a concept. Multiple perceptual exemplars of a concept map onto a common conceptual representation. The concept is further mapped onto a word label, which enables a speaker to activate the same concept in a listener using the label. The one-way connections between representational layers prevent the word label to have an influence on the conceptual representations. (B): A schematic view of the label-feedback hypothesis. All representational layers are recurrently connected, which allows the word label to affect the conceptual representations through feedback. Reprinted from: Lupyan, G. (Reference Lupyan and Thompson-Schill2012). What do words do? Toward a theory of language-augmented thought. In B. H. Ross (ed.), The psychology of learning and motivation, Vol. 57, pp. 255–297. Waltham, MA: Academic Press.
There is ample evidence that supports the label-feedback hypothesis. Studies have suggested that categories are learned more effectively when the categories are accompanied by their labels (Fulkerson & Waxman, Reference Fulkerson and Waxman2007; Robinson, Best, Deng & Sloutsky, Reference Robinson, Best, Deng and Sloutsky2012; Lupyan & Casasanto, Reference Lupyan and Casasanto2015; Lupyan et al., Reference Lupyan, Rakison and McClelland2007). Once a category is learned, key features of the category are more effectively activated by a verbal label than other nonlinguistic cues (Boutonnet & Lupyan, Reference Boutonnet and Lupyan2015; Edmiston & Lupyan, Reference Edmiston and Lupyan2015; Lupyan & Thompson-Schill, Reference Lupyan and Thompson-Schill2012; Gervits, Johanson & Papafragou, Reference Gervits, Johanson and Papafragou2016). For example, Lupyan (Reference Lupyan2008) presented participants with pictures of common objects such as chairs and tables, and then asked them to label some with their basic-level name (e.g., “chair”), and to provide a nonverbal response to others (e.g., indicating whether they liked that particular chair). In the subsequent recognition memory test, participants had substantially worse memory for the objects they had labeled. Lupyan proposed that labeling resulted in the activation of prototypical features and thus made the representations of stimuli more categorical.
Other supporting evidence for the label-feedback hypothesis comes from studies investigating the effects of labelling on perception. Studies have found that two stimuli that share a verbal label are perceived as more similar than stimuli that do not share a label. Evidence was found in the domains of colour perception (Athanasopoulos, Reference Athanasopoulos2009; Athanasopoulos, Dering, Wiggett, Kuipers & Thierry, Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010; Forder & Lupyan, Reference Forder and Lupyan2017; Hu et al., Reference Hu, Hanley, Zhang, Liu and Roberson2014; Roberson, Reference Roberson, Gelfand, Chiu and Hong2012; Roberson & Davidoff, Reference Roberson and Davidoff2000; Thierry et al., Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009; Winawer, Witthoft, Frank, Wu, Wade & Boroditsky, Reference Winawer, Witthoft, Frank, Wu, Wade and Boroditsky2007; Xia, Xu & Mo, Reference Xia, Xu and Mo2019), pitch perception (Dolscheid, Shayan, Majid & Casasanto, Reference Dolscheid, Shayan, Majid and Casasanto2013), time and spatial perception (Boroditsky, Reference Boroditsky2001; Boroditsky, Fuhrman & McCormick, Reference Boroditsky, Fuhrman and McCormick2011; Casasanto & Boroditsky, Reference Casasanto and Boroditsky2008; Choi & Hattrup, Reference Choi and Hattrup2012; Fuhrman, McCormick, Chen, Jiang, Shu, Mao & Boroditsky, Reference Fuhrman, McCormick, Chen, Jiang, Shu, Mao and Boroditsky2011; Lai & Boroditsky, Reference Lai and Boroditsky2013; Y. Li et al., Reference Li, Jones and Thierry2018), and object perception (Boutonnet, Athanasopoulos & Thierry, Reference Boutonnet, Athanasopoulos and Thierry2012; Boutonnet, Dering, Viñas-Guasch & Thierry, Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013; Jouravlev, Taikh & Jared, Reference Jouravlev, Taikh and Jared2018; Masuda, Ishii, Miwa, Rashid, Lee & Mahdi, Reference Masuda, Ishii, Miwa, Rashid, Lee and Mahdi2017). Some studies have used event-related brain potentials (ERPs) and a visual-oddball paradigm (e.g., Boutonnet et al., Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013; Jouravlev et al., Reference Jouravlev, Taikh and Jared2018; Thierry et al., Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009; Xia et al., Reference Xia, Xu and Mo2019).
Evidence from the visual-oddball paradigm with monolinguals
In a typical visual-oddball paradigm, participants identify infrequent visual target stimuli within a continuous flow of rapidly presented stimuli. The critical stimuli in this design are nontarget stimuli. Within the critical stimuli, a standard stimulus is presented with a high probability (e.g., 80%), and a deviant stimulus is presented with a low probability (e.g., 15%). The presentation of a deviant stimulus in a sequence of standards is known to evoke a visual mismatch negativity (vMMN) response in ERP brainwaves in an early time window (usually peaking at 150–250 ms, see Czigler, Reference Czigler2014; Pazo-Alvarez, Cadaveira & Amenedo, Reference Pazo-Alvarez, Cadaveira and Amenedo2003). The vMMN is considered to be an index of an automatic preattentive detector of change to a novel visual stimulus (e.g., Czigler, Reference Czigler2007; Winkler, Czigler, Sussamn, Horváth & Balázs, Reference Winkler, Czigler, Sussman, Horváth and Balázs2005). Consequently, the magnitude of vMMN has been used as an index of perceived difference/similarity between objects. The vMMN has been interpreted as a purely perceptual phenomenon because it is thought that specific lexical information is unlikely to be available in this early time window (Strijkers, Holcomb & Costa, Reference Strijkers, Holcomb and Costa2011; Thierry, Reference Thierry2016). The vMMN is assessed by comparing the average amplitude of the waveform for the deviant in the time window of interest to that for the standard. To ensure that these comparisons reflect deviancy and not inherent perceptual differences between the standards and deviants, studies typically counterbalance stimuli such that a stimulus that is a standard in one block of trials is the deviant in a second block, and data are averaged across the blocks.
Thierry et al. (Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009) used a visual-oddball paradigm to test the effects of language on colour perception. They tested English and Greek individuals, who differ in how they label the colour blue. Greek differentiates a darker blue called ble and a lighter blue called ghalazio. However, this differentiation does not exist for green. A visual-oddball task was used where shades of dark and light blue were used as standards and deviants in two experimental blocks, and shades of dark and light green were used in two other blocks. Results showed that the vMMN effect (deviants elicited a more negative vMMN than standards) was larger for dark and light blues than for the greens in Greek participants, but this difference was not observed in English participants, suggesting that using two labels for the colour blue made Greek speakers perceive a greater difference between light and dark blues than English speakers. Similarly, Boutonnet et al. (Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013) investigated the effects of labeling on object perception. English has different words to refer to a cup and a mug, while Spanish uses the same label taza for these two objects. Pictures of cups and mugs were used as standards and deviants in a visual-oddball task. Results showed that the vMMN effect was larger in English compared to Spanish participants, suggesting that English speakers perceived a greater difference between the two objects than Spanish speakers.
More recently, Jouravlev et al. (Reference Jouravlev, Taikh and Jared2018) used a visual-oddball paradigm to investigate the perception of objects that share a common verbal label but belong to different conceptual categories. In their Experiment 1, pictures of orange as a fruit and orange as a colour were used as critical stimuli. Pictures of an apple and the colour red were used as controls. In Experiment 2, pictures of bat as an animal and as a baseball bat were used as critical stimuli. Pictures of a bird and a hockey stick were used as controls. Results showed that English speakers had a larger vMMN effect for controls than critical stimuli pairs, suggesting that they perceived two objects as more similar if they share a label even when the two objects are very dissimilar and from different conceptual categories.
According to the label-feedback hypothesis (Lupyan, Reference Lupyan2008, Reference Lupyan and Ross2012), in a visual-oddball task, participants’ ongoing perceptual processes could be influenced by the feedback from the on-line activation of the corresponding labels for pictures. More specifically, a sequence of the standard stimulus would activate the label for that stimulus, and the label would then activate a range of typical perceptual features of the object. Then when a deviant stimulus is presented, the picture would activate a range of perceptual features of the deviant object. When the standard and deviant objects share a verbal label, the relevant features activated from the feedback from the label for the standard would overlap with the features activated for the deviant, resulting in the standard and deviant objects being perceived more similarly. For example, in Boutonnet et al.'s study (Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013), a sequence of cup standards would activate in Spanish participants the label taza, and the label would then activate a range of features associated with taza, like holds liquid, has a handle. When a mug deviant is presented, many features of a mug would have already been activated, thus producing a small vMMN. In contrast, when the standard and deviant have different labels, the relevant features activated from the feedback from the label for the standards would have less overlap with the features activated for the deviant, resulting in the standard and deviant objects being perceived less similarly. Continuing with the cup/mug example, a sequence of cup standards would activate in English participants the label cup, and the label would activate typical features of the object cup, only some of which are also features of a mug. When a mug deviant is presented, it would activate some unique features that were not previously activated by the label cup (e.g., straight sides, tall), thus producing a large vMMN.
While the above explanation is based on the assumption that the activation of a label has a transient influence on the ongoing perceptual processing, it is also possible that, through our daily usage of language, the feedback from labels gradually changes the organization of our conceptual representations. Objects that share a label, such as taza, become more strongly associated with the typical features of the category, with the result that they are represented closer together in conceptual space, and objects with different labels are represented further away from each other. As a result, in an oddball task, Spanish speakers would produce only a small surprise when they saw a mug deviant followed by cup standards, because cup and mug are stored close together in semantic space, thus producing a reduced vMMN effect.
Implications for bilinguals
The label-feedback hypothesis provides a new perspective on language-thought interaction. However, to our knowledge, the majority of the research exploring this hypothesis has focused on monolinguals, despite the fact that much of the world's population is bilingual (Grosjean & Li, Reference Grosjean and Li2013). Bilinguals have two sets of labels, and these labels often do not have a one-to-one correspondence with conceptual and perceptual referents across languages. Therefore, the labels from each language could potentially have a somewhat different influence on the conceptual representations of bilinguals. Indeed, studies investigating naming patterns of common household objects in bilinguals and monolinguals have provided evidence that the structure of those categories in bilinguals is different from that of monolinguals in either language (Ameel, Malt, Storms & Van Assche, Reference Ameel, Malt, Storms and Van Assche2009; Ameel, Storms, Malt & Sloman, Reference Ameel, Storms, Malt and Sloman2005; Malt, Li, Pavlenko, Zhu & Ameel, Reference Malt, Li, Pavlenko, Zhu and Ameel2015; Malt & Sloman, Reference Malt and Sloman2003; Pavlenko & Malt, Reference Pavlenko and Malt2011). The results of these studies suggest that feedback from labels can produce long-term changes to conceptual representations.
One study that has used the visual-oddball paradigm with bilinguals again made use of the fact that Greek has two labels for blue, ghalazio and ble, whereas English has just one (Athanasopoulos et al., Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010). Recall that Thierry et al. (Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009) observed that the vMMN effect was larger for dark and light blues than for dark and light greens in Greek participants, but no such difference was seen in English participants. Athanasopoulos et al. (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) then investigated whether exposure time to an L2 (English) environment in Greek–English bilinguals affects their colour perception. Greek–English bilingual participants were divided into two groups of 10 participants each based on the amount of time they had lived in an English-speaking country. Results showed that short-stay bilinguals (M = 7.2 months, range = 5–12 months) showed larger vMMN effects for dark and light blues than for greens, whereas long-stay bilinguals (M = 42.6 months, range = 18–60 months) did not, resembling English monolinguals. Furthermore, these group differences remained when L2 proficiency, years of formal L2 instruction, and frequency of L2 use were included as covariates. The results suggested that, with more immersion in an English-speaking environment, Greek–English bilinguals eventually lose the ble/ghalazio distinction. Therefore, both languages of a bilingual appear to influence colour perception.
Athanasopoulos et al.'s findings (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) can be understood by two alternative explanations based on the label-feedback hypothesis (Lupyan, Reference Lupyan2008, Reference Lupyan and Ross2012). First, in the visual-oddball task, when a sequence of light blue standards was presented to a long-stay Greek–English bilingual, the English label blue would be activated to a higher degree than the Greek label ghalazio, because long-stay bilinguals are generally highly proficient in English and have more opportunity to use English than Greek. This would result in the long-stay bilinguals getting more feedback from the English label to the perceptual level than the Greek label. The English label blue would activate a range of perceptual features for a prototypical blue. Then when a dark blue deviant is presented, some relevant perceptual features would have already been activated, thus producing a small vMMN. On the contrary, for short-stay bilinguals, a sequence of light blue standards would activate the Greek label ghalazio to a higher degree than the English label blue, and the Greek label ghalazio would then activate only light blue perceptual features. When a dark blue deviant is presented, it would activate some unique features that do not overlap with the features activated for the light blue standards, thus producing a larger vMMN. The alternative explanation is that the long-stay Greek–English bilinguals’ organization of conceptual representations of the colour blue could have changed under the influence of the relatively large amounts of English usage. Light blue and dark blue become more strongly associated with the features of a prototypical blue through the feedback from using the English label blue for both of them. As a result, light blue and dark are represented closer together in conceptual space. Therefore, in an oddball task, long-stay bilinguals would produce only a small surprise when they saw a dark blue deviant followed by light blue standards, thus producing a small vMMN.
It is difficult to adjudicate between these explanations using an oddball task, which taps very early online processing. Evidence that there are lasting changes in conceptual representations comes from the results of a similarity judgement task that the same Greek–English bilinguals were asked to perform on various blue colour chips. Short-stay bilinguals rated pairs of chips falling within a category (galazio or ble) as more similar than did long-stay bilinguals, and they rated pairs of chips from different categories as less similar than long-stay bilinguals. These findings, along with the previously cited findings of Malt and colleagues, suggest that conceptual representations have been altered by the acquisition of another language. However, it may be the case that the mechanism described as the first alternative is also operating and helps to produce long-term changes. When explaining the processing that we think is happening in our vMMN experiment, we give an account from both perspectives, but do not attempt to adjudicate between them. Indeed, they may not be mutually exclusive.
The present study
The present study extended the work of Athanasopoulos et al. (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) on the impact of verbal labels on perception in bilinguals. In that study, and in the aforementioned visual-oddball studies with monolinguals, the researchers made use of lexical ambiguity: that is, they examined the effects of sharing a label for two stimuli compared to using two different labels for the stimuli. Here, we instead examined the impact of Chinese word construction on the performance of Chinese–English bilinguals in the visual-oddball task. This choice was inspired by Liu et al. (Liu, Tardif, Mai, Gehring, Simms & Luo, Reference Liu, Tardif, Mai, Gehring, Simms and Luo2010) who investigated the effects of verbal labels on object categorization by capitalizing on differences in the way that words are constructed in Chinese and English. In English, most nouns do not provide linguistic clues to their categories (exceptions are sunflower and bluebird), whereas in Chinese, most nouns provide explicit category information morphologically (e.g., the morpheme 鸟 bird in the noun 鸵鸟 ostrich). In Liu et al.'s study (Reference Liu, Tardif, Mai, Gehring, Simms and Luo2010), images of objects that have category level cues in their Chinese names were used. Chinese and English participants judged the membership of images of typical and atypical exemplars and non-category exemplars of a category. Results showed that English speakers categorized atypical items with more difficulty than typical items, whereas Chinese speakers showed no such difference. The authors argued that these results demonstrated that the category cue provided in nouns facilitated the categorization process for Chinese speakers and reduced the influence of typicality.
Here we investigated whether sharing a category level cue in objects’ Chinese names enhances their perceived similarity by Chinese–English bilinguals in a visual-oddball task. The critical stimuli were two pairs of pictures of birds; one pair of pictures shared a category level cue in their Chinese names (robin, ostrich) whereas the other pair did not share a category level cue in their Chinese names (pigeon, penguin). Furthermore, one member of each pair was a typical exemplar of the category bird (robin, pigeon), and the other member of each pair was an atypical exemplar (ostrich, penguin). If a shared category cue in the Chinese name enhances perceived similarity, then Chinese speakers should show a smaller vMMN response with the robin/ostrich pair than the pigeon/penguin pair whereas English speakers should show no difference between the pairs because they are unaware of the category cue in the Chinese name. Such a finding would be akin to Athanasopoulos et al.'s (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) finding that Greek speakers showed a smaller vMMN to the light blue/dark blue pair than light green/dark green pair and English speakers did not. However, it would extend their findings from colours, which have a single visual feature, to objects, which have multiple features, and would also extend their research to a different property of the verbal label. Like Athanasopoulos et al. (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010), we tested two groups of bilinguals, one group who were relatively recent arrivals in Canada, and the second who had lived in Canada for a longer time. We used similar criteria as in their study to define the groups. The aim was to test their claim that, with a longer stay in an English-speaking environment, the influence from bilinguals’ first language on object perception diminishes.
Based on the label-feedback hypothesis, Chinese participants’ ongoing perceptual processes could be influenced by the feedback from the on-line activation of the object label when they are doing the visual-oddball task. A sequence of standards would activate the label of the object, and the label would then activate a range of features of the object. If the label has a category cue embedded (e.g., ostrich in Chinese), it would activate the most common features of the object category to a higher degree than the uncommon features. This effect of category cue would be even more notable when the object is an atypical exemplar of the category than a typical exemplar because a typical exemplar would activate many of the most common features of the category without the help of a category cue. Then when a deviant object of the category (e.g., robin) is presented, many of the perceptual features would have already been activated, thus producing a reduced vMMN effect. In contrast, if the label of the standards does not have a category cue embedded (e.g., penguin), the feedback from the label would activate only perceptual features of the object, without any additional boost in activation of the most common features of the category, thus enhancing the difference between the standards and the deviant.
Alternatively, the organization of Chinese speakers’ conceptual representations could have been changed through everyday usage of verbal labels. Having a category cue in an object's name would result in the most common features of the category being activated each time the object is encountered. Through everyday usage of labels, objects that have a category cue in their names (e.g., ostrich) would become more strongly associated with the most common features of the category and would be pulled closer together in the center of the category, even if it is an atypical exemplar of the category. As a result, objects that share a category cue would be perceived more similarly in a visual-oddball task because they are represented closer together in the semantic space, thus producing a reduced vMMN effect.
In summary, the current study used a visual-oddball task with ERP to examine the impact of verbal labels on object perception in bilinguals. Specifically, the study investigated whether there is an effect of sharing a category level cue in objects’ Chinese names on the perception of typical and atypical exemplars of a category in Chinese–English bilinguals compared to monolingual English participants, and whether any such effect in bilinguals is influenced by increased time in an English-speaking country.
Method
Participants
The data from 28 short-stay Chinese–English bilinguals (mean age 19 years, range 18–28, 20 female), 28 long-stay Chinese–English bilinguals (mean age 19 years, range 17–22, 20 female) and 28 native English speakers without any knowledge of Chinese (mean age 18 years, range 17–20, 15 female) were included in the analyses. Data from three additional short-stay bilinguals (poor ERP recording), and four additional long-stay bilinguals (two did not complete the experiment, two had poor ERP recording) were excluded from analyses. Participants were undergraduates at the University of Western Ontario. The L1 of the bilinguals was Mandarin; bilinguals were born in China or Taiwan. They rated their English language skills on a scale of 1 (none) to 10 (native-like fluency); they also self-reported the percentage of time that they were currently exposed to each of their language in their daily activities (see Table 1). Participants received course credit.
Demographic information for bilingual participants.
Materials
The stimuli were images of a robin, an ostrich, a pigeon, a penguin, and a squirrel. The birds were critical stimuli; the image of a squirrel was the target to which participants responded. Robin (zhi geng niao in Pinyin) and ostrich (tuo niao) share the category cue bird (niao) in their Chinese names while pigeon (ge zi) and penguin (qi e) do not. Robin and pigeon were typical items; ostrich and penguin were atypical items. The stimuli were selected based on typicality rating data and name agreement data collected in two pilot studies with participants from the same participant pool as in the main study. In pilot study 1, category label and item pairs (e.g., BIRD-robin) were presented to 34 English speakers and 18 Chinese–English bilinguals. Bilinguals had lived at least 2 years in Canada. Their mean self-ratings of proficiency in Chinese were 9.1/10 for understanding and 8.7/10 for speaking. Participants rated the typicality of the item using a 0 (Atypical) to 100 (Typical) slide scale. Items with the highest ratings for a category were selected as typical, items with the lowest ratings were selected as atypical. In pilot study 2, name agreement data for pictures were collected to ensure that images activated the expected names. Images were presented to 58 English speakers and 46 Chinese–English bilinguals. Bilinguals had lived from 2 months to 8 years in Canada. Their mean self-ratings of proficiency in Chinese were 9.7/10 for understanding and 9.7/10 for speaking. Participants were asked to type in a name for each image in their L1. Table 2 shows typicality ratings and name agreement for the birds used in the current study.
Mean typicality ratings and name agreements for the critical stimuli.
Note. Some participants named robin (24% in English monolinguals, 34% in bilinguals) and pigeon (29% in English monolinguals) as bird, which resulted in the relatively low name agreement data for them.
Procedure
Participants viewed 4 blocks of 400 images. Within each block, a standard stimulus was presented with a high probability (80% of trials) and a deviant stimulus was presented with a low probability (15% of trials). On 5% of trials, the target object (squirrel) was presented. Each image was presented for 300 ms. Variable interstimulus intervals, ranging from 400–600 ms were used. Participants were instructed to view the images passively and to press a button whenever they detected a squirrel. The order of items in each block of trials was pseudorandomized in such a way that (a) two deviants or two targets did not appear next to each other and (b) there were at least three standards in a row before a deviant was presented. To control for perceptual differences between the stimuli and ensure that comparisons reflect deviancy and not inherent perceptual differences between the standards and deviants, each of the four pictures served as both standards and deviants. In Block 1, robins were used as standards and ostriches as deviants. In Block 2, ostriches were used as standards and robins as deviants. In Block 3, pigeons were used as standards and penguins as deviants. In Block 4, penguins were used as standards and pigeons were used as deviants (see Figure 2). The order of blocks was counterbalanced across participants. At the end of the experiment, participants were asked to fill in a questionnaire about their language background. They were asked if they had been saying the names of the pictures to themselves during the task and none reported that they did. They were surprised to hear that we were looking at the effect of the labels for the nontarget pictures.
Experimental procedure.
EEG Recording and Preprocessing
Continuous EEG activity was recorded at 32 scalp sites using ActiveTwo BioSemi active Ag/AgCl electrodes embedded in a custom elastic cap (BioSemi, Amsterdam, The Netherlands). The electro-oculogram (EOG) was recorded with electrodes placed above and below the right eye (vertical), and on the outer canthus of each eye (horizontal). Data were recorded using ActiView software (BioSemi) in the frequency range of 0.1–100 Hz at a sampling rate of 512 Hz. All EEG electrode impedances were maintained below 5 kΩ.
Off-line analysis was performed using ERPlab toolbox (Lopez-Calderon & Luck, Reference Lopez-Calderon and Luck2014). All data were re-referenced to the mean electrical activity of the mastoids and bandpass filtered with cutoffs of 0.1 and 30 Hz. The epochs of interest for target images were established to be from -100 to 600 ms post-stimulus onset. Data were baseline corrected to the prestimulus baseline. The data were filtered of eye-movement artifacts that were identified by running an independent component analysis (ICA). Trials contaminated with activity greater than ± 75 microvolts (μΩ) were excluded from the analysis (2.64% of the trials were excluded for the short-stay bilinguals who were included in the analyses, 5.24% for the long-stay bilinguals, 4.65% for English monolinguals).
Results
The vMMN effect is observed in the N1 component. Here, the negative-going N1 component peaked at around 160 ms and was measured in the 140–180 ms time window. The data from 5 electrodes in the parieto-occipital region (PO3, PO4, O1, Oz, and O2) were included in the analyses (see Figure 3). These electrodes were selected for the analyses because the vMMN is typically greatest over posterior areas (Czigler, Reference Czigler2007; Holmes, Franklin, Clifford & Davies, Reference Holmes, Franklin, Clifford and Davies2009; Pazo-Alvarez et al., Reference Pazo-Alvarez, Cadaveira and Amenedo2003). For each participant, the data from the 5 electrodes were averaged. This focus on parieto-occipital electrodes is consistent with recent vMMN studies which also have presented only data averaged across electrodes in this area (e.g., Athanasopoulos et al., Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010; Boutonnet et al., Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013; Jouravlev et al., Reference Jouravlev, Taikh and Jared2018; Mo, Xu, Kay & Tan, Reference Mo, Xu, Kay and Tan2011; Thierry et al., Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009). To ensure that ERPs reflect only the deviancy effect rather than specific perceptual characteristics of stimuli, ERPs elicited by robin and ostrich were averaged when they were presented as standards and, further, when they were presented as deviants. Thus, there was one standard and one deviant ERP response to the robin/ostrich pair. The same procedure was used for the pigeon/penguin pair. Figure 4 shows the grand average waveforms in microvolts (μV) to standards and deviants elicited by each pair and voltage maps for the vMMN effect for the three participant groups (waveforms for 15 other electrodes appear in the Supplementary Materials, Figures S1–3). To test for any potential perceptual differences between robin/ostrich and pigeon/penguin pairs, the mean amplitudes of the P1 evoked just by standard stimuli in each of the four blocks were analyzed. The positive going P1 component peaked at around 120 ms and was measured in the 100–140 ms time window.
Electrode montage. Circles indicate electrodes included in the analysis.
Grand average waveforms in microvolts (μV) to standards and deviants elicited by Robin/Ostrich and Pigeon/Penguin pairs for (A) Short-stay Chinese-English bilinguals, (B) Long-stay Chinese-English bilinguals, and (C) English monolinguals. Voltage maps of the vMMN effect (deviants-standards). Waveforms correspond to mean responses of electrodes PO3, PO4, O1, Oz, and O2. Note that negative is plotted down. Voltage maps show data only from electrodes of interest.
N1/vMMN (140–180 ms)
In the first analysis, mean amplitudes of standard and deviant responses between 140 to 180 ms from short-stay bilinguals and English monolinguals were analyzed with linear mixed effects (LME) models in R (version 3.4.1, R Core Team, 2017) using the lme4 package (version 1.1–8–1, Bates, Mächler, Bolker & Walker, Reference Bates, Mächler, Bolker and Walker2015). Model 1 was fitted with Trial-Type (Standard vs. Deviant, sum coded), Word-Type (Cue vs. No-Cue, sum coded), and Participant-Group (English vs. Short-stay Bilingual, sum coded) as fixed effects, participants as random intercepts, and by-participant random slopes for the effects of Trial-Type and Word-Type (without interaction). The significance of the fixed effects was determined with effect coding and type-II Wald tests using the Anova function provided by the car package (version 2.1–5; Fox & Weisberg, Reference Fox and Weisberg2011). Results of the tests evaluating fixed effects are presented in Table 3; full model outputs appear in the Supplementary Materials (Tables S1–6).
Analysis of Variance Table for Model 1 (vMMN in short-stay bilinguals and English monolinguals).
There was a significant effect of Trial-Type (p < .001). Deviant images elicited a more negative N1 than standard images (the classic vMMN effect). Importantly, there was a significant three-way interaction of Trial-Type, Word-Type, and Participant-Group (p = .04). Separate analyses on each language group revealed that the Trial-Type x Word-Type interaction was significant for short-stay bilinguals, χ2(1) = 7.54, p = .006, but not for monolinguals, χ2(1) = 0.11. The vMMN effect was smaller for the robin/ostrich (Cue) pair than the pigeon/penguin (No-cue) pair in short-stay bilinguals, but the vMMN effect did not differ for the two pairs in English monolinguals. This result for monolinguals is important because there was a slightly smaller difference in the typicality ratings for the robin vs. ostrich than for the pigeon vs. penguin pair (see Table 2). If the vMMN results for bilinguals were the consequence of this small discrepancy, then the monolinguals should also have produced a smaller vMMN response for the robin/ostrich pair than for the pigeon/penguin pair. Likelihood ratio tests were conducted to measure the effect size of the three-way interaction. The relative likelihood was computed by comparing the Akaike information criteria (AIC; Akaike, Reference Akaike, Petrov and Caski1973, Reference Akaike1974) of two models: the full model with the triple interaction and the reduced model without the interaction using the formula: exp((AIC(model.reduced) – AIC(model.full))/2) (Burnham & Anderson, Reference Burnham and Anderson1998, Reference Burnham and Anderson2004). The relative likelihood indicates the likelihood that each model would minimize information loss compared to the other model. Here a relative likelihood of 2.96 was found, indicating that the full model (with the triple interaction) was 2.96 times more likely than the reduced model (without the interaction) to minimize information loss.
In the second analysis, mean amplitude of standard and deviant responses between 140 to 180 ms from long-stay bilinguals and English monolinguals were included. Model 2 had the same variables as Model 1 (see Table 4). There was a significant effect of Trial-Type (p < .001). Deviant images elicited a more negative N1 than standard images. No other main effects or interactions reached significance. The three-way interaction did not reach significance (p > .50), suggesting that long-stay bilinguals treated the difference between robin/ostrich deviants and standards similarly to that between pigeon/penguin deviants and standards, resembling the English monolinguals.
Analysis of Variance Table for Model 2 (vMMN in long-stay bilinguals and English monolinguals).
And finally, vMMN responses for long-stay and short-stay bilinguals were compared. Model 3 had the same variables as Models 1 and 2 (see Table 5). There was a significant main effect of Trial-Type (p < .001). Deviant images elicited a more negative N1 than standard images. No other main effects or interactions reached significance.
Analysis of Variance Table for Model 3 (vMMN in short-stay bilinguals and long-stay bilinguals).
In summary, although the pattern of vMMN responses for short-stay bilinguals differed from that of English monolinguals, the pattern for long-stay bilinguals did not differ significantly from either English monolinguals or short-stay bilinguals, but was between the two. This finding suggests that the performance of bilinguals becomes more like that of monolinguals the longer they are in an English-speaking country. However, our use of an arbitrary split between individuals who had been in an English-speaking country for less than two years vs two years or more was not sensitive enough to provide strong statistical support for such a developmental change.
Instead of treating length-of-stay as a categorical variable, we tried treating it as a continuous variable. When all bilinguals were included in the analysis with length-of-stay as a continuous predictor, the triple interaction between Word-Type, Trial-Type and Participant-Group did not reach significance (p > .15). This result for length-of-stay may have occurred because the distribution of scores on that variable was not ideal. Most participants had a stay of a year or less, or three years and more, with only 5 having a stay between those values. It may be the case that with a larger sample of bilinguals having a better distribution of scores on the length-of-stay variable, stronger evidence of a developmental trend might be found. It might also be the case, however, that, contrary to the claims of Athanasopoulos et al. (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010), length-of-stay in an L2-speaking country is not the most critical variable, but is correlated with another variable that does provide stronger evidence of a development trend. We explored the percentage of the day that participants are exposed to English (the correlation between the length of stay and percentage of English exposure was r = .31). We had asked our participants to estimate their current exposure to English, and those values ranged from 4–90% (SD = 20.8). Therefore, this variable has a more suitable distribution for use as a continuous variable in our models. When all of the bilingual participants were included in a single analysis and percentage of current use of English was included instead of Participant-Group, the triple interaction of English-Use, Word-Type, and Trial-Type was very close to significant, p < .07 (see Table 6). With more exposure to English, the category cue in Chinese had less impact on the pattern of vMMN results.
Analysis of Variance Table for Effect of Current English Exposure on vMMN in Bilinguals.
P1 (100–140 ms, Standards only)
To test for any potential perceptual differences in the two pairs of stimuli, mean amplitudes of the P1 evoked just by standard stimuli were analyzed in LME models. In the first analysis, mean amplitudes of standard responses between 100–140 ms from short-stay bilinguals and English monolinguals were analyzed. Model 4 was fitted with Word Type (Cue vs. No-Cue, sum coded), Stimulus-Type (Typical vs. Atypical, sum coded), and Participant-Group (Short-stay Bilingual vs. English, sum coded) as fixed effects, and participants as random intercepts (see Table 7).
Analysis of Variance Table for Model 4 (P1 mean amplitudes, standards only in short-stay bilinguals and English monolinguals).
There was a significant effect of Word-Type (p = .007), and a significant effect of Stimulus-Type (p < .001). Participants’ visual system responded with a larger P1 to standard stimuli without a category cue in their Chinese name (pigeon and penguin) than to those with category cue (robin and ostrich), and with a larger P1 to atypical items than typical items. There was also a significant two-way interaction between Word-Type and Stimulus-Type (p < .001), suggesting that there were some low-level perceptual visual differences between robin and ostrich vs. pigeon and penguin. Most importantly, the three-way interaction between Word-Type, Stimulus-Type, and Participant-Group was not significant (p > .40), suggesting that the low-level perceptual differences between robin and ostrich vs. pigeon and penguin were not significantly different for two language groups.
Model 5 was just like Model 4 except that the mean amplitudes of standard responses between 100–140 ms from long-stay bilinguals and English monolinguals were analyzed (see Table 8). There was a significant effect of Word-Type (p = .008), and a significant effect of Stimulus-Type (p < .001). Participants’ visual system responded with a larger P1 to standard stimuli without a category cue in their Chinese name (pigeon and penguin) than to standard stimuli with category cue (robin and ostrich), and with a larger P1 to atypical items than typical items. There was also a significant two-way interaction between Word-Type and Stimulus-Type (p = .001), suggesting that there were some low-level perceptual differences between robin and ostrich vs. pigeon and penguin. Most importantly, the three-way interaction between Word-Type, Stimulus-Type, and Participant-Group was not significant (p > .70), suggesting that the low-level perceptual visual differences between robin and ostrich vs. pigeon and penguin were not significantly different for two language groups.
Analysis of Variance Tablefor Model 5 (P1 mean amplitudes, standards only in long-stay bilinguals and English monolinguals).
Discussion
The current study explored the effects of Chinese word structure on object perception in Chinese–English bilinguals using a visual-oddball task. Specifically, the study investigated whether pairs of pictures whose Chinese names share a category label (bird) are perceived as more similar than pairs of pictures whose names do not contain a category label. The vMMN effect, in which deviant stimuli elicit a more negative ERP response than standard stimuli in an early time window (140–180 ms), was taken as an index of perceived difference/similarity between objects. The responses of Chinese–English bilinguals were compared to those of English monolinguals who were unaware of the category cue in the Chinese names. Furthermore, as in Athanasopoulos et al.'s study (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010), both short-stay and long-stay bilinguals were included to examine whether increased exposure to English has an impact on the vMMN response to pairs of pictures whose Chinese names share a category label.
Short-stay bilinguals vs. English monolinguals
Short-stay bilinguals and English monolinguals showed a differential vMMN response pattern. The vMMN effect was smaller for typical and atypical exemplars that share a category level cue (robin and ostrich) than exemplars that do not (pigeon and penguin) in short-stay bilinguals, but no such difference was observed in English monolinguals. These findings indicate that short-stay bilinguals perceived the robin/ostrich pair of birds as more similar than pigeon/penguin pair, presumably because the former shared the category cue bird in their Chinese names. More broadly, these findings suggest that two objects that share a category cue in their Chinese names are perceived by Chinese speakers as more similar than two objects that do not share a category cue. This result is consistent with Athanasopoulos et al.'s (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) finding that short-stay Greek–English bilinguals produced a larger vMMN response to light and dark blues than did English monolinguals.
The differences observed here between short-stay bilinguals and monolinguals can be explained by the label-feedback hypothesis (Lupyan, Reference Lupyan2008, Reference Lupyan and Ross2012). In this view, participants’ ongoing perceptual processes are influenced by feedback from the on-line activation of the object label from the previous stimulus. For example, in the visual-oddball task, a sequence of frequently presented penguin standards would have activated the label penguin, and the label would have then activated a range of perceptual features of a penguin. When the pigeon deviant was presented, the picture would have activated perceptual features of a pigeon. Because penguin is an atypical bird and pigeon is a typical bird, the perceptual features that were activated by feedback from the label for the penguin standard would have had less overlap with the perceptual features that were activated by the pigeon picture, thus producing a large vMMN effect. Similarly, a sequence of frequently presented ostrich standards would have activated the label ostrich, and the label would have then activated the perceptual features of an ostrich. In addition, an ostrich's Chinese name has the category cue bird embedded, which would have activated the most common features of the category bird (e.g., has wings, has feathers) to a higher degree than the uncommon features. When the robin deviant was presented, the picture would have activated the perceptual features of a robin (e.g., has wings, red belly, has feathers). As a result, the relevant features activated from the feedback from the Chinese label for the ostrich would be highly overlapping with the features activated for the robin, making robin less of a surprise when it appeared as a deviant and thus producing a reduced vMMN effect. One might expect a smaller vMMN with atypical standards (ostrich) and typical deviants (robin) than the reverse because the category label bird that would be activated from the standard would likely activate more features of robin than of ostrich. Indeed, in a post-hoc analysis we confirmed that there was a significant interaction of Block x Word-Type x Stimulus-Type for short-stay bilinguals (p < .005) in which the vMMN effect was smaller with ostrich standards than with robin standards. However, such a finding needs to be interpreted cautiously because it could simply reflect smaller perceptual differences from ostrich to robin than robin to ostrich. It is to control for such differences that visual-oddball experiments typically have two blocks of critical stimuli in which each stimulus serves as a standard and a deviant in a different block.
The alternative way to understand the current findings is that bilingual participants’ organization of the conceptual representations of four birds: robin, ostrich, pigeon, and penguin could have been changed under the long-term influence of the feedback from daily usage of the birds’ Chinese labels. Robin and ostrich become more strongly associated with the most common features of the category bird through the feedback from the category cue in their Chinese names, resulting in them being stored closer together in the center of the category. On the contrary, atypical birds that do not have a category cue in their Chinese names (e.g., penguin) are stored in the periphery of the category bird, while typical birds without a cue (e.g., pigeon) are stored closer to the center of the category, because the most common features of the category are more salient for typical members than atypical members. In the visual-oddball task, bilingual participants would have produced a smaller surprise when they saw an ostrich deviant followed by robin standards than when they saw a penguin deviant followed by pigeon standards, because robin and ostrich were stored closer together in the semantic space than pigeon and penguin. As a result, robin/ostrich pair were perceived as more similar to each other than pigeon/penguin pair in bilinguals, thus producing a reduced vMMN effect.
In the current study, the vMMN effect appeared in the same early window (140–180 ms) and in the same scalp location (posterior) as in previous studies investigating vMMN effects (e.g., Athanasopoulos et al., Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010; Jouravlev et al., Reference Jouravlev, Taikh and Jared2018). The early timing of the vMMN component is one reason that it is believed to reflect only perceptual processes (Boutonnet et al., Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013); lexical access for a newly-presented item is not likely to have occurred at this early time window. Several previous studies have suggested that the earliest effects of lexical information occur at around 200 ms after stimulus onset in picture naming (Costa, Strijkers, Martin & Thierry, Reference Costa, Strijkers, Martin and Thierry2009; Indefrey & Levelt, Reference Indefrey and Levelt2004; Strijkers, Costa & Thierry, Reference Strijkers, Costa and Thierry2010) and even later in tasks with pictures that do not require naming (Strijkers et al., Reference Strijkers, Holcomb and Costa2011). The visual-oddball task used here did not require naming or any processing of images that were not targets. Therefore, the vMMN component observed in the current study likely reflects the early perceptual processing of the just-presented objects, but not subsequent linguistic processes.
The difference observed between the robin/ostrich pair and the pigeon/penguin pair in short-stay bilinguals is also very unlikely to be due to some low level perceptual differences between two stimuli pairs. In the current study, we used a design in which stimuli that were used as deviants in one block were used as standards in another, and vice versa, and then averaged across blocks. This ensured that within each word type (Cue vs. No-Cue) condition, standard and deviant stimuli were exactly matched. In addition, P1 responses to standards were examined, and no significant interaction between Word-Type, Stimulus-Type, and Participant-Group was observed. This means that the perceptual difference when switching from standard to deviant in the Cue vs No-Cue conditions was the same for bilingual groups and English monolingual group.
The effect of exposure to English
We also tested a group of long-stay bilinguals to investigate how the influence of L1 on object perception changes with increased L2 exposure. Our choice of length-of-stay as the criterion for forming groups was based on Athanasopoulos et al.'s (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) study. Long-stay bilinguals and English monolinguals showed a similar vMMN response pattern. That is, there was no difference in vMMN effects for the pigeon/penguin pair and robin/ostrich pair in long-stay bilinguals and English monolinguals, suggesting that the two groups perceived similar-sized differences between robin and ostrich vs. pigeon and penguin. These findings are consistent with Athanasopoulos et al.'s study (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010), in which no differences were found between long-stay Greek–English bilinguals and English monolinguals in perceiving shades of light and dark blue, and provides some evidence that object perception is influenced by labels in both of a bilingual's languages. As bilinguals stay in an L2-speaking country for a longer time, the influences from their L1 on object perception gradually diminish. However, the results were not as clear as anticipated because the pattern of results for long-stay bilinguals was not significantly different from that of short-stay bilinguals either.
We first sought firmer evidence for an effect of exposure to English by treating length of stay as a continuous rather than a categorical variable. This was not successful, likely because the distribution of scores was not ideal for such an analysis. We then used participants’ estimates of the percentage of the day that they were currently exposed to English and found a nearly significant triple interaction (p < .07) with Word-Type and Trial-Type. This result provides promising evidence that exposure to English weakens the impact of a Chinese category cue on bilinguals’ object perception, although, again, it is not as strong a result as one would hope. We think that this variable only partly captures what likely has an impact on the influence of Chinese on object perception, and that is substantial English exposure over a prolonged period of time. We asked participants only about current exposure.
Athanasopoulos et al. (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) argued that the effect of length of stay in the L2 country is likely to be fundamentally a matter of use of language. The more bilinguals stay in the L2 country, the less opportunity they have to use their L1 words to refer to objects. While this is likely to be generally true, there may be considerable variability among bilingual individuals in the extent to which they maintain their interactions with speakers of their L1. This could be a reason why length-of-stay was more weakly related to the diminished influence of L1 in our study than in Athanasopoulos et al.'s study, and why percentage of current English use worked somewhat better here than length-of-stay. Our participants may have maintained more extensive interactions with other L1 speakers than did theirs. Another possible reason for the difference in results between the studies is that the Athanasopoulos et al. study involved colours, for which each additional year living in an English-speaking country would likely result in thousands more exposures to the relevant labels, whereas we used birds, for which each additional year living in an English-speaking country might result in only dozens more exposures to the relevant labels.
Assuming that increased English exposure did diminish the influence of Chinese on object perception, we offer the following explanation based on the on-line influence of labels account. In the visual oddball task, when a sequence of ostrich standards was presented to a bilingual with extensive English exposure, the English label ostrich would be activated to a higher degree than the Chinese label 鸵鸟. This would result in these bilinguals getting more feedback from the English label ostrich to the perceptual level than the Chinese label. Because the English label ostrich does not have the category cue bird embedded in it, bilinguals with extensive exposure to English get little boost in activation of the most common features of the category bird when they saw the ostrich standards. When the robin deviant was presented, the picture would have activated perceptual features of a robin. As a result, the relevant features activated for an ostrich would have little overlap with the features activated for a robin, thus producing a large vMMN difference between robin/ostrich standards and deviants. Alternatively, with extensive exposure to English, bilinguals’ organization of the conceptual representations of four birds: robin, ostrich, pigeon, and penguin could have changed. Because the English label ostrich does not have the category cue bird embedded in it, these bilinguals get little boost in activation from the most common features of the category bird when they use the English label. This would result in the strong association between ostrich and the most common features of the category being gradually weakened, and ostrich being pushed away from the center of the category. On the contrary, robin, as a typical bird, would still be represented close to the center of the category. As a result, bilinguals with extensive exposure to English perform more like English monolinguals in the oddball task in that they produce similar vMMN effects for the robin/ostrich pair and pigeon/penguin pair.
Relation to previous literature
This study has extended in several ways the previous studies that have investigated the impact of language on perception using the visual-oddball paradigm. Athanasopoulos et al.'s study (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010) is the most similar, also having tested bilinguals, but here we have extended their finding that both languages of a bilingual can impact perception from the domain of colours, which have a single visual feature, to objects, which have multiple features. Several previous oddball studies have shown that verbal labels can impact object perception, in a single group of monolinguals (Jouravlev et al., Reference Jouravlev, Taikh and Jared2018), and in a comparison of monolinguals speaking two different languages (Boutonnet et al., Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013). Here we extended this research to bilinguals, providing evidence that both languages of a bilingual can impact object perception specifically. Furthermore, all of these studies made use of lexical ambiguity, either within a language (Jouravlev et al., Reference Jouravlev, Taikh and Jared2018), or across languages (Athanasopoulos et al., Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010; Boutonnet et al., Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013; Thierry et al., Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009). In these studies, two stimuli sharing a common label were perceived as more similar than stimuli having different labels. Here we extended the findings from ambiguity to a different feature of verbal labels: that is, whether the verbal label contains a category cue. There are only a few such words in English (e.g., sunflower, bluebird), but this is a common feature of Mandarin Chinese. The current results provided evidence that two objects can also be perceived as more similar if they share a category cue in their names (robin/ostrich) compared to objects that do not share a cue (pigeon/penguin). Our study has demonstrated this phenomenon using a small set of stimuli because of the large number of trials (1600) needed to obtain enough data from the deviants. Many other sets could have been chosen because a large number of Chinese words contain a category cue; future research will need to examine whether the results generalize to other stimuli.
As in Athanasopoulos et al.'s study (Reference Athanasopoulos, Dering, Wiggett, Kuipers and Thierry2010), we demonstrated the impact of English L2 on perception by showing that the influence of Chinese word structure is observed for bilinguals with a limited exposure to English, but this influence is diminished for those who have more exposure to English. Both studies tested distinct groups of participants based on years of living in an L2 speaking country; future research could track the time course of the influence of L2 by testing a much larger number of participants and treating length-of-stay in an L2 environment as a continuous variable. However, length-of-stay in an English-speaking country is only one indication of exposure to English, and it is likely not the best measure for groups of bilinguals where some members maintain extensive interactions with other L1 speakers. Here, bilinguals’ rating of their current exposure to English was somewhat related to the size of their vMMN effect. A more sensitive measure of L2 exposure would be one that captures the amount of exposure to L2 in each of the years that participants have lived in an L2 speaking country.
Conclusion
The current study provides supporting evidence for Luypan's (Reference Lupyan2008, Reference Lupyan and Ross2012) label-feedback hypothesis which states that the effects of labels can penetrate perceptual processes and influence object perception. Here we used an interesting characteristic of Chinese labels, and demonstrated that perception of an object is not only influenced by whether or not it has a verbal label, but it is also influenced by how a verbal label is constructed. Furthermore, the current study extended previous ERP oddball studies (e.g., Boutonnet et al., Reference Boutonnet, Dering, Viñas-Guasch and Thierry2013; Jouravlev et al., Reference Jouravlev, Taikh and Jared2018) to bilinguals and suggests that the language-perception interaction is more complex in bilinguals than in monolinguals. Bilinguals’ perceptual processes are constantly under the influence of two languages, and the relative impact of each language may change as bilinguals are exposed more to L2 and less to L1. The label-feedback hypothesis provides a useful framework in which to understand the mechanisms of language-perception interactions in bilinguals.
Supplementary Material
For supplementary material accompanying this paper, visit https://doi.org/10.1017/S1366728920000206
Acknowledgements
This research was supported by a grant from the Natural Sciences and Engineering Research Council of Canada to Debra Jared. We thank Steve Lupker and Paul Minda for their feedback on an earlier version of this work.
