Introduction
A challenge for bilingual individuals, from new second language learners to skilled translators, is that the referents of what appear to be translation equivalents can differ across languages. De Groot (Reference De Groot2011, p. 132) noted that “it is a well-known fact that complete meaning equivalence of the two terms in a translation pair is a rare phenomenon”. For example, if the French word balle is typed into a computer translator, it produces the English word ball, but as Paradis (Reference Paradis, De Groot and Kroll1997) points out, balle can be used to refer to tennis balls but not to basketballs or footballs. Many years ago, Kolers (Reference Kolers1963) investigated conceptual representations of translation-equivalent words by asking bilinguals to produce associates of words, first in one language and then in their other. He found that only about one-third of responses to translation-equivalent words were similar, for example, king–queen, rey–reina “king–queen”, but house–window, casa–madre “house–mother”. More recently, Malt and colleagues (Ameel, Storms, Malt & Sloman, Reference Ameel, Storms, Malt and Sloman2005; Malt, Sloman & Gennari, Reference Malt, Sloman and Gennari2003; Malt, Sloman, Gennari, Shi & Wang, Reference Malt, Sloman, Gennari, Shi and Wang1999) investigated cross-language differences in conceptual categories by examining how speakers of different languages label pictures of 60 common household containers. Among their findings is that Spanish speakers gave seven different names to the 16 containers that English participants called bottles, and Chinese speakers gave a single word for 40 different objects that were variously labeled jar, bottle, and container by English speakers. These cross-language differences not only pose a challenge to language users but also to researchers who attempt to understand how bilinguals represent meaning and how they link concepts to their verbal referents in each language (Ameel et al., Reference Ameel, Storms, Malt and Sloman2005; Ameel, Malt, Storms & Van Assche, Reference Malt, Sloman, Gennari, Shi and Wang2009; Bassetti & Cook, Reference Bassetti, Cook, Cook and Bassetti2011; Pavlenko & Malt, Reference Pavlenko and Malt2011).
In the next sections we review several models of conceptual processing in bilinguals, specifically examining whether they can account for the existence of different conceptual representations for translation-equivalent words. We first consider models in which conceptual knowledge is assumed to be represented by modality-neutral features (i.e., abstracted from perceptual representations), which is the case for most bilingual models, and we raise concerns about such representations. We then turn to models with modality-specific representations (i.e., representations specific to each perceptual modality). In particular, we focus on the Bilingual Dual Coding Theory (Paivio & Desrochers, Reference Paivio and Desrochers1980), and in the experiment that follows, we test an assumption of that theory.
The Revised Hierarchical Model (RHM)
The RHM (Kroll & Stewart, Reference Kroll and Stewart1994) was developed to explain how bilinguals activate word meanings in each of their languages and how they translate words from one language to another. In the RHM model, bilinguals are assumed to have a separate lexical store for words in each language and a common conceptual store. Connections between the lexical stores are assumed to be stronger from L2 to L1 than the reverse. The links between conceptual representations and words in the second language are assumed to be weak at first but become stronger with experience with the language. Once bilinguals are proficient in their second language, it is assumed that the same underlying conceptual representations would be accessed from both languages. However, the precise nature of the conceptual representations was left unspecified (Kroll, Van Hell, Tokowicz & Green, Reference Kroll, Van Hell, Tokowicz and Green2010), although we assume that the authors of the RHM had abstract feature representations in mind. The model does not tackle the issue of how translation-equivalent words might link to somewhat different underlying conceptual representations in each language.
The Distributed Conceptual Feature Model
De Groot and colleagues (De Groot, Reference De Groot, Frost and Katz1992a; Kroll & De Groot, Reference Kroll, De Groot, De Groot and Kroll1997; Van Hell & De Groot, Reference Van Hell and De Groot1998) developed a distributed conceptual feature model to account specifically for differences in how translation-equivalent words map to conceptual representations (see Figure 1). Conceptual memory is assumed to be shared across languages and to consist of elementary meaning units, or features. Pairs of translation equivalents may activate all or many of the same features, as in Figure 1 (top panel) for the words father and the Dutch word vader, or they may activate fewer of the same features, as in Figure 1 (bottom panel) for the words idea and idee. De Groot suggested that concrete words and their translations are likely to activate the same or a very similar set of conceptual features because concrete words refer to perceptual referents that are largely shared across languages, whereas abstract words and their translations are likely to activate a smaller set of common features. This conjecture is supported by studies showing that concrete words were more easily translated than abstract words (De Groot, Reference De Groot1992b; De Groot & Poot, Reference De Groot and Poot1997), concrete translation pairs more often generated the same word association response than abstract translation pairs (Kolers, Reference Kolers1963; Van Hell & De Groot, Reference Van Hell and De Groot1998), and cross-language semantic priming effects were larger for concrete word pairs than for abstract words (Jin, Reference Jin1990). De Groot and colleagues do not specify the nature of these conceptual features, although in Van Hell and De Groot (Reference Van Hell and De Groot1998) they noted that some monolingual models of memory use labeled conceptual features such as “has legs” (see McRae, De Sa & Seidenberg, Reference McRae, de Sa and Seidenberg1997).
The Distributed Conceptual Feature Model. Distributed conceptual representations for translations that have exactly the same meaning in English and Dutch (top panel) and translations that differ somewhat in the two languages (bottom panel). Reprinted from A. M. B. De Groot (Reference De Groot, Frost and Katz1992a), Bilingual lexical representation: A closer look at conceptual representations. In R. Frost & L. Katz (eds.), Orthography, phonology, morphology, and meaning, p. 393. Elsevier, Amsterdam.
The Shared (Distributed) Asymmetrical Model
Dong, Gui and MacWhinney (Reference Dong, Gui and MacWhinney2005) also claimed that bilinguals have both shared and separate conceptual representations for translation-equivalent words, and in addition they made some proposals about changes that occur as proficiency in L2 develops. When L2 is first being learned, associations to a new L2 word include both the elements of the conceptual representation that are shared across languages as well as the elements specific to L1. As the learner becomes more proficient in L2, the links between L2 and the conceptual elements specific to L1 weaken, and the links between L2 and the L2 conceptual elements strengthen. Learning a second language is also assumed to impact L1 in that connections develop between L1 and conceptual elements specific to L2. It is not entirely clear what the authors mean by “conceptual elements”, but one interpretation is that these are features that are abstracted from perceptual experience.
Dong et al. (Reference Dong, Gui and MacWhinney2005) provided support for their view from a study of students in China who were learning English (as well as Chinese and English monolinguals). The participants were given target words and were asked to rate how similar each was to eight other words that were chosen specifically for that target. The task ratings were collected for both Chinese words and their English translation equivalents. For nine of the 16 target words (desk, car, tea, religion, red, colony, bread, crane, and green), there were clear differences in the similarity ratings across the groups. Ratings of less proficient Chinese–English bilinguals on the English version were quite different from English monolinguals and more similar to those produced by Chinese monolinguals on the Chinese version. This finding suggests that the conceptual representations that were connected to English words were largely similar to those connected to Chinese words. In contrast, the ratings of more proficient bilinguals on the English version were a little more similar to the English monolinguals and quite a bit less similar to Chinese monolinguals on the Chinese version. In Dong et al.'s model, this finding suggests that proficient bilinguals had lost or weakened some of their associations between L2 words and L1-specific conceptual elements. For both groups, the associations between L2 words and some L2-specific elements were either missing or weak. Furthermore, the ratings on the Chinese version of the task for both bilingual groups differed from those of the Chinese monolinguals, which indicates that conceptual representations associated with Chinese words were influenced by their knowledge of English. In Dong et al.'s model, this is reflected in connections from L1 words to conceptual elements that are unique to L2. Dong et al.'s findings provide further evidence that translation-equivalent words can have somewhat different conceptual representations in each language, and also provide evidence that these representations are dynamic and change with increasing proficiency in the L2.
Problems with feature models
Distributed feature models face several serious challenges (see Paivio, Reference Paivio1986, Reference Paivio2007, pp. 217–226). One challenge is to determine what information is coded by the features. For example, what features are involved in our ability to distinguish thousands of different faces? We can usually correctly report the number of windows in our homes, but would we have this stored as a feature? How detailed are features? For the concept dog we could say a feature is “has four legs”. But surely we would want to be able to distinguish between a Corgi, which has very short legs, and a Great Dane, which has very long legs. We could use the features “short legs” and “long legs” but what about leg lengths in between (e.g., a Collie)? In addition, we need to know something about leg shape for front and back legs to understand the expression crooked as a dog's hind leg. Is leg shape a feature? Determining the features, then, for even a simple concrete, concept such as dog is not easy.
A second challenge for feature models is to explain how the distributed features are integrated into conceptual wholes (the binding problem). For example, we know that the legs are attached to the underside of the dog with two on each side, and that the paws of the dog are attached to the end of the leg and not somewhere in the middle. How is this spatial information encoded? Where is the model that allows us to put the parts together correctly?
Several pieces of evidence cast doubt on the view that our representations of concepts consist solely of sets of features such as “has legs”. For example, studies have found that inverted faces (e.g., Yin, Reference Yin1969) and inverted line drawings of objects (e.g., Jolicoeur, Reference Jolicoeur1985) are identified more slowly than upright versions. Jolicoeur reasoned, “Clearly, if recognition is based entirely on the extraction of orientation-invariant features, then patterns should be recognized equally quickly in any orientation. The results argue strongly against this possibility” (Jolicoeur, Reference Jolicoeur1985, p. 300). Graf and Schneider (Reference Graf and Schneider2001, p. 484) concluded from object recognition studies such as these that “considering the orientation- and size-dependency of recognition performance, the most reasonable strategy is to integrate the concept of part-structure into an image-based framework of recognition”.
Furthermore, there is evidence for holistic processing of both faces and objects from effects such as the composite effect (for a review see Maurer, Le Grand & Mondloch, Reference Maurer, Le Grand and Mondloch2002). The composite effect is the finding that the top half of a face or object is more difficult to recognize when it is presented with a different bottom half than in the original if the two halves are aligned in a composite (i.e., appear as a complete face) than when they are misaligned (i.e., the bottom half is shifted sideways). If recognition is based on feature extraction, recognition should be equally easy in the two conditions because all of the same features are present in the aligned and misaligned conditions. The aligned composite is thought to engage holistic processing making it difficult to extract information about individual features. Gauthier and Tarr (Reference Gauthier and Tarr2002) and Wong, Palmeri and Gauthier (Reference Wong, Palmeri and Gauthier2009) provided evidence that holistic processing increases with experience differentiating subordinate category members. Scott (Reference Scott2011) has extended this finding to object learning in infants.
Barsalou (Reference Barsalou2003) argued that categories do not have underlying summary feature representations, although an individual can derive them by first constructing a holistic simulation of a target category (e.g., a particular dog), and then interpreting this simulation using property and relation simulators. With respect to whether conceptual representations are amodal (i.e., abstracted from perceptual experience) or multimodal (i.e., representations are specific to each perceptual modality), McNorgan, Reid and McRae (Reference McNorgan, Reid and McRae2011) note that although there is still a debate on this issue, the bulk of recent evidence from neuroimaging favours the multimodal account.
The Modified Hierarchical Model
Pavlenko (Reference Pavlenko and Pavlenko2009) rejected the feature-based approach to conceptual representation. Following a review of the current literature on the representation of lexical concepts in bilinguals, and taking into account Barsalou's (Reference Barsalou2003) view of conceptual representation as a dynamic, distributed and emergent phenomenon that operates in a context-dependent manner, she proposed a modified version of the hierarchical model (see Figure 2). In her model, the conceptual store is not unified, but instead includes conceptual representations that are either fully shared across languages, partially shared, or are specific to each language. Concepts that are specific to one language may not be linked to a lexical item in the other language (e.g., personal space does not have a conceptual equivalent in Russian, nor a translation equivalent), with the consequence that expressing that concept may require code-switching or lexical borrowing. Lexical concepts are assumed to be “multimodal mental representations that include visual (mental imagery), auditory (sound), perceptual (texture) and kinesthetic (sensori-motor) information” (Pavlenko, Reference Pavlenko and Pavlenko2009, p. 132), although the precise nature of these representations was not specified.
The Modified Hierarchical Model. Reprinted from: A. Pavlenko (Reference Pavlenko and Pavlenko2009), Conceptual representation in the bilingual lexicon and second language vocabulary learning. In A. Pavlenko (ed.), The bilingual mental lexicon: Interdisciplinary approaches, p. 147. Multilingual Matters, Buffalo, NY.
The model that Pavlenko (Reference Pavlenko and Pavlenko2009) outlined is remarkably similar to the Bilingual Dual Coding Theory proposed by Paivio and Desrochers (Reference Paivio and Desrochers1980) almost three decades earlier. That theory, and especially its parent, Dual Coding Theory, were developed in some detail (see Paivio, Reference Paivio1986, Reference Paivio2007, Reference Paivio2010). We think that it would be useful to revisit the Bilingual Dual Coding Theory and bring it into current discussion about the representation of lexical concepts in bilinguals because it was initially published long before the current surge of interest in cognitive research on bilinguals and may not be widely known among researchers of bilingualism (although see Hamers & Blanc, Reference Hamers and Blanc2000; Heredia, Reference Heredia, Altarriba and Heredia2008). The theory makes an interesting prediction concerning the relationship between translation equivalents and their referents that was tested in the current study. Before turning to that study, we first present an overview of Dual Coding Theory followed by a discussion of the bilingual version of the theory.
Dual Coding Theory
Dual Coding Theory (DCT) was developed as a contrast to single code cognitive theories according to which thinking goes on in the form of internal language alone or activation of more abstract mental representations variously called propositions, semantic representations, computational descriptions and the like. DCT asserts instead that all cognition involves activation and use of two modality-specific systems (see Figure 3). The nonverbal system directly represents the perceptual properties and affordances (uses) of nonverbal objects and events, whereas the verbal system uses linguistic symbols in thinking and communication. The systems are functionally independent but partly interconnected so that they can be activated and used separately or together, depending on task demands. They come into play when activated directly by corresponding perceptual stimuli or indirectly by spreading activation from already-activated representations. Patterns of dual coding activity mediate task relevant behaviors.
The Dual Coding Theory. Reprinted from A. Paivio (Reference Paivio2010), Dual Coding Theory and the mental lexicon. The Mental Lexicon, 5, p. 209. With kind permission by John Benjamins Publishing Company, Amsterdam/Philadelphia. www.benjamins.com.
The DCT representational units are called imagens and logogens. Activated imagens give rise to conscious imagery and can operate unconsciously to mediate performance in recognition, memory, language, and other tasks. They come in different sensorimotor modalities and sizes, so that we have visual, auditory, tactile, and motor imagens corresponding to objects and their attributes. Concepts, are therefore, assumed to be grounded in the perceptual systems that are used in interacting with the world (see also Barsalou, Reference Barsalou1999, Reference Barsalou2003, Reference Barsalou2010). In DCT, features are holistic parts of larger wholes. These parts are synchronously organized into perceptual hierarchies or nested sets (see Paivio, Reference Paivio1986, p. 60). For example, a human face consists of eyes, nose, lips, and other components that are themselves composed of still smaller parts such as iris, pupil, and nostril, and so on. This assumption appears to fit well with Graf and Schneider's (Reference Graf and Schneider2001) more recent conclusion that the concept of part-structure in object recognition should be integrated into an image-based framework. Tastes, smells, and emotions also are nonverbal experiences that can be indirectly activated by memory images of tasty, smelly, and emotional objects. Logogens (adapted from Morton, Reference Morton, Kolers, Wrolstead and Bouma1979) also are modality-specific entities that vary in size and are describable as visual, auditory, or haptic logogens corresponding to phonemes (or letters), morphemes, words, fixed phrases, and longer linguistic structures that can be remembered and expressed as holistic units. Importantly, multilingual individuals have logogen systems for their different languages, as discussed further below.
Activation occurs via pathways that connect the units to the external world and to each other. Recognition of familiar objects and language stimuli requires relatively direct activation of corresponding imagens and logogens via intermediate sensory systems. Referential connections between imagens and concrete word logogens permit objects to be named and names to evoke mental images. Associative connections between imagens and between logogens enable activity to spread within systems. Complex tasks could involve patterns of activity that engage both dual-coding systems in a probabilistic fashion. For example, abstract-word logogens lack direct referential connections to imagens but they can indirectly activate imagens via associations to concrete logogens (e.g., religion–church). A conceptual representation consists of the juxtaposition between the verbal representation and the associative image and verbal representations that are linked to it.
Dual Coding Theory has been supported by many kinds of evidence over the years. Numerous studies have shown additive memory effects of dual coding resulting, for example, from presentation of target items as pictures and their names (see Paivio, Reference Paivio1986). A comprehensive early summary (Paivio, Reference Paivio and Yuille1983) described 60 independent findings that were predicted or explained by DCT but not by any single code theory. The support has increased subsequently, especially with the addition of brain correlates of DCT representational units, structures, and adaptive functions (summarized in Paivio, Reference Paivio2007, Reference Paivio2010).
Bilingual DCT
A bilingual version of DCT is shown in Figure 4 (see Paivio, Reference Paivio1986, Reference Paivio and Reynolds1991; Paivio & Desrochers, Reference Paivio and Desrochers1980; and for a comprehensive update and extended applications see Paivio, Reference Paivio2007). The notable differences from the general DCT model are the separate logogen systems for two languages (L1 and L2) and the direct connections between them. The connections are assumed to be primarily between logogens for translation equivalents. The model also shows that concrete word logogens for the two languages are connected to a common imagen system, which is an alternate route for translation of concrete words. The theory makes a further interesting assumption that was tested in the current study. It specifies that there can be connections from L1 and L2 to separate and shared imagens, depending on the way the two languages are learned. Learning two languages in the same context (e.g., more or less concurrently in the same country) would result in more shared imagens, whereas learning in separate contexts (e.g., at different ages and/or different countries) would result in some differences in referential imagens for L1 and L2. Paivio (Reference Paivio and Reynolds1991) provided a personal anecdote to illustrate this idea. He is the son of Finnish immigrants to Canada and learned Finnish as his first language. He noted that the word church elicits an image of a typical Canadian Protestant or Catholic church, but the Finnish equivalent kirkko elicits an image of the white, wood frame, Finnish Lutheran church in the community in which he grew up. Bugelski (Reference Bugelski and Nicholas1977, as cited in Paivio & Desrochers, Reference Paivio and Desrochers1980) similarly observed that stimulus words in his first language, Polish, evoked images from his childhood in Poland, whereas English words elicited images of objects and events experienced after his move to North America. Such culture-specific differences in the nature of the imagery aroused by a bilingual's two languages were also suggested by Winograd, Cohen and Barresi (Reference Winograd, Cohen and Barresi1976).
Bilingual Dual Coding Theory. Reprinted from: A. Paivio & A. Desrochers (Reference Paivio and Desrochers1980). A dual-coding approach to bilingual memory. Canadian Journal of Psychology, 34, p. 391.
Several studies have provided experimental evidence supporting Bilingual Dual Coding Theory (see Paivio, Reference Paivio1986, Reference Paivio and Reynolds1991; Paivio, Clark & Lambert, Reference Paivio, Clark and Lambert1988; Paivio & Lambert, Reference Paivio and Lambert1981; Vaid, Reference Vaid1988). For example, Paivio and Lambert (Reference Paivio and Lambert1981) tested French–English bilinguals who were (a) shown pictures, French words, and English words one at a time and were required to, respectively, write the English name of each picture, translate each French word into English, and simply copy each English word; or (b) shown only English words accompanied by cues that prompted them to image to one third of the words, translate one third into French, and copy the remainder. Participants were then given an unexpected memory test in which they were required to recall either the English words they had written down or were given. The results of both experiments clearly showed that recall was highest for items in the verbal–nonverbal dual coding conditions, intermediate for translated items, and lowest for copied items. Especially relevant here is the fact that the bilingually coded items were recalled twice as well as the monolingually coded (copied) items, supporting the hypothesis that the two language codes were independent and additive in their joint effect on recall. The equally large further increase with verbal–nonverbal dual coding provided evidence that pictures or images contributed more to recall than did an additional verbal code. Bilingual Dual Coding Theory can also account for the concreteness effects noted above.
Bilingual Dual Coding Theory is not the only bilingual model to include an image store. Potter, So, Von Eckardt and Feldman (Reference Potter, So, Eckardt and Feldman1984) presented two alternative bilingual models, the word association model and the concept mediation model, and in both an image store was directly connected to a conceptual store, which in turn was connected to lexical store(s) for one or both languages. They did not address the issue of whether an image could be more strongly associated with a word in one language than the other, and it is not clear how that could happen if activation must first flow through a common conceptual store without more specific details on the nature of the conceptual representations.
The present study
The present study investigated the mapping between real-world referents and verbal forms in bilinguals. More specifically, it tested the assumption of Bilingual Dual Coding Theory that when two languages are learned at different times and in different contexts, some imagens may be more strongly associated with one language than the other. To accomplish this goal, Mandarin–English bilinguals completed a picture-naming task. Pavlenko (Reference Pavlenko and Pavlenko2009) noted that the picture-naming task is the only task in the traditional array that taps into the mapping between words and their real-world referents. A limitation of existing picture-naming studies is that they typically use single pictures of prototypical objects, avoiding cross-linguistic differences. Such stimuli may have overestimated the extent to which conceptual representations are shared across languages (Kroll & Tokowicz, Reference Kroll, Tokowicz, Kroll and De Groot2005; Pavlenko, Reference Pavlenko and Pavlenko2009). In contrast, we sought pictures that differed between Chinese and Canadian/Western culture. For example, in China, dragons are depicted as serpent-like creatures, whereas in Canadian/Western culture they are depicted more like dinosaurs (see Figure 5 for other examples). The prediction of the Bilingual Dual Coding Theory is that pictures from Chinese culture should be named more quickly in Mandarin than in English because the links between Chinese imagens and the Mandarin lexicon would be stronger than links between Chinese imagens and the English lexicon (assuming that participants do not have extensive experience speaking English in China). Similarly, pictures from Canadian/Western culture should be named more quickly in English than in Mandarin, assuming that participants have had sufficient experience in Canada and regularly speak English here (see Figure 6). The study, thus, provides evidence concerning the nature of mental representations in bilinguals of words that are considered to be translation equivalents.
Examples of culturally-specific images for mailbox, birdcage, cabbage, and mask.
A simplified figure of the Bilingual Dual Coding Theory showing the assumption of differential connections between imagens and their verbal referents in each language. Connections from the verbal store in each language to the nonverbal store are not shown.
Method
Participants
The participants were 40 Mandarin–English bilingual students at the University of Western Ontario. The sample included 13 males and 27 females, with a mean age of 20.7 years. They were all born in China and had lived there for a minimum of 9 years (M = 15.4). The mean length of their residency in Canada was 5.2 years. The mean age of first exposure to English was 8.8 years, and was typically at school in China. On average, they estimated that they currently spoke Mandarin 40% of the time and English 60% of the time. All indicated that Mandarin was the language that they spoke most often at home with their families. They rated their Mandarin language skills higher (on a scale out of 10) than their English skills for understanding (Mandarin M = 9.6, English M = 8.4), speaking (Mandarin M = 9.5, English M = 8.1), reading (Mandarin M = 9.3, English M = 8.3), and writing (Mandarin M = 8.5, English M = 7.7). Participants were given a research credit, or compensated $10 per hour for their participation.
Materials
The stimuli used in this experiment consisted of 44 pairs of culturally-biased experimental images and 44 pairs of culturally-unbiased filler images (for a list of picture names, see the Appendix). All images were real life images that were presented in colour. Each pair of biased images consisted of a Canadian/Western-biased image and a Chinese-biased image (e.g., a typical Canadian/Western mask vs. a typical Chinese mask) of the same object. The objects were chosen based on how different the objects or items were in the two different cultures. The unbiased fillers were pictures of common objects that are the same in both cultures. These unbiased filler images were included to disguise the experimental manipulation, and were chosen in pairs (e.g., a red apple and a green apple) so that the number of times the participants named each object was the same for experimental and filler items.
Two lists of 176 pictures each were created. All 44 pairs of biased images and 44 pairs of unbiased images were on both lists. The lists were divided into two sub-lists such that each member of a pair of images, both biased and unbiased items, was on a different sub-list (e.g., the Canadian/Western mask on list A, the Chinese mask on list B). Half of the Canadian/Western-biased images and half of the Chinese-biased images were on sub-list A, and the other half of each were on sub-list B. A second version of each list was created by mixing the order of the images. Two further lists were created by placing the B sub-list before the A sub-list on each list. That is, there were lists 1A–1B, 1B–1A, 2A–2B, and 2B–2A. Each sub-list started with four filler images to ensure that participants were focused on the task before the first experimental picture was shown. There were also 10 images used in the practice trials.
Procedure
Participants were given instructions verbally in English. They were informed that pictures would be presented one at a time on the center of a computer screen. They were instructed to name each picture aloud into a microphone that was interfaced to the computer. Half of the participants were asked to name the pictures in their first list (1A–1B or 1B–1A) in English and the other half named the pictures in Mandarin. The participants were given short breaks after 88 images (i.e., after naming each sub-list). They were then asked to name the pictures in their second list (2A–2B or 2B–2A) in the other language. In summary, the participants had to name a total of 352 pictures (176 biased, 176 unbiased fillers) in the main part of the experiment. Chinese-biased, Canadian/Western-biased and neutral pictures were mixed within lists, and language of naming changed between lists. There were also 10 practice trials before the start of the main experiment to allow the participants to familiarize themselves with the naming procedure.
Stimulus presentation and response timing was done using the E-Prime 2.0 software (Schneider, Eschman & Zuccolotto, Reference Schneider, Eschman and Zuccolotto2002). Each naming trial started with the presentation of a blank screen for 500 ms. A pictorial stimulus was then shown and remained on the screen until a verbal response was given. The inter-stimulus-interval (ISI) was 1500 ms. Participants’ reaction times were measured, in milliseconds, as the time taken between the onset of the pictorial stimulus and the onset of the verbal response. The experimenter, who could speak both Mandarin and English, had a list of the expected names for each picture. The experimenter put a check mark beside a picture name if the participant gave the expected response and otherwise wrote down the response that was given (or that no response was given).
Participants were then asked to complete a questionnaire to collect information about their language history. Lastly, participants completed a picture-rating questionnaire. They were given pages on which the experimental pictures were printed in colour. On separate sheets they were asked to rate each of the biased images on a seven-point Likert scale on how good of an example they thought each picture was of that particular object in Canada (Canadian/Western-biased images) and in China (Chinese-biased images). On the rating sheet, the names of the Canadian/Western-biased images were given in English and the names of the Chinese-biased images were given in Mandarin.
Results
Four of the experimental pictures had very high error/omission rates in at least one of the languages (duster, flute, puppets, and well), and were excluded from the analyses. The remaining 40 Chinese-biased images were given a mean typicality rating of 6.46 (out of 7) and the 40 Canadian/Western-biased images were given a mean typicality rating of 6.33. ANOVAs were performed on reaction time and the proportion error data from the experimental pictures, using both participant (F1) and item (F2) means as units of analysis. Data were excluded from the reaction time (RT) analyses for trials that contained mechanical errors, such as the microphone picking up sounds that were not a response (2.6% of trials), and from trials on which an error was made or the picture name was not known. Response latencies that were above 3500 ms (> 2.75 SDs from the mean and 3.8% of the data), were truncated and replaced by 3500 ms. This procedure was used so that as many correct trials could be included in the analyses as possible without having a small number of extreme scores distort the means. There were two variables in the analyses, Naming Language (Mandarin, English), and Image Version (Chinese-biased, Canadian/Western-biased). In the analysis by participants, both variables were within-participant variables. In the analysis by items, Naming Language was a within-item variable, while Image Version was a between-item variable. Below we focus on effects of Naming Language and interactions of Naming Language with Image Version because these are within-picture comparisons. It is very difficult to match different pictures across versions for ease of naming. The mean reaction times are illustrated in Figure 7.
Mean picture-naming latencies in Mandarin and English for Chinese and Canadian pictures.
There was no main effect of Naming Language on latencies for experimental images, Fs < 1. Critically, the interaction of Naming Language and Image Version was significant, F1(1,39) = 19.12, p < .001, MSE = 16755.9, F2(1,78) = 9.77, p < .01, MSE = 41349.1. Planned one-tailed t-tests were performed to examine the effect of language for each picture type. Participants named Chinese-biased images 87 ms faster in Mandarin than in English, t1(1,39) = 1.93, p < .05, t2(1,39) = 2.31, p < .05, and they named Canadian/Western-biased images 92 ms faster in English than in Mandarin, t1(1, 39) = 2.02, p < .05, t2(1,39) = 2.11, p < .05.
Fewer errors/omissions were made when naming experimental images in Mandarin (7.2%) than when naming them in English (14.2%), F1(1,39) = 51.11, p < .001, MSE = .004, F2(1,78) = 34.86, p < .001, MSE = .006, reflecting better Mandarin than English vocabularies. Furthermore, the interaction between Naming Language and Image Version was significant, F1(1,39) = 24.50, p < .001, MSE = .001, F2(1,78) = 5.79, p < .05, MSE = .006. Consistent with the latency data, fewer errors/omissions were made when naming Chinese-biased pictures in Mandarin (7.8%) than in English (17.7%), t1(1,39) = 8.27, p < .001, t2(1,39) = 5.21, p < .001. Surprisingly, fewer errors/omissions were also made when naming and Canadian/Western-biased pictures in Mandarin (6.6%) than in English (10.8%), t1(1,39) = 3.85, p < .001, t2(1,39) = 2.90, p < .01, but the impact of naming language was smaller for Canadian/Western-biased pictures (4.2% difference) than for Chinese-biased pictures (9.9% difference).
Participants named the unbiased filler pictures somewhat faster in English (M = 1161 ms) than in Mandarin (M = 1209 ms) but made slightly more errors in English (2.4%) than in Mandarin (1.7%). However neither of these differences were significant (RT: t(39) = 1.49, ns; Errors: t(39) = 1.85, ns).
Discussion
Regardless of naming language, culturally-biased pictures were named more quickly in the culturally-congruent language than in the culturally-incongruent language. Specifically, pictures from Chinese culture were named more quickly in Mandarin than in English. This finding alone could simply be due to greater fluency in Mandarin than in English. Critically, pictures from Canadian/Western culture were named more quickly in English than in Mandarin. The interaction between Naming Language and Picture Version was highly significant. These data provide evidence that the links between translation-equivalent words and their underlying referents can differ for a bilingual's two languages. Furthermore, they suggest that representations of lexical concepts are not necessarily the same for translation-equivalent words in two languages, even for concrete concepts.
The finding that the latency and error/omission data were in opposite directions for Canadian/Western-biased images (faster naming latencies during English naming than Mandarin naming but more errors) does not, we feel, reflect a speed accuracy trade-off in responding. The reason is that most responses were either omissions or indicated a lack of the vocabulary item, such as calling the cabbage a vegetable, or the drum a musical instrument, rather than responses that might have been started before the picture was fully processed. Our participants spoke English quite fluently, but appear to lack the vocabulary for some low-frequency concepts.
Choosing the culturally-biased pictures was challenging, particularly for the Chinese-biased images. There are many pictures that we could have chosen, such as Chinese-style clothing, that may not reflect the current experience of young Chinese people as China has become more Westernized. To ensure that we selected pictures that accurately reflected their experiences in China, participants were asked to rate how typical our pictures were of each lexical concept. Our participants gave the Chinese-biased pictures a rating of 6.46/7, indicating that our pictures were fairly typical exemplars of the lexical concepts as experienced in China. Their rating of the Canadian/Western pictures of 6.33/7 indicates that they these were typical exemplars of the concepts as experienced in Canada.
Theoretical implications
The experiment was designed to test the assumption of Bilingual Dual Coding Theory that when two languages are learned at different times and in different contexts, some imagens may be more accessible to one language than the other. Bilingual Dual Coding Theory provides the following account for how Mandarin–English translation-equivalent words could link to somewhat different underlying conceptual representations in each language. Experiences when living in China would have led to the creation of imagens that were strongly linked to Mandarin words. Here we specifically examined visual imagens, but the theory proposes that there are imagens for the other senses as well. There may have been some linking of the same imagens to English words from experiences while learning English at school, but these are likely to be much weaker than links to Mandarin words. When our participants moved to Canada, they would have created many new imagens that were quite different from their existing ones. Some of these would have been linked to Mandarin words, as all continued to speak Mandarin once they arrived. However, as they spoke more and more English, the links between the imagens that were created in Canada and English words would have become stronger and stronger. There would likely be much less opportunity to strengthen links between imagens of Chinese culture-specific objects and English words. Thus, links become strongest between a language and culturally-congruent imagens, and they are weaker between a language and culturally-incongruent imagens. In some cases, culturally-incongruent pictures might be named by first activating the word in the congruent language and then activating the name in the other language through links between the verbal systems.
The Bilingual Dual Coding Theory assumes separate, but interconnected stores for each language, as does the Revised Hierarchical Model (Kroll & Stewart, Reference Kroll and Stewart1994). In a recent critique of the RHM, Brysbaert and Duyck (Reference Brysbaert and Duyck2010) argued that such an assumption is no longer tenable given evidence collected in the last decade showing that L1 lexical representations influence the recognition of L2 words and vice versa. In a reply to the critique, Kroll et al. (Reference Kroll, Van Hell, Tokowicz and Green2010) point out that evidence for parallel access of the lexicons does not necessarily imply an integrated lexicon. They note that “[i]t could very well be the case that the two lexicons are functionally separate but with parallel access and sublexical activation that creates resonance among shared lexical features” (Kroll et al., Reference Kroll, Van Hell, Tokowicz and Green2010, p. 374). This debate is beyond the scope of the present investigation. The focus of the present study and of the Bilingual Dual Coding Theory is on the relationships between lexical and conceptual representations. We acknowledge that further development of the prelexical components of the Bilingual Dual Coding Theory may be needed to fully account for bilingual language processing.
The distributed feature model proposed by De Groot and colleagues (De Groot, Reference De Groot, Frost and Katz1992a; Kroll & De Groot, Reference Kroll, De Groot, De Groot and Kroll1997; Van Hell & De Groot, Reference Van Hell and De Groot1998) also claims to be able to account for differential links between translation-equivalent words and concepts in a bilingual's two languages. It does so by assuming that the features that are linked to words in each language can differ. Concrete words are assumed to activate largely overlapping sets of features in each language. Presumably for some concrete concepts, such as the culturally-biased ones chosen here, the features associated with the word in each language are somewhat different. For the concept cabbage, the feature “round” could be connected to the English word cabbage and “oblong” could be connected to its Chinese equivalent. Faster naming latencies for congruent than incongruent pictures could be explained by proposing that the picture of the Chinese cabbage activates more feature nodes which are connected to the Mandarin lexical node (e.g., the “oblong” feature) than are connected to the English lexical node (e.g., “round” is not activated), and vice versa for the Western cabbage. More activated feature nodes result in faster naming times. Similarly for mailbox, “red” and “rectangular” could be connected just to the English word mailbox and “green” and “cylindrical” just to the Chinese translation, producing faster naming times in English than in Mandarin when a red rectangular mailbox is shown, and faster naming times in Mandarin than English when a green cylindrical mailbox is presented. A similar explanation of our findings would be given by Dong et al.'s (Reference Dong, Gui and MacWhinney2005) view.
One problem with this account is to determine what information is represented in the abstract features that would allow a distinction to be made between the pairs of pictures used here. It seems fairly easy to come up with features that distinguish the two cabbages and two mailboxes. It is much harder, however, to come up with a list of features that would distinguish a Chinese and Canadian/Western depiction of a lion or a sword or a mask. Information about relative size of features and about spatial relationships among parts appears to be needed. And even the features for the seemingly straight forward case of the cabbages are not so clear on further consideration. The relationship between the leaves is quite different for the two cabbages (wrapped right around one another vs. adjacent to one another), and a person who has seen these two kinds of cabbages could describe the different relationships. But how likely is it that this information is stored as features (see Barsalou, Reference Barsalou2003)? Furthermore the model would need to explain how the features are rapidly extracted from the pictures that were shown and then reassembled correctly to activate the concept that those abstract features represent. These and other problems with feature accounts were more fully discussed in the introduction. Distributed feature accounts will have to tackle these issues if they are to be an adequate account of bilingual conceptual representation.
Paivio's (Reference Paivio1986, Reference Paivio2007, Reference Paivio2010) Dual Coding Theory (and Bilingual Dual Coding Theory) addresses the problem of how features are combined to represent whole objects (the binding problem) by assuming that features are holistic parts of larger wholes, which are organized into perceptual hierarchies or nested sets (e.g., iris, eye, face, head). Furthermore, Dual Coding Theory's proposal of imagens for each of the sensory modalities that represent the visual, auditory, tactile, and motor perceptual properties of concepts fits with recent evidence on grounded cognition which has shown that the brain's modal systems become active when people perform cognitive tasks (see Barsalou, Reference Barsalou2010). Barsalou (Reference Barsalou1999) has interpreted such findings as support for his view that concepts are represented by recreating patterns of activation that are associated with actual perception and action. In Dual Coding Theory, conceptual representations consist of a verbal representation and the associative image and verbal representations that are linked to it. More recently, Barsalou, Santos, Simmons and Wilson (Reference Barsalou, Santos, Simmons, Wilson, De Vega, Glenberg and Graesser2008) proposed the Language and Situated Simulation (LASS) theory in which linguistic forms and situated simulations interact to produce conceptual processing. They acknowledged that “Dual code theory and LASS have much in common” (p. 253).
Conclusion
We believe that the field of bilingualism needs to consider alternatives to abstract feature-based conceptual representations to best capture the results for bicultural bilinguals presented here and other emerging findings, such as those by Malt and colleagues (e.g., Ameel et al., Reference Ameel, Storms, Malt and Sloman2005, Reference Ameel, Malt, Storms and Van Assche2009; Malt et al., Reference Malt, Sloman, Gennari, Shi and Wang1999, Reference Malt, Sloman and Gennari2003; Pavlenko & Malt, Reference Pavlenko and Malt2011), and by Athanasopoulos and colleagues on colour perception (Athanasopoulos, Reference Athanasopoulos2009, Reference Athanasopoulos, Cook and Bassetti2011; Athanasopoulos, Damjanovic, Krajciova & Sasaki, Reference Athanasopoulos, Damjanovic, Krajciova and Sasaki2011; Thierry, Athanasopoulos, Wiggett, Dering & Kuipers, Reference Thierry, Athanasopoulos, Wiggett, Dering and Kuipers2009). These findings indicate that the referents for translation-equivalent words can differ in subtle ways that are not easy to capture with abstract features. We think that Bilingual Dual Coding Theory deserves serious consideration as an alternative explanation.
Appendix. Picture names for the culturally-biased and culturally-unbiased stimuli
