Bimodal bilinguals, the fluent users of a spoken and a signed language, occupy only a small proportion of the bilingual population. However, they provide a unique window into the cognitive and neural structures of language processing. In an article in this issue, Emmorey and colleagues carefully examined the recent empirical evidence from hearing bimodal bilinguals and presented clear and interesting discussion on the implications to our understanding of bilingual language organization and processing (Emmorey, Giezen & Gollan, Emmorey, Giezen & Gollan). Here I would like to add discussion on code-blending and language control in bimodal bilinguals.
In the article, the authors primarily focused on code-blending, a unique feature of bimodal bilinguals. Obviously, code-blending is only possible for bimodal bilinguals, since signed and spoken languages use different sensory and motor systems. For unimodal bilinguals, given their spoken languages share the same sensory and motor system, only one language is allowed for production and comprehension at a particular time. Then, rather than code-blending, the unimodal bilinguals may have to apply a process of code-switching to shift the language in use. The authors also pointed out that bimodal bilinguals, both adults and children, showed a strong preference for code-blending over code-switching. Notably, the authors clearly distinguished ‘code-blending’ from ‘simultaneous communication’, in which both languages are parallel processed for mixed audiences.
Code-blending may not be comparable to code-switching in terms of the inner cognitive mechanism. Instead, it is more comparable to co-speech gestures. But, since there is no comprehensive cognitive model depicting the processing of either code-blending, code-switching or co-speech gestures, it may be difficult to make any comparison among these processes. Here, I venture to make a comparison from a single aspect: whether the process is goal-directed. Generally, language production or comprehension is a goal-directed process (Berger, Roskos-Ewoldsen & Monahan, Reference Berger, Roskos-Ewoldsen and Monahan2007), but it also includes many spontaneous components. As Emmorey et al. pointed out, code-blends in an utterance include a matrix language which provides the syntactic structure for the utterance and accompanying signs or words of the non-matrix language (Emmorey et al., Emmorey et al.). It seems that the processing of the matrix language is mainly goal-directed, though based on many automatic components. However, the activation of the accompanying non-matrix signs or words, particularly in the daily communication of bimodal bilinguals, is mainly spontaneous. Code-blending seems more like the process of speech accompanied with gestures in unimodal bilinguals or monolinguals, in which the speech is mainly goal-directed while the co-speech gestures are spontaneous. But code-switching is obviously different. It mainly occurs due to the change of dialogue partner(s) or situation(s), and the process is mainly goal-directed and controlled. Of course, the processing of co-speech signs is not identical to that of co-speech gestures, since the former have lexical representations with semantic, morphosyntactic, and phonological specifications but the latter do not, as Emmorey et al. have suggested (Emmorey et al., Emmorey et al.).
Differentiation of goal-directed process from spontaneous process may be helpful in clarifying the implications of the experimental findings on code-blending. For the code-blending in daily life, the production of the matrix language is goal-directed while that of the accompanying words or signs in the non-matrix language is mainly spontaneous. However, this might not be the case for code-blending in experimental conditions. When code-blending is required as the bilinguals participate in an experiment, the processing of both languages could be goal-directed. In other words, ‘code-blending’ in experimental conditions may be more like ‘simultaneous communication’ rather than the code-blending in daily life. In addition, the authors suggested that dual lexical retrieval does not incur a processing cost, based on their findings which show that response times for ASL (American Sign Language) produced alone were not different from response times for code-blend ASL(Emmorey et al., Emmorey et al.). Since ‘lexical retrieval’ mainly refers to a goal-directed process, the authors’ conclusion can be better supported if the processing of the two languages is both goal-directed.
The other issue I would like to discuss is language control in bimodal bilinguals. The authors concluded that because of the absence of perceptual and articulatory competition at the phonological level, bimodal bilinguals may experience weaker demands on language control compared to unimodal bilinguals. The first part of the conclusion is true, but the second part is arguable. There is no doubt that bimodal bilinguals do not apply the same mechanism as unimodal bilinguals in language control, but it is still not clear what the differences are. Bimodal bilinguals may actually have to deal with much more complex situations in controlling their languages. For unimodal bilinguals, only one language is allowed to be produced or perceived; they just need to select the target language and inhibit the non-target language during speech, or switch from one language to the other. Bimodal bilinguals have to choose between single code and code-blends. For language-switching, they could switch among the signed, the spoken and the code-blends. Even for code-blends, the matrix language may also be switched if the partner or situation changes. These more complex situations may call for a more flexible and effective control mechanism in bimodal bilinguals compared to unimodal bilinguals.
In addition, code-blending requires dynamic and efficient coordination between the two languages, which may not be necessary for unimodal bilinguals. In the article the authors have given several helpful examples of code-blends which reveal some important features of code-blending, including the synchronization between corresponding signs and words, the ability to distribute distinct syntactic constituents (Example 9, 10) or to produce distinct word orders across languages (Example 11, 12). These features suggest that the coordination between two languages is very complex and is not limited to semantical systems but also engages syntax systems. Such complex coordination during language production may also require a powerful and effective control system (Zou, Ding, Abutalebi, Shu & Peng, Reference Zou, Ding, Abutalebi, Shu and Peng2012).
Furthermore, if an inhibition of one language is required, unimodal bilinguals may actually benefit from the antagonistic mechanism for two languages in the production/perception systems due to the shared articulatory or perceptive organs. In such cases, the low-level ‘hardware’ constraint in unimodal bilinguals may alleviate the competition at lemma or higher levels compared to bimodal bilinguals.
Then, while there is no direct perceptual and articulatory competition at the phonological level, it does not necessarily mean bimodal bilinguals experience weaker demands than unimodal bilinguals on language control. It may be difficult to make a comparison of control demands between the two groups due to the absence of clear and comprehensive models for language control in both groups. Future studies are required to address this issue.
You have
Access
Bimodal bilinguals, the fluent users of a spoken and a signed language, occupy only a small proportion of the bilingual population. However, they provide a unique window into the cognitive and neural structures of language processing. In an article in this issue, Emmorey and colleagues carefully examined the recent empirical evidence from hearing bimodal bilinguals and presented clear and interesting discussion on the implications to our understanding of bilingual language organization and processing (Emmorey, Giezen & Gollan, Emmorey, Giezen & Gollan). Here I would like to add discussion on code-blending and language control in bimodal bilinguals.
In the article, the authors primarily focused on code-blending, a unique feature of bimodal bilinguals. Obviously, code-blending is only possible for bimodal bilinguals, since signed and spoken languages use different sensory and motor systems. For unimodal bilinguals, given their spoken languages share the same sensory and motor system, only one language is allowed for production and comprehension at a particular time. Then, rather than code-blending, the unimodal bilinguals may have to apply a process of code-switching to shift the language in use. The authors also pointed out that bimodal bilinguals, both adults and children, showed a strong preference for code-blending over code-switching. Notably, the authors clearly distinguished ‘code-blending’ from ‘simultaneous communication’, in which both languages are parallel processed for mixed audiences.
Code-blending may not be comparable to code-switching in terms of the inner cognitive mechanism. Instead, it is more comparable to co-speech gestures. But, since there is no comprehensive cognitive model depicting the processing of either code-blending, code-switching or co-speech gestures, it may be difficult to make any comparison among these processes. Here, I venture to make a comparison from a single aspect: whether the process is goal-directed. Generally, language production or comprehension is a goal-directed process (Berger, Roskos-Ewoldsen & Monahan, Reference Berger, Roskos-Ewoldsen and Monahan2007), but it also includes many spontaneous components. As Emmorey et al. pointed out, code-blends in an utterance include a matrix language which provides the syntactic structure for the utterance and accompanying signs or words of the non-matrix language (Emmorey et al., Emmorey et al.). It seems that the processing of the matrix language is mainly goal-directed, though based on many automatic components. However, the activation of the accompanying non-matrix signs or words, particularly in the daily communication of bimodal bilinguals, is mainly spontaneous. Code-blending seems more like the process of speech accompanied with gestures in unimodal bilinguals or monolinguals, in which the speech is mainly goal-directed while the co-speech gestures are spontaneous. But code-switching is obviously different. It mainly occurs due to the change of dialogue partner(s) or situation(s), and the process is mainly goal-directed and controlled. Of course, the processing of co-speech signs is not identical to that of co-speech gestures, since the former have lexical representations with semantic, morphosyntactic, and phonological specifications but the latter do not, as Emmorey et al. have suggested (Emmorey et al., Emmorey et al.).
Differentiation of goal-directed process from spontaneous process may be helpful in clarifying the implications of the experimental findings on code-blending. For the code-blending in daily life, the production of the matrix language is goal-directed while that of the accompanying words or signs in the non-matrix language is mainly spontaneous. However, this might not be the case for code-blending in experimental conditions. When code-blending is required as the bilinguals participate in an experiment, the processing of both languages could be goal-directed. In other words, ‘code-blending’ in experimental conditions may be more like ‘simultaneous communication’ rather than the code-blending in daily life. In addition, the authors suggested that dual lexical retrieval does not incur a processing cost, based on their findings which show that response times for ASL (American Sign Language) produced alone were not different from response times for code-blend ASL(Emmorey et al., Emmorey et al.). Since ‘lexical retrieval’ mainly refers to a goal-directed process, the authors’ conclusion can be better supported if the processing of the two languages is both goal-directed.
The other issue I would like to discuss is language control in bimodal bilinguals. The authors concluded that because of the absence of perceptual and articulatory competition at the phonological level, bimodal bilinguals may experience weaker demands on language control compared to unimodal bilinguals. The first part of the conclusion is true, but the second part is arguable. There is no doubt that bimodal bilinguals do not apply the same mechanism as unimodal bilinguals in language control, but it is still not clear what the differences are. Bimodal bilinguals may actually have to deal with much more complex situations in controlling their languages. For unimodal bilinguals, only one language is allowed to be produced or perceived; they just need to select the target language and inhibit the non-target language during speech, or switch from one language to the other. Bimodal bilinguals have to choose between single code and code-blends. For language-switching, they could switch among the signed, the spoken and the code-blends. Even for code-blends, the matrix language may also be switched if the partner or situation changes. These more complex situations may call for a more flexible and effective control mechanism in bimodal bilinguals compared to unimodal bilinguals.
In addition, code-blending requires dynamic and efficient coordination between the two languages, which may not be necessary for unimodal bilinguals. In the article the authors have given several helpful examples of code-blends which reveal some important features of code-blending, including the synchronization between corresponding signs and words, the ability to distribute distinct syntactic constituents (Example 9, 10) or to produce distinct word orders across languages (Example 11, 12). These features suggest that the coordination between two languages is very complex and is not limited to semantical systems but also engages syntax systems. Such complex coordination during language production may also require a powerful and effective control system (Zou, Ding, Abutalebi, Shu & Peng, Reference Zou, Ding, Abutalebi, Shu and Peng2012).
Furthermore, if an inhibition of one language is required, unimodal bilinguals may actually benefit from the antagonistic mechanism for two languages in the production/perception systems due to the shared articulatory or perceptive organs. In such cases, the low-level ‘hardware’ constraint in unimodal bilinguals may alleviate the competition at lemma or higher levels compared to bimodal bilinguals.
Then, while there is no direct perceptual and articulatory competition at the phonological level, it does not necessarily mean bimodal bilinguals experience weaker demands than unimodal bilinguals on language control. It may be difficult to make a comparison of control demands between the two groups due to the absence of clear and comprehensive models for language control in both groups. Future studies are required to address this issue.