The commentaries on our Keynote article “Psycholinguistic, cognitive, and neural implications of bimodal bilingualism” were enthusiastic about what can be learned by studying bilinguals who acquire two languages that are understood via distinct perceptual systems (vision vs. audition) and that are produced with distinct linguistic articulators (the hands vs. the vocal tract). The authors also brought out several new ideas, extensions, and issues related to bimodal bilingualism, which we discuss in this reply.
Because the input-output systems do not conflict for bimodal bilinguals (in contrast to unimodal bilinguals), investigations into the nature of language control for bimodal bilinguals can provide novel insights into the psycholinguistic and cognitive mechanisms that all bilinguals utilize to manage their two languages. In this regard, Kroll and Bice (Reference Kroll and Bice2015) point out the need to distinguish between multiple components of inhibitory control when comparing bimodal and unimodal bilinguals. In particular, they argue that it is important to distinguish between local inhibition which is tied to specific patterns of lexical activation (e.g., as assessed in switching tasks) and global inhibition which is associated with inhibitory control of the language itself. Thus far, only local inhibitory control mechanisms have been investigated experimentally in bimodal bilinguals, with results suggesting reduced inhibitory demands for bimodal bilinguals because they can (and prefer to) code-blend (i.e., produce a word and a sign at the same time) rather than code-switch, and there appear to be no processing costs for producing or perceiving code-blends. What is still to be determined is whether unimodal and bimodal bilinguals differ with respect to the more sustained control associated with global inhibition of a non-target language.
However, the fact that bimodal bilinguals produce ‘co-speech signs’ and grammatical facial expressions when speaking with monolinguals suggests weaker global inhibition demands than for unimodal bilinguals. For example, bimodal bilinguals produce the ASL conditional marker (a grammatical facial expression) nearly 80% of the time when producing English conditional clauses in a monolingual context (Pyers & Emmorey, 2009). Green (Reference Green2015) asks how such inadvertent code-blending might affect conversational synchrony, e.g., multi-modal alignment across conversational partners (e.g., Pickering & Garrod, Reference Pickering and Garrod2013). We agree that it would be interesting to determine whether sign-influenced gestures or facial expressions produced by bimodal bilinguals impact the nature or the extent of alignment with a monolingual conversational partner. Anecdotal evidence from hearing signers from deaf families (‘Children of Deaf Adults’ or Codas) suggests that inadvertent code-blending could disrupt conversational synchrony because monolingual speakers are reported to sometimes misinterpret their facial expressions (Preston, Reference Preston1994).
Like Kroll and Bice (Reference Kroll and Bice2015), Poarch (Reference Poarch2015) argues that cognitive control mechanisms are complex and that there is likely not a simple ‘on/off’ switch for activating a target language and inhibiting a non-target language, as implied in our description of the switching study by Emmorey, Petrich and Gollan (Reference Emmorey, Petrich and Gollan2014). We agree that language control is more complex than flipping a switch and that a language is never completely turned ‘off’. Nonetheless, the fact that bimodal bilinguals switch between speaking (or signing) and code-blending offers a unique way to tease apart specific aspects of language control, e.g., ‘adding’ a language instead of switching to a different language. Adding a language in a code-blend requires releasing inhibition (if we assume that the non-target language was previously inhibited during speaking or signing). The results of Emmorey et al. (Reference Emmorey, Petrich and Gollan2014) indicate that for bimodal bilinguals, the process of bringing a new language on-line as part of a code-blend appears to be cost free.
Ding (Reference Ding2015) points out that experiments that force switching or code-blending via cues differ from spontaneous language mixing in natural settings. He argues that natural code-blending is akin to the production of spontaneous co-speech gesture and thus differs from code-switching, which he argues is more goal-directed and controlled. We agree that cued-switching and cued-blending tasks tap goal-directed rather than spontaneous processes (see Gollan & Ferreira, Reference Gollan and Ferreira2009; Gollan, Kleinman & Wierenga, Reference Gollan, Kleinman and Wierenga2014 for experimental paradigms that tap spontaneous code-switching). However, we suggest that code-blending can be goal-directed (i.e., intentional), as in example (8) of the target article in which an ASL–English bilingual says “So it was a lot of this” and produces the ASL sign CONNECTION while saying “this.” Nonetheless, we agree that differentiating goal-directed from spontaneous processes is important to understanding the factors that impact language control for both bimodal and unimodal bilinguals.
Ding (Reference Ding2015) and Green (Reference Green2015) both suggest that code-blending uniquely requires coordination and synchronization between sign and speech – perhaps similar to co-speech gesture, which might also draw on cognitive control processes. Green (Reference Green2015) wonders whether sign and gesture might actually compete for production in bimodal bilinguals. However, if this were the case, one might predict that bimodal bilinguals would gesture less than monolinguals due to gesture suppression from signs that are competing for manual output. Instead, the evidence indicates that bimodal bilinguals may gesture more than monolingual speakers (Casey & Emmorey, Reference Casey and Emmorey2009; Casey, Larrabee & Emmorey, Reference Casey, Emmorey and Larrabee2012).
Green (Reference Green2015) argues that a mechanism is required to control the serial output of both spoken words and signs, particularly for multi-word code-blends that have different syntactic structures in each modality (although these are relatively rare). He suggests that a competitive queuing mechanism offers a neurally plausible solution to the serial order problem. Although possible, it is clear that such a solution is particularly complex for code-blending because if speech and sign have distinct competitive queuing mechanisms, then the synchronization problem must be solved. And if there is a single competitive queuing mechanism, then as Green (Reference Green2015) notes, it must be able to simultaneously sample from two parallel (and not always identical) sentence plans.
Several commentaries discussed population factors that have yet to be adequately investigated in bimodal bilinguals. Kroll and Bice (Reference Kroll and Bice2015) argue that Codas are heritage language speakers (i.e., they acquire a signed language in early childhood and are then immersed and educated in the dominant spoken language). They suggest that the failure to observe advantages in inhibitory control in this population might reflect their status as heritage speakers who use their two languages in distinct contexts, which requires less inhibitory control between languages. On the other hand, it is possible that the requirement to maintain strict separation between two languages in different contexts might actually require more cognitive control, and cognitive control advantages have been observed in heritage language speakers (e.g., Tao, Taft & Gollan, Reference Tao, Taft and Gollan2015). Tang (Reference Tang2015) suggests that language proficiency, language dominance and age of acquisition may all interact to affect the extent of language co-activation and degree of inhibitory control in ways that are not yet understood in bimodal bilinguals. We agree and suggest that bimodal bilinguals provide a unique population to study these various effects on bilingual language processing and control because language dominance can remain stable (i.e., the spoken language for hearing bilinguals) while proficiency in the non-dominant sign language can vary across both early and late bilinguals (like other heritage language speakers, not all Codas are proficient in their early first language).
Hearing status is another important population factor that deserves further attention, as discussed by Woll and MacSweeney (Reference Woll and MacSweeney2015) and by Anible and Morford (Reference Anible and Morford2015). Almost all deaf signers who are educated in the spoken language of their communities are bilingual to some extent, but their status as bilinguals is often overlooked. Woll and MacSweeney (Reference Woll and MacSweeney2015) question the “bimodal” label for deaf bilinguals because they access both speech and sign primarily via the visual modality. Nonetheless, the output modality of their two languages involves different primary articulators (the hands vs. the mouth), and the suggested alternative label for deaf bimodal bilinguals may simply be too unwieldy (“deaf sign language and spoken/written bilinguals”). Anible and Morford (Reference Anible and Morford2015) also point out that deaf people who are bilingual in two signed languages constitute a different type of unimodal bilingual, and the study of these bilinguals may provide unique insights into cross-language activation due to the fact that distinct sign languages often utilize similar forms to express semantically-related concepts. Such “pseudo-cognates” (Adam, 2013) do not arise by historical relationships (like cognates in spoken languages) but because the form of these signs may be conceptually motivated (e.g., signs related to mental processes are often produced at the forehead across many historically unrelated languages).
Finally, Woll and MacSweeney (Reference Woll and MacSweeney2015) raise the interesting possibility that mouthings, i.e., the silent production of (parts of) spoken words with signs, constitute a form of code-blending – particularly, for deaf signers. As Woll and MacSweeney (Reference Woll and MacSweeney2015) note, the linguistic status of mouthings is unclear, with some arguing that mouthings form part of the non-manual phonological representation of signs (integrated into the sign lexicon via borrowing), while others argue that mouthings are represented in the spoken language lexicon and are accessed independently of signs. Only the latter view would support mouthings as a form of code-blending between a spoken and signed language. We agree that this issue deserves further research, but we note that voiced code-blends should be kept distinct from silent mouthings because they have different distributions. Hearing bimodal bilinguals produce silent mouthings when signing with either deaf or hearing people, but voiced code-blends are only produced with other hearing bimodal bilinguals.
In sum, the commentaries raised important questions that should be addressed by future research, with particular focus on the nature of language control mechanisms in bimodal bilinguals, population factors such as hearing status, proficiency, and language dominance, as well as possible effects deriving from unique aspects of language in the visual-manual modality (e.g., mouthings, motivated form-meaning mappings).
