No CrossRef data available.
Article contents
Sign, language, and gesture in the brain: Some comments
Published online by Cambridge University Press: 26 April 2017
Abstract
In contrast with two widely held and contradictory views – that sign languages of deaf people are “just gestures,” or that sign languages are “just like spoken languages” – the view from sign linguistics and developmental research in cognition presented by Goldin-Meadow & Brentari (G-M&B) indicates a more complex picture. We propose that neuroscience research suggests that a similar approach needs to be taken and offer some examples from research on the brain bases of sign language perception.
Information
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © Cambridge University Press 2017
References
Atkinson, J., Campbell, R., Marshall, J., Thacker, A. & Woll, B. (2004) Understanding “not”: Neuropsychological dissociations between hand and head markers of negation in BSL. Neuropsychologia
42(2):214–29.Google Scholar
Baker, C. & Padden, C. (1978) Focussing on the non-manual components of ASL. In: Understanding language through sign language research, ed. Siple, P., pp. 27–57. Academic Press.Google Scholar
Capek, C. M., Grossi, G., Newman, A. J., McBurney, S. L., Corina, D., Roeder, B. & Neville, H. J. (2009) Brain systems mediating semantic and syntactic processing in deaf native signers: Biological invariance and modality specificity. Proceedings of the National Academy of Sciences of the United States of America
106(21):8784–89.Google Scholar
Capek, C. M., Waters, D., Woll, B., MacSweeney, M., Brammer, M. J., McGuire, P. K., David, A. S. & Campbell, R. (2008) Hand and mouth: Cortical correlates of lexical processing in British Sign Language and speechreading English. Journal of Cognitive Neuroscience
20(7):1220–34.Google Scholar
Capek, C. M., Woll, B., MacSweeney, M., Waters, D., McGuire, P. K., David, A. S. & Campbell, R. (2010) Superior temporal activation as a function of linguistic knowledge: Insights from deaf native signers who speechread. Brain and Language
112(2):129–34.Google Scholar
Cardin, V., Orfanidou, E., Kästner, L., Rönnberg, J., Woll, B., Capek, C. M. & Rudner, M. (2015) Monitoring different phonological parameters of sign language engages the same cortical language network but distinctive perceptual ones. Journal of Cognitive Neuroscience
1:20–40.Google Scholar
Corina, D., Chiu, Y. S., Knapp, H., Greenwald, R., San Jose-Robertson, L., & Braun, A. (2007) Neural correlates of human action observation in hearing and deaf subjects. Brain Research
1152:111–29.Google Scholar
Corina, D. P., Poizner, H., Bellugi, U., Feinberg, T., Dowd, D. & O'Grady-Batch, L. (1992) Dissociation between linguistic and nonlinguistic gestural systems: A case for compositionality. Brain and Language
43(3):414–47.Google Scholar
Cormier, K., Schembri, A. & Woll, B. (2013) Pronouns and pointing in sign languages. Lingua
137:230–47.Google Scholar
Emmorey, K., Damasio, H., McCullough, S., Grabowski, T., Ponto, L. L., Hichwa, R. D. & Bellugi, U. (2002) Neural systems underlying spatial language in American Sign Language. NeuroImage
17(2):812–24.Google Scholar
MacSweeney, M., Campbell, R., Woll, B., Giampietro, V., David, A. S., McGuire, P. K., Calvert, G. A. & Brammer, M. J. (2004) Dissociating linguistic and nonlinguistic gestural communication in the brain. NeuroImage
22(4):1605–18.CrossRefGoogle ScholarPubMed
Newman, A. J., Supalla, T., Fernandez, N., Newport, E. L. & Bavelier, D. (2015) Neural systems supporting linguistic structure, linguistic experience, and symbolic communication in sign language and gesture. Proceedings of the National Academy of Sciences of the United States of America
112(37):11684–89.Google Scholar
Özyürek, A. (2014) Hearing and seeing meaning in speech and gesture: Insights from brain and behaviour. Philosophical Transactions of the Royal Society B: Biological Sciences
369(1651):20130296.Google Scholar
Goldin-Meadow & Brentari's (G-M&B's) article uses evidence from sign linguistics and cognitive development to indicate how sign language processing may be accommodated within cognitive science. We propose that, in addition to perspectives from those disciplines, brain imaging has provided some important leads, as have neuropsychological case studies of patients who sign. In one brain imaging study, for example, deaf native signers were shown signed utterances and brachial gestures (specifically, the traditional tic-tac signalling system used by bookies to communicate odds), which could not be interpreted linguistically (MacSweeney et al. Reference MacSweeney, Campbell, Woll, Giampietro, David, McGuire, Calvert and Brammer2004). Whereas tic-tac activated posterior and inferior regions of the temporal lobe, signed utterances additionally activated superior temporal and inferior frontal sites, implicating left-hemisphere-dominant perisylvian regions associated with language processing in hearing nonsigners. Further studies also point to a distinction between those manual actions that can be construed as language and those that do not (i.e., signers vs. nonsigners; (Cardin et al. Reference Cardin, Orfanidou, Kästner, Rönnberg, Woll, Capek and Rudner2015; Newman et al. Reference Newman, Supalla, Fernandez, Newport and Bavelier2015). Again, these implicate greater activation in left-hemisphere-dominant perisylvian regions for the processing of linguistically structured gestures contrasting with greater activation in posterior/inferior regions for manual gestures that are not linguistically construed. Such studies also lead us to distinguish regions that provide essential sensory input to the language system, and which may be distinct from language processing itself (Corina et al. Reference Corina, Chiu, Knapp, Greenwald, San Jose-Robertson and Braun2007,Reference Corina, Poizner, Bellugi, Feinberg, Dowd and O'Grady-Batch1992; Emmorey et al. Reference Emmorey, Damasio, McCullough, Grabowski, Ponto, Hichwa and Bellugi2002).
Such sign-focussed studies set up an interesting context for considering co-speech gestures in hearing speakers. Co-speech gestures can be “sign-like” and carry communicative significance. These include emblems such as “thumbs-up,” as well as gestures indicating action dynamics and spatial relationships. Behavioral studies show that such gestures modulate the spoken message. Left-hemisphere-dominant perisylvian language regions are specifically activated by co-speech gestures such as these (Özyürek Reference Özyürek2014). So, and in contrast with the studies that indicate different processing networks for (sign) language and gesture, these findings emphasize some common processing circuitry for gestures and (heard) speech, and raise issues of interpretation. For example, might gestures that sign linguists have appropriated to language analysis be actions that, in people who do not sign, are nevertheless processed by brain circuits associated with language processing? Such considerations drive us to take a more integrative view of language-as-communication.
Co-sign gesture
In sign language, face and head actions accompany manual signs and can be construed as “co-sign gestures” (Baker & Padden Reference Baker, Padden and Siple1978). Are such co-sign gestures readily dissociated cortically from sign-as-language? The question has not yet been tackled directly, but we offer two insights from our own work. The first is a case-series study of three left-hemisphere-lesioned signers (LH-lesion) and three right-hemisphere-lesioned signers (RH-lesion), who were early or native users of British Sign Language (BSL). Their pattern of sparing and impairment led to a specific interpretation of the linguistic status of an utterance (Atkinson et al. Reference Atkinson, Campbell, Marshall, Thacker and Woll2004).
Is negation in sign syntactic?
In BSL, as in many other sign languages, headshake can indicate negation of a manually expressed statement, with no accompanying manual marker of negation. We hypothesised that if this is syntactically processed, a headshake accompanying a manual expression should not be interpreted accurately in the LH-lesioned patients, in line with their other linguistic deficits. By contrast, if negation is managed as a prosodic feature, then accuracy may not be predicted by site of lesion, or RH lesions may adversely affect the perception of headshake negation.
In contrast to their many other language perception deficits, LH-lesioned patients were accurate at distinguishing video clips of signed negative (marked with headshake-only) utterances. RH-lesioned patients were unable to identify headshake-only negation utterances – although all of their other (manual) sign-language-processing skills were unimpaired. These findings suggest that headshake negation is not processed syntactically – at the very least, that it is not readily accommodated by left hemisphere language processing hitched to a damaged right hemisphere. In this way, neuropsychological investigations may constrain and help develop conceptualisations of those processes that may be construed as “core linguistic” (which, nevertheless, make use of some gestures) and others that may function in rather different ways, with greater reliance on right hemisphere processes.
Mouth actions can be more or less linguistic in sign language
Using functional magnetic resonance imaging (fMRI), we investigated facial actions accompanying the presentation of lexical manual signs in native deaf users of BSL (Capek et al. Reference Capek, Waters, Woll, MacSweeney, Brammer, McGuire, David and Campbell2008). One type of facial action, where mouth actions disambiguate sign homonyms (in BSL, the signs UNCLE and BATTERY have the same manual expression but different mouth actions), has “lexical” import. This type of display generated activation in the same LH-dominant perisylvian regions as manual signs, which had no accompanying face action. However, a different type of face action was also investigated. Here, the facial action mimics the dynamics of the manual action (“echo-phonology”). For example, the BSL sign SHUT is a manual gesture of the hands coming together, accompanied by mouth closure. This may not be obligatory for lexical interpretation. In contrast to the “disambiguating mouth” signs, the echo-phonology signs showed less activation in LH perisylvian regions and more activation, bilaterally, in posterior and inferior regions associated with the analysis of visual, nonlinguistic signals (Capek et al. Reference Capek, Waters, Woll, MacSweeney, Brammer, McGuire, David and Campbell2008).
While these findings can be interpreted in other ways (Capek et al. Reference Capek, Grossi, Newman, McBurney, Corina, Roeder and Neville2009, Reference Capek, Woll, MacSweeney, Waters, McGuire, David and Campbell2010), they suggest that conceptual processes underlying linguistic processing can be expressed and interpreted through means other than speech/manual sign and that these processes leave their mark in the brain of the language user. Questions remain about how gestural elements interact with linguistic elements within the manual component of sign language (see Cormier et al. Reference Cormier, Schembri and Woll2013 on pointing gestures in BSL). It will be interesting to explore gesture more systematically in relation to sign language using brain imaging techniques – alongside linguistic and cognitive explorations, which, as the target article shows, now offer powerful models of cognitive and linguistic function.