The integrative machinery that language contributes to human cognition in either the gestural-visual modality (sign) or in the mainly vocal-auditory modality (speech) is more inclusive than Goldin-Meadow & Brentari (G-M&B) propose. We suggest that language includes components that are (1) discrete and categorical, (2) imagistic, (3) topographic (location) and gradient (movement), and, finally, (4) deictic. G-M&B, on the other hand, do not single out location(-movement) (3) or deixis (4), but include them under the co-sign imagistic gesture component (2), which in turn is denied a true linguistic status. We suggest, to the contrary, that all four components are inherently linguistic and that location(-movement) (3) and deixis (4), in contrast to co-speech/sign (2), are used for purposes of grammatical reference – with location(-movement) (3) contributing to supralexical iconicity, a hallmark of sign languages. Finally, gestural deixis (4) and co-speech/sign gesture (2) are common to sign and speech, with both having concurrent oral counterparts in the latter (pronouns, demonstratives, etc., and prosody, respectively). In their gestural versions, these two components are then a natural nexus between both modalities although it is deixis (4) that is inherently integrated with grammatical development.
In the view of long-standing debates on the divide between sign and gesture and on the degree of similarity between linguistic modalities, G-M&B attempt to satisfy both the side that gives primacy to gesture in the signed modality and the side contending that sign “is just like spoken language.” They propose that the comparison between modalities “needs to include both the more categorical (sign or speech) and the more imagistic (gestural) components regardless of modality” (para. 5) – that is, our discrete and categorical (1), imagistic (2), components mentioned above. However, G-M&B nonetheless wish to keep them distinct, as mismatches between these components observed in the learning of hearing and deaf children could otherwise not be seen. Still, they would be part of a “single gesture-speech system” (sect. 5, para. 15) at least in the spoken modality (Alibali & Goldin-Meadow Reference Alibali and Goldin-Meadow1993), rather than “two distinct sets of representations”. However, what this single system amounts to is not clear. Is it language, as said occasionally (para. 5; sect. 8, para. 2); is it communication with language as a subset (section 8, para. 1) or is it communication taken as coextensive with language (section 8, para. 2)? Neither is it clear whether the distinctness and integration of speech and co-speech gesture extends to sign and co-sign gesture (in this regard, the interrogative title of sect. 6 contrasts with the assertion made in sect. 5). In fact, the idea of an overall linguistic system embracing sign and co-sign gesture is already questioned in section 4.2, where G-M&B restrict language to sign in the signed modality. Signs are linguistic as they are “discrete” and “categorical”; gestures, instead, are imagistic. But what about indicating gestures, whether incorporated into lexical roots as in agreement verbs or independent, as in deictic gestures?
G-M&B, citing Liddell (2003), take these as gestural. However, indicating gestures could not technically qualify as gestures by G-M&B's definition, being discrete and categorical. Only through a distortion of what “discrete” means, namely by wrongly equating discrete with “finite or listable” (sect. 4.2, para. 2), can G-M&B conclude that indicating gestures, being nonlistable, are nonlinguistic. With regard to “categorical,” G-M&B go for a relaxation of the proposed divide by accepting “categorical gestures” that would almost reach the status of sign (“spontaneous sign” sect. 4.3, para. 2) – the case of homesigners and silent gesturers.
Our alternative would avoid these problems while maintaining G-M&B's insights. Indicating gestures, (4), are part and parcel of language irrespective of modality. Goldin-Meadow et al. (Cartmill et al. Reference Cartmill, Hunsicker and Goldin-Meadow2014) have shown that the individual onset of different deictic gestures, (4), combined with first words (e.g., dog or eat), (1), predicts the individual onset of correlating phrases (determiner phrases, e.g., the dog, and sentences, respectively) in speech (Goldin-Meadow & Morford Reference Goldin-Meadow and Morford1985; Mattos & Hinzen Reference Mattos and Hinzen2015; Özçalışkan & Goldin-Meadow Reference Özçalışkan and Goldin-Meadow2009).Therefore, not only mismatches between imagistic gestures (2) and speech are able to make predictions about learning in different domains but also right combinations of speech and both imagistic (2), and deictic (4), gestures make predictions about learning in the domain of language. Furthermore, these predictions about linguistic learning point to a single system integrating these different gestural formats, an integration that is not only toward depiction but also grammar. Problems with deixis in both gestures and spoken words in autism suggest that language, and grammar itself, is multiformat – and bimodal (sound and gesture) in speech.
As for the topographic component (3), should we really wait for a “motiongraph” that does for sign what the spectrograph has done for speech? Actually, the spectrograph showed that categorically perceived segments were not acoustically categorical, whereas G-M&B speculate in the opposite direction, namely on uncovering potentially hidden categorical components in the continuous perception of movement. Following Klima & Bellugi (Reference Klima and Bellugi1979), we contend that “the structured use of space and movement” which goes along with classifiers is a hallmark of sign languages. Neither its absence in speech nor the fact that right-hemisphere processing is involved only when classifiers are used to convey information on location(-motion) suggest its nonlinguistic nature (Emmorey Reference Emmorey and Reilly2013). Handshape is categorical, mainly processed by the left hemisphere, and then linguistic. Classifiers have a handshape, but they go with nonlistable locations (and continuous movement). Are they then linguistic and nonlinguistic at once? This hardly makes sense.
The right hemisphere's normal contribution (suprasegmentals, deixis [Mizuno et al. Reference Mizuno, Liu, Williams, Keller, Minshew and Just2011], space/movement in sign) is contrastively illuminated when language malfunctions. Deaf schizophrenic patients natively signing in British Sign Language have difficulties in depicting dynamic or static locations with classifiers. They make many more handshape errors than location or movement errors, while being free of errors in lexical (without classifiers) descriptions of the same locative relationships (static, in this case) (Chatzidamianos Reference Chatzidamianos2013). This suggests that a well-functioning language and cognition obtains when the four components we distinguish interact correctly, and across hemispheres.
In sum, taking the four components at face value is less problematic than thinking of language as exhausted by sign/speech and co-sign/speech gesture. Conceptual inconsistencies, exceptions regarding deixis, and unlikely hopes of a hidden categoricity for the location(-motion) component would all vanish, and we could focus on how the four representational formats unite to yield human language.
The integrative machinery that language contributes to human cognition in either the gestural-visual modality (sign) or in the mainly vocal-auditory modality (speech) is more inclusive than Goldin-Meadow & Brentari (G-M&B) propose. We suggest that language includes components that are (1) discrete and categorical, (2) imagistic, (3) topographic (location) and gradient (movement), and, finally, (4) deictic. G-M&B, on the other hand, do not single out location(-movement) (3) or deixis (4), but include them under the co-sign imagistic gesture component (2), which in turn is denied a true linguistic status. We suggest, to the contrary, that all four components are inherently linguistic and that location(-movement) (3) and deixis (4), in contrast to co-speech/sign (2), are used for purposes of grammatical reference – with location(-movement) (3) contributing to supralexical iconicity, a hallmark of sign languages. Finally, gestural deixis (4) and co-speech/sign gesture (2) are common to sign and speech, with both having concurrent oral counterparts in the latter (pronouns, demonstratives, etc., and prosody, respectively). In their gestural versions, these two components are then a natural nexus between both modalities although it is deixis (4) that is inherently integrated with grammatical development.
In the view of long-standing debates on the divide between sign and gesture and on the degree of similarity between linguistic modalities, G-M&B attempt to satisfy both the side that gives primacy to gesture in the signed modality and the side contending that sign “is just like spoken language.” They propose that the comparison between modalities “needs to include both the more categorical (sign or speech) and the more imagistic (gestural) components regardless of modality” (para. 5) – that is, our discrete and categorical (1), imagistic (2), components mentioned above. However, G-M&B nonetheless wish to keep them distinct, as mismatches between these components observed in the learning of hearing and deaf children could otherwise not be seen. Still, they would be part of a “single gesture-speech system” (sect. 5, para. 15) at least in the spoken modality (Alibali & Goldin-Meadow Reference Alibali and Goldin-Meadow1993), rather than “two distinct sets of representations”. However, what this single system amounts to is not clear. Is it language, as said occasionally (para. 5; sect. 8, para. 2); is it communication with language as a subset (section 8, para. 1) or is it communication taken as coextensive with language (section 8, para. 2)? Neither is it clear whether the distinctness and integration of speech and co-speech gesture extends to sign and co-sign gesture (in this regard, the interrogative title of sect. 6 contrasts with the assertion made in sect. 5). In fact, the idea of an overall linguistic system embracing sign and co-sign gesture is already questioned in section 4.2, where G-M&B restrict language to sign in the signed modality. Signs are linguistic as they are “discrete” and “categorical”; gestures, instead, are imagistic. But what about indicating gestures, whether incorporated into lexical roots as in agreement verbs or independent, as in deictic gestures?
G-M&B, citing Liddell (2003), take these as gestural. However, indicating gestures could not technically qualify as gestures by G-M&B's definition, being discrete and categorical. Only through a distortion of what “discrete” means, namely by wrongly equating discrete with “finite or listable” (sect. 4.2, para. 2), can G-M&B conclude that indicating gestures, being nonlistable, are nonlinguistic. With regard to “categorical,” G-M&B go for a relaxation of the proposed divide by accepting “categorical gestures” that would almost reach the status of sign (“spontaneous sign” sect. 4.3, para. 2) – the case of homesigners and silent gesturers.
Our alternative would avoid these problems while maintaining G-M&B's insights. Indicating gestures, (4), are part and parcel of language irrespective of modality. Goldin-Meadow et al. (Cartmill et al. Reference Cartmill, Hunsicker and Goldin-Meadow2014) have shown that the individual onset of different deictic gestures, (4), combined with first words (e.g., dog or eat), (1), predicts the individual onset of correlating phrases (determiner phrases, e.g., the dog, and sentences, respectively) in speech (Goldin-Meadow & Morford Reference Goldin-Meadow and Morford1985; Mattos & Hinzen Reference Mattos and Hinzen2015; Özçalışkan & Goldin-Meadow Reference Özçalışkan and Goldin-Meadow2009).Therefore, not only mismatches between imagistic gestures (2) and speech are able to make predictions about learning in different domains but also right combinations of speech and both imagistic (2), and deictic (4), gestures make predictions about learning in the domain of language. Furthermore, these predictions about linguistic learning point to a single system integrating these different gestural formats, an integration that is not only toward depiction but also grammar. Problems with deixis in both gestures and spoken words in autism suggest that language, and grammar itself, is multiformat – and bimodal (sound and gesture) in speech.
As for the topographic component (3), should we really wait for a “motiongraph” that does for sign what the spectrograph has done for speech? Actually, the spectrograph showed that categorically perceived segments were not acoustically categorical, whereas G-M&B speculate in the opposite direction, namely on uncovering potentially hidden categorical components in the continuous perception of movement. Following Klima & Bellugi (Reference Klima and Bellugi1979), we contend that “the structured use of space and movement” which goes along with classifiers is a hallmark of sign languages. Neither its absence in speech nor the fact that right-hemisphere processing is involved only when classifiers are used to convey information on location(-motion) suggest its nonlinguistic nature (Emmorey Reference Emmorey and Reilly2013). Handshape is categorical, mainly processed by the left hemisphere, and then linguistic. Classifiers have a handshape, but they go with nonlistable locations (and continuous movement). Are they then linguistic and nonlinguistic at once? This hardly makes sense.
The right hemisphere's normal contribution (suprasegmentals, deixis [Mizuno et al. Reference Mizuno, Liu, Williams, Keller, Minshew and Just2011], space/movement in sign) is contrastively illuminated when language malfunctions. Deaf schizophrenic patients natively signing in British Sign Language have difficulties in depicting dynamic or static locations with classifiers. They make many more handshape errors than location or movement errors, while being free of errors in lexical (without classifiers) descriptions of the same locative relationships (static, in this case) (Chatzidamianos Reference Chatzidamianos2013). This suggests that a well-functioning language and cognition obtains when the four components we distinguish interact correctly, and across hemispheres.
In sum, taking the four components at face value is less problematic than thinking of language as exhausted by sign/speech and co-sign/speech gesture. Conceptual inconsistencies, exceptions regarding deixis, and unlikely hopes of a hidden categoricity for the location(-motion) component would all vanish, and we could focus on how the four representational formats unite to yield human language.