We agree that gesture and sign, like gesture and speech, form an integrated communication system. Communicative acts in both spoken and signed languages combine categorical linguistic (words or signs) with imagistic (gesture) components. Still, gestures contribute different types of information from signs or spoken words. Our analysis of communicative acts should respect this. Therefore, we should aim to tell apart categorical from imagistic components.
In spoken language, the distinction between them is rather straightforward as two different modalities are involved, forming two different channels. Linguistic components are conveyed in the auditory domain; imagistic ones are conveyed visuo-spatially. For communicative acts in sign language, however, the distinction is not so straightforward. Gesture and sign occur in the same modality such that linguistic and imagistic components get blended together. However, a comparative analysis of signed and spoken communication will only get off the ground once we can disentangle linguistic and gestural components in this single visuo-spatial channel.
To achieve this, we propose including a third category of components in the analysis of communicative acts: modifying components. Rather than independent gestures, modifying components are imagistic modifications of linguistic components. Like gestures, they can convey additional information iconically. Unlike gestures, however, they are not independent imagistic units. They are bound to linguistic components exploiting their available degrees of freedom to add meaning without changing the core categorical meaning altogether. As a simple illustration in spoken language, consider extending the length of a vowel (e.g., l-o-o-o-o-ong) to emphasize duration of an event (Okrent Reference Okrent, Meier, Quinto-Pozos and Cormier2002). While this modification is merely amplifying the meaning of the linguistic component “long,” we can also think of modifications adding information. Depending on how a speaker modulates voice and intonations, the utterance “Sarah be-e-e-e-e-ent the metal” conveys not only that Sarah bent the metal, but also that it took her a lot of effort to do so.
Analogously, signers can modify, for example, the handshape, movement, or location of signs while retaining categorical linguistic morphemes (as Duncan Reference Duncan2005 discussed in the context of signers’ descriptions of Tweety Bird cartoons). This way, essential categorical information is preserved while iconic information is added through spontaneous and idiosyncratic modifications. Signers intuitively know the degrees of freedom they can exploit in their variations of signs as speakers know what the permissible variations of words are in the preceding examples. The modified signs embody both lexical and imagistic features; they form units that have both linguistic (lexical) and imagistic (gestural) aspects. Importantly, here the gestural aspects are not separable from the linguistic ones; they are bound to their linguistic base morphemes.
Not all variations of signs are spontaneous and idiosyncratic, though. Just as in spoken language, there are systematic differences between dialects, accents, and so on. For current purposes, however, we are not concerned with these. Rather, we are interested in how speakers purposefully modify individual signs imagistically. There are cases where variations in how a sign is performed lead to lexicalized changes in meaning while the basic linguistic morpheme remains unchanged. Consider the verb “to email” in British Sign Language. Here, the agent-patient relation is determined by the orientation of the sign: If the index finger of the signer's dominant hand is flicking at you and the index finger of the nondominant hand is flicking at her, she signs “I e-mail you.” If she performs the sign the other way around (with the dominant finger flicking at her), she signs “You e-mail me.” These, too, are modifying components, but they have become conventionalized; they are no longer idiosyncratic.
In spoken language, we find modifying components not only in vocal gestures like extending a vowel; there are also cross-modal modifications. For illustration, consider demonstratives accompanied by pointing gestures. Suppose we are looking at a set of three keys on a table. We will usually expect an instruction like “Take this key!” to be accompanied by some kind of pointing, handing over a key, nodding at a key, and so on. Whatever we do to indicate which of the keys we refer to, it is a modification of “this” which forms a single, cross-modal unit. Again, the gestural and lexical aspects may be easier to dissociate in spoken than signed language, but the phenomenon is the same.
A critic may object that we should simply consider the pointing at a key as a gestural component. But this would be to ignore that the demonstrative “this” is lexically required to be accompanied by some kind of indication (though the exact format can vary significantly). Besides, such a move would render gestural components much too inclusive. In principle, gestures also convey information independently of the linguistic components they accompany. For the pointing in the scenario at issue, this is not the case. Pointing here only gets its meaning through combination with the spoken demonstrative.
Modifying components may occur in different modalities in sign language, too. Take the mouth forms that Sandler (Reference Sandler2009) reported in signers describing Tweety Bird cartoons. These are, like the pointing that accompanies spoken demonstratives, imagistic aspects directly modifying linguistic components. Again, the reason to treat them as modifying components rather than imagistic components is that they add imagistic aspects to a lexical base morpheme rather than functioning as an additional imagistic component within a communicative act.
The crucial characteristic of modifying components is that they deliver imagistic information as part of a modified lexical unit. They can be lexically required to form meaningful units (as in the case of demonstratives) or they can be added spontaneously. They can take different forms, some of which may (have) become conventionalized, and they may even occur in different modalities from the base morphemes. But they always remain bound to a linguistic component.
By adopting modifying components as a third category, we are much better placed to disentangle linguistic and imagistic components of communicative acts. This allows us to better assess and systematically compare the role that gesture plays in both signed and spoken languages.
You have
Access
We agree that gesture and sign, like gesture and speech, form an integrated communication system. Communicative acts in both spoken and signed languages combine categorical linguistic (words or signs) with imagistic (gesture) components. Still, gestures contribute different types of information from signs or spoken words. Our analysis of communicative acts should respect this. Therefore, we should aim to tell apart categorical from imagistic components.
In spoken language, the distinction between them is rather straightforward as two different modalities are involved, forming two different channels. Linguistic components are conveyed in the auditory domain; imagistic ones are conveyed visuo-spatially. For communicative acts in sign language, however, the distinction is not so straightforward. Gesture and sign occur in the same modality such that linguistic and imagistic components get blended together. However, a comparative analysis of signed and spoken communication will only get off the ground once we can disentangle linguistic and gestural components in this single visuo-spatial channel.
To achieve this, we propose including a third category of components in the analysis of communicative acts: modifying components. Rather than independent gestures, modifying components are imagistic modifications of linguistic components. Like gestures, they can convey additional information iconically. Unlike gestures, however, they are not independent imagistic units. They are bound to linguistic components exploiting their available degrees of freedom to add meaning without changing the core categorical meaning altogether. As a simple illustration in spoken language, consider extending the length of a vowel (e.g., l-o-o-o-o-ong) to emphasize duration of an event (Okrent Reference Okrent, Meier, Quinto-Pozos and Cormier2002). While this modification is merely amplifying the meaning of the linguistic component “long,” we can also think of modifications adding information. Depending on how a speaker modulates voice and intonations, the utterance “Sarah be-e-e-e-e-ent the metal” conveys not only that Sarah bent the metal, but also that it took her a lot of effort to do so.
Analogously, signers can modify, for example, the handshape, movement, or location of signs while retaining categorical linguistic morphemes (as Duncan Reference Duncan2005 discussed in the context of signers’ descriptions of Tweety Bird cartoons). This way, essential categorical information is preserved while iconic information is added through spontaneous and idiosyncratic modifications. Signers intuitively know the degrees of freedom they can exploit in their variations of signs as speakers know what the permissible variations of words are in the preceding examples. The modified signs embody both lexical and imagistic features; they form units that have both linguistic (lexical) and imagistic (gestural) aspects. Importantly, here the gestural aspects are not separable from the linguistic ones; they are bound to their linguistic base morphemes.
Not all variations of signs are spontaneous and idiosyncratic, though. Just as in spoken language, there are systematic differences between dialects, accents, and so on. For current purposes, however, we are not concerned with these. Rather, we are interested in how speakers purposefully modify individual signs imagistically. There are cases where variations in how a sign is performed lead to lexicalized changes in meaning while the basic linguistic morpheme remains unchanged. Consider the verb “to email” in British Sign Language. Here, the agent-patient relation is determined by the orientation of the sign: If the index finger of the signer's dominant hand is flicking at you and the index finger of the nondominant hand is flicking at her, she signs “I e-mail you.” If she performs the sign the other way around (with the dominant finger flicking at her), she signs “You e-mail me.” These, too, are modifying components, but they have become conventionalized; they are no longer idiosyncratic.
In spoken language, we find modifying components not only in vocal gestures like extending a vowel; there are also cross-modal modifications. For illustration, consider demonstratives accompanied by pointing gestures. Suppose we are looking at a set of three keys on a table. We will usually expect an instruction like “Take this key!” to be accompanied by some kind of pointing, handing over a key, nodding at a key, and so on. Whatever we do to indicate which of the keys we refer to, it is a modification of “this” which forms a single, cross-modal unit. Again, the gestural and lexical aspects may be easier to dissociate in spoken than signed language, but the phenomenon is the same.
A critic may object that we should simply consider the pointing at a key as a gestural component. But this would be to ignore that the demonstrative “this” is lexically required to be accompanied by some kind of indication (though the exact format can vary significantly). Besides, such a move would render gestural components much too inclusive. In principle, gestures also convey information independently of the linguistic components they accompany. For the pointing in the scenario at issue, this is not the case. Pointing here only gets its meaning through combination with the spoken demonstrative.
Modifying components may occur in different modalities in sign language, too. Take the mouth forms that Sandler (Reference Sandler2009) reported in signers describing Tweety Bird cartoons. These are, like the pointing that accompanies spoken demonstratives, imagistic aspects directly modifying linguistic components. Again, the reason to treat them as modifying components rather than imagistic components is that they add imagistic aspects to a lexical base morpheme rather than functioning as an additional imagistic component within a communicative act.
The crucial characteristic of modifying components is that they deliver imagistic information as part of a modified lexical unit. They can be lexically required to form meaningful units (as in the case of demonstratives) or they can be added spontaneously. They can take different forms, some of which may (have) become conventionalized, and they may even occur in different modalities from the base morphemes. But they always remain bound to a linguistic component.
By adopting modifying components as a third category, we are much better placed to disentangle linguistic and imagistic components of communicative acts. This allows us to better assess and systematically compare the role that gesture plays in both signed and spoken languages.