Goldin-Meadow & Brentari (G-M&B) have taken us on a tour of the highlights of the history of the study of sign language and gesture, concluding that sign languages are full-fledged languages like spoken ones, and that like speakers, signers gesture when they sign. A full understanding of face-to-face language, whether spoken or signed must consider gesture as part of the meaning-making package. I couldn't agree more.
It is tempting to stop here – what can be added to such a thoughtful and convincing analysis? But like any good analysis, this one keeps you thinking. Just as in real space, I took detours. For example, the discussion of whether sign is pantomime took me to the development of written languages (Gelb Reference Gelb1963). The alphabet, mapping sounds to marks, was invented only once, but written languages that mapped meaning to marks were invented many times. Reminiscent of sign, the forms began as crude depictions but were gradually schematized so that it became difficult to recognize their depictive origins, and there is little similarity across languages. Similarly, depictive schematizations were never sufficient to express all meanings.
Another place that set me wondering and wandering was a central claim of the target article: “But speech and gesture convey meaning differently – whereas speech uses primarily categorical devices, gesture relies on devices that are primarily imagistic and analog. Unlike spoken sentences in which lower constituents combine into higher constituents, each gesture is a complete holistic expression of meaning unto itself (McNeill 1992)” (sect. 5, para. 3).
Points are undoubtedly the simplest of gestures, accomplished in many ways: a finger, a hand, a nod of the head. Perhaps these are synonyms. The familiar deictic point mastered early by babies refers to something in the world. Those points seem to stand alone as “complete holistic expression[s] of meaning”; perhaps it's easier and more natural to point than to say. Yet other points “combine into higher constituents.” As people study descriptions of environments (Jamalian et al. Reference Jamalian, Giardino, Tversky, Knauff, Pauen, Sabenz and Wachsmuth2013) or describe environments to others (Emmorey et al. Reference Emmorey, Tversky and Taylor2000), they point successively not to things in the world but to the imagined locations of places integrated in an imaginary space. Landmarks of any kind receive the same undifferentiated point. Similarly, when solving problems alone in a room, people use points to represent the structure of the problem: for example, an array of glasses (Tversky & Kessell Reference Tversky and Kessell2014). Points can be used to indicate contrasting arguments – on the one hand, on the other – or successive events of a story, both integrated at a higher level. These examples challenge the claim that gestures are “holistic … unto itself” that do not “combine into higher constituents” as well as the claim that gestures are imagistic or analog.
Similar issues arise for line-like gestures, gestures used to interrelate points, the relationship between two contrasting arguments, the events of a story, or the paths between landmarks. Like lines in sketch maps, they could be analog but are not (Tversky & Lee Reference Tversky, Lee, Freksa and Mark1999). Rather, gestured lines seem to fall into approximate categories, parallel to the ways language coarsely categorizes quantitative concepts: some, few, many, or close, near, far (Talmy Reference Talmy, Pick and Acredolo1983).
Such gestures are not “imagistic” or “analog.” The same point-like gestures are used to individuate and arrange a broad range of entities, without differentiating the entities. These points are like the points used in counting. What is being counted makes no difference; all that counts is their quantity. The same line-like gesture is used to indicate a path in the world, a timeline of events, a relationship between arguments. There is no semblance of resemblance as in iconic gestures, nor are the meanings metaphoric. Rather than being “imagistic” or “analog,” these point-like and line-like gestures are diagrammatic (Tversky Reference Tversky2011; Tversky et al. Reference Tversky, Heiser, Lee, Daniel, Coventry, Tenbrink and Bateman2009) or, in Talmy's term, schematic (Talmy Reference Talmy, Pick and Acredolo1983). They represent entities that can be conceived of as points or lines (e.g., Talmy Reference Talmy, Pick and Acredolo1983). They are not metaphoric as they are devoid of content; instead, the points and lines and a few other forms common in gesture and graphics are abstractions, a point or dot for a place or an idea, a line for a relationship between places or ideas, a blob to contain a set of them (e.g., Tversky Reference Tversky2011; Tversky et al. Reference Tversky, Heiser, Lee, Daniel, Coventry, Tenbrink and Bateman2009). Such abstract gestures are ambiguous, similar to the analogous words point, line, and set; for both, context clarifies.
As noted, speech can take on characteristics of gesture. Soooo can be elongated just as very's can be concatenated to indicate intensity on any dimension. Sooner or later, someone will produce a video titled “50 Shades of No.” If many gestures are not imagistic or analog and can form integrated constituents of higher-level structures, if speech can be overlaid with sounds that act like gesture and sign with gesture, then how can gesture be distinguished from speech? Even common categories are slippery: what's a slipper, what's a sandal, what's a shoe? They slide into each other. The arts stretch our categories further. There is conceptual art, which is typically only words, and visual poetry, which often lacks them. Wittgenstein (Reference Wittgenstein1953), and Rosch (Reference Rosch, Rosch and Lloyd1978) on his heels, observed that common categories don't appear to have defining features that cleanly assign instances into categories; rather, common categories have a family resemblance structure, a set of features that members are more likely to share.
Categories: We can't live with them, we can't live without them.
These observations blur some of the distinctions G-M&B meant to sharpen in their thoughtful and thought-provoking analysis. Yet their primary intent was to show some of the subtleties and varieties of meaning-making. Everyday communication is a harmonious blend of words, spoken or signed; gestures, prosody, and more creating rich, textured, and innovative meanings. Thinking about what does what and how, and how they combine is endlessly fascinating.
Goldin-Meadow & Brentari (G-M&B) have taken us on a tour of the highlights of the history of the study of sign language and gesture, concluding that sign languages are full-fledged languages like spoken ones, and that like speakers, signers gesture when they sign. A full understanding of face-to-face language, whether spoken or signed must consider gesture as part of the meaning-making package. I couldn't agree more.
It is tempting to stop here – what can be added to such a thoughtful and convincing analysis? But like any good analysis, this one keeps you thinking. Just as in real space, I took detours. For example, the discussion of whether sign is pantomime took me to the development of written languages (Gelb Reference Gelb1963). The alphabet, mapping sounds to marks, was invented only once, but written languages that mapped meaning to marks were invented many times. Reminiscent of sign, the forms began as crude depictions but were gradually schematized so that it became difficult to recognize their depictive origins, and there is little similarity across languages. Similarly, depictive schematizations were never sufficient to express all meanings.
Another place that set me wondering and wandering was a central claim of the target article: “But speech and gesture convey meaning differently – whereas speech uses primarily categorical devices, gesture relies on devices that are primarily imagistic and analog. Unlike spoken sentences in which lower constituents combine into higher constituents, each gesture is a complete holistic expression of meaning unto itself (McNeill 1992)” (sect. 5, para. 3).
Points are undoubtedly the simplest of gestures, accomplished in many ways: a finger, a hand, a nod of the head. Perhaps these are synonyms. The familiar deictic point mastered early by babies refers to something in the world. Those points seem to stand alone as “complete holistic expression[s] of meaning”; perhaps it's easier and more natural to point than to say. Yet other points “combine into higher constituents.” As people study descriptions of environments (Jamalian et al. Reference Jamalian, Giardino, Tversky, Knauff, Pauen, Sabenz and Wachsmuth2013) or describe environments to others (Emmorey et al. Reference Emmorey, Tversky and Taylor2000), they point successively not to things in the world but to the imagined locations of places integrated in an imaginary space. Landmarks of any kind receive the same undifferentiated point. Similarly, when solving problems alone in a room, people use points to represent the structure of the problem: for example, an array of glasses (Tversky & Kessell Reference Tversky and Kessell2014). Points can be used to indicate contrasting arguments – on the one hand, on the other – or successive events of a story, both integrated at a higher level. These examples challenge the claim that gestures are “holistic … unto itself” that do not “combine into higher constituents” as well as the claim that gestures are imagistic or analog.
Similar issues arise for line-like gestures, gestures used to interrelate points, the relationship between two contrasting arguments, the events of a story, or the paths between landmarks. Like lines in sketch maps, they could be analog but are not (Tversky & Lee Reference Tversky, Lee, Freksa and Mark1999). Rather, gestured lines seem to fall into approximate categories, parallel to the ways language coarsely categorizes quantitative concepts: some, few, many, or close, near, far (Talmy Reference Talmy, Pick and Acredolo1983).
Such gestures are not “imagistic” or “analog.” The same point-like gestures are used to individuate and arrange a broad range of entities, without differentiating the entities. These points are like the points used in counting. What is being counted makes no difference; all that counts is their quantity. The same line-like gesture is used to indicate a path in the world, a timeline of events, a relationship between arguments. There is no semblance of resemblance as in iconic gestures, nor are the meanings metaphoric. Rather than being “imagistic” or “analog,” these point-like and line-like gestures are diagrammatic (Tversky Reference Tversky2011; Tversky et al. Reference Tversky, Heiser, Lee, Daniel, Coventry, Tenbrink and Bateman2009) or, in Talmy's term, schematic (Talmy Reference Talmy, Pick and Acredolo1983). They represent entities that can be conceived of as points or lines (e.g., Talmy Reference Talmy, Pick and Acredolo1983). They are not metaphoric as they are devoid of content; instead, the points and lines and a few other forms common in gesture and graphics are abstractions, a point or dot for a place or an idea, a line for a relationship between places or ideas, a blob to contain a set of them (e.g., Tversky Reference Tversky2011; Tversky et al. Reference Tversky, Heiser, Lee, Daniel, Coventry, Tenbrink and Bateman2009). Such abstract gestures are ambiguous, similar to the analogous words point, line, and set; for both, context clarifies.
As noted, speech can take on characteristics of gesture. Soooo can be elongated just as very's can be concatenated to indicate intensity on any dimension. Sooner or later, someone will produce a video titled “50 Shades of No.” If many gestures are not imagistic or analog and can form integrated constituents of higher-level structures, if speech can be overlaid with sounds that act like gesture and sign with gesture, then how can gesture be distinguished from speech? Even common categories are slippery: what's a slipper, what's a sandal, what's a shoe? They slide into each other. The arts stretch our categories further. There is conceptual art, which is typically only words, and visual poetry, which often lacks them. Wittgenstein (Reference Wittgenstein1953), and Rosch (Reference Rosch, Rosch and Lloyd1978) on his heels, observed that common categories don't appear to have defining features that cleanly assign instances into categories; rather, common categories have a family resemblance structure, a set of features that members are more likely to share.
Categories: We can't live with them, we can't live without them.
These observations blur some of the distinctions G-M&B meant to sharpen in their thoughtful and thought-provoking analysis. Yet their primary intent was to show some of the subtleties and varieties of meaning-making. Everyday communication is a harmonious blend of words, spoken or signed; gestures, prosody, and more creating rich, textured, and innovative meanings. Thinking about what does what and how, and how they combine is endlessly fascinating.