INTRODUCTION
At an early age, children use gesture and speech together to convey meanings that they cannot yet express in speech (Greenfield & Smith, Reference Greenfield and Smith1976; Goldin-Meadow & Butcher, Reference Goldin-Meadow, Butcher and Kita2003; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005a). They, for example, point at a cookie while describing the action to be performed on the cookie (e.g., ‘eat’+point at cookie), or point at a book while naming the person to act on the book (e.g., ‘mommy’+point at book). In these early combinations the object (e.g., cookie, book) is almost always indicated by a deictic gesture (Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005a). In contrast, combinations in which object information is conveyed by an iconic gesture appear much later, typically soon after children begin to show steady increases in their iconic gesture production – roughly around three years of age (Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2009, Reference Özçalışkan, Goldin-Meadow, Stam and Ishino2011). In this study we ask whether children understand the meaning of gesture+speech combinations – in which an iconic gesture adds object information to the accompanying speech – during the period when they begin to produce similar gesture+speech combinations with iconic gestures in their everyday interactions.
Emergence of iconicity in the child's developing gesture+speech system
Gesture and speech form an integrated system both semantically and temporally in adult speakers; 90% of the gestures produced by adults accompany speech and convey distinct but related information (McNeill, Reference McNeill1992; Reference McNeill2005). In fact, previous work with adult speakers suggests bi-directional interactions between gesture and speech in both comprehension and production, with speech influencing the processing of the accompanying gesture and gesture influencing the processing of the accompanying speech (Kita & Özyürek, Reference Kita and Özyürek2003; Kelly, Özyürek & Maris, Reference Kelly, Özyürek and Maris2010). That is, adult speakers routinely use gesture, particularly iconic gestures, to further extend or to clarify what they convey in speech; similarly they also rely on gesture and speech together to interpret communicative acts directed at them.
Not surprisingly, children learn the gesture+speech system at an early age as well (Goldin-Meadow, Reference Goldin-Meadow2003). Soon after they begin to produce their first words around the age of one, children use gesture along with speech to convey more complex meanings that they cannot yet express by using speech alone (Goldin-Meadow & Butcher, Reference Goldin-Meadow, Butcher and Kita2003; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005a). During this period, deictic gestures, such as pointing at an object to indicate or request that object, constitute the most common gesture type, accounting for roughly 80% of children's overall gesture production (e.g., Iverson, Capirci & Caselli, Reference Iverson, Capirci and Caselli1994; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005a). Thus, between ages one and three, children routinely combine a pointing gesture with a spoken word to convey simple sentence-like meanings (‘drink’+point at milk, ‘baby’+point at milk), and these early gesture+speech combinations precede and predict the emergence of similar sentence-like meanings in their speech (Iverson & Goldin-Meadow, Reference Iverson and Goldin-Meadow2005; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005b, Reference Özçalışkan and Goldin-Meadow2010; see also Greenfield, Lyn & Savage-Rumbaugh, Reference Greenfield, Lyn and Savage-Rumbaugh2008 for a similar report on frequent use of deictic gesture+lexigram combinations in captive bonobos and chimps, our most closely related living primate relatives).
In contrast to pointing gestures, the incidence of spontaneous iconic gestures in young children's communicative repertoire is quite rare, typically accounting for less than 5% of the gestures that young children produce (Iverson, Capirci & Caselli, Reference Iverson, Capirci and Caselli1994; Nicoladis, Mayberry & Genesee, Reference Nicoladis, Mayberry and Genesee1999; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005b, Reference Özçalışkan and Goldin-Meadow2009). The initial significant increase in children's iconic gesture production comes in around age 2;2 (Özçalışkan & Goldin-Meadow, Reference Özçalışkan, Goldin-Meadow, Stam and Ishino2011; Özçalışkan, Gentner & Goldin-Meadow, Reference Özçalışkan, Gentner and Goldin-Meadow2013). This spurt in iconic gesture production is followed by increased use of gesture+speech combinations in which the iconic gesture conveys additional meanings not conveyed in speech (e.g., ‘Me see it’+make a V-shape with index and middle fingers to convey RABBIT; ‘Talk like this’+place fisted hand next to ear to convey TELEPHONE; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2009). But do we observe a similar increase in children's comprehension of gesture+speech combinations with iconic gestures around age three as well?
Previous research on children's comprehension of gesture+speech combinations, mostly with deictic gestures, suggests early comprehension. When presented with a gesture+speech combination involving either a deictic (‘push’+point at ball) or a conventional gesture (‘clock’+GIVE gesture) in a direct requesting context, children aged 1;3 to 2;4 were able to understand the meaning of the gesture+speech combination and offer the requested object to the experimenter (Morford & Goldin-Meadow, Reference Morford and Goldin-Meadow1992). Three- to five-year-old children could even grasp the meaning of an indirect request conveyed in a gesture+speech combination, also with a pointing gesture. For example, upon hearing an adult say, ‘It is going to get loud in here’ along with a point at the door, children would get up and close the door (Kelly, Reference Kelly2001). Most of this earlier work focused on children's understanding of gesture+speech combinations in which the gesture was either a pointing gesture indicating a concrete referent or a conventional gesture conveying a culturally agreed upon emblematic meaning, thus leaving comprehension of gesture+speech combinations involving iconic gestures relatively unexplored.
We know from previous research that even though children understand the referent of a deictic gesture as early as age 1;0 (Butterworth & Grover, Reference Butterworth, Grover and Weiskrantz1988), understanding the iconicity of a gesture is a relatively late achievement, beginning in earnest around age 2;2 (Namy & Waxman, Reference Namy and Waxman1998; Namy, Reference Namy2001; Namy, Campbell & Tomasello, Reference Namy, Campbell and Tomasello2004). This late-emerging sensitivity to iconicity might in turn lead to later comprehension of gesture+speech combinations with iconic gestures that convey additional information about an object not found in the accompanying speech. In this study, we explored this possibility by studying two-, three-, and four-year-old children as they participated in a gesture+speech comprehension task depicting familiar everyday objects with iconic gestures and words. In line with previous work that shows late-emerging sensitivity to iconicity, we expected children to show understanding of gesture+speech combinations with iconic gestures at a later age (∼three), roughly around the same time they have been shown to use such iconic gestures to produce gesture+speech combinations conveying sentence-like meanings.
METHODS
Participants
Thirty-six children participated in the study, including 12 two-year-olds (M = 2;2, range = 2;0–2;4), 12 three-year-olds (M = 3;5 months, range = 3;1–3;8), and 12 four-year-olds (M = 4;4 months, range = 3;11–4;8), with equal numbers of boys and girls in each age group. The children came from middle- to upper-middle-class families and were predominantly Caucasian (79%) or African American (19%). Data from two additional children were excluded due to experimental error (N = 1) or parent prompting (N = 1) during data collection.
Procedure for data collection
Each child was tested individually in our laboratory. The experimenter presented the child with six gesture+speech combinations, one at a time. Each combination was composed of an iconic gesture conveying an action characteristic of an object along with a verbal description (e.g., ‘I am eating’+move cupped palms towards mouth in parallel as if eating a sandwich). After presenting each gesture+speech combination twice, the experimenter placed a pair of pictures depicting two different objects (e.g., a bowl of cereal vs. a sandwich) on the table, each at an equal distance from the child, and asked the child to choose the picture that matched the description (e.g., ‘What did I eat?’). One picture always matched the object depicted in the iconic gesture. For each object pair, half of the children in each age group watched the iconic gesture that matched the action characteristic of the first object (e.g., bowl of cereal) and the other half observed the iconic gesture that matched the action characteristic of the second object (e.g., sandwich). The presentation order of the six picture pairs and the placement of the correct picture (right side vs. left side) were counterbalanced across children. All responses were videotaped.
The picture stimuli consisted of six pairs of laminated cards (11·5 cm×11·5 cm) depicting different objects. For each pair, the verbal description matched the action associated with both objects (e.g., ‘I am eating’). However, the iconic gesture matched only one of the objects in the pair (e.g., scooping movement with a closed fist to convey eating cereal vs. moving cupped hand towards mouth with fingers and thumb extended in a U-shape to convey eating sandwich). A detailed summary of the spoken and gestural descriptions used for each of the six picture pairs is provided in Table 1.
Verbal and gestural descriptions used for the stimulus objects pairs
To ensure that children chose correctly based on their comprehension of the gestures and not because of subtle prompting by the experimenter, we examined the question segment in 25% of the collected data (N = 54 test trials, 2 test trials per child). Five independent coders (M age = 22), blind to the child's age and research hypothesis, watched the experimenter's behaviors during the question segment in each of the fifty-four selected test trials (i.e., the segment where the experimenter asks ‘Which one did I eat?’ after placing two pictures in front of the child); the coders were then asked to choose the correct picture (e.g., sandwich vs. cereal) based on what they saw. None of the five coders chose the correct picture significantly above chance (M correct = 28 [SD = 1·48], M incorrect = 26 [SD = 1·48], t(4) ⩽·81, p = ·15), and the agreement on picture choice between coders was low across test trials (kappas ⩽·29), thus showing that children's responses during testing were not likely to be influenced by subtle prompting by the experimenter.
Procedure for data analysis
We assessed children's responses by identifying which picture the child first indicated, either by pointing, touching, or labeling, from the videos. Each response was scored as correct (1) or incorrect (0), with the total score ranging from 0 to 6 for each child across six test trials. We assessed differences from chance performance with independent t-tests, separately for each age group; we also computed differences between age groups by a one-way ANOVA with age as a between-subjects factor. The effect sizes were computed by using eta-squared (hereafter η2) for the ANOVA comparison and Cohen's d (hereafter d) for t-tests. Reliability on scoring was assessed by re-coding a randomly chosen 25% of the data by a second coder who was blind to the age group and the research hypothesis. Agreement between coders was 100%. Our preliminary analyses showed no significant effect of item, presentation order, or child's sex on the choice score; therefore we collapsed across these factors for subsequent analyses.
RESULTS
Our results showed that children's comprehension of object meanings conveyed through iconic gestures in a gesture+speech combination became evident by age three. As can be seen in Figure 1A, both the three-year-olds and the four-year-olds chose the picture that matched the meaning of the gesture+speech combination significantly above chance; 63% correct for the three-year-olds (t(11) = 2·69, p = ·02, d = ·77) and 72% correct for the four-year-olds (t(11) = 3·55, p = ·01, d = 1·02). Two-year-olds, on the other hand, showed no reliable differences from chance performance in choosing the correct picture (51%; t(11) = ·29, p = ·77). The number of children who provided correct responses above chance (i.e., ⩾60% correct response rate) also increased with age: only 42% (5 out of 12) of the two-year-olds chose the correct picture above chance levels, while majority of both the three-year-olds (7/12; 58%) and the four-year-olds (9/12; 75%) were able to identify the correct picture at levels above chance (see Figure 1B).
Mean percentage of children's correct response rate (Panel A) and mean percentage of children providing correct responses above chance levels (Panel B) in the picture-choice task.
Children's comprehension of gesture+speech combinations with iconic gestures also improved with age (F(2, 33) = 3·89, p = ·03, η2 = ·19), from a mean correct response rate of 3·08 (SD = 1·0) at age two to a mean correct response rate of 4·45 (SD = 1·29) at age four, revealing a significant difference between ages two and four (Student-Newman-Keuls, p <·05).
DISCUSSION
In this study we examined whether the comprehension of gesture+speech combinations with iconic gestures emerges around the same time as the production of such gesture+speech combinations in children's communications, and found this to be true. Three-year-olds, but not two-year-olds, were significantly above chance in extracting the meaning of iconic gestures that conveyed unique information not found in the accompanying speech, and this understanding improved with age.
Why do children understand the meaning of gesture+speech combinations with iconic gestures later than they grasp the meaning of similar gesture+speech combinations with deictic gestures? One possible explanation could be the relative cognitive complexity of these two types of gestures. Compared to deictic gestures, the mapping between symbol and referent is less straightforward for iconic gestures. Deictic gestures map onto the world in a direct way by indicating perceptually cohesive entities such as people, objects, or locations. In contrast, iconic gestures select their referents from a set of relational concepts (i.e., associated actions and attributes) and, as such, might impose additional cognitive challenges for young children (Gentner, Reference Gentner, Hirsh-Pasek and Michnick Golinkoff2006; Özçalışkan et al., Reference Özçalışkan, Gentner and Goldin-Meadow2013).
A second possible explanation for the late comprehension of gesture+speech combinations with iconic gestures might be the lack of familiarity with such gestures at the early ages. Previous research has shown that even though children can associate iconic gestures with objects at age 1;6, it is not until age 2;2 that they truly understand the iconic relation between gesture and referent (Namy & Waxman, Reference Namy and Waxman1998; Namy, Reference Namy2001; Namy, Campbell & Tomasello, Reference Namy, Campbell and Tomasello2004). In fact, deictic gestures typically precede children's first words (Iverson & Goldin-Meadow, Reference Iverson and Goldin-Meadow2005), whereas iconic gestures appear several months after children produce their first words (Özçalışkan et al., Reference Özçalışkan, Gentner and Goldin-Meadow2013). Moreover, the incidence of iconic gestures in children's immediate environment is very scarce; parents of young children rarely use iconic gestures when communicating with their young children, and instead rely predominantly on pointing gestures (Iverson et al., Reference Iverson, Capirci and Caselli1994; Iverson, Capirci, Longobardi & Caselli, Reference Iverson, Capirci, Longobardi and Caselli1999; Özçalışkan & Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005b, Reference Özçalışkan, Goldin-Meadow, Stam and Ishino2011). As such, comprehension of gesture+speech combinations with iconic gestures might be particularly challenging for young children who have not yet themselves produced or observed others produce a substantial amount of iconic gestures.
In fact, even the four-year-olds in our study were performing below ceiling levels, suggesting a more extended developmental trajectory for children's comprehension of gesture+speech combinations with iconic gestures. Previous work on gestures that accompany children's early narrative productions shows that children go through a steep increase in their production of iconic co-speech gestures – particularly the ones that convey story events from the story character's viewpoint – around age five (Stites & Özçalışkan, Reference Stites and Özçalişkan2012; see also McNeill, Reference McNeill1992: 295–328 for a related discussion). A similar trajectory might be true for the comprehension of gesture+speech combinations with iconic gestures as well, with initial significant improvements in comprehension becoming evident around age three – at the same time as the initial spurt in iconic gesture production (Özçalışkan et al., Reference Özçalışkan, Gentner and Goldin-Meadow2013), which is then followed by a more full-fledged understanding of such combinations around age five.
Previous work shows clear evidence of early comprehension abilities of gesture+speech combinations with deictic gestures (e.g., ‘push’+point to ball; Morford & Goldin-Meadow, Reference Morford and Goldin-Meadow1992). Our study extends this finding by showing that children also develop the ability to understand gesture+speech combinations with iconic gestures, but at a slightly later age, during the period when they also begin to produce such combinations in their communications. Overall, our results suggest that the complexity of a gesture (iconic vs. deictic) as well as its relative frequency in the input might serve as important contributors to children's developing ability to unpack multimodal communications addressed to them at the early ages.
