THEORETICAL FRAMEWORKS REGARDING THE RELATIONSHIP BETWEEN GESTURES AND SPEECH

There are several frameworks to explain the connections between gestures and speech. All these frameworks address the nature of representations underlying the processing of gestures. One way to differentiate between models is to consider the relevance of visuospatial and linguistic information. Some views suggest that the representation of gestures is based on visuospatial images (e.g., the sketch model, de Ruiter Reference De Ruiter2000; the interface model, Kita & Özyürek, Reference Kita and Özyürek2003; the gestures-as-simulated-action [GSA] framework, Hostetter & Alibali, Reference Hostetter and Alibali2008). In other models, the emphasis is placed on the close relationship between the representation of gestures and linguistic information (e.g., the interface model, Kita & Özyürek, Reference Kita and Özyürek2003; the growth point theory, McNeill, Reference McNeill1992, Reference McNeill2005). Another way to differentiate between models is by attending to the way gestures and speech are processed (separate vs. unitary processing). In some models, it is considered that gestures and speech are processed in two separate systems (e.g., lexical gesture process model; Krauss, Chen, & Gottesman, Reference Krauss, Chen and Gottesman2000) that interact when communicative intentions are formed (sketch model; de Ruiter, Reference De Ruiter2000), or at the conceptualization stage (interface model; Kita & Özyürek, Reference Kita and Özyürek2003) to produce effective communication. In other models, however, it is assumed that gestures and speech work together as two parts of the same system (the growth point theory, McNeill, Reference McNeill1992, Reference McNeill2005; the GSA, Hostetter & Alibali, Reference Hostetter and Alibali2008). The gesture-in-learning-and-development model (Goldin-Meadow, Reference Goldin-Meadow2000, Reference Goldin-Meadow2003), for example, considers that children process gestures and speech independently and they become part of an integrated system in mature speakers (Butcher & Goldin-Meadow, Reference Butcher and Goldin-Meadow2000; Özçaliskan, Gentner, & Goldin-Meadow, Reference Özçaliskan, Gentner and Goldin-Meadow2014).

In addition to the specific mechanism by which gestures and speech are processed, a relevant question concerns the role of gestures in communication. It has been confirmed that listeners glean information from gestures (Alibali, Flevares, & Goldin-Meadow, Reference Alibali, Flevares and Goldin-Meadow1997; Cassell, McNeill, & McCullough, Reference Cassell, McNeill and McCullogh1999; Goldin-Meadow, Reference Goldin-Meadow2003; Goldin-Meadow, Wein, & Chang, Reference Goldin-Meadow, Wein and Chang1992; Holler, Shovelton, & Beattie, Reference Holler, Shovelton and Beattie2009; Singer & Goldin-Meadow, Reference Singer and Goldin-Meadow2005; see Hostetter & Alibali, Reference Hostetter and Alibali2008, for a review; Hostetter, Reference Hostetter2011, for a meta-analysis demonstrating the benefits of gestures for communication). Because gestures usually arise during speech planning, many models defend the communicative value of gestures (e.g., the sketch model, de Ruiter, Reference De Ruiter2000; the interface model, Kita & Özyürek, Reference Kita and Özyürek2003; the growth point theory, McNeill, Reference McNeill1992, Reference McNeill2005; and the gesture-in-learning-and-development framework, Goldin-Meadow, Reference Goldin-Meadow2000, Reference Goldin-Meadow2003). In the next section, we address the role of gestures in L2 learning.

GESTURES AND L2 LEARNING

The role of different types of gestures in L2 learning has been emphasized in several studies (e.g., De Grauwe, Willems, Rueschemeyer, Lemhöfer, & Schriefers, 2014; Gullberg, Reference Gullberg2014; Kelly, Özyürek, et al., Reference Kelly, Özyürek and Maris2010; Macedonia & Knösche, Reference Macedonia and Knösche2011; Macedonia & Kriegstein, Reference Macedonia and Kriegstein2012; McCafferty & Stam, Reference McCafferty and Stam2008; Morett, Reference Morett2014; So et al., Reference So, Sim Chen-Hui and Low Wei-Shan2012, for reviews). In general, it is widely assumed that gestures have a positive effect on vocabulary acquisition and they should be used in foreign language instruction and embedded in a natural approach of language teaching (Asher & Price, Reference Asher and Price1967; Carels, Reference Carels1981; Krashen & Terrell, Reference Krashen and Terrell1983; Macedonia, Bergmann, & Roithmayr, Reference Macedonia, Bergmann and Roithmayr2014; however, see Hirata, Kelly, Huang, & Manansala, Reference Hirata, Kelly, Huang and Manansala2014; Kelly, Hirata, Manansala, & Huang, Reference Kelly, Hirata, Manansala and Huang2014, for evidence about the limited effect of gestures on learning segmental phonology).

In cognitive psychology, three main perspectives might account for the facilitative role of iconic gestures in L2 vocabulary learning. The self-involvement explanation states that gestures might favor the involvement of the participant in the learning task and, therefore, gestures would have a key role in the L2 vocabulary learning process (Helstrup, Reference Helstrup1987) and could facilitate enhanced attention to learning material. In particular, the impact of gestures on vocabulary learning is caused primarily by increased perceptual and attentional processes occurring during proprioception of movements associated with gestures or when individuals use objects to perform the action (Bäckman, Nilsson, & Chalom, Reference Bäckman, Nilsson and Chalom1986). However, under this view, the benefits associated with the use of gestures does not come from enactment itself because the motor component is not crucial (Kormi-Nouri & Nilsson, Reference Kormi-Nouri and Nilsson2001); rather, it is the multisensory information conveyed about a word that leads to greater semantic processing and higher attention level (Knopf, Reference Knopf1992; Knudsen, Reference Knudsen2007). Therefore, according to the self-involvement explanation, the learning of new words with gestures facilitates vocabulary acquisition regardless of whether a gesture is usually produced within a language or it denotes the same meaning of the word to be learned. Attention increases the retention of words (Craik & Tulving, Reference Craik and Tulving1975; see Muzzio et al., Reference Muzzio, Levita, Kulkarni, Monaco, Kentros, Stead and Kandel2009, for the role of attention in the encoding and retrieval of information at the neurobiological level). Thus, individuals using gestures would learn L2 words in an attentive manner, which would favor the learning process. The motor-trace perspective suggests that the physical component of gestures is coded in the learning process, leaving a motor trace in memory that aids the acquisition of new words in L2 (Engelkamp & Zimmer, Reference Engelkamp and Zimmer1984, Reference Engelkamp and Zimmer1985). According to this view, the physical enactment is crucial because it allows the formation of a motor trace associated with the meaning of the word. There is recent evidence from neuroscientific studies (e.g., repetitive transcranial magnetic stimulation) supporting the role of the motor cortices in the understanding of written words (Vukovic, Feurra, Shpektor, Myachykov, & Shtyrov, Reference Vukovic, Feurra, Shpektor, Myachykov and Shtyrov2017). Moreover, there is evidence that the use of familiar gestures might engage procedural memory as they involve well-defined motor programs (Macedonia & Mueller, Reference Macedonia and Mueller2016). Thus, the involvement of procedural memory along with declarative memory used for word processing might enhance vocabulary learning. From this perspective, familiar gestures that have been routinely practiced and are within the repertoire of the speaker (e.g., the gesture of answering the phone) have strong motor traces so that they produce more facilitation than less familiar gestures (e.g., the gesture of moving the finger from the mouth to the ear). Thus, the facilitation effect would depend on the extent to which an individual is familiar with certain gestures. However, according to this view, the effect of gestures would operate independently of their meaning; that is, regardless of the congruency between the gesture and the word meaning, familiar (well-practiced) gestures would facilitate the learning process relative to less familiar gestures. Finally, the motor-imagery perspective indicates that gestures are associated with motor images that form part of a word’s meaning (Denis, Engelkamp, & Mohr, Reference Denis, Engelkamp and Mohr1991). To be more specific, performing a gesture when individuals process a word promotes the creation of a visual image associated with the meaning of this word, which would enrich the semantic content of the word to be learned. This image would be a mental representation of the action associated with the word during encoding (Macedonia & Knösche, Reference Macedonia and Knösche2011; Macedonia et al., Reference Macedonia, Müller and Friederici2011). Neurobiological evidence obtained with functional connectivity analyses suggests the involvement of the hippocampal system in binding visual and linguistic representations of words learned with pictures (Takashima, Bakker, Van Hell, Janzen, & McQueen, Reference Takashima, Bakker, Van Hell, Janzen and McQueen2014). Hence, according to this view, the facilitation effect observed with gestures would be greater for iconic gestures related to the meaning of words to be learned than when there is a mismatch between the gestures and the meaning of the word. In addition, the learning of words accompanied by gestures with incongruent meanings would produce semantic interference and reduced recall (Cook, Yip, & Goldin-Meadow, Reference Cook, Yip and Goldin-Meadow2012; Yap, So, Yap, Tan, & Teoh, Reference Yap, So, Yap, Tan and Teoh2010).

It is important to note that the three perspectives described above are not mutually exclusive but instead emphasize different aspects of the effect of gestures in L2 learning. Hence, a gesture accompanying a word might increase self-involvement (co-speech gestures as movements that enhance attention to the L2 learning), create a motor trace (co-speech gestures as meaningful movements) due to the physical enactment associated with the production of gestures, or create a semantic visual image that could become integrated with the meaning of the word (co-speech gestures as semantically congruent movements).

EMPIRICAL EVIDENCE REGARDING THE IMPACT OF GESTURES ON L2 LEARNING

The first empirical study concerned with the impact of gestures on L2 learning was conducted by Quinn-Allen (Reference Quinn-Allen1995). In this study, English participants had to learn French expressions under two conditions. In the control condition, the participants were presented with sentences in French (e.g., Veux-tu quelque chose à boire? Do you want something to drink?), which they had to repeat in French. In the experimental condition, the learners received the sentences with an emblematic gesture illustrating the meaning (e.g., pointing the thumb toward the open mouth), and they were then required to reproduce the gesture. The results showed that sentences presented with gestures were associated with better recall in comparison with control sentences.

The facilitation effect produced by gestures in Quinn-Allen’s (Reference Quinn-Allen1995) study can be accommodated within the self-involvement account (Helstrup, Reference Helstrup1987). Participants would be more engaged in the learning process when they received and produced gestures relative to the control condition without gestures. However, the facilitation effect could also be explained by the motor-trace account and the motor-imagery explanation. The gestures used by Quinn-Allen, such as the act of drinking, are conventional gestures frequently produced in social communication, which could explain the facilitation effect according to the motor-trace view. In addition, this gesture is congruent in meaning with the sentence to be learned (do you want something to drink?) and, hence, a facilitation effect would be expected on the basis of the motor-imagery account. In short, a disadvantage of comparing only a condition with gestures with a condition without gestures is that it is not possible to distinguish whether the benefits observed in the gesture condition come from the use of familiar gestures, gestures with related meanings, and/or the mere fact of performing gestures. Additional experimental work has resolved this problem by using paradigms in which several gesture conditions are compared (Kelly, McDevitt, & Esch, Reference Kelly, McDevitt and Esch2009; Krönke, Mueller, Friederici, & Obrig, Reference Krönke, Mueller, Friederici and Obrig2013; Macedonia & Knösche, Reference Macedonia and Knösche2011; Macedonia et al., Reference Macedonia, Müller and Friederici2011).

In Macedonia et al.’s (2011) study, a group of German speakers were trained in Vimmi, an artificial language that has the advantage of controlling for several linguistic variables such as word length, familiarity of L2 words, and phonotactic factors (see Jalbert, Neath, Bireta, & Surprenant, Reference Jalbert, Neath, Bireta and Surprenant2011, for a discussion of these variables in vocabulary learning). The authors compared the learning of concrete nouns presented with iconic gestures (e.g., the word suitcase appeared with the gesture of an actor lifting an imaginary suitcase) or meaningless gestures (e.g., the word suitcase and the gesture of touching one’s own head). The results showed better recall for words learned with iconic gestures relative to words accompanied by meaningless gestures. The findings of this study appear to indicate that gestures involve something additional to the self-involvement of the participant in the task, as participants were exposed to gestures in the two experimental conditions. Both the motor-trace and the motor-imagery accounts could explain the better memory performance in the iconic gesture condition compared with meaningless gestures. Iconic gestures would favor L2 learning because they are semantically rich and because they are produced more frequently than meaningless gestures, hence producing strong motor activation. The results found in Macedonia et al.’s study favor both explanations. The authors observed activity in the premotor cortices for words encoded with iconic gestures, which is compatible with the motor-trace account. Moreover, words learned with meaningless gestures produce activity in a network engaged in cognitive control, suggesting that individuals detected the mismatch between the meaning of the word and the gesture.

Other studies have made use of additional experimental conditions to differentiate between explanations based strictly on the motor component of gestures as opposed to those based on the motor-imagery account. Studies performed in monolingual contexts have explored congruity effects during communication by mismatching the information between the semantics of words and the meanings of the gestures (Barbieri, Buonocore, Dalla Volta, & Gentilucci, Reference Barbieri, Buonocore, Dalla Volta and Gentilucci2009; Bernardis & Gentilucci, Reference Bernardis and Gentilucci2006; Bernardis, Salillas, & Carameli, Reference Bernardis, Salillas and Carameli2008; Chieffi, Secchi, & Gentilucci, Reference Chieffi, Secchi and Gentilucci2009; Feyereisen, Reference Feyereisen2006; see Kircher et al., Reference Kircher, Strube, Leube, Weibs, Sachs, Willmes and Greena2009, for congruity effects in the context of an unknown language). To illustrate, Kelly, Creigh, and Bartolotti (Reference Kelly, Creigh and Bartolotti2010; see also, Kelly, Healey, Özyürek, & Holler, Reference Kelly, Healey, Özyürek and Holler2015) conducted an event-related potential study combined with the use of a Stroop-like paradigm. The participants were presented with words (e.g., “cut”) and gestures that could be congruent (e.g., the act of cutting) or incongruent (e.g., the act of drinking). The authors found reduced N400 to words accompanied by congruent gestures relative to incongruent gestures (a semantic integration effect; Kutas & Hillyard, Reference Kutas and Hillyard1980). In addition, response times were faster in the congruent condition compared with the incongruent condition. The pattern of results found in Kelly et al.’s study is similar to what is observed in other Stroop tasks and seems to indicate that gestures are integrated with the meaning of words, producing interference when the word meaning does not match the meaning of the gesture (incongruent condition). In our view, the results of this study strongly agree with the motor-imagery account of gestures (see also Macedonia et al., Reference Macedonia, Müller and Friederici2011). No differences would be expected according to the self-involvement account as both conditions, congruent and incongruent, involved gestures. Moreover, in the congruent and incongruent conditions, familiar gestures were used so they would promote strong activation of motor-trace representations, and no differences between the two conditions would be found. On the contrary, clear differences between the congruent and incongruent condition would be predicted based on the motor-imagery view as they would differ in the degree to which the gesture could be integrated with the word meaning.

Nevertheless, there is a limitation with using the results of Stroop-like studies to draw the conclusion that gestures integrate with the meaning of words. In the standard Stroop color–word tasks (see MacLeod, Reference MacLeod1991, for a review) three conditions are implemented: a congruent condition, an incongruent condition, and a meaningless condition. The usual finding is a facilitation effect when the congruent condition is compared to the meaningless condition and an interference effect when the incongruent condition is compared to the meaningless condition. However, in some of the studies (cited above) regarding the impact of gestures on L2 learning, the meaningless condition is omitted, so it is not possible to determine the amount of facilitation and interference that results from the direct comparison of the congruent and incongruent gesture conditions.

Taken together, the results of previous studies appear to confirm the influence of gestures on L2 vocabulary learning. However, in spite of the merits of previous work in the field, a detailed comparison across several conditions is needed to distinguish the various theoretical explanations. In particular, the acquisition of L2 words should be compared between conditions with and without gestures in order to evaluate the self-involvement account. Furthermore, a condition with familiar gestures (e.g., iconic gestures) might be compared to a condition with unfamiliar gestures in order to evaluate whether the motor trace of gestures modulates vocabulary learning. Finally, congruent and incongruent conditions need to be compared with a meaningless condition to fully determine the motor-imagery account of gestures in L2 vocabulary acquisition. We acknowledge that some of these comparisons have been made in separate studies. For example, previous work has explored the comparison between congruent and incongruent gestures (e.g., Kelly, Creigh, & Bartolotti, Reference Kelly, Creigh and Bartolotti2010), as well as the use of a gestural control condition (Cook et al., Reference Cook, Yip and Goldin-Meadow2012; Wagner, Nusbaum, & Goldin-Meadow, Reference Wagner, Nusbaum and Goldin-Meadow2004). However, if we assume that several explanations might work together to explain how gestures interact with L2 vocabulary acquisition, it would be desirable to evaluate in the same study the role of congruent, incongruent, meaningless, and no gesture conditions.

Moreover, if we consider the overall pattern of results found in previous research, almost all studies have revealed a positive effect of gestures on L2 vocabulary learning. However, this observation contrasts with previous work showing that dual task conditions, in which individuals have to perform two tasks simultaneously, can hinder language learning (e.g., learning at the same time the meaning and form of aural input; see Van Patten, Reference Van Patten1990; Wong, Reference Wong2001). A L2 learning condition in which individuals have to process words and gestures concurrently is a dual task that might negatively influence the learning process. However, in some previous studies concerning the role of gestures in L2 vocabulary acquisition, this possible negative effect has not been captured. Previous studies show beneficial effects of gesturing during the L1 and L2 language processing (Holle & Gunter, Reference Holle and Gunter2007; Marstaller & Burianová, Reference Marstaller and Burianová2014; Özyürek, Reference Özyürek2014). For example, Quinn-Allen (Reference Quinn-Allen1995) observed better retrieval of L2 expressions when these were learned with semantically related gestures relative to learning without gestures. In our opinion, the possible negative effect of a dual task condition (the concurrent processing of a L2 sentence and a gesture) might not be observed in this study because it overlapped with a positive effect of congruency between the sentence and the gesture meaning. Therefore, a direct comparison between a no gesture condition and a meaningless gesture condition is required in order to determine the possible negative effect of gestures in L2 vocabulary learning.

Another important question concerns the type of words used as learning material in previous studies. To assess the effect of gestures on vocabulary learning, most of the studies have employed verbs as training material (De Grauwe et al., Reference De Grauwe, Willems, Rueschermeyer, Lemhöfer and Schriefers2014; Kelly et al., Reference Kelly, McDevitt and Esch2009; Kircher et al., Reference Kircher, Strube, Leube, Weibs, Sachs, Willmes and Greena2009). The use of verbs seems to be the best option if we consider that representational gestures depict actions. There is a more direct mapping between gestures and the semantic characteristics of verbs, in comparison with other types of words such as nouns (Childers & Tomasello, Reference Childers and Tomasello2002; De Grauwe et al., Reference De Grauwe, Willems, Rueschermeyer, Lemhöfer and Schriefers2014; Kelly et al., Reference Kelly, McDevitt and Esch2009; Kircher et al., Reference Kircher, Strube, Leube, Weibs, Sachs, Willmes and Greena2009).

However, apart from gestures, it has been corroborated that nouns are easier to learn than verbs, at least for children. Different studies have shown that children are able to learn English nouns easier than verbs in their natural context (Fernald & Morikawa, Reference Fernald and Morikawa1993; Goldfield, Reference Goldfield1993; Tardif, Shatz, & Naigles, Reference Tardif, Shatz and Naigles1997). It is probably due to the fact that English speakers place special emphasis on nouns when they interact with children while acquiring their L1. However, this advantage is not present in other cultures (see Gentner & Boroditsky, Reference Gentner and Boroditsky2001; Gopnik & Choi, Reference Gopnik and Choi1995; Tardif et al., Reference Tardif, Shatz and Naigles1997, for the absence of this advantage in Korean and Mandarin languages). Concerning the role of gestures during the learning of nouns and verbs, the GSA framework makes concrete predictions about what types of words could be more influenced by gestures. This theory states that gestures occur as a result of simulated action and perception, which are the basis of mental imagery and language production (Hostetter & Alibali, Reference Hostetter and Alibali2008). Even if thinking about the size or shape of a particular object (nouns) involves simulating movements, the relationship between verbs and movements is stronger. Gestures would therefore have a greater influence on verb learning than on noun learning. However, to our knowledge, there are no previous studies in which the role of gestures is examined during the learning of nouns and verbs. This comparison was performed in the current study.

THE CURRENT STUDY

Previous research seems to indicate that gestures facilitate L2 vocabulary learning. The data suggest that gestures help to create a motor trace associated with the meaning of words that involves procedural memory, which fosters L2 learning (Macedonia & Mueller, Reference Macedonia and Mueller2016). Nevertheless, we cannot discard other possible explanations, and these accounts might work together to explain the role of gestures in L2 vocabulary acquisition. Moreover, it might be possible that gestures could also have a negative effect on the learning process as individuals learning vocabulary with gestures would be involved in a dual task situation. The current study aimed at evaluating these questions.

In our study, monolingual speakers of Spanish (L1) learned words in an artificial language (Vimmi; L2) on 3 consecutive training days. The words to be learned were presented alone (no gesture condition), coupled with meaningless gestures or unfamiliar gestures (meaningless gesture condition), or they were presented with iconic gestures whose meanings were semantically related (congruent condition) or unrelated (incongruent condition) to the meanings of words.

In two experiments, we compared the learning of L2 nouns (Experiment 1) and verbs (Experiment 2). These word classes were used because it has been suggested that verbs are more difficult to learn than nouns (Childers & Tomasello, Reference Childers and Tomasello2002; Gentner, Reference Gentner1981; Gentner & Boroditsky, Reference Gentner and Boroditsky2001). In addition, it has been proposed that compared with nouns, action verbs intrinsically contain a gestural/motor component in their representation (Boulenger, Hauk, & Pulvermüller, Reference Boulenger, Hauk and Pulvermüller2009; Hauk, Johnsrude, & Pulvermüller, Reference Hauk, Johnsrude and Pulvermüller2004). Hence, differences between these two types of words might depend on whether they engage overt body movements (e.g., action verbs). For example, De Grauwe et al. (Reference De Grauwe, Willems, Rueschermeyer, Lemhöfer and Schriefers2014) found that the comprehension of motor verbs (e.g., to throw) in L2 produced activation of motor and somatosensory brain areas. Hence, it might be possible that the effect of using gestures in L2 vocabulary learning would be greater with verbs than with nouns.

If we consider the theoretical accounts concerning the role of gestures in L2 vocabulary learning, the self-involvement account (Helstrup, Reference Helstrup1987), the motor-trace account (Engelkamp & Zimmer, Reference Engelkamp and Zimmer1984, Reference Engelkamp and Zimmer1985), and the motor-imagery account (Denis et al., Reference Denis, Engelkamp and Mohr1991), then it is possible to generate specific predictions. If gestures only promote the self-involvement of the participant in the learning tasks, all conditions with gestures would favor L2 vocabulary learning relative to the condition without gestures. If a motor trace of gestures helps participants to acquire new words, familiar gestures would be associated with better L2 vocabulary learning (congruent condition and incongruent condition) relative to the learning of L2 words accompanied by less familiar gestures (meaningless condition). Moreover, the motor-imagery explanation would suggest that the learning of gestures with meaning would produce a facilitation effect or an interference effect depending on the convergence between the meaning of the gesture and the meaning of the word to be learned in the L2. More specifically, congruent gestures could facilitate vocabulary learning while incongruent gestures impede the acquisition of new words. Alternatively, incongruent gestures might become distinctive and benefit the encoding and recall of L2 words (Worthen, Reference Worthen2006). Furthermore, the positions described above are not mutually exclusive. For instance, the acquisition of L2 words coupled with congruent gestures might involve a balance between the positive effect of promoting the connection between semantic information and L2 words, and the negative impact of engaging the participants in a dual task situation.

EXPERIMENT 1: LEARNING L2 NOUNS

In Experiment 1, we evaluated the impact of gestures on the learning of L2 nouns.

Method

Participants

Twenty-five individuals participated in the experiment (21 women and 4 men). Their mean age was 21.72 years (SD = 3.17). All of them were native Spanish speakers. Each subject provided written informed consent before performing the experiment.

Design and materials

Four L2 vocabulary learning conditions were manipulated within participants as follows: no gesture condition: Spanish (L1)–Vimmi (L2) word pairs had to be learned without gestures (e.g., cuchara, spoon in Spanish, and deschoga, a Vimmi word); meaningless condition: L1-L2 word pairs to be learned were coupled with unfamiliar gestures (e.g., cuchara-deschoga and the gesture of moving the hand from the forehead to the ear); congruent condition: L1-L2 word pairs were accompanied with gestures that reflected the common use of objects whose names had to be learned in L2 (e.g., cuchara-deschoga and the gesture of holding an invisible spoon and raising it to the mouth); and incongruent condition: L1-L2 word pairs were coupled with gestures associated with the use of an object different from that denoted by the L1 word (e.g., cuchara-deschoga and the gesture of lighting a match; Figure 1; see online-only Supplemental Material for the complete set of material used in the study).

Figure 1

Conditions used during the study of Spanish (L1)–Vimmi (L2) words (nouns in Experiment 1 and verbs in Experiment 2). In the example, cuchara (spoon in Spanish)–dechoga (a Vimmi word) was accompanied by (a) the gesture of holding an invisible spoon and raising it to the mouth (congruent condition), (b) the gesture of lighting a match (incongruent condition), or (c) the meaningless gesture of moving the hand from the forehead to the ear (meaningless condition), and (d) the word pairs were presented without gestures (no gesture condition).

The congruent and incongruent gestures presented along with the L1-L2 word pairs were iconic gestures (McNeill, Reference McNeill1992), which have been also called representational gestures (Kendon, Reference Kendon1981) that usually illustrate a concrete physical object by using the hands to show the properties or details of the item. For example, for the meaning of pencil, the gesture would involve holding a pencil with the fingers and doing handwriting movements. The gestures used in the meaningless condition were small movements performed with the hand that did not have iconic or metaphoric associations with the meaning of physical items (e.g., to form a fist with one hand and raise the fingers of the other hand). We took care to select meaningless gestures with similar properties to those of meaningful gestures (e.g., hand configuration, the use of simple movement trajectory, and spatial location). For the meaningless condition, we selected 10 different movements that were the same for all the participants.

In addition, 40 words were selected in Spanish. These words were concrete nouns denoting objects that could be manipulated with the hands (e.g., spoon, pen, etc.). Forty words were also selected from an artificial language, Vimmi (Macedonia & Knösche, Reference Macedonia and Knösche2011; Macedonia et al., Reference Macedonia, Müller and Friederici2011). The corpus of Vimmi words is constructed so that it avoids factors that might favor the learning of specific items (co-occurrence of syllables and similarity with words from languages such as Spanish, English, and French). Vimmi words were carefully selected so that they were pseudowords with legal orthography and phonology in Spanish but without meaning. To create the learning material, the 40 Spanish nouns were randomly paired with the 40 Vimmi words previously selected. The resulting 40 word pairs (L1-Spanish/L2-Vimmi word pairs) were randomly divided into 4 sets of 10 word pairs. Each set of 10 pairs was associated to one gesture condition (congruent condition, incongruent condition, meaningless condition, or no gesture condition). In order to counterbalance the gestures conditions across word sets, a total of 4 lists of material were created. In this way, a word pair (e.g., “cuchara-deschoga”) was coupled with a congruent gesture in List 1, an incongruent gesture in List 2, a meaningless gesture in List 3, and it was presented without gesture in List 4. Each participant was randomly assigned to one of the four lists. Hence, across lists, the 40 words were counterbalanced over the four training conditions, so that all word pairs appeared in all training conditions.

We equated the Spanish words across the four sets of word pairs in lexical variables (Davis & Perea, Reference Davis and Perea2005). There were no differences across word sets in terms of number of graphemes, F (3, 36)=1.40, p=.26 (M=6.57, SD = 1.72), number of phonemes, F (3, 36)=1.06, p=.38 (M=6.27, SD = 1.63), number of syllables, F (3, 36)=1.58, p=.21 (M=2.77, SD = 0.77), lexical frequency, F (3, 36)=2.67, p=.06 (M=16.07, SD = 35.74, per one million count), familiarity, F (3, 36)=1.59, p=.21 (M=3.95, SD = 2.85), and concreteness, F (3, 36)=1.42, p=.25 (M=4.06, SD = 2.88). Similarly, Vimmi words in the four sets were equated in number of graphemes, F<1 (M=5.15, SD = 1.61), number of phonemes, F<1 (M=5.10, SD = 1.53), and number of syllables, F<1 (M=2.47, SD = 0.78). Finally, we controlled for the similarity between the Spanish words and Vimmi words across sets of word pairs. The number of shared phonemes between the Spanish and Vimmi words was the same across the four sets both when the phoneme position was considered, F (3, 36)=1.59, p=.21 (M=0.36, SD = 0.54), and when it was not, F<1 (M=1.82, SD = 0.93).

The gestures used in all gesture conditions involved hand movements. The congruent and incongruent gesture conditions included iconic gestures that represented frequent movements that individuals usually perform when they manipulate objects (e.g., the gesture of playing the flute, the gesture of typing on a keyboard, etc.). The meaningless gestures involved similar small movements with the hands, but they did not convey any meaning (e.g., to move a closed hand from right to left in front of the face). These gestures were carefully selected to ensure that they were not emblems or gestures with metaphorical meaning. In addition, we wanted to make sure that the congruent condition, incongruent condition, and meaningless condition differed in the degree to which the semantics of the word was associated with the paired gesture. To this end, a set of 15 Spanish participants who did not participate in the main experiment took part in a pilot study. The participants received a video with a gesture (without sound) at the top of the screen and a word written in Spanish at the bottom of the screen, and they were instructed to rate the degree to which there was a match between the meaning of the word and the gesture, which ranged from 1 (high mismatch) to 9 (high match). There were differences between the congruent condition (M=8.23, SD = 2.34), meaningless condition (M=3.96, SD = 1.80), and incongruent condition (M=1.55, SD = 1.52), F (2, 28)=146.39, p<.001, η²=.91. The gesture-word pairs were rated higher in the congruent condition compared to the meaningless condition, F (1, 14)=94.80, p<.001, η²=.87, and the incongruent condition, F (1, 14)=170.64, p<.001, η²=.92. The incongruent condition and the meaningless condition also differed, F (1, 14)=43.62, p<.001, η²=.96. Therefore, the three conditions with gestures used in the study differed in terms of the association between the meaning of the word and the gesture.

Procedure

L2 vocabulary learning involved three training sessions conducted on 3 consecutive days. In each session, participants performed, first, the L2 training and, afterward, the assessment of the L2 learning. The two phases were separated by a 15-min break. E-prime experimental software was used for stimulus presentation and data acquisition (Schneider, Eschman, & Zuccolotto, Reference Schneider, Eschman and Zuccolotto2002).

L2 training

We employed a stimulus presentation procedure grouped by experimental condition similar to that used in other studies with various gesture conditions (e.g., Macedonia et al., Reference Macedonia, Müller and Friederici2011). Participants were presented with a block of 40 Spanish–Vimmi word pairs. These word pairs were grouped (10 word pairs in each group) according to the four learning conditions (no gestures, meaningless gestures, congruent gestures, and incongruent gestures). This block was repeated 12 times. Hence, a participant received 480 trials where the 40 word pairs were presented 12 times. A short break was introduced between learning blocks. The word pairs were randomly presented within each condition. In addition, the order in which the learning conditions were presented within a block was counterbalanced. This procedure blocked by learning condition avoids the cognitive cost associated with the continuous change between situations in which the participants have to perform gestures and the learning condition without gestures. In all experimental conditions, on each trial, the participant received a Spanish–Vimmi (L1-L2) word pair visually presented at the bottom of the screen. These word pairs were presented alone or they were accompanied by a gesture (see Figure 1). Gestures were recorded on video by the experimenter, and they were congruent, incongruent, and meaningless, depending on the learning condition. The duration of each recorded gesture was 5 s and the gesture was repeated twice. In all learning conditions, participants received verbal instruction to read aloud each L1-L2 word twice. In the gesture conditions, participants had to produce the gesture presented on the trial each time they said aloud the L1-L2 word pair. Participants started the production of the gesture when they began the production of the L1-L2 word pair so that each gesture and L1-L2 word pair was produced twice. For example, when participants received the word pair cuchara-deschoga along with the congruent gesture (Figure 1a), they had to say aloud this word pair at the time they produced the gesture of holding an invisible spoon and raising it to the mouth. Once the participants had produced the word pair twice, they had to press the space bar to continue to the next trial. The training lasted approximately 1 hr.

L2 learning assessment

Two tests were used to evaluate the acquisition of L2 words: translation from Spanish into Vimmi (forward translation from L1 to L2) and translation from Vimmi into Spanish (backward translation from L2 to L1). These tasks have been used in previous studies to evaluate L2 vocabulary acquisition (Kroll & de Groot, Reference Kroll and de Groot2005; Poarch et al., Reference Poarch, Van Hell and Kroll2014). Other L2 learning tasks could have been used; however, previous studies have found a positive correlation between L2 proficiency and performance on translation tasks and lexical decision tasks (Prior, MacWhinney, & Kroll, Reference Prior, MacWhinney and Kroll2007).

The order in which the translation tests were presented was randomized across the three training sessions and across participants. In each translation task, the 40 words to be learned were presented, and participants were instructed to translate them. On each trial, a word was presented in the middle of the screen until the participant produced its translation. Oral translations were recorded for later analyses of recall accuracy. Reaction times (RTs) from the presentation of the word until the beginning of the oral translation were also registered. The learning assessment lasted approximately 10 min.

Results

The main index of L2 vocabulary learning was the percentage of words recalled in the forward and backward translation tasks; however, RTs were also examined. Translation accuracy across the three training sessions was 48.73% (48.80% in the forward translation test and 48.67% in the backward translation test). The RTs associated with correct translations were trimmed following the procedure described by Tabachnick and Fidell (Reference Tabachnick and Fidell2001) to eliminate univariate outliers. Raw scores were converted to standard scores (z scores). Data points that, after standardization, were 3 SD outside the normal distribution, were considered outliers. After removing outliers from the distribution, z scores were calculated again. The filter was applied in recursive cycles until no observations were outside 3 SD. The percentage of outliers was 5.20%. Next, we report the results found in the L2 evaluation tests (forward translation and backward translation) for RTs and correct recall. In all analyses, we adopted a significance level of α=0.05. Only correct responses were included in the analyses of the RTs. Data points were excluded from the RT analyses if (a) the participants produced nonverbal sounds that triggered the voice key, (b) the participants stuttered or hesitated in producing the word, or (c) the participants produced something different than the word required. Some small errors were allowed and considered correct responses depending on the length of the correct word to be produced: (a) for monosyllabic words, the replacement of a vowel; (b) for disyllabic words, the replacement of a vowel or a consonant but not both; or (c) for words with three or more syllables, the inversion of a vowel and a consonant or the replacement of a vowel or a consonant.

Reaction times

The RTs were subject to analysis of variance (ANOVA) with translation test (forward translation, backward translation), training session (first session, second session, third session) and learning condition (no gestures, meaningless gestures, congruent gestures, incongruent gestures) as within-participants factors. The order in which participants received the translation tasks (forward-backward translation order vs. backward-forward translation order) was initially entered in the analyses as a between-subject factor. However, the main effect of order was not significant, and this variable did not interact with any other factor (all ps > .05); thus, the order of the translation task was not considered any further.

Table 1 shows the mean RTs across conditions. The main effect of session was significant, F (2, 10)=5.45, p=.02, η_p²=.52. Mean translation latency was 1380 ms (SE = 95) in the first session, 1277 ms (SE = 84) in the second session, and 1184 ms (SE = 77) in the third session. Linear trend analysis was significant, F (1, 5)=7.27, p=.04, η²=.52, and thus there was a practice effect with faster recall of words at the end of the training relative to the beginning of the learning process. No other main effects or interactions were significant (all ps > .05).

Table 1

Reaction times in the noun translation tasks (Experiment 1)

Note: Mean reaction times (in milliseconds) obtained in Experiment 1 (learning L2 nouns) as a function of the translation test (L1 to L2 translation, L2 to L1 translation), the training session (first session, second session, third session), and the learning condition (congruent gestures, incongruent gestures, meaningless gestures, no gestures). Standard errors are in brackets.

Recall performance

The Translation Test × Session × Learning Condition ANOVA revealed a significant main effect of session, F (2, 48)=210.17, p<.001, η_p²=.90. The recall percentage of L2 words was 25% (SE = 3.31) in the first session, 52% (SE = 4.12) in the second session, and 69% (SE = 3.75) in the third session. Linear trend analysis showed that the recall of L2 words was higher on the final session relative to the beginning of training, F (1, 24)=263.69, p<.001, η²=.92. The main effect of learning condition was significant, F (3, 72)=11.80, p<.001, η_p²=.33, and this effect was modulated by the translation test, so that the Learning Condition × Translation Test interaction was significant, F (3, 72)=3.99, p=.01, η_p²=.14. No other main effects or interactions were significant (all ps > .05). When the Translation Test × Learning Condition interaction was examined, we found differences due to the translation test in the no gesture condition, F (1, 24)=4.74, p=.04, η_p²=.16. In particular, better recall was observed in the backward translation (54%, SE = 4.57) relative to the forward translation (49%, SE = 4.05). The differences between the two translation directions were not significant in the congruent condition, incongruent condition, and meaningless condition (all ps > .05). Next, we examined recall performance in each translation test.

Forward translation (from Spanish-L1 to Vimmi-L2)

The results found in the forward translation test are presented in Figure 2. When session and learning condition were entered in the ANOVA, the main effect of session was significant, F (2, 48)=217.65, p<.001, η_p²=.90. Recall percentage was 24% (SE = 3.32) in the first session, 53% (SE = 4.02) in the second session, and 70% (SE = 3.47) in the third session. Linear trend analysis was significant, F (1, 24)=326.53, p<.001, η²=.93. Thus, L2 vocabulary learning increases as a function of the training. In addition, the main effect of learning condition was significant, F (3, 72)=10.09, p<.001, η_p²=.30. The Session × Learning Condition interaction was not significant, but there was a trend toward significance, F (6, 144)=1.90, p=.08, η_p²=.07. When learning conditions were compared (see Table 2), we observed better recall in the congruent gesture condition (59%, SE = 3.80) relative to the meaningless gesture condition (43%, SE = 3.58). The recall performance in the incongruent condition (45%, SE = 4.19) was similar to the meaningless condition and the no gesture condition. However, compared with the no gesture condition (49%, SE = 4.05), the recall percentage was lower in the meaningless condition (see Figure 3).

Figure 2

Recall percentages (% Recall) obtained in Experiment 1 during the translation of nouns (a) from L1 to L2 and (b) from L2 to L1 across training sessions (first, second, and third) and gesture conditions (congruent, incongruent, meaningless, and no gestures). Standard error is plotted in vertical lines.

Table 2

Comparison between learning conditions in the noun and verb translation tasks

Note: Comparison of recall percentages across learning conditions obtained in the translation tasks of Experiment 1 (learning of L2 nouns) and Experiment 2 (learning of L2 verbs). ^*p≤.05.

Figure 3

Recall percentage (% Recall) of (a) nouns in Experiment 1 and (b) verbs in Experiment 2 as a function of translation direction (L1 to L2, L2 to L1) and gesture conditions (congruent, incongruent, meaningless, and no gestures). Standard error is plotted in vertical lines.

Backward Translation (from Vimmi-L2 to Spanish-L1)

The ANOVA conducted with session and learning condition as variables revealed a significant main effect of session, F (2, 48)=122.55, p<.001, η_p²=.84. The recall of Spanish words from Vimmi words was 27% (SE = 3.90) in the first session, 51% (SE = 4.48) in the second session, and 68% (SE = 4.26) in the third session. Linear trend analysis indicated that recall performance increased as a function of training, F (1, 24)=141.98, p<.001, η²=.85. The main effect of learning condition was also significant, F (3, 72)=10.05, p<.001, η_p²=.30. Finally, the Session × Learning Condition interaction was not significant, F<1. When the learning conditions were compared, we observed better recall in the congruent condition (56%, SE = 4.07) relative to the meaningless condition (43%, SE = 4.10). The difference in recall between the incongruent condition (41%, SE = 5.06) and the meaningless condition was not significant. However, compared with the no gesture condition (54%, SE = 4.57), recall was lower in the incongruent condition and the meaningless condition.

EXPERIMENT 2: LEARNING L2 VERBS

In Experiment 1, we observed that gestures could either benefit or hinder the learning of nouns in a foreign language. This observation needed replication and further examination, which was the aim of Experiment 2.

When comparing the learning of nouns and verbs, it has been found that verbs are more difficult to acquire than nouns (Childers & Tomasello, Reference Childers and Tomasello2002; Gentner, Reference Gentner1981; Gentner & Boroditsky, Reference Gentner and Boroditsky2001). A possible way of remediating the intrinsic difficulty associated with the acquisition of verbs would be the use of gestures during the learning process. It has been theorized that the semantic representation of verbs involves an intrinsic motor component (Boulenger et al., Reference Boulenger, Hauk and Pulvermüller2009; Hauk et al., Reference Hauk, Johnsrude and Pulvermüller2004). Hence, in Experiment 2 we evaluated the role of gestures when participants learned verbs in L2.

Results

The recall percentage across the three training sessions was 42.46% (40.44% in the forward translation test, and 44.48% in the backward translation test). The RTs associated with correct translations were trimmed as described in Experiment 1. The percentage of outliers was 7.86%.

Reaction time

An ANOVA was performed with translation test (forward translation, backward translation), training session (first session, second session, third session) and learning condition (no gestures, meaningless gestures, congruent gestures, and incongruent gestures) as within-participant factors. Table 3 shows mean RTs across conditions.

Table 3

Reaction times in the verb translation tasks (Experiment 2)

Note: Mean reaction times (in milliseconds) obtained in Experiment 2 (learning L2 verbs) as a function of the translation test (L1 to L2 translation, L2 to L1 translation), the training session (first session, second session, third session), and the learning condition (congruent gestures, incongruent gestures, meaningless gestures, no gestures). Standard errors are in brackets.

The main effect of session was significant, F (2, 62)=23.05, p<.001, η_p²=.43. Mean translation latency was 2951 ms (SE = 139) in the first session, 2680 ms (SE = 115) in the second session, and 2350 ms (SE = 97) in the third session. Linear trend analysis was significant, F (1, 31)=39.37, p<.001, η²=.56, with faster responses at the end of the learning process relative to the beginning of training. The main effect of learning condition was also significant, F (3, 93)=9.96, p<.001, η_p²=.24. The mean RTs were 2494 ms (SE = 114) in the congruent condition, 2777 ms (SE = 111) in the incongruent condition, 2760 ms (SE = 130) in the meaningless condition, and 2776 ms (SE = 119) in the no gesture condition. RTs were faster in the congruent condition compared with the incongruent condition, F (1, 31)=29.96, p<.001, η²=.49, the meaningless condition, F (1, 31)=13.45, p<.001, η²=.30, and the no gesture condition, F (1, 31)=9.57, p=.004, η²=.23. No differences were found between the incongruent and the meaningless conditions, F<1, and the meaningless condition did not differ from the no gesture condition, F<1. No other main effects or interactions were significant (all ps > .05).

Recall performance

In the Translation Test × Session × Learning Condition ANOVA, the main effect of session was significant, F (2, 62)=240.80, p<.001, η_p²=.89. The recall percentage was 20% (SE = 1.91) in the first session, 43% (SE = 2.92) in the second session, and 64% (SE = 3.10) in the third session. Linear trend analysis was significant, F (1, 31)=325.47, p<.001, η²=.91; the recall of L2 words was higher at the end of training relative to the beginning of learning. The main effect of learning condition was significant, F (3, 93)=23.06, p<.001, η_p²=.43, and the main effect of translation test was also significant, F (1, 31)=9.23, p=.005, η_p²=.23. Moreover, the Translation Test × Learning Condition interaction was significant, F (3, 93)=9.14, p<.001, η_p²=.23. No other main effect or interactions were significant (all ps > .05). When we examined the Translation Test × Learning Condition interaction, we obtained differences due to the translation test in the no gesture condition, F (1, 31)=32.84, p<.001, η²=.51. This analysis revealed better recall in the backward translation (45%, SE = 3.09) relative to the forward translation (34%, SE = 2.83). There were no significant differences between the two translation directions in the congruent condition, incongruent condition, and meaningless condition (all ps > .05). As in Experiment 1, we examined the learning condition effect across training sessions in the two tests used to measure the learning of L2 words.

Forward translation (from Spanish-L1 to Vimmi-L2)

The main effect of session was significant, F (2, 62)=209.66, p<.001, η_p²=.87. Recall percentage was 18% (SE = 1.76) in the first session, 40% (SE = 2.72) in the second session, and 63% (SE = 3.14) in the third session. Linear trend analysis was significant, F (1, 31)=301.32, p<.001, η²=.91. Thus, the vocabulary learning increased throughout the course of training. Moreover, a significant learning condition effect was found, F (3, 93)=25.27, p<.001, η_p²=.45. The Session × Learning Condition interaction was not significant, F (6, 186)=1.22, p=.30, η_p²=.04. When learning conditions were compared (see Table 2), we observed better recall in the congruent gesture condition (57%, SE = 3.24), relative to the meaningless gesture condition (36%, SE = 2.70) and the no gesture condition (34%, SE = 2.83). The recall performance was similar in the incongruent condition (35%, SE = 3.10) and the meaningless condition. Finally, compared with the no gesture condition, the recall percentage was similar in the meaningless condition and the no gesture condition (Figure 4).

Figure 4

Recall percentages (% Recall) obtained in Experiment 2, during the translation of verbs (a) from L1 to L2 and (b) from L2 to L1 across training sessions (first, second, and third) and gesture conditions (congruent, incongruent, meaningless, and no gestures). Standard error is plotted in vertical lines.

Backward translation (from Vimmi-L2 to Spanish-L1)

Figure 4 shows the results obtained in the backward translation test. The ANOVA conducted with session and learning condition as variables revealed a significant main effect of session, F (2, 62)=173.80, p<.001, η_p²=.85. The recall of Spanish words from Vimmi words was 23% (SE = 2.26) in the first session, 46% (SE = 3.45) in the second session, and 65% (SE = 3.33) in the third session. Linear trend analysis indicated that recall performance increased as a function of training, F (1, 31)=247.57, p<.001, η²=89. The main effect of condition was also significant, F (3, 93)=16.32, p<.001, η_p²=.34. Finally, the Session × Learning Condition interaction was not significant, F (6, 186)=1.32, p=.25, η_p²=.04. When the learning conditions were compared (Table 2), we observed better recall in the congruent condition (57%, SE = 3.30) relative to the meaningless condition (39%, SE = 3.22) and the no gesture condition (45%, SE = 3.10). The difference in recall between the incongruent condition (36%, SE = 3.76) and meaningless condition was not significant. However, compared with the no gesture condition, the recall was lower in the incongruent condition and the no gesture condition.

The role of gestures in the learning of nouns and verbs

We evaluated the differential effect of gestures on the learning of L2 nouns (Experiment 1) and L2 verbs (Experiment 2; Figure 5). In the forward translation task, the main effect of word type was significant, F (1, 55)=4.50, p=.04, η_p²=.08. Participants recalled more L2 nouns (49%, SE = 2.95) than L2 verbs (40%, SE = 2.61). Furthermore, the Condition × Word Type interaction was significant, F (3, 165)=2.85, p=.04, η_p²=.05. No other interactions involving the word type variable were significant (ps > .05). The recall of verbs was lower relative to the recall of nouns in the no gesture condition, F (1, 55)=9.49, p=.003, η_p²=.15, and the incongruent condition, F (1, 55)=3.97, p=.05, η_p²=.07. However, no differences due to word type were found in the meaningless condition, F (1, 55)=2.18, p=.15, η_p²=.04, and the congruent condition, F<1. When the backward translation was taken into account, the main effect of word type was not significant, F<1, nor did this variable interact with any other factor (all ps> .05).

Figure 5

Comparison of recall percentages (% Recall) of nouns (Experiment 1) and verbs (Experiment 2) across gesture conditions (congruent, incongruent, meaningless, and no gestures). Standard errors are plotted in vertical lines.

To specify further the nature of the differences in L2 learning depending on the word type, we evaluated possible differences between nouns and verbs in lexicosemantic variables reported in the method section of Experiments 1 and 2. The lexical frequency of nouns (16.07, SD = 35.74) and verbs (15.48, SD = 23.62) was similar, t (78)=0.09, p=.93. The number of graphemes of nouns (6.57, SD = 1.72) and verbs (6.53, SD = 1.48) did not differ, t (78)=0.13, p=.89. Further, the number of phonemes was equated in the case of nouns (6.27, SD = 1.63) and verbs (6.30, SD = 1.44), t (78)=0.07, p=.94. The familiarity of words was similar for nouns (3.95, SD = 2.85) and verbs (3.50, SD = 3.09), t (78)=0.67, p=.50. However, differences between nouns and verbs emerged when concreteness was considered, t (78)=1.91, p=.05. The concreteness value was higher for nouns (4.06, SD = 2.88) than for verbs (2.90, SD = 2.53).

GENERAL DISCUSSION

Native English speakers beginning school have a vocabulary of around 4,000 to 5,000 word families, adding about 1,000 word families each year until reaching adulthood with a vocabulary of around 20,000 word families (Nation & Waring, Reference Nation and Waring1997). This observation means that learners of L2 face a considerable challenge, and there is hence a need to develop training methods that can facilitate L2 vocabulary acquisition. The use of gestures might be a good tool to foster this learning process (see Gullberg, Reference Gullberg2014, for a review). Previous empirical studies support this premise by showing that L2 vocabulary learning is facilitated by the inclusion of gestures in the learning context (Macedonia et al., Reference Macedonia, Müller and Friederici2011; Masumoto et al., Reference Masumoto, Yamaguchi, Sutani, Tsuneto, Fujita and Tonoike2006; Quinn-Allen, Reference Quinn-Allen1995; Tellier, Reference Tellier2008).

In previous studies, several conditions have been considered to evaluate the role of gestures. For instance, the comparison between a congruent and an incongruent condition using speech or gestures has been addressed during the learning of verbs (Kelly et al., Reference Kelly, McDevitt and Esch2009) and nouns (Macedonia et al., Reference Macedonia, Müller and Friederici2011). Other studies have compared congruent gestures relative to meaningless gestures presented with written words (Krönke et al., Reference Krönke, Mueller, Friederici and Obrig2013) and sentences (Straube et al., Reference Straube, Green, Weis and Kircher2012). Previous work has also compared conditions with gestures relative to conditions without gestures (Macedonia & Knösche, Reference Macedonia and Knösche2011). Moreover, congruent, incongruent, and no gesture conditions have also been considered (Feyereisen, Reference Feyereisen2006). However, to our knowledge, there are no previous studies in which four conditions have been evaluated at the same time (congruent, incongruent, meaningless, and no gesture conditions). The simultaneous comparison of these conditions in a single experiment would be of use in evaluating models about the role of gestures in L2 vocabulary learning. The current study set out to achieve this aim by evaluating four training procedures during the learning of nouns and verbs in a foreign language.

The results found in the experiments reported here showed facilitation and interference effects associated with the use of gestures in L2 learning (see also Kelly et al., Reference Kelly, McDevitt and Esch2009). This pattern of results runs counter to the self-involvement account of the role of gestures in L2 vocabulary acquisition (Helstrup, Reference Helstrup1987). According to this view, gestures might foster the involvement of the participant in the learning task, and thus, whenever gestures are used, an improvement in word acquisition would be found. Focusing on the facilitation effect, we observed that participants learned more words in a foreign language when the word pairs to be learned were accompanied by congruent gestures (Feyereisen, Reference Feyereisen2006; Kelly, Creigh, & Bartolotti, Reference Kelly, Creigh and Bartolotti2010; Macedonia & Klimesch, Reference Macedonia and Klimesch2014).Footnote ¹ This finding agrees with the motor-imagery account of the role of gestures in L2 acquisition (Denis et al., Reference Denis, Engelkamp and Mohr1991). In the congruent condition, the motor trace associated with the gesture was part of the meaning of the word. Hence, semantic processing was enhanced, which promoted the acquisition of L2 words. The facilitation effect observed with congruent gestures could also be explained by the motor-trace perspective (Engelkamp & Zimmer, Reference Engelkamp and Zimmer1984, Reference Engelkamp and Zimmer1985). According to this view, familiar gestures would have a strong motor-trace representation in memory, which would favor L2 learning. A way of dissociating between the motor-trace and the motor-imagery account would be the use of beats and deictic gestures. Previous studies have evaluated the role of these gestures in L2 learning (Kushch, Igualada, & Prieto, Reference Kushch, Igualada and Prieto2018; Morett, Reference Morett2014; So et al., Reference So, Sim Chen-Hui and Low Wei-Shan2012). For example, Morett showed that the use of beat and deictic gestures facilitated the recall of words in a language not familiar to the participants. Beats and deictic gestures are typically produced with language, and they have meaning associated with them. However, their meaning may not be part of a word’s meaning (particularly for beats). Therefore, the comparison between iconic gestures versus beats would be of interest to dissociate between theoretical perspectives. A facilitation effect with beats would favor the motor-trace theory, and a greater facilitation effect in case of iconic gestures would be explained by the motor-imagery account.

It is important to stress that different explanations might work together to explain the role of gestures in L2 vocabulary learning. Macedonia and Mueller (Reference Macedonia and Mueller2016) found neural evidence of the relationship between higher cognitive processes associated with the learning of words with iconic gestures (i.e., attention, language, sensory and motor processes, and declarative and procedural memory). Thus, it would be simplistic and reductionist to limit the effect of gestures on L2 learning to only one cognitive mechanism. In addition, in our study, the beneficial effect of using gestures in L2 vocabulary acquisition was considered during the processing of isolated words. However, different studies have found that word processing in L2 is affected by local sentence context and more global discourse context (Van Assche, Duyck, & Hartsuiker, Reference Van Assche, Duyck and Hartsuiker2016, for a review). Thus, future studies should evaluate the effect of gestures on the acquisition and processing of L2 words in semantically rich contexts.

Nevertheless, not all well-practiced gestures facilitated vocabulary learning in our study. In particular, incongruent gestures (familiar gestures with an easily recognizable meaning that mismatched the meaning of the word pair to be learned) made L2 learning difficult (Feyereisen, Reference Feyereisen2006; Kelly, Creigh, & Bartolotti, Reference Kelly, Creigh and Bartolotti2010; Macedonia et al., Reference Macedonia, Müller and Friederici2011). The interference effect found with incongruent gestures relative to the learning of L2 words without gestures might be difficult to explain merely in terms of semantic processing. This account would predict reduced L2 learning in the incongruent condition (meaningful gestures) relative to the meaningless condition (meaningless gestures) because in the incongruent condition there was a mismatch between the meaning of the gestures and the meaning of the words to be learned in L2. However, there were no differences between these two conditions. The similar pattern of results found in the incongruent iconic gesture condition and the meaningless gesture condition can be explained by assuming that meaningless gestures functioned as self-adaptor gestures (self-touching movements unconnected to the content of the speech). Furthermore, previous studies have shown that both iconic gestures and self-adaptor gestures play a similar role during lexical processing in speech production (Beattie & Coughlan, Reference Beattie and Coughlan1999). In addition, both the incongruent and the meaningless condition were associated with lower L2 vocabulary learning than that found in the no gesture condition. The only difference between the incongruent/meaningless conditions compared with the no gesture condition was the involvement of the participant in a dual versus single learning context, respectively. Specifically, participants in the incongruent and meaningless conditions were immersed in a dual task situation in which they had to code the gesture and the L1 word at the time they learned the corresponding word in the foreign language. The cost associated with this dual coding is also found in other studies, which have confirmed the difficulty of coding the meaning of a message in L2 at the time at which the learners are required to perform a concurrent task (Bransdorfer, Reference Bransdorfer1991; Van Patten, Reference Van Patten1990; Wong, Reference Wong2001). Thus, the use of meaningless or incongruent gestures in the learning of L2 would result in a dual task that makes L2 learning difficult. However, this idea contrasts with the absence of cognitive cost when words and gestures are processed concurrently in everyday interactions. Previous studies suggest that the cognitive cost of a concurrent task depends on the familiarity that people have with the words that are learned. For example, Padilla, Bajo, and Macizo (Reference Padilla, Bajo and Macizo2005) showed that the adverse effect of a concurrent task (articulatory suppression) depended on the participants’ word knowledge with the learning material. A cognitive cost of the concurrent task was observed in the learning of pseudowords but not in the retention of words already known to the participants. Thus, in normal circumstances, long-term knowledge associated with known words provides support for the retention of information even in dual task situations. In our study, however, this long-term knowledge of words in L2 was absent, so gestures would have an adverse effect on vocabulary acquisition unless the gestures were congruent with the learning material.

In the current study, we also evaluated the differential role of gestures in the learning of two types of words, that is, nouns (Experiment 1) and verbs (Experiment 2). Verbs are more difficult to learn than nouns, at least for children. Children acquire verbs slower than nouns and adults usually perform better with nouns than verbs on a range of tasks (Gentner, Reference Gentner1981, Reference Gentner1982; Gentner & Boroditsky, Reference Gentner and Boroditsky2001). In addition, L2 learners show better acquisition of nouns than verbs (Lennon, Reference Lennon1996). We confirmed this pattern of results in the current study. In the forward translation task, the recall of verbs was lower relative to the recall of nouns in the no gesture condition. In addition, when examining the impact of gestures, we found similar facilitation and interference effects during the learning of L2 nouns and verbs. However, upon closer inspection, we observed that the difficulty associated with the learning of verbs disappeared when congruent gestures were included in the training. Thus, the use of gestures in L2 vocabulary acquisition appears to remediate the intrinsic difficulty associated with the learning of verbs. This effect appears to agree with the motor-imagery account as differences between nouns and verbs disappeared when the gestures denoted the same meaning as the words to be learned in L2. Moreover, the differential effect of gestures on the acquisition of verbs in a foreign language can be accommodated within the gestures for conceptualization hypothesis (Kita, Alibali, & Chu, Reference Kita, Alibali and Chu2017). According to this view, gestures are rooted in practical actions that involve body movements and motoric contents, and the meaning of verbs also intrinsically denotes motoric information. Thus, gestures might involve the direct simulation of the meaning of verbs, which would facilitate the learning of this category of words.

It is important to note that the conclusions drawn from the differential effect of the use of gestures during the learning of verbs and nouns in L2 should be moderated. In Experiment 2, the mapping between representational gestures, which involve depicted actions, and the semantic characteristics of verbs, which refer to actions, also is stronger than that between gestures and nouns (Experiment 1). This may explain why the vast majority of studies on gestures have used verbs as learning material (Childers & Tomasello, Reference Childers and Tomasello2002; De Grauwe et al., Reference De Grauwe, Willems, Rueschermeyer, Lemhöfer and Schriefers2014; Kelly et al., Reference Kelly, McDevitt and Esch2009; Kircher et al., Reference Kircher, Strube, Leube, Weibs, Sachs, Willmes and Greena2009). Thus, it might be possible that the benefit associated with the use of congruent gestures during the learning of verbs might reflect this strong semantic overlap between the actions described by the gestures and the meaning of the verbs. Nevertheless, besides the semantic differences between nouns and verbs, both types of words differ in grammatical category. To explore this point, the role of gestures during the acquisition of L2 could be evaluated by mixing nouns and verbs as learning material. It is possible that the mixing of nouns and verbs produced a greater salience of their grammatical category and possibly it would maximize the differences found in the role of gestures during the learning of these two types of words. Future studies will shed light on this point.

In this section, we argue that the benefit of using congruent gestures in L2 vocabulary learning might be accounted for by a motor-imagery explanation (Denis et al., Reference Denis, Engelkamp and Mohr1991). According to this view, gestures promote semantic processing of the material to be learned. However, there remains some uncertainty regarding the specific mechanism by which the increased semantic processing associated with the use of gestures promoted the learning of L2 words. The impact of gestures on L2 vocabulary learning can be accommodated within the revised hierarchical model (Kroll & Stewart, Reference Kroll and Stewart1994). As described in the introduction section, in this model, L1 words, L2 words, and a shared semantic system are interconnected. However, differences in the weight of these connections emerge depending on the stage of L2 vocabulary acquisition. In the early stages, the links between the semantic system and the L2 words are weak and L2 learners preferentially use a lexical route of processing from L2 to L1. Furthermore, when proficiency increases, the links between L2 words and the semantic system develop while diminishing the weight of the lexical route. The model has been widely supported by previous work. For example, unbalanced bilinguals show an asymmetry in translation tasks with faster performance in backward translation (lexical route from L2 to L1) than in forward translation (semantic route from L1 to L2; Kroll & Stewart, Reference Kroll and Stewart1994).

Increased vocabulary learning has been demonstrated with L2 learning methods that foster a semantic route of processing (e.g., presenting a word to learn with a picture denoting its content, Lotto & de Groot, Reference Lotto and de Groot1998; Van Hell & Candia-Mahn, Reference Van Hell and Candia-Mahn1997; or imagining the meaning of a word to be learned, Ellis & Beaton, Reference Ellis and Beaton1993; Wang & Thomas, Reference Wang and Thomas1995). Previous studies demonstrate that gestures enrich the encoding of the words to be learned by adding sensorimotor networks and procedural memory to the semantic/declarative memory associated with the meaning of the words (Macedonia & Mueller, Reference Macedonia and Mueller2016). Hence, gestures enhance semantic processing of words. The findings obtained in the current study have provided evidence suggesting the use of semantic-L2 connections associated with the use of gestures. As mentioned previously, gestures abolished the difficulty associated with the learning of verbs but not nouns. This effect was not captured in backward translation but in forward translation, a task that is semantically mediated to a greater extent than backward translation (see Kroll, Van Hell, Tokowicz, & Green, Reference Kroll, Van Hell, Tokowicz and Green2010, for a critical review). Moreover, when we explored the characteristics of nouns and verbs in our material, concreteness produced a difference: nouns were associated with a higher concreteness value relative to verbs, a variable that has been shown to modulate L2 vocabulary learning. For example, concrete words activate the semantic system more robustly than abstract words (Van Hell & de Groot, Reference Van Hell and de Groot1998), with the former being more readily acquired by L2 learners (Kaushanskaya & Rechtzigel, Reference Kaushanskaya and Rechtzigel2012).

To conclude, the current study suggests that the use of gestures that are congruent in meaning with the word to be learned facilitates vocabulary acquisition in a foreign language. Furthermore, congruent gestures reduce the difficulty associated with the learning of verbs relative to the learning of nouns. Other methods based on conceptual analysis of the material foster L2 learning (e.g., the use of pictures paired with the words to be learned; e.g., Tellier, Reference Tellier2008). It would be interesting in the future to evaluate whether these two supporting materials might have additive effects on L2 vocabulary acquisition.