Speakers show cross-linguistic differences in the expression of key components of spatial motion, both in speech (Slobin, Reference Slobin, Strömqvist and Verhoeven2004; Talmy, Reference Talmy and Shopen1985, Reference Talmy2000) and in co-speech gestures (Gullberg, Hendriks, & Hickmann, Reference Gullberg, Hendriks and Hickmann2008; Kita & Özyürek, Reference Kita and Özyürek2003; Kita et al., Reference Kita, Özyürek, Allen, Brown, Furman and Ishizuka2007; McNeill, Reference McNeill2000; McNeill & Duncan, Reference McNeill, Duncan and McNeill2000; Özçalışkan, Reference Özçalışkan2012; Özçalışkan & Slobin, Reference Özçalışkan, Slobin, Greenhill, Littlefield and Tano1999; Özyürek et al., Reference Özyürek, Kita, Allen, Brown, Furman and Ishizuka2008; Özyürek, Kita, Allen, Furman, & Brown, Reference Özyürek, Kita, Allen, Furman and Brown2005). Speakers of verb-framed languages (V-languages; e.g., Turkish) typically express path of motion using verbs (e.g., eve girdi: house-to entered). By contrast, they express manner of motion (i.e., how one moves) less frequently, and when they do, manner tends to be encoded with optional constituents such as adverbs (e.g., eve hızla girdi: house-to rapidly entered) or subordinate verbs (e.g., eve koşarak girdi: house-to by running entered). Speakers of satellite-framed languages (S-languages; e.g., English) typically express manner in the verb (e.g., run or crawl) and use optional linguistic elements outside the verb (i.e., “satellites”) to express path of motion (e.g., out or down; Slobin, Reference Slobin, Strömqvist and Verhoeven2004; Talmy, Reference Talmy and Shopen1985, Reference Talmy2000). Typological distinctions are also reflected in gesture. Speakers of S-languages typically synthesize manner and path components into a single gesture (e.g., wiggling fingers left to right to convey running left to right), while speakers of V-languages usually express each component using separate gestures (wiggling fingers in place to convey the manner of running; tracing a line with index finger from left to right to convey the path of rightward trajectory; e.g., Kita & Özyürek, Reference Kita and Özyürek2003).
Cross-linguistic differences in the expression of motion can also influence the way that people represent different types of motion events (e.g., Gennari, Sloman, Malt, & Fitch, Reference Gennari, Sloman, Malt and Fitch2002; Kersten et al., Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010; Naigles & Terrazas, Reference Naigles and Terrazas1998). For example, speakers of Spanish, a V-language, tend to rate events that display the same path (e.g., up) but different manners of motion (e.g., running vs. walking) as being more similar than events that display the same manner but different paths. Moreover, this finding is particularly robust when speakers are asked to verbally describe those events while performing the task (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002), suggesting that they are more attuned to the path than to the manner information of motion events. However, it remains unclear whether this pattern reflects the relative importance of the information encoded within the verb (i.e., path in V-languages vs. manner in S-languages; Papafragou, Massey, & Gleitman, Reference Papafragou, Massey and Gleitman2002) or whether it reflects a more pervasive effect of the habitual expression of each motion component in one's native language (e.g., relative salience of manner in English; Slobin, Reference Slobin, Strömqvist and Verhoeven2004). It is generally agreed upon that speakers of V-languages should be more likely to show a path of motion bias, as opposed to a manner of motion bias, particularly when events are accompanied by verbal descriptions (Naigles & Terrazas, Reference Naigles and Terrazas1998) or when participants are asked to generate a verbal description rather than engaging in a purely nonverbal task (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002).
There is less agreement on the nature of motion event representation among speakers of S-languages, such as English. There is evidence for a manner of motion bias in English (the most commonly used S-language in these studies), a pattern that complements the path of motion bias in V-languages such as Spanish. For example, according to Naigles and Terrazas (Reference Naigles and Terrazas1998), English speakers preferentially choose motion events that depict the same manner as being more similar as opposed to the same path, similar to Gennari et al. (Reference Gennari, Sloman, Malt and Fitch2002). Authors who propose a manner of motion bias in English have highlighted differences in grammatical coding of manner versus path information. Whereas path is often relegated to an auxiliary (“satellite”) phrase in English, manner is typically encoded within the verb, which is often regarded as “an ‘informationally privileged’ element” (Papafragou et al., Reference Papafragou, Massey and Gleitman2002).
In contrast, researchers (e.g., Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002) have noted that both path and manner are encoded frequently in English, albeit by different linguistic devices (i.e., verbs vs. satellites). Perhaps the salience of a feature depends on how easily it is expressed, regardless of where it is expressed (i.e., inside or outside the verb). If so, then English speakers may be equally sensitive to manner and path (no bias).
A final possibility is that speakers of all languages, including English, are more sensitive to path than to manner (path of motion bias). According to Talmy (Reference Talmy2000), path is a core component of motion events that must be explicitly encoded for events that involve translational (as opposed to stationary) motion, even in a language such as English, whereas the overt expression of manner is typically optional. If this is the case, path may be equally salient in all languages. Consequently, speakers of S-languages and V-languages may differ only in the prominence with which they attend to manner (Kersten et al., Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010; Slobin, Reference Slobin, Strömqvist and Verhoeven2004).
Previous work gives some support for each of the three views: manner of motion bias, path of motion bias, and no bias. The goal of the current paper is to test these competing views in an experiment that requires English speakers to learn new words for motion events that vary either in path or in manner. Word-learning outcomes are expected to support one of the three views described earlier. Thus, the present findings can contribute new insights into motion event processing in English.
Finally, this study considers the role of gesture in motion word learning. Half of the participants were assigned to a speech+gesture condition, whereas others were assigned to a speech-only condition. We predict that English speakers will learn labels for events more readily when the training for the labels are accompanied by iconic gestures that depict either the manner or the path component of the event (speech+gesture), as compared with a speech-only training condition. We test these predictions by exposing learners to pseudowords, together with animated clips that depict novel motion events with variations in either path or manner.
TYPOLOGY OF MOTION EVENTS
The world's languages can be divided into two types based on their prototypical ways of expressing the different components of motion (Talmy, Reference Talmy and Shopen1985, Reference Talmy2000). In V-languages, such as Spanish, the verb is predominantly used to express path of motion (e.g., entrar “enter”; see Example (1)). Conversely, in S-languages, such as English, the path of motion is typically expressed outside of the verb, usually with a particle or a preposition, while the verb is reserved for the expression of manner of motion as in Example (2). This cross-linguistic difference, however, reflects tendencies, not absolutes. Speakers of V-languages can (and sometimes do) express manner of motion. This can be achieved by appending an optional manner adverbial or a subordinate clause, as in Example (3). However, because manner of motion requires the addition of an optional phrase or clause in V-languages, it is often omitted in V-languages (Naigles, Eisenberg, Kako, Highter, & McGraw, Reference Naigles, Eisenberg, Kako, Highter and McGraw1998; Özçalışkan, Reference Özçalışkan2004, Reference Özçalışkan2005; Özçalışkan & Slobin, Reference Özçalışkan, Slobin, Greenhill, Littlefield and Tano1999, Reference Özçalışkan, Slobin, Özsoy, Nakipoglu-Demiralp, Erguvanlı-Taylan and Aksu-Koç2003; Slobin, Reference Slobin, Strömqvist and Verhoeven2004).
1. El perro entra a la casa
“The dog enters the house”
2. The dog runsinto the house
3. El perro entra a la casa corriendo
“The dog enters the house running”
By contrast, in S-languages, verbs encode manner of motion as a default, leading to a wide variety of manner words that capture a broader and more fine-grained set of distinctions than that which is encoded by the limited set of manner verbs available in V-languages (Özçalışkan, Reference Özçalışkan2005; Özçalışkan & Slobin, Reference Özçalışkan, Slobin, Greenhill, Littlefield and Tano1999; Slobin, Reference Slobin, Strömqvist and Verhoeven2004). For instance, English speakers can express variations on jumping by using many different verbs, including hop, skip, jump, leap, bound, bounce, pop, bob, vault, and spring, while Spanish speakers have only three words to express variations on jumping: brincar “jump,” saltar “jump,” and botar “bounce.” It is important that, although S-language speakers have a greater repertoire of manner words and use them with greater frequency than do V-language speakers, intratypological variation also exists within a typological group. For example, Russian speakers express a greater amount and variety of manner verbs than English speaker even though Russian, like English, has been classified as an S-language (Slobin, Reference Slobin, Strömqvist and Verhoeven2004).
Language-specific patterns are evident not only in speech but also in the co-speech gestures that speakers produce. Prior work on co-speech gestures suggests two possible ways gesture might reflect typological patterns. One set of studies shows that speakers use gestures to express information that is not conveyed in speech (McNeill, Reference McNeill2000; McNeill & Duncan, Reference McNeill, Duncan and McNeill2000). For example, Spanish (V-language) speakers often produce a manner gesture along with a path description in speech, using gesture to convey additional motion information that they do not typically convey in speech (McNeill, Reference McNeill2000; McNeill & Duncan, Reference McNeill, Duncan and McNeill2000). In contrast, other studies suggest that the information conveyed in gesture is typically similar to the information expressed in speech (Gullberg et al., Reference Gullberg, Hendriks and Hickmann2008; Kita & Özyürek, Reference Kita and Özyürek2003; Kita et al., Reference Kita, Özyürek, Allen, Brown, Furman and Ishizuka2007; Özçalışkan, Reference Özçalışkan2012; Özyürek & Kita, Reference Özyürek, Kita, Hahn and Stoness1999; Özyürek et al., Reference Özyürek, Kita, Allen, Furman and Brown2005, Reference Özyürek, Kita, Allen, Brown, Furman and Ishizuka2008). For example, information that is omitted from speech (e.g., manner of motion) is often omitted from gestures as well. Gestures also reflect syntactic features, such as the degree of clause coherence (Givón, Reference Givón1989, Reference Givón2001). For example, when path and manner are expressed in the same clause (as is common in English), gestures tend to conflate both elements. However, when the elements are expressed in separate clauses (as is common in the V-languages; e.g., Japanese and Turkish), each gesture tends to express a single motion element, such as path or manner of motion (Kita & Özyürek, Reference Kita and Özyürek2003). As is evident across a broad range of studies, patterns in gesture are closely linked with patterns in speech in the expression of motion events in typologically distinct languages.
Influence of typological differences on cognition
Cross-linguistic differences in the expression of motion, both in speech and in gesture, also appear to have an effect on cognition, particularly if those motion events are verbalized. According to Slobin's (Reference Slobin, Gumperz and Levinson1996) thinking for speaking hypothesis, the act of verbalizing an event may bias speakers to conceptualize the event in a manner that is consistent with the language that they use to express it. Any event carries with it numerous semantic features, such as tense (e.g., “the dog ran” vs. “the dog runs”), perfectivity (e.g., in Spanish “el perro corrí” in which the dog ran at a particular moment vs. “el perro coria” in which the dog used to run), and evidentiality (e.g., in Turkish “kopek kaç-ıyor-du” in which the dog's running was witnessed vs. “kopek kaç-ıyor-mus” in which the dog's running is inferred). Languages differ, however, in whether and how they encode each of these features. For example, evidentiality is part of the morphosyntax of Turkish, requiring its obligatory expression, a pattern unlike English, where it is optional and encoded at the lexical level. As such, Slobin argues that the decision to verbally express certain features while omitting others is based not only on the relevance of the feature for the conceptualization of the event but also on how readily encodable that feature is in one's language. This means that languages may attune their speakers toward encoding certain features of an event while excluding others. Therefore, in verbalizing an event, a speaker is likely to attend more to features that are linguistically available and easier to encode in the language that they are speaking.
Other work takes this hypothesis one step further, asking whether cross-linguistic differences in the expression of motion events lead to cognitive differences in representing and remembering various components of the event. Two such studies (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002; Papafragou et al., Reference Papafragou, Massey and Gleitman2002) examined memory for motion events using a variation on the classic old/new memory task. Participants were asked to view a series of motion events with both manner and path components. After a delay period of either 10 to 20 min or 2 days, they were presented with a second set of motion events; half of these events matched the initially observed motion events (i.e., old stimuli) and the other half were altered so that either the path or the manner was different from the original stimulus (i.e., new stimuli). The participants’ task was to identify whether each motion event was old (i.e., one that they had seen before) or new (i.e., one that they had not seen before). The authors predicted that speakers would make more false alarms (i.e., incorrectly classify a new stimulus as old) if the stimulus feature that was altered was one that was not routinely encoded in their native language (e.g., speakers of V-languages should be better able to recognize a change in path than a change in manner). Contrary to the predictions of the thinking for speaking hypothesis, neither study reported language-specific effects in participants’ identification of events as new or old, even in the conditions where participants produced a verbal description during the first viewing of the stimuli. These results thus suggest that cross-linguistic differences in the expression of motion may not influence speakers’ long-term memory for motion events.
In addition to the aforementioned long-term memory tasks, Gennari et al. (Reference Gennari, Sloman, Malt and Fitch2002) and Papafragou et al. (Reference Papafragou, Massey and Gleitman2002), as well as several other studies (i.e., Cardini, Reference Cardini2010; Hohenstein, Reference Hohenstein2005; Naigles & Terrazas, Reference Naigles and Terrazas1998), used forced-choice similarity judgments to explore the effects of language on cognition in processing motion events in S-languages and V-languages. While neither study found language-specific differences on the memory task, several did show evidence of language-specific effects but only for those studies in which a verbal description of the events was involved (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002; Hohenstein, Reference Hohenstein2005; Naigles & Terrazas, Reference Naigles and Terrazas1998). In these studies, participants were presented with motion events that were characterized by path and manner cues. After each event presentation, they viewed two alternatives of the same event and were asked to choose the one that was “most similar” to the original event. In each case, one of the choices involved an alteration of path with no change in manner (i.e., same-path alternative), and the other involved an alteration of manner with no change in path (i.e., same-manner alternative). The authors hypothesized that speakers should categorize based on the feature most commonly expressed in their language type; that is, V-language speakers should choose same-path events as more similar, while S-languages speakers should choose same-manner events as more similar. The studies by Cardini (Reference Cardini2010) and Papagfragou et al. (Reference Papafragou, Massey and Gleitman2002) showed that speakers of S-languages and V-languages were equally likely to choose the manner or path alternatives as being more similar to the original event when not asked to verbalize the original event prior to making their similarity judgment. However, when participants were asked to produce a verbal description of the event before making their decision, Spanish speakers (V-language) were more likely to choose same-path events as being more similar to the original event than were English (S-language) speakers (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002), suggesting that for Spanish speakers, path was the more salient feature. Similar language-specific effects were found in both adults (Naigles & Terrazas, Reference Naigles and Terrazas1998) and children (i.e., 7-year-olds; Hohenstein, Reference Hohenstein2005) when the original video was presented with a verbal description containing a novel nonsense verb (see the following section for further discussion). The overarching finding across these studies was that the conceptualization of an event (as reflected in judgments of similarity) is affected when verbalizations are present during the encoding process but not when verbalizations are absent. Thus, it would appear that the act of verbalizing an event has short-term effects on cognition, at least for judgments of similarity, for motion events, in line with the thinking for speaking hypothesis.
Learning labels for motion events in typologically distinct languages
Several studies have examined how speakers of typologically distinct languages learn new words for different types of motion events. Naigles and Terrazas (Reference Naigles and Terrazas1998) showed adults live-action motion videos paired with spoken sentences that contained pseudoverbs (e.g., kradding for English, mercando for Spanish). The descriptions encoded the motion event using one of three grammatical frames: path, manner, or neutral. After hearing these statements, participants were asked to choose which of the two subsequent videos (changed in either manner or path) depicted the same type of motion as the previous video. Speakers showed the expected cross-linguistic differences in their sensitivity to manner in English and path in Spanish, but the syntactic framing of the motion description (i.e., path, manner, or neutral frame) also affected speakers’ performance. Speakers chose same-path videos more often than same-manner videos in response to path-biasing descriptions (i.e., either transitive sentences in both Spanish and English, e.g., “Ella está bechando el puente”/“She's zubbing the bridge,” or sentences which included the general path preposition “a” before a ground nominal in Spanish, e.g., “El está filando a la mesa”/“He's blicking the table”); in contrast, they opted for same-manner videos more often in response to the manner-biasing descriptions (i.e., nontransitive sentences with a specific path preposition, e.g., “Ella está klitando hacia la carpa”/“She's pimming into the tent”) and neutral descriptions (i.e., a short intransitive frame without a path preposition, e.g., “Ella está mecando”/“She's kradding”; cf. Feist, Reference Feist2010).
Hohenstein (Reference Hohenstein2005) examined eye movements of English- and Spanish-speaking children (ages 3.5 and 7) in response to videos that depicted same-manner and same-path motion events. When the videos were accompanied by path- or manner-biasing statements, 3-year-olds, regardless of language, fixated longer on videos matching the sentence bias (e.g., longer fixations for same-manner video with a manner-biasing statement), suggesting that they were using the sentence frame to determine the meaning of the novel verbs while ignoring language-specific patterns. In contrast, the fixations of the 7-year-olds suggested that these children utilized both the sentence frame and language-specific patterns in making predictions about word meaning, a pattern similar to what has been shown for adult Spanish and English speakers (Naigles & Terrazas, Reference Naigles and Terrazas1998). These findings suggest that children as young as 7, as well as adults, can use linguistic cues, along with language-internal biases, to guide their learning of new words for motion events.
Kersten et al. (Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010) also examined motion classification and word learning among English (S-language) and Spanish (V-language) speakers but by using a categorization task. The task was to classify videos of buglike creatures into categories by using a pseudoverb, a pseudonoun, or a numeral. The creatures in each video differed in both their appearance (e.g., number of legs) and motion (i.e., path and manner); participants had to learn through trial and error that the creatures could only be categorized by either their path or their manner. Kersten et al. (Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010) found that English speakers were quicker than Spanish speakers to identify a creature's category when the relevant feature was manner but were equally quick as Spanish speakers when the relevant feature was path. It is important that syntactic category (pseudoverb, pseudonoun, or numeral) did not affect the rate of learning. At first glance, these findings might appear to provide contrary evidence to the uniqueness of verbs in attuning speakers to either manner or path of motion (as suggested by Papafragou et al., Reference Papafragou, Massey and Gleitman2002). However, it is important to note that Kersten et al. (Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010) used a subtler manipulation: they varied the syntactic category of a single word rather than the grammatical structure of an entire clause. It is possible that the presentation of a word within a larger syntactic context provides additional cues that combine to have a greater influence on speakers’ attention to the different semantic components of an event. Therefore, the absence of word category effects in Kersten et al. (Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010) does not necessarily challenge these earlier findings.
Kersten (Reference Kersten1998) performed a similar word-learning task using novel verbs and nouns embedded within sentences. In this study, English speakers were more likely to associate the novel verbs with path than manner of motion, suggesting a path of motion bias for newly learned verbs even among English speakers. This finding has rarely been emphasized, perhaps because it does not comport with the theory that speakers of S-languages, unlike speakers of V-languages, should display a manner of motion bias.
In summary, the nature of motion event processing among English speakers remains an open issue. Some studies provide evidence for language-specific effects in categorization and similarity judgment tasks, particularly when the event is verbalized prior to or during the task (e.g., Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002; Kersten et al., Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010; Naigles & Terrazas, Reference Naigles and Terrazas1998). In contrast, other studies show evidence against language-specific effects, particularly in the memory for such events (e.g., Cardini, Reference Cardini2010; Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002; Papagfragou et al., Reference Papafragou, Massey and Gleitman2002). One possible reason for the discrepancy in results could be the way different researchers conceptualize relative salience of path versus manner within an S-language. Although it is widely agreed upon that path should be more salient than manner to V-language speakers, the relative ease with which S-language speakers attend to manner versus path is less clear and might explain some of the discrepancy in the pattern of results in earlier work.
Furthermore, studies of co-speech gesture could help clarify this picture as gesture offers an alternative way to encode meaning. To the best of our knowledge, there is no prior work that specifically focuses on gesture as an aid in learning new motion words. However, there is evidence that gestures can aid word learning in general. For example, Kelly, McDevitt, and Esch (Reference Kelly, McDevitt and Esch2009) demonstrated that foreign words, when accompanied by a gesture that expressed congruent information with speech (e.g., “Nomu means drink”+“drink” gesture) were learned better than words trained without gestures (e.g., “Nomu means drink”+“Nomu means drink”) or with incongruent gestures (e.g., “Nomu means drink”+“wash” gesture). Moreover, words learned with gestures, compared to those that were learned without gestures, elicited a larger P600, an EEG pattern that is correlated with successful recall (Rugg & Curran, Reference Rugg and Curran2007). Furthermore, Tellier (Reference Tellier2008) demonstrated that the richer memory traces left by gestures could not be simply attributable to the inclusion of an extra modality for encoding the new words. She found that 5-year-olds who had learned words with gestures showed greater performance in recalling and producing these words than did children who had been taught the same words with another nonverbal modality, namely, pictures. Thus, it is likely that iconic co-speech gestures portraying path and manner may provide a particularly effective tool for teaching novel motion words to adult speakers as well.
THE CURRENT STUDY
Prior cross-linguistic research has shown that when verbalizations are present during the task, language may have short-term effects on the perception of the different components of motion events. Judgment tasks with verbal manipulations show clear differences between speakers of typologically distinct languages and in the performance of speakers in V-languages. However, these studies do not provide a cohesive view of motion event processing in S-languages. In this study, we focus on English (an S-language) and examine three possible trajectories in novel word learning in this language among adult speakers.
The first possible trajectory is that English speakers would show no bias, meaning that there would be no differences in the learning of words encoding manner versus path variations. Such a finding would suggest that both elements are equally likely to be encoded in speech and therefore must be equally represented in cognition (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002).
The second possible trajectory follows a manner of motion bias. In this trajectory, participants would show better learning of words for manner than for path events. This could suggest a cognitive bias toward manner over path. Because manner of motion is primarily encoded in the verb in English and because the verb is “an ‘informationally privileged’ element” (Papafragou et al., Reference Papafragou, Massey and Gleitman2002), English speakers might find words encoding variations on manner easier to learn.
The third possibility is a path of motion bias, which suggests that English speakers would learn words encoding path more easily than words encoding manner. According to Talmy (Reference Talmy2000), path is a core component of any motion event. The path of motion bias, in turn, might suggest one of two things: (a) what matters most for motion word learning is the underlying cognitive universal bias to attend to path over manner regardless of the typological class of one's language (Talmy, Reference Talmy2000), or (b) the greater frequency with which path is expressed compared to manner (even in S-languages) when all elements of a motion description are taken into account (i.e., both the motion verb and each of its associated path arguments; Emerson, Çörekli, & Özçalışkan, Reference Emerson, Çörekli and Özçalışkan2013).
In this study, we examine the effect of motion type and modality on word learning in English (an S-language), using a novel word-learning task that encodes variations on either path or manner of motion. We predict that speakers of English might show a path or a manner bias, or no bias at all, in learning novel words for motion events. We also predict that modality might influence learning: we expect that speakers will show better learning when instructed in both speech and gesture than when instructed only in speech. Overall, our study will shed new light on the relative biases speakers of English (an S-language) show in learning words for manner and path and the role gesture might play in this learning process.
METHODS
Participants
The participants included 89 adult native speakers of English, all attending college at the time of the study. Six participants were excluded because of computer malfunction during data collection, resulting in a total of 83 participants (M age = 20.45, range = 18–46, 16 males). All participants received course credit for their participation.
Materials
Motion animations
Participants were presented with animations of a star-shaped character that moved with a particular manner and path in relation to a stationary spherical object. The animations were adapted from a stimuli set originally developed by Pulverman, Golinkoff, Hirsh-Pasek, and Buresh (Reference Pulverman, Golinkoff, Hirsh-Pasek and Buresh2008). The stimuli animations consisted of 12 path variations with the same manner and 12 manner variations with the same path (see Appendix A for all path animations and Appendix B for all manner animations used in the study). All stimuli were created using Strata Design 3D CX 6 software.
Pseudowords
Twelve different pseudowords were created to be used in both the path and the manner conditions. All of the pseudowords ended with the phoneme /u/ to exclude any morphophonemic cues to syntactic class, thereby preventing speakers from encoding them as either nouns or verbs. In addition, pseudowords were presented on their own without any sentential context, which further eliminated any cues as to their syntactic class. We used Wuggy (Keuleers & Brysbaert, Reference Keuleers and Brysbaert2010) to generate possible pseudowords that were disyllabic; from this initial pool, we then chose a final set of pseudowords that were maximally distinct from each other phonologically. The final set of pseudowords was also matched to one another in terms of the number of phonemes and the type of surrounding phonological neighborhood, using PSIMETRICA (Mueller, Seymour, Kieras, & Meyer, Reference Mueller, Seymour, Kieras and Meyer2003). All pseudowords were recorded either with gestures for the speech+gesture condition or without gestures for the speech-only condition as part of the instructional videos; they were then converted into WAV audio files with Final Cut Pro such that all audio files used during testing exactly matched the audio presented during training.
Instructional videos
The word-learning training was conducted by instructional videos, which depicted the torso of a male instructor wearing a long-sleeved black shirt against a dark backdrop. In the speech-only condition, the instructor verbally produced the pseudoword for the motion animation (e.g., “blomu”) without moving his hands. In the speech+gesture condition, he produced the pseudoword for the animation with an accompanying iconic gesture. The co-speech gesture depicted either the path (e.g., “blomu”+move index finger left to right) or the manner (e.g., “blomu”+closing and opening fingers rhythmically) of the motion shown in the animation, resulting in a total of 24 instructional videos for path (12 speech-only, 12 speech+gesture; Appendix A) and 24 instructional videos for manner (12 speech-only, 12 speech+gesture; Appendix B).
Data collection
Each participant was tested on a computer in a laboratory setting and was randomly assigned to either the path (N = 41) or the manner (N = 42) word-learning condition. Within each condition, half of the participants received the training only in speech using pseudowords (speech-only) and the other half received the training in both speech and gesture using pseudowords with iconic gestures (speech+gesture) depicting either the manner or the path of the motion shown in the animation.
The training consisted of the presentation of a motion animation, followed by an instructional video with or without a gesture (see Figure 1a and b). Each participant completed four repeated training blocks of the same 12 animations (either manner or path variations depending on the condition they were assigned), presented in random order. At the end of each of the four training blocks, participants were given a block test that assessed their learning of all 12 words presented in the block (see Figure 1c). During testing, the participant observed two animations side by side accompanied by a pseudoword that was the correct label for only one of the displayed animations. The participant was asked to either press the red key on the left (the “z” button on a keyboard marked with a red sticker) if they thought that the pseudoword matched the animation on the left or the yellow key on the right (the “m” button on a keyboard marked with a yellow sticker) if they thought that the pseudoword matched the animation on the right. Test trials were designed so that for each subject each pseudoword was presented once and each motion animation was presented twice (once as the correct choice and once as the incorrect choice). All test trials were presented in random order across subjects; participants’ accuracy was recorded during testing.
Experimental paradigm wherein participants (a) first see a motion animation (either manner or path) followed by (b) its associated instructional video (either speech-only or speech+gesture). After three variations of motion animations and their instructional videos have been shown, (c) the participant views a short mini-test displaying two of the previous three motion animations and one of the associated pseudowords. This process is repeated 4 times to display all 12 motion animations, producing a block.
In addition, to ensure that the participants were attending to the stimuli throughout the training phase and to also improve retention, we tested each participant four times within each block. The procedure for the within-block testing was identical to the procedure for the between-block testing, except that participants were given only one test trial, which contained two of the three previous animations. Their responses in these within-block tests were not included in the analyses.
Stimulus validation
In order to control for the level of difficulty participants had in distinguishing pairs of animations within a motion type, we presented each possible combination of manner and path pairs to a separate group of participants (N = 57, M age = 20.43, range = 18–39, 15 males), all native English speakers. Each participant viewed either path variants (N = 28) or manner variants (N = 29), following the same between-subject design that we used in the main experiment. After the presentation of each stimulus pair, participants were asked to rate how similar the two animations were to one another on a 7-point Likert scale and then to indicate how confident they were in their rating also on a 7-point Likert scale. The scales ranged between not at all similar and identical for similarity rating and not at all confident and very confident for the confidence rating. The ratings on each scale were counterbalanced so that identical and very confident were indicated by “7” for half of the participants and by “1” for the other half. Responses were then rescaled for analysis so that a “7” would indicate identical and very confident for all participants. Participants’ similarity and confidence ratings in the manner and path condition were compared using independent t tests. The mean similarity ratings for each test item pair of path and manner stimuli are provided in Appendix C and Appendix D, respectively.
On average, the participants’ mean similarity rating was 3.28 (SD = 0.71) for path pairs and 3.03 (SD = 0.74) for manner pairs, revealing no reliable difference, t (55) = 1.30, p = .198, in similarity ratings for manner versus path. Similarly, their average confidence rating was 6.49 (SD = 0.51) for path pairs and 6.50 (SD = 0.61) for manner pairs, also showing no reliable difference between manner and path, t (55) = –0.03, p = .977. Overall, these results showed that participants did not view the stimuli of one motion type (path or manner) as being more difficult to distinguish (i.e., more similar) than the other. At the same time, we observed substantial individual variability in similarity ratings for stimulus pairs within each motion type, with a similarity rating range of 1.50 to 6.43 for the path condition, and a similarity rating range of 1.33 to 5.36 for the manner condition. We therefore decided to compare models that did or did not include similarity rating as a predictor (i.e., item similarity) in the data analysis.
Data analysis
We analyzed correlations between accuracy and our predictor variables, which included motion type (path or manner), modality (speech-only or speech+gesture), test block (one, two, three, or four), and item similarity (i.e., mean ratings ranging from 1 to 7), using Spearman correlations.
In our main analysis, we asked whether differences in accuracy were dependent on our two variables of interest: motion type and modality. Because our study design included both categorical and continuous variables and because the outcome was a binary variable (correct vs. incorrect), we analyzed the effect of motion type and modality on accuracy, while controlling for both the fixed and random effects of test block and item similarity, using generalized linear mixed modeling (GLMM) with logit (log-odds) as the link function (Jaeger, Reference Jaeger2008). GLMMs are an extension of linear mixed models and generalized linear models that allow for the analysis of nonnormally distributed and categorical data and include both fixed and random effects. GLMMs that use logit as the link function can be applied to analyze data involving binary outcomes (e.g., accuracy), thus providing an advantage over linear mixed models as well as analyses of variance. Furthermore, GLMMs include both fixed and random effects, unlike generalized linear models. The inclusion of random effects allows the model to identify and account for differences in the data that are specific to variation both in the baseline performance (i.e., intercept) and in the rate of change over time (i.e., slope) for each participant.
GLMM models for a study with more than two predictors can quickly become very complex, both computationally and conceptually. Thus, rather than beginning with the full model and backward fitting, as recommended by some authors (Barr, Levy, Scheepers, & Tily, Reference Barr, Levy, Scheepers and Tily2013), we used a forward-fitting logic starting with the simplest possible model and adding additional predictors in a stepwise fashion consistent with our hypotheses (Baayen, Davidson, & Bates, Reference Baayen, Davidson and Bates2008). Models were generated in R (R Development Core Team, 2010) using the lme4.glmer function (Bates, Maechler, Bolker, & Walker, Reference Bates, Maechler, Bolker and Walker2014); these models were then fit by maximum likelihood using adaptive Gauss–Hermite quadrature (nAGQ = 1).
Altogether, we evaluated a sequence of seven models, which are represented in Table 1 (first two columns). Model 1 had two factors, a fixed intercept and a random by-subjects intercept. The by-subjects intercept was added to account for individual differences in the baseline performance of our participants. Next, we added fixed and random factors to account for the two control variables, test block and item similarity. To be specific, we added a fixed effect for test block in Model 2 to determine whether the probability of accurate responding changed over time, that is, whether there was significant learning across blocks. In Model 3 we added a random slope for test block (conditioned on subjects). The inclusion of random slopes by subject in this model enabled it to account for the variance caused by the different rates of improvement in accuracy across the four blocks separately for each participant. In Model 4 we added a fixed effect for item similarity (i.e., mean ratings from the “stimulus validation” study) to account for variability in outcomes due to test item difficulty (as measured by mean similarity rating). Model 5 included a random intercept for item similarity to account for different baseline levels in performance for the varying levels of mean similarity ratings.Footnote 1 Finally, in Models 6 and 7, we added fixed effects for our two main variables of interest, motion type and modality, respectively. The fixed effect for motion type (Model 6) was used to determine whether participants in the path condition (coded as baseline) performed differently from participants in the manner condition. Likewise, the fixed effect for modality (Model 7) was added to account for group differences that reflected assignment to the speech-only condition (coded as baseline) versus the speech+gesture condition. We did not include interactions in any of the models for two reasons: we had no a priori hypotheses concerning interactions. In particular, we had no theoretical or methodological grounds to expect an interaction between motion type and modality, the two main factors of interest. Moreover, adding interactions would have substantially increased the conceptual and computational complexity of the models.
Generalized linear mixed models and comparisons using log likelihood ratio test
Note: AIC, Akaike information criterion; BIC, Bayesian information criterion; LL, log-likelihood value.
*p < .05. **p < .01. ***p < .001.
The estimated goodness of fit for each model was calculated with the Akaike information criterion (AIC) and Bayesian information criterion measures. These widely used measures reward models that give a better fit to the observed data, while penalizing more complex models (i.e., those with more degrees of freedom) because those are more prone to overfitting. For each of these measures, a lower score represents better overall model fit. Each consecutive model was then compared to the previous model by conducting likelihood ratio tests on the AIC using the anova function in R (see columns 1 and 2 in Table 1). Finally, we report the fixed effects for each predictor from the final model. For all analyses using motion type, the path condition was coded as the baseline; and for all analyses involving modality, the speech-only condition was coded as the baseline. As such, negative coefficients for these variables suggest better performance in the baseline condition (i.e., path or speech-only, respectively) while positive coefficients indicate better performance in the nonbaseline conditions (i.e., manner or speech+gesture, respectively).
RESULTS
We examined correlations between accuracy (i.e., choosing the correct animation match for a pseudoword) and other key variables in the study. We found that accuracy was positively correlated with test block (Spearman r s = .160, p < .001), showing that the probability of participants making accurate choices increased with each successive test block. At the same time, we found that the probability of accurate responses was negatively correlated with both item similarity (r s = –.033, p = .036) and motion type (r s = –.042, p = .008), suggesting that the probability of an accurate response was lower for animation pairs that were rated as more similar and that involved variations in manner. We found no reliable relationship between accuracy and modality (r s = –.005, p = .758).
Next we compared the different GLMM models to test the effects of motion type and modality on word learning, as outlined in Table 1. The comparison of Model 1 to Model 2 (which added test block) showed a significant decrease in AIC, χ2 (1) = 110.03, p < .001. Comparison of Models 2 and 3 showed that the addition of the random slope for test block also improved model fit, χ2 (2) = 60.05, p < .001. Next, Model 4 added in the fixed effect for item similarity as a covariate to account for the added variability due to the discriminability of the animation pairs in each test item, which also improved model fit for Model 4 versus Model 3, χ2 (1) = 6.64, p = .010. Likewise, the addition of the random intercept for item similarity in Model 5 improved model fit in comparison with Model 4, χ2 (1) = 13.18, p < .001.
The last two models added our two main variables of interest. The addition of the fixed effect for motion type in Model 6 significantly improved model fit over Model 5, χ2 (1) = 4.10, p = .043. Finally, the addition of the fixed effect for modality in Model 7 did not improve model fit in comparison with Model 6, χ2 (1) = 0.53, p = .466.
Table 2 displays the fixed effects for Model 7, the full model. The effect of test block was significant (B = 0.47, SE = 0.06, p < .001), showing an improvement in accuracy from Block 1 to Block 4, suggesting greater learning over time. Item similarity was also significant (B = –0.11, SE = 0.05, p = .031), showing worse accuracy on items that were rated as more similar and underscoring the importance of accounting for item variability (e.g., the similarity between animation pairs within a test item) within the analysis. Turning next to our main variables of interest, the fixed effect of motion type was significant (B = −0.30, SE = 0.15, p = .043) with participants in the path condition showing higher accuracy than participants in the manner condition. This finding provides support for the path of motion bias hypothesis. Finally, the fixed effect for modality was not significant (B = –0.09, SE = 0.12, p = .465). Thus, contrary to our second hypothesis, the presence of gestures during training did not lead to improved performance (see Table 3 and Figure 2 for distribution of accuracy scores by motion type and modality).
Fixed effects for Model 7 (full model)
*p < .05. **p < .01. ***p < .001.
Mean (standard deviation) percentage of correct responses per participant by motion type, modality, and test block
Mean percentage of items answered correctly per block in the path and manner conditions (collapsed across speech-only and speech+gesture conditions).
DISCUSSION
In this study, we asked whether English speakers would show different patterns of learning for novel words that encode manner or path of motion. Based on previous work, we predicted that speakers would show one of three patterns: no bias, a manner of motion bias, or a path of motion bias. Our results showed that English speakers had more difficulty learning words for manner than words for path, thus supporting a path of motion bias in English.
Researchers have generally assumed that S-language speakers should show either a manner of motion bias, largely due to the greater verbal expression of this motion component in these languages (Papafragou et al., Reference Papafragou, Massey and Gleitman2002), or no bias in representing and processing different types of motion events because both path and manner are expressed frequently in English speakers’ description of motion in English (Gennari et al., Reference Gennari, Sloman, Malt and Fitch2002). However, it is interesting to note that these studies have tended to use within-subjects, forced-choice tasks, which require that speakers choose between manner and path variations. Consequently, manner and path necessarily compete for salience (Kersten, et al., Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010). By contrast, the present study used a between-subjects design. This design tests how each speaker processes either path or manner variations, instead of choosing one over the other. We suggest that this design adds an important new perspective on the relative salience of path versus manner for English speakers, because they can more easily focus exclusively on one of the two features during the word-learning task.
The present study also highlights the important role of discriminability between testing items (i.e., similarity), which has a substantive effect on learner performance. The stimulus validation that we included in our study showed that, on average, speakers did not differ in their perception of similarity between pairs of animations for path or for manner. At the same time, we observed a considerable range of variability in similarity ratings for item pairs within both manner and path of motion stimuli, suggesting that the ability to discriminate between stimuli might become a confounding variable in a study. For example, in our experiment, animations for path in which the figure's motion remained close to the landmark (i.e., the sphere) passing either in front of or behind it were rated as very similar to one another (see Appendixes A and C); in contrast, animations for manner that showed the figure rotating along the x or y axis were rated as very dissimilar from each other (see Appendixes B and D). If a study were to contain only stimuli of these two types, the greater distinction between the two manners compared to the two paths might have caused these stimuli to be more salient to the learner, thus confounding the results toward a manner bias. Our study, which included similarity ratings between animations within test item pairs as a predictor in its design, not only avoided such a confound, but also provided evidence for the important role similarity between animations could play in identifying biases for different motion components.
In addition, our study used pseudowords with an ambiguous inflectional marking so that the speakers would not be able to identify the syntactic class of the word. The goal was to activate verbal representations of motion events without priming specific parts of speech (e.g., nouns or verbs), which could bias attention toward path or manner (see Hohenstein, Reference Hohenstein2005; Kersten, Reference Kersten1998; Naigles & Terrazas, Reference Naigles and Terrazas1998). The use of such native or nativelike phonemes has been shown to assist in word learning (e.g., Ellis & Beaton, Reference Ellis and Beaton1993; Papagno, Valentine, & Baddeley, Reference Papagno, Valentine and Baddeley1991; Service & Craik, Reference Service and Craik1993), and the use of ambiguous syntactic class assignment further prevents biases based on syntactic cues (Kersten, Reference Kersten1998), particularly when they are presented in isolation, rather than embedded in a sentential context (Kersten et al., Reference Kersten, Meissner, Lechuga, Schwartz, Albrechtsen and Iglesias2010).
In contrast with earlier work (e.g., Kelly et al., Reference Kelly, McDevitt and Esch2009; Tellier, Reference Tellier2008), our study found no difference in the learning of motion words in the speech+gesture and speech-only conditions. Participants who received training in speech and gesture (“blomu”+move index finger left to right) did not differ reliably from participants who received similar training only in speech (“blomu”). One possible explanation for the lack of a gesture effect in the study could be the nature of the animations that were used. In our study, we held the manner constant in the path condition and the path constant in the manner condition, such that there was only one possible referent for each novel psuedoword. This in turn made the identification of information conveyed through gesture relatively easy, possibly eliminating any added effect gesture might have on the word-learning process. It is possible that if we used more complex motion animations that varied both in manner and in path (instead of varying only in manner or in path), then gestures might have contributed more to the word-learning process. There is empirical work that suggests that gestures are particularly influential on learning for tasks that are cognitively challenging for the learner (e.g., Alibali & Goldin-Meadow, 1993; Church & Goldin-Meadow, 1986; Perry & Elder 1997). As such, the inclusion of gestures in a word-learning task that is cognitively more challenging might have added benefits to learning, a possibility that remains to be examined in future work.
An alternative, but not mutually exclusive, explanation for the lack of a gesture effect could be that our participants were not sufficiently engaged in the gestures for them to have had an added impact on learning. The participants in our study only watched the gestures but did not gesture themselves. As shown in previous work, doing rather than just seeing gesture leads to better learning in both adults (Engelkamp & Dehn, Reference Engelkamp and Dehn2000; Macedonia, Müller, & Friederici, Reference Macedonia, Müller and Friederici2011) and children (Cook & Goldin-Meadow, Reference Cook and Goldin-Meadow2006; Gerson & Woodward, Reference Gerson and Woodward2014; Goldin-Meadow et al., Reference Goldin-Meadow, Levine, Zinchenko, Yip, Hemani and Factor2012; Tellier, Reference Tellier2008). Future research examining the effects of seeing versus producing gesture during word-learning tasks could shed new light onto this question.
It may also be informative to extend our paradigm to examine other S-languages and V-languages, as well as equipollently framed languages (e.g., Thai and Mandarin), which carry features of both S-languages and V-languages. Based on our results, one likely outcome of such research would be a path bias for all languages within each typological class as well, because path constitutes an obligatory component of a motion event (Talmy, Reference Talmy2000), making it a cognitive universal. At the same time, the path bias might show variability in its strength, particularly across different S-languages and equipollently framed languages, largely due to the variations in the extent of path versus manner expression in these languages (Slobin, Reference Slobin, Strömqvist and Verhoeven2004). Slobin showed that speakers of some S-languages such as Dutch and German expressed manner less frequently than did English speakers, while speakers of some equipollently framed languages such as Mandarin and Thai used manner more frequently than English speakers. Examination of additional S-languages as well as equipollently framed languages could therefore further elucidate the reasons for the path of motion bias found in this experiment. If path is an obligatory component of all translational motion (as suggested by Talmy, Reference Talmy2000), we would expect the path of motion bias to prevail in these languages as well. In addition, if the effect is driven by the absolute rate of path expression relative to manner expression, we would expect speakers’ abilities to learn path and manner words to be modulated by their rates of path and manner expression in their native languages, respectively.
In conclusion, the present results suggest that English speakers have an advantage (or bias) in learning new words for path of motion as compared to words for manner of motion. These findings thus suggest a path of motion bias even among English speakers. If this finding can be replicated and generalized to other language types, it could support a universal bias in representing path of motion, even among speakers of S-languages.
APPENDIX A
Path animations and gesture instructional videos.
APPENDIX B
Manner animations and gesture instructional videos.
Mean similarity ratings for path of motion item pairs
Mean similarity ratings for manner of motion item pairs
ACKNOWLEDGMENTS
This work is supported by a Language and Literacy Graduate Fellowship (to S.N.E.). We thank the two anonymous reviewers and the action editor, Theres Grüter, for their valuable comments. We also thank Iria-Gael Romay-Fernández, Neil Gordon, G. Taylor Brooks, and Michael Sovik for their help in the creation of stimuli and collection of data, and Scott R. Weaver, Lee Branum-Martin, Leslie E. Hodges, and particularly Kevyn Collins-Thompson for their help with the statistical analysis.
