1. Introduction
The privileged status of arbitrariness as a defining feature of human language has given way to the recognition that iconicity too is a fundamental organizing feature of human language (Imai & Kita, Reference Imai and Kita2014; Padden, Meir, Hwang, Lepic, Seegers, & Sampson, Reference Padden, Meir, Hwang, Lepic, Seegers and Sampson2013; Perniss, Thompson, & Vigliocco, Reference Perniss, Thompson and Vigliocco2010). Iconicity works alongside arbitrariness (and systematicity) to structure the grammars of languages in both the spoken and signed modality (Dingemanse, Blasi, Lupyan, Christiansen, & Monaghan, Reference Dingemanse, Blasi, Lupyan, Christiansen and Monaghan2015; Occhino, Reference Occhino2017). Still, while iconicity is now a mainstream topic in theoretical circles, additional experimental investigations are needed to understand iconicity’s role in the organization and processing of natural linguistic systems.
The difficulty in studying iconicity within an experimental context may be due in part to the multi-faceted nature of iconicity, making it a difficult parameter to operationalize (Motamedi, Little, Nielsen, & Sulik, Reference Motamedi, Little, Nielsen and Sulik2019). While many theoretical papers in the European semiological tradition have examined, in depth, the nature of iconicity (Cuxac, Reference Cuxac1996; Pietrandrea, Reference Pietrandrea2002; Pizzuto & Volterra, Reference Pizzuto, Volterra, Emmorey and Lane2000; Russo, Reference Russo2004), within experimental paradigms investigating signed languages, iconicity is most frequently defined as a resemblance relationship between the form and the meaning of a linguistic unit (e.g., Bosworth & Emmorey, Reference Bosworth and Emmorey2010; Ormel, Knoors, Hermans, & Verhoeven, Reference Ormel, Knoors, Hermans and Verhoeven2009; Thompson, Vinson, & Vigliocco, Reference Thompson, Vinson and Vigliocco2009, Reference Thompson, Vinson and Vigliocco2010; Vinson, Cormier, Denmark, Schembri, & Vigliocco, Reference Vinson, Cormier, Denmark, Schembri and Vigliocco2008). Cognitive Linguistics frameworks however, understand iconicity in a different way. As Wilcox asserts: “Iconicity is not a relation between the objective properties of a situation and the objective properties of articulators. Rather, the iconic relation is between construals of real-world scenes and construals of form” (2004, p. 123). By construal, we intend the technical meaning of the term, a process by which humans select one way to portray an object/event, from the many different possible ways in which they can conceptualize the object/event (Langacker, Reference Langacker1991, 2008). Humans can think about language, conceptualize its pieces, and make connections between language-internal (between linguistic units) and language-external (between language and non-linguistic experiences) patterns. Under this definition, iconicity is dynamic rather than static, it is subjective rather than objective, and it is based in individual embodied experiences rather than universal properties.
Prior experimental investigations of iconicity in signed languages have focused on language-external connections built on shared cultural and sensorimotor experiences (Ormel et al., Reference Ormel, Knoors, Hermans and Verhoeven2009; Thompson et al., Reference Thompson, Vinson and Vigliocco2010; Vinson, Thompson, Skinner, & Vigliocco, Reference Vinson, Thompson, Skinner and Vigliocco2015; see Ortega, Reference Ortega2017, for a review of iconicity in acquisition). These shared individual, embodied experiences seem to allow members of groups to tap into the same sources of motivation, creating the illusion of ‘transparent’ objective properties of a linguistic unit. This type of iconicity has also been called ‘absolute’ iconicity by Ullman (Reference Ullmann1962) and ‘imagic’ iconicity by Haiman (Reference Haiman1980). Language-internal connections, on the other hand, are part of the systematic relations within the language, but can also be grounded in embodied experiences. As speakers become more proficient in a language, they build up more systematic language-internal connections so that motivations based on construals of the world interact with construals of the forms and relations within a language.
We have previously tested whether iconicity effects are mediated by language proficiency in Occhino, Anible, Wilkinson, and Morford (Reference Occhino, Anible, Wilkinson and Morford2017), in which American Sign Language (ASL) and German Sign Language (DGS) users were asked to rate the iconicity of signs in both their native and in a foreign sign language. If language proficiency mediated the perception of iconicity, we expected the signers to differ in their iconicity judgements of signs from the two languages. Overall, Germans rated DGS signs as being more iconic than ASL signs, and Americans rated ASL signs as more iconic than DGS signs, even when signs were rated as highly iconic by both native and non-native signers. This is to say that signers always considered signs in their own language as more iconic than signs matched for meaning, form similarity, or native signer iconicity rating than signs in the unfamiliar sign language. We take these findings as evidence that iconicity, rather than an inherent objective property of signs, is more likely a perceived relationship between a signer’s construal of form and construal of meaning, which is mediated by a signer’s experience with the language. We also take these results as support for the idea that proficient signers of a language are more sensitive to language-internal patterns of iconicity across signs within a language than are less-proficient signers, who are primarily sensitive to language-external patterns of iconicity.
Dingemanse and colleagues (2015) contrast iconicity with systematicity to distinguish motivated relationships between word forms and their meanings that are rooted in embodied experience, as opposed to statistical patterns across a language that are associated with abstract categories, such as phonological differences between nouns and verbs. Language-external iconicity, as we define it, is comparable to iconicity in this framework. However, language-internal iconicity is not equivalent to Dingemanse et al.’s systematicity. The language-external vs. language-internal iconicity dichotomy seeks to capture the dynamic and diachronic nature of the relationships between form and meaning within the lexicon, acknowledging not only phonological and prosodic overlap between forms, but a more broadly defined set of mappings including diagrammatic iconicity.
Diachronically, the relationship between language-internal motivations can flex and change. Language-internal mappings may overtake or subsume language-external mappings, which originally motivated the form–meaning pairing. A straightforward example of the internal versus external mapping argument can be seen in ASL signs such as DANCE, which is produced with an inverted V-handshape (Figure 1A). DANCE has been used in several studies as an example of a non-iconic sign, because when shown to a non-signer, the relationship between the form and meaning is not straightforward. However, as the signer learns more signs that involve the inverted V-handshape it may become clear that DANCE in fact participates in a network of constructions which use the same form–meaning mapping, such as the sign STAND (Figure 1B). These signs all involve the construal of a pair of legs mapped to a pair of extended fingers (Anible, Reference Anible2016). Thus, while on its own the sign DANCE may not be iconic for a signer, the network of constructions which make use of the mapping of legs-to-fingers strengthens language-internal associations which enhance diagrammatic iconicity.
Examples of ASL signs A) DANCE and B) STAND (signs used with permission and adapted from <www.spreadthesign.com>).
Likewise, newly construed language-external form–meaning mappings can arise through reanalysis creating iconic mappings from previously unmotivated signs. Note that these connections between forms are not necessarily dependent on morphological relations based in etymological histories or in faithfulness to original language-external motivation; they are based in perceived resemblances between forms in the language and between form and meaning pairings within words. This process can proliferate or extend iconicity where there previously was none, or where mappings were previously idiosyncratic. Thus, experience with a language increases speakers’ language-internal network of form–meaning mappings and attenuates reliance on language-external motivations. Iconicity in this way becomes part of the systematic organization of the language (Lepic & Padden, Reference Lepic, Padden, Bowern, Horn and Zanuttini2017).
In spoken language, the construct of construal has helped to elucidate the ways in which speaker perspective, viewing arrangements, or stance, are encoded in the grammar. For example, one’s proclivity to utter the phrase “the glass is half-full” or “the glass is half-empty” is really a matter of how their perspective is encoded, via lexical selection, as a means to indicate one of two different construals of the same situation. This particular aspect of construal, selection of what part of a scene to highlight, is called profiling. As Langacker points out, the simple observation of a glass containing water can be construed in multiple ways depending on which part of the scene is profiled: “(1) the glass with water in it designates the container; (2) the water in the glass designates the liquid it contains; (3) the glass is half-full designates the relationship wherein the volume occupied by the liquid is just half of its potential volume; and (4) the glass is half-empty designates the relationship wherein the volume occupied by the void is just half of its potential volume” (2008, p. 43). Whether one says “The tree behind the house” or “The tree next to the house” can also be a matter of construal, related to physical viewing arrangement, dependent on where one stands relative to the house and tree. The grammatical encoding of spatial relationships can and does depend on the relationship between the viewer and the addressee, both in terms of physical proximity, but also personal familiarity (whether one is speaking to their partner or their neighbor about said tree). Finally, a particular construal may be an indicator of a particular stance, for example, whether one selects the pronoun “I” or “we” as a marker of whether or not the speaker claims affiliation with a group.
While the discussion of construal generally focuses on grammatical constructions, construal operations can also apply to the production and perception of articulatory gestures (Occhino, Reference Occhino2016, Reference Occhino2017; Wilcox, Reference Wilcox2004). Construal of the articulators, or of the articulation itself, occurs in both signed and spoken modalities. A speaker of English, for example, can construe different parts of the articulatory gesture of [p] focusing on the bilabial closure or on the puff of air expelled upon release as corresponding to different event structures. These articulatory gestures can then be co-opted and cluster around concepts which might relate to quickness or release of pressure as seen in English sound-symbolic forms such as pop, ping, or pow. For signed languages, articulatory gestures are produced in plain sight, visible to the signer and the recipient; thus, articulatory events in the visual modality easily lend themselves to becoming objects of construal. An extended pointer finger makes an excellent slender stiff object and signers in fact use extended fingers to map to missiles, swords/knives, and screwdrivers, while a closed fist handshape makes an excellent compact object, such as a rock or, of course, a closed fist. The way in which signed language users map construal of articulatory forms to construal of the world is referred to as cognitive iconicity (Wilcox, Reference Wilcox2004), a concept that has been extended in Occhino’s work on Embodied Cognitive Phonology (2016, 2017) to include signed and spoken languages. The remainder of our paper focuses on handshape construal as it relates to signed languages and the ways in which construal interacts with iconicity.
Occhino (Reference Occhino2017) investigated cognitive iconicity of a single handshape in ASL, Claw-5 (see Figure 2). Traditionally, this handshape has been assumed to be phonologically equivalent across all signs in which it appears. However, across the ASL lexicon the Claw-5 handshape participates in multiple distinct but consistent form–meaning mappings based on individual construals of different features of the handshape paired with schematic construals of the world. In each of the construals, different viewing-arrangements of the hand are profiled leading to a many-to-one mapping of a set of construals related to a single handshape. Claw-5 can be construed, for example, as a curved surface, as in the ASL sign BALL (Figure 2A), as a whole hand as in the ASL signs TYPE-ON-KEYBOARD (Figure 2B) and PIANO (Figure 2C), or the extended fingers of the handshape can be construed as individual members of a bounded group, as seen in the ASL signs RAIN (Figure 2D) and SPIDER (Figure 2E). Whereas previous studies have appealed to characteristics of language form such as handshape salience and markedness to account for differences in the distribution and processing of handshape, a usage-based constructional approach requires one to consider form and meaning simultaneously.
Examples of ASL signs articulated with the claw-5 handshape: A) BALL, B) TYPE-ON-A-KEYBOARD, C) PIANO, D) RAIN, and E) SPIDER (signs used with permission and adapted from <www.spreadthesign.com>).
Here we introduce the construct of construal density to measure the relationship between a given form and the number of possible/probable motivated mappings. Thus, construal density provides a kind of type-frequency for the number of possible form–meaning mappings a particular handshape participates in. Another way to think of construal density is the number of constructions in a language which map different semantic schemas onto different phonological features of a given form. Signs like PIANO and SPIDER may be rated as ‘iconic’ by non-signers due to the ‘transparency’ of the relationship of form and meaning, because the articulatory form of both PIANO and SPIDER look very much like the movement of fingers on a keyboard or the legs of a spider moving (Caselli, Sehyr, Cohen-Goldberg, & Emmorey, Reference Caselli, Sehyr, Cohen-Goldberg and Emmorey2017). Importantly, even as these signs are more ‘transparent’, i.e., are clearly motivated by language-external mappings, as a whole they still participate in systematic language-internal networks of form and meaning which serve to strengthen their relationship to other signs sharing the same form–meaning mappings. The notion of construal of the articulators helps to elucidate how signers are able to conceptualize a single handshape as having multiple form–meaning mappings, while at the same time profiling, or highlighting, individual construals of the handshape in specific constructions.
In the current study, we test whether signers’ online processing of handshape is a function of both language proficiency and the number of different construals in which a handshape participates. In order to include language proficiency as an independent variable in a study of signed language processing, we selected a task that even novice signers would be able to carry out. Participants completed a phoneme monitoring task in which they merely needed to respond each time that the target handshape appeared in a signed stimulus. The phoneme monitoring task has been used to investigate a broad range of issues about spoken language processing (Connine & Titone, Reference Connine and Titone1996; Foss, Reference Foss1998; Howell & Bernstein Ratner, Reference Howell and Bernstein Ratner2018; Hutson, Damian, & Spalek, Reference Hutson, Damian and Spalek2013; Manoiloff, Segui, & Hallé, Reference Manoiloff, Segui and Hallé2016; Sobczak & Gaskell, Reference Sobczak and Gaskell2019). Evidence from the phoneme monitoring task demonstrates that listeners activate sublexical units while comprehending language online. Lexical factors such as word frequency can influence phoneme monitoring performance, but only when target phonemes are presented in isolated words, in non-words or in words that are embedded in syntactically or semantically incoherent utterances (Eimas & Nygaard, Reference Eimas and Nygaard1992). Because our questions concerned sublexical processing, we presented the target handshapes in short, coherent sentences so that they would be immune to lexical effects.
The phoneme monitoring task has also been used in two investigations of signed languages (Grosvald, Lachaud, & Corina, Reference Grosvald, Lachaud and Corina2012; Morford & Carlson, Reference Morford and Carlson2011). These studies were designed to investigate differences in perceptual processing by deaf and hearing signers and non-signers. Neither study found differences in the speed of handshape monitoring relative to signing experience or age-of-acquisition among signers with different backgrounds. However, both previous studies relied on isolated signs or nonce signs for stimuli, whereas full utterances are used in the current study. Proficiency differences may not be as easily detected in tasks that rely on single signs.
Grosvald et al. (Reference Grosvald, Lachaud and Corina2012) was the only study to include non-signers as a comparison group. They found that non-signers were slower than signers to detect handshape. They also report that participants detected marked handshapes faster than unmarked handshapes in one-handed signs, but they detected unmarked handshapes faster than marked handshapes in two-handed symmetrical signs. The same pattern was found for both signers and non-signers, suggesting that the study was actually tapping a non-linguistic dimension of handshapes such as perceptual complexity rather than markedness. The only difference other than speed between signers and non-signers in Grosvald et al.’s study was that signers tended to ignore the base hand in two-handed asymmetrical signs, whereas non-signers did not. In asymmetrical signs, the dominant hand performs an action on the non-dominant hand. The non-dominant hand in these signs is restricted to one of a few basic handshapes, such as the flat palm, or the fist. For this reason, we chose not to include any target signs in which the base hand alone was the phoneme target.
Thompson et al. (Reference Thompson, Vinson and Vigliocco2010) used a task that is very similar to phoneme monitoring to explore whether iconicity affects online processing of British Sign Language (BSL). Specifically, they asked signers to monitor the handshape features of straight vs. curved fingers in signs differing in iconicity. They found that iconicity slowed signers’ ability to detect handshape features. They also found no differences in sensitivity to iconicity between native and non-native signers, concluding that “although native signers were faster to respond overall, they did not exhibit a greater iconicity effect compared with non-native signers, suggesting that iconicity is not encoded differently between these two groups and that iconicity effects are not dependent on the age at which BSL is learned” (2010, p. 1022).
These findings are consistent with the view that iconicity is an objective characteristic of a signed form and that signers should exhibit uniform behavior with respect to the same sign irrespective of language proficiency and with respect to the relative place of that sign in its distribution with other signs in the language. By contrast, if iconicity is a construal process, then language proficiency and construal density, i.e., how many form–meaning construals a handshape participates in, should interact with iconicity to shape phonological processing of signs. We predict that signers will recognize handshapes that participate in fewer construals more rapidly than handshapes that participate in many construals. The reason for this prediction is that, as a handshape participates in more and more construals, learners detect more schematic patterns across the variety of usage events in which the handshape occurs. The more the semantics are attenuated across usage events, the more difficult it becomes to detect the handshape in a meaningful utterance. Proficiency will also impact handshape detection because novice L2 signers rely primarily on language-external motivations while experienced signers detect language-internal associations between signs as much or more than language-external motivations.
How can we distinguish between our view that iconicity is rooted in the construal of language form and language meaning from the more common view that iconicity is commensurate with transparency? If one takes the view that iconicity is an objective property of a sign which is able to be tapped into by people who are not users of a language, as is the assumption in many iconicity judgement tasks which rely on non-signers to rate the iconicity of signs, then one would predict that all individuals, regardless of language knowledge, will be influenced in the same way, if not to the same degree, by iconicity. Let us call this view iconicity as transparency and base the following set of hypotheses on these assumptions:
(H1) Iconicity as transparency
(1a) Proficient signers will be faster to detect handshapes than novice signers, but proficiency will not interact with iconicity.
(1b) Handshapes in iconic signs will be detected more slowly regardless of proficiency, consistent with the findings of Thompson et al. (Reference Thompson, Vinson and Vigliocco2010) for handshape features.
(1c) Because iconicity is objective, there will be no effect of construal density.
If, however, one rejects this view and instead assumes that iconicity is subjective and depends on language experience, and on the number of possible construal relationships between a given form and meaning, one will make an entirely different set of predictions. In our view of iconicity as construal, none of the predictions from H1 are tenable. In hypothesis two, construal density is a necessary measure for understanding the ways that proficient signers relate signs within networks of form–meaning mappings. While iconicity, operationalized as non-signer iconicity ratings, emphasizes language-external mappings between form and meaning, construal density captures the network of related form–meaning mappings within a signer’s grammar. Thus, in hypothesis two, construal density is predicted to interact with signer proficiency and iconicity:
(H2) Iconicity as construal
(2a) Proficient signers will be influenced by construal density more than iconicity because within language mappings become more salient than language-external mappings with increasing language experience.
(2b) Less-proficient signers will be influence by iconicity more than construal density because they will be less sensitive to language-internal patterns.
In sum we predict that proficiency in a language will shift a signer’s sensitivity from language-external to language-internal iconic mappings, and that a sign’s iconicity is dependent on the distribution of its phonological parameters in the language relative to other signs.
2. Method
2.1. participants
Twenty-eight participants (21 female, 7 male) were compensated either monetarily or with course credit. Participants with a broad range of signing ability were recruited. Half of the participants were hearing English–ASL bilinguals (12 female, 2 male), 21 to 47 years old (M = 31.4, SD = 11.2), and half were deaf ASL–English bilinguals (9 female, 5 male), 24 to 50 years old (M = 34.72, SD = 8.4). The deaf participants all reported a hearing loss of 80dB or greater in the better ear. ASL was a second language for all of the hearing participants, who were enrolled in a signed language interpreting program and who had completed at least one year of study of ASL. ASL was the dominant language of all of the deaf participants. Half of the deaf participants acquired ASL from their deaf parents; half acquired ASL from hearing parents or at school. ASL proficiency assessments are not widely available. In the past, investigators have often relied on participants to provide ratings of their own production and comprehension skills or have used parental hearing status as a proxy for ASL fluency (cf. Anible et al., Reference Anible, Twitchell, Waters, Dussias, Piñar and Morford2015). We used a direct measure of ASL proficiency, the ASL Sentence Reproduction Test, 20-item version (ASL-SRT; Hauser et al., Reference Hauser, Paludneviciene, Riddle, Kurz, Emmorey and Contreras2016; Hauser, Paludneviciene, Supalla, & Bavelier, Reference Hauser, Paludneviciene, Supalla, Bavelier and de Quadros2008), allowing us to enter language proficiency into our statistical model without relying on proxy variables such as parental hearing status. Participants were included with proficiency levels across the full range of the test. Only one participant received the lowest and the highest possible scores on the test, respectively, so floor and ceiling effects are not a concern in the distribution of participants’ signing proficiency. Scores ranged from 0 to 20 (M = 7.57, SD = 5.96). Hearing English–ASL bilinguals had scores ranging from 0 to 8 (M = 2.58, SD = 2.09). Deaf ASL–English bilinguals had scores ranging from 6 to 20 (M = 12.5, SD = 4.2).
2.2. materials
Four target handshapes were selected on the basis of their distribution in the ASL lexicon. Two handshapes (B, 1) were selected because they are highly frequent unmarked handshapes, and two other handshapes (F, V) were selected because they are less frequent marked handshapes (Lepic, Reference Lepic2015, p. 114). To estimate the construal density for each of these handshapes, we first identified the signs in the ASL-LEX database with these handshapes (Caselli et al., Reference Caselli, Sehyr, Cohen-Goldberg and Emmorey2017), and then coded each sign for handshape construal. For example, the signs BIOLOGY, BOOK, and PRAYER all are produced with the B-handshape, but the handshape is associated with different construals in each case. In BIOLOGY, the handshape maps to the letter B; in BOOK the handshape is construed as a flat object, and in PRAYER it is construed as a human hand (see Figure 3).
The ASL Signs for A) BIOLOGY, B) BOOK, and C) PRAYER are articulated using the B-handshape but construed as the letter B in BIOLOGY, a flat surface in BOOK, and as human hands in PRAYER (signs used with permission and adapted from <www.spreadthesign.com>).
After coding each sign for construal type, we calculated the total number of construals per handshape, giving us an estimated construal-density, a kind of type-frequency of how many different construals are mapped to a given handshape for the four target handshapes in the ASL-LEX database. See Table 1 for summary statistics of each handshape. For the remainder of the paper, we refer to stimuli with the first two target handshapes (B, 1) as the dense construal condition, and stimuli with the latter two target handshapes (F, V) as the sparse construal condition, to distinguish their distributional characteristics in the lexicon.
Distribution of target handshapes relative to meaning in ASL-LEX database
Twelve signs were selected from the core ASL lexicon (Brentari, Reference Brentari1998) for each handshape for a total of 48 target signs. Signs were restricted to one-syllable constructions with no hand-internal movement and no sign-internal hand configuration change. Most signs (65%) were two-handed – both symmetrical and asymmetrical – with the remaining targets consisting of one-handed signs. Not all the target signs have entries in an existing database of ASL iconicity ratings, such as ASL-LEX (Caselli et al., Reference Caselli, Sehyr, Cohen-Goldberg and Emmorey2017). Therefore, iconicity ratings were collected from non-signers using Mechanical Turk following equivalent methods.Footnote 1 Specifically, we asked workers, “How much does this sign resemble its meaning”, on a seven-point scale. These ratings ranged from 1.04 to 6.96 (M = 4.5, SD = 1.94), see the ‘Appendix’.
Target signs were embedded within short utterances produced by a native signer with four to six signs per sentence (see Figure 4). Foss (Reference Foss1969) found that phoneme monitoring times speed up as targets are moved to later words in an utterance. To control for possible effects of sentence position on monitoring times, the location of the target sign was manipulated such that it never occurred in sentence-initial position. More than half of the targets occurred after the midpoint in the sentence (n = 31) while 17 targets appeared before the midpoint of the sentence. Although including our target signs in sentential contexts introduces additional variation due to semantic and syntactic processing, the more naturalistic data generated by the task allows us to be more confident in extending task dependent conclusions to more general linguistic behavior.
Stimulus sentence for the target handshape F (present in the sign HAIR).
2.3. procedure
Participants viewed 192 (48 experimental, 144 filler) ASL sentences in four blocks, one for each handshape target, and responded with a keypress when they detected target handshapes. Participants were run on one of four versions of the experiment in E-prime (Psychology Software Tools, 2012) where the order of target handshapes was systematically varied using a Latin square design. In experimental sentences the target handshape occurred only once; 128 filler sentences did not include any instances of the target handshapes, and 16 filler sentences included a classifier construction with the target handshape so that participants would not be strategic in looking only for core lexical items. Participants were reminded before each block that the actual token of the handshape could vary in pronunciation slightly, but that they should nevertheless respond when they saw these natural variations of the target. Participants were subsequently shown pictures of target handshapes in addition to possible variations, such as extended thumbs, that could be found in that block (see Figure 5).Footnote 2 Each sentence was presented in succession, separated by a 1000ms transition slide. The transition slide was blank with a fixation cross in the center of the screen.
Target handshapes and variation instructions.
Response Time (RT) was measured as the amount of time from target sign onset until the response key was pressed. Since target signs were presented in sentential context, sign onset was measured as the point at which the hands transitioned out of the preceding sign’s handshape, as sign transitions can include coarticulatory cues to the next sign. In each experimental sentence, the timestamp of the target sign onset was calculated by two researchers with high agreement (93%).
Inaccurate trials, where the response key was not pressed or was pressed before the onset of the sign, were removed. Additionally, RTs that were faster than 200ms or slower than 5000ms were removed from the analysis as outliers because very fast and very slow RTs are often an indication of anticipatory responses or attentional lapses, respectively (Ratcliff, Reference Ratcliff1994). Five observations, less than 0.5% of the data, were trimmed in this way.
Mixed-effects models were estimated using the lme4 package version 1.1-20 (Bates, Mächler, Bolker, & Walker, Reference Bates, Mächler, Bolker and Walker2015). P-values were calculated using the lmerTest package version 3.1-0 (Kuznetsova, Brockhoff, & Christensen, Reference Kuznetsova, Brockhoff and Christensen2017). Data and R (R Core Team, 2019) code used for the analyses is available via the Open Science Framework at <osf.io/3gdm6>.
3. Results
We estimated a model for log-transformed RTs with interactions for iconicity ratings, construal density (dense vs. sparse), and proficiency (high vs. low, mean-split) as fixed effects. Both categorical variables were sum coded and iconicity ratings were centered and standardized. Random effects were estimated for participants and items as well as random slopes for iconicity and proficiency.Footnote 3 Table 2 presents the model estimates.
Main effects and interactions for Response Time (RT). Construal density sum coded as Dense = 1, Sparse = –1. Proficiency sum coded as Low = 1, High = –1. Iconicity z-scored.
There was a marginal main effect of construal. Signers were slower to detect handshapes that participate in dense construals than handshapes that participate in sparse construals. There was a significant three-way interaction of iconicity, construal, and proficiency. Starting first with the lower-proficiency group (Figure 6, left panel), novice signers were faster to detect handshapes when the signs were highly iconic. The handshapes in the dense construal condition were somewhat more difficult to detect than the handshapes in the sparse construal condition, but the ease of associating meaning and form via iconic mappings aided these developing signers in identifying the target handshapes in both conditions. By contrast, the more proficient signers show a different pattern. For these signers, iconicity was only helpful for the handshapes that participate in dense construals. High-proficiency signers were slow to detect the B- and 1-handshapes in signs that had been judged to be non-iconic (GOOD, HAPPEN), and faster to detect these handshapes in signs judged to be iconic by non-signers (BOOK, BRUSH-TEETH). Response times for high-proficiency signers to signs that participate in sparse construals were not affected by iconicity ratings (see Figure 6, right panel).
Three-way interaction of iconicity, construal, and proficiency on RT. RTs are log transformed and iconicity ratings are z-scored. Error bars show the standard error of the estimate.
An accuracy model with identical parameters to the RT model was also constructed, but no significant effects were found. Results mirror the RT results such that signers were generally more accurate under the same conditions in which they provided faster responses, indicating that there were no speed–accuracy trade-offs (Reed, Reference Reed1973).
4. Discussion
Our study provides new evidence that the processing of phonological form is influenced by iconicity. However, our results show that effects of iconicity are not uniform across language proficiency levels nor across handshapes. Many of the recent psycholinguistic investigations have operationalized the construct of iconicity as an inherent relationship between the physical form of a sign (or word) and the sign’s meaning, that is, a resemblance relationship which is equally accessible to all individuals, i.e., transparency. However, the complex interactions between iconicity, language proficiency, and construal density uncovered in the current study are better accounted for by acknowledging that iconicity is not an objective property of the lexical item, but is instead a cognitive construct, generated by individuals on the basis of their own construal of a specific form and a specific meaning. When signers in the current study viewed the signed sentences, their ability to detect the presence of a specific handshape was dependent on their construal of the handshape form relative to the meaning and on their proficiency in the language.
Consistent with prior studies (Ormel et al., Reference Ormel, Knoors, Hermans and Verhoeven2009; Thompson et al., Reference Thompson, Vinson and Vigliocco2010; Vinson et al., Reference Vinson, Thompson, Skinner and Vigliocco2015), our experiment replicated the finding that iconicity impacts phonological processing in signers. What is unique about the current findings is that participants were faster to detect handshapes within signs judged to be highly iconic, relative to signs judged to have lower levels of iconicity, despite the fact that they were making judgements about sign form. Many previous studies have found that psycholinguistic tasks requiring a semantic decision are facilitated by iconicity (Grote & Linz, Reference Grote, Linz, Müller and Fischer2003; Thompson et al., Reference Thompson, Vinson and Vigliocco2009). For example, Grote and Linz (Reference Grote, Linz, Müller and Fischer2003) found that DGS signers were faster to decide that a picture matched a sign (EAGLE) when the picture highlighted the iconic base of the DGS sign (eagle’s beak) than when the picture highlighted some other characteristic of the concept (eagle’s wings). However, in tasks that required participants to attend to phonetic features of the signs, such as whether the fingers are straight or bent, iconicity slowed performance (Thompson et al., Reference Thompson, Vinson and Vigliocco2010). Our study is the first to show faster performance for iconic signs even though participants were not required to process the stimuli for meaning. This discrepancy between findings may be a result of study design. While Thompson and colleagues presented signers with isolated signs, our current study was designed so that the stimuli were presented within full sentences. Our stimuli most likely engaged naturalistic language processing to a greater extent than prior studies. This suggests that, in the course of comprehending signed utterances, the activation of meaning does not interfere with making form-based decisions as proposed by Thompson and colleagues. Instead, the alignment of form and meaning within the signer’s conceptualization may aid the signer in becoming conscious of the signed form.
The effect of iconicity interacted with the proficiency level of the signers. Novice signers were more susceptible to the effects of iconicity than experienced signers. A simple explanation of this result would be to point out that adult learners take advantage of motivated form–meaning mapping during the acquisition of signs (Baus, Carreiras, & Emmorey, Reference Baus, Carreiras and Emmorey2013). However, if this alone explained why iconicity impacts processing, the more proficient signers should have shown little to no effect of iconicity on their performance. This was not the case. Instead, the experienced signers showed selective effects of iconicity whereas the novice signers showed more uniform effects of iconicity. This pattern suggests that increasing experience with a language leads to changes in signers’ perception of iconicity (cf. Occhino et al., Reference Occhino, Anible, Wilkinson and Morford2017). How is this shift in iconicity best explained?
Experienced signers show stronger effects of iconicity for handshapes that participate in many construals relative to handshapes that participate in fewer construals. On first reflection, this may seem counter-intuitive. If experienced signers show less sensitivity to iconicity overall, why do they show iconicity effects for handshapes that participate in many construals? Frequency is an acknowledged factor shaping language processing, but the frequency in question here is quite different from lexical frequency, which typically erodes phonetic form and semantic specificity (Börstell, Hörberg, & Östling, Reference Börstell, Hörberg and Östling2016; Bybee, Reference Bybee2001, Reference Bybee2006). As signers acquire a larger and more diverse vocabulary, they have a larger set of linguistic exemplars across which to identify regularities and patterns.
We propose that handshapes that participate in more construals are more likely to contain form–meaning mappings rooted in sensorimotor experiences which are then extended to different domains to create further mappings. But it is the sensorimotor schemas at the base of these mappings which make them unique, because mappings which are grounded in embodied processes maintain stronger connections to language-external sources of iconicity. Signs which have handshapes participating in less dense construal networks tend not to have their basic construals rooted in these embodied properties and thus are more subject to the pull of language-internal relationships between signs. Thus, the signs in less dense construals create tight-knit motivated mappings without a high degree of variation or schematicity within construal clusters because there are less possibilities for mappings. This lack of variation makes signs more similar to one another, which allows the signs to become subject to the pull of language-internal patterns.
Handshapes in dense construal networks, on the other hand, form more varied and loosely assembled clusters which contain many tokens of a variety of construal-types leading to more schematic associations between signs. However, the embodied sensorimotor nature of the mappings within these dense clusters are able to maintain strong language-external associations even after many usage events and high degrees of schematization. For instance, the flat-handshape (B handshape) will never stop being able to be easily mapped to a flat-object, and a fist-handshape (S handshape) will never stop being able to be easily mapped to a compact or round object, regardless of how dense the construal network for the handshape becomes.
In contrast to the current study, former handshape monitoring studies found no differences in the speed of handshape monitoring relative to signing experience or age-of-acquisition among signers with different backgrounds (Grosvald et al., Reference Grosvald, Lachaud and Corina2012; Morford & Carlson, Reference Morford and Carlson2011). The use of a direct measure of ASL proficiency rather than relying on self-reporting may have been an essential methodological change necessary to detect effects of proficiency on handshape monitoring. While our study found significant effects of proficiency, it was only in interaction with construal density and iconicity. This result is consistent with our proposal that construal density taps a signer’s sensitivity to language-internal form–meaning mappings which are dynamic and undergo change with increasing language experience.
While this study focuses on handshape, the multidimensional nature of iconicity explored here should not be restricted to the shape of the hand relative to lexical meaning. Lepic, Börstell, Belsitzman, and Sandler (Reference Lepic, Börstell, Belsitzman and Sandler2016) provide evidence that construal of the hands, as a dual set of articulators, maps onto semantic construals of dual entities such that two-handed signs can encode lexical plurality and Figure–Ground relations by simultaneously foregrounding and backgrounding information. Börstell, Lepic, and Belsitzman (Reference Börstell, Hörberg and Östling2016) confirm that this finding holds across ten signed languages from five different language families. Most recently, Östling, Börstell, and Courtaux (Reference Östling, Börstell and Courtaux2018), surveyed 120,000 sign videos in 31 different sign languages, and found that this pattern extends well beyond these initial studies. Johnston and Schembri (Reference Johnston and Schembri1999) have shown that two-handed articulation can also be construed as intensification. The phenomenon of languages utilizing the construal of more form equals more meaning has been well documented in the case of spoken languages as well (Haiman, Reference Haiman1980; Lakoff & Johnson, Reference Lakoff and Johnson1980; Moravcsik, Reference Moravcsik and Greenberg1978; Sapir, Reference Sapir1921). Importantly, these consistencies of duality and doubling of form to indicate increased meaning demonstrate that effects of iconicity that are phonological in nature are not restricted to either the auditory or visual modality.
Returning to the central focus of this investigation, this study raises several important considerations for the study of iconicity in language processing despite the preliminary nature of the findings. The results demonstrate that iconicity is not an objective, fixed, or unidimensional property of a given sign. Iconicity interacts with proficiency (experience of the signer) and construal density (distribution of form and meaning across the lexicon). This finding supports the notion that iconicity is not objective, but that it is affected by an individual’s experience with a specific language. Iconicity is also a dynamic, rather than fixed feature of linguistic units and exists on both lifetime and moment-to-moment timescales. Spoken language research has documented similar findings regarding speakers’ varying relationships with iconic form–meaning mappings across timescales of acquisition. For example, Kantartzis (Reference Kantartzis2011) found that child and adult speakers of English and Greek had different sensitivities to language-specific and cross-linguistic patterns of sound symbolism, suggesting that adults may lose their sensitivity to the more ‘universal’ sound symbolisms as language-specific patterns take over. Imai and Kita (Reference Imai and Kita2014) have also discussed the development of universal versus language-specific sound symbolism across a variety of spoken languages, which echo the points raised here that not all relationships between form–meaning units should be treated as the same. These studies suggest that form–meaning mappings are directly affected by language use and have the potential to change over time.
The existence of iconicity in grammars may provide different processing advantages at different stages of proficiency. Language users detect both language-internal and language-external iconic patterns. Hence, we find that handshapes are processed differently depending on their distribution in a language. Handshapes participating in fewer construals are highly salient to signers regardless of the iconicity of the signs they are used in, while handshapes that participate in many construals become salient only in iconic signs.
Given the innovative nature of the study, alternative explanations of our findings should be explored in future work. Our operationalization of construal density, for example, has yet to be used in other psycholinguistic studies of signed language processing. More work is also needed to examine the extent to which construal density interacts with other linguistic measures such as phonological neighborhood density, which is based on formal characteristics of the sign, and iconicity ratings, which focus on the meaning relative to the form. What is clear is that looking at form and meaning together requires new ways of designing and operationalizing our parameters for psycholinguistic experiments, but taking on this challenge has the potential to offer new insights into old problems.
Appendix
Target signs and their iconicity estimates
