A greatly debated topic in bilingual research is whether the two languages in the mental lexicon of bilinguals operate as separate systems (Gerard & Scarborough, Reference Gerard and Scarborough1989) or in an interactive manner (Dijkstra, Grainger & van Heuven, Reference Dijkstra, Grainger and van Heuven1999; Dijkstra, van Heuven & Grainger, Reference Dijkstra, van Heuven and Grainger1998; de Groot, Delmaar & Lupker, Reference Groot, Delmaar and Lupker2000; Kroll, Bobb & Wodniecka, Reference Kroll, Bobb and Wodniecka2006; Kroll & Stewart, Reference Kroll and Stewart1994). The evidence points in the direction of the latter (van Hell & Dijkstra, Reference Hell and Dijkstra2002; Marian & Spivey, Reference Marian and Spivey2003). Recognition of words in one language is affected by lexical knowledge in the other (Dijkstra, van Jaarsveld & ten Brinke, Reference Dijkstra, van Jaarsveld and Brinke1998; Gollan, Forster & Frost, Reference Gollan, Forster and Frost1997). The languages used to study cross-language interaction have been languages with varying degrees of orthographic and phonological overlap (Bijeljac-Babic, Biardeau & Grainger, Reference Bijeljac-Babic, Biardeau and Grainger1997; de Groot et al., Reference Groot, Delmaar and Lupker2000; van Hell & Dijkstra, Reference Hell and Dijkstra2002; van Heuven, Dijkstra & Grainger, Reference Heuven, Dijksta and Grainger1998; Kandil & Jiang, Reference Kandil and Jiang2004; Lemhöfer, Dijkstra & Michel, Reference Lemhöfer, Dijkstra and Michel2004; Wang, Koda & Perfetti, Reference Wang, Koda and Perfetti2003; van Wijnendaele & Brysbaert, Reference Wijnendaele and Brysbaert2002). Indeed, Thierry and Wu (Reference Thierry and Wu2004; also Wu & Thierry, Reference Wu and Thierry2010) report cross-language interaction in two languages (Chinese and English) with minimal overlap. The present study examines interaction where the languages in question are produced in different modalities without any sharing of orthography and only minimal sharing of phonology. We asked: Does cross-language interaction occur for deaf children who use a sign language and who can read Dutch?
Sign languages are the natural languages of deaf people and have their own grammars. Dutch and Sign Language of the Netherlands (NGT), for example, are rather independent of each other and have fundamentally different grammars. In contrast to most spoken languages, sign languages also do not have an accompanying written system (Evans, Reference Evans2004; Padden & Ramsey, Reference Padden and Ramsey1998). This means that there is no orthographic overlap between a signed language and the written form of a spoken language with the exception of a set of finger-spelled loan words and initialized handshapes. Initialized handshape refers to the use of a finger-spelled letter as the handshape for a sign, which represents the first letter of the word translation equivalent. Furthermore, only minimal phonological overlap between signed and spoken languages is possible because signed languages rely upon visual and spatial information (although a number of signs are accompanied by specific mouth movements that relate to a spoken word). Hearing and deaf people who know a sign language and who can read therefore provide an excellent venue for the study of interactive cross-language activation. Recently, in several studies with bimodal deaf adults, cross-language written word-recognition effects have been observed (Grote & Linz, Reference Grote, Linz, Muller and Fisher2003; Morford, Wilkinson, Villwock, Piñar & Kroll, Reference Morford, Wilkinson, Villwock, Piñar and Kroll2011). The main question in the present study is whether or not signs and sign elements are activated during the written word recognition of deaf children, whose proficiency in either language is still developing. The answer to this question is important given the major implications this may have for deaf children's literacy development.
Interactive bilingual language access in deaf individuals
Studies of possible cross-language effects during the written word recognition of bimodal bilinguals are limited (Grote & Linz, Reference Grote, Linz, Muller and Fisher2003; Hanson & Feldman, Reference Hanson and Feldman1989, Reference Hanson and Feldman1991; Morford et al., Reference Morford, Wilkinson, Villwock, Piñar and Kroll2011; Treiman & Hirsh-Pasek, Reference Treiman and Hirsh-Pasek1983). Treiman and Hirsh-Pasek (Reference Treiman and Hirsh-Pasek1983) assessed the activation of signs during the reading of English sentences by native deaf adult signers. They found some activation of sign during sentence processing but no activation of the spoken language phonology or finger spelling.
Two decades later, Grote and Linz (Reference Grote, Linz, Muller and Fisher2003) investigated the effects of sign iconicity. They used a task in which deaf and hearing signers judged the semantic relatedness of stimulus pairs (a semantic relatedness judgment task). Pairs of pictures and written words and pairs of pictures and signs were presented on a computer screen, and participants were instructed to decide whether or not the items in the pairs were semantically related. Picture–sign pairs were presented to deaf participants and hearing participants who knew sign language (i.e., bimodal participants). Picture–word pairs were presented to non-signing hearing participants and to bimodal hearing participants. Most interestingly, not only within-language effects in the picture–sign task but also cross-language effects in the picture–word task were found for the bimodal hearing participants. Responses were faster when the iconic features of the sign translation equivalent of the word were visible in the picture (for example, the word EAGLE and a picture of the beak of an eagle, where the sign for eagle in German Sign Language (DGS) shows the shape of a beak of an eagle) in comparison to the same word EAGLE in combination with a picture that does not show the iconic features of the sign translation (for example, the word EAGLE and a picture of a wing of an eagle).
Recently, Morford et al. (Reference Morford, Wilkinson, Villwock, Piñar and Kroll2011) examined cross-modal bilingual effects during written word recognition in a group of deaf bilingual adults. In a semantic relatedness judgment task, pairs of written words were presented sequentially on a computer screen and participants decided whether or not the words were semantically related. For semantically similar word pairs and semantically dissimilar word pairs, half of the items had phonologically related American Sign Language (ASL) translation equivalents and half had phonologically unrelated ASL translation equivalents. For the semantically related pairs of words, those with phonologically related ASL translation equivalents were responded to faster than those with no such ASL phonological relation. For the semantically unrelated pairs of words, those with a phonologically related ASL translation equivalent were responded to slower than those with no such phonological relation. The results thus showed that deaf adults activated ASL translation equivalents when asked to process pairs of written English words (but see Hanson & Feldman, Reference Hanson and Feldman1989, Reference Hanson and Feldman1991, who found no evidence of cross-language activation of sign translation equivalents during the reading of English word primes followed by English target words by deaf adults).
In sum, the above findings provide evidence for cross-language influences of signs during the comprehension of written words. Treiman and Hirsch-Pasek (Reference Treiman and Hirsh-Pasek1983) provided evidence with regard to the written sentence processing of deaf adults, Grote and Linz (Reference Grote, Linz, Muller and Fisher2003) provided evidence for cross-language activation (sign iconicity) with regard to written word processing by hearing adult signers, and Morford et al. (Reference Morford, Wilkinson, Villwock, Piñar and Kroll2011) provided evidence for cross-language activation (sign phonology) with regard to written word processing by deaf adults.
The present study concerning deaf children is built upon the results obtained in a study by Ormel, Hermans, Knoors and Verhoeven (Reference Ormel, Hermans, Knoors and Verhoeven2009) on sign recognition by deaf children. In that study, we found that at least two elements of signs, namely sign iconicity and sign phonology, affected the recognition of sign language. In the present work, we examine whether the same two elements may affect the written word recognition of bilingual deaf children in a cross-language interactive manner, in line with the findings for deaf and hearing adults (Grote & Linz, Reference Grote, Linz, Muller and Fisher2003; Morford et al., Reference Morford, Wilkinson, Villwock, Piñar and Kroll2011).
In all sign languages, the individual signs are composed of parameters, which thus constitute the sign phonology (e.g., Klima & Bellugi, Reference Klima and Bellugi1979; Stokoe, Reference Stokoe1980). The most relevant parameters include handshape, movement of the hands and arms, location of the hands relative to the body, and hand-palm orientation. In Figure 1, two sign pairs in NGT are presented. The first pair shows extensive phonological overlap. The second pair shows no phonological overlap. In the first pair (DOG–CHAIR),Footnote 1 there is overlap for location (neutral), overlap for hand-palm orientation (downward), and large overlap in movement (straight line downward, although the movement in DOG is repeated) but a distinct handshape (open B-hand versus closed S-hand). In the second pair (DOG–COMB), there is a distinct location (neutral versus head), a distinct hand-palm orientation (downward versus sideways), a distinct handshape (open B-hand versus closed “money-hand”) and a partly distinct movement (straight line downward, although only in DOG, the movement is repeated).
Examples of strong and no sign phonological relatedness.
The parameters of handshape, hand-palm orientation, movement and location thus help distinguish signs in a manner similar to how the English phonemes /b/ and /w/ distinguish the spoken words ball and wall. Many models of spoken and visual word recognition assume that the presentation of a spoken or written word (e.g., dog) leads to the activation of its sublexical (i.e., phonological or orthographic) features (/d/, /o/ and /ɡ/). These sublexical features may automatically activate related words (e.g., dog, doll), and these words can compete for selection during the word recognition process. In other words, it is assumed in many models of spoken and written word recognition, including the Cohort model of lexical access, that word recognition can be construed as a process of competition in which a cohort of phonologically related words compete for selection as the spoken word information becomes available (e.g., Gaskell & Marslen-Wilson, Reference Gaskell and Marslen-Wilson2002; Zwitserlood, Reference Zwitserlood1996). Similarly, for sign language, the recognition of a sign may start with the activation of various elements (i.e., handshape, hand-palm orientation, movement and location) and several competing lexical candidates (i.e., neighbour signs) may be activated as a result (see also Clark & Grosjean, Reference Clark and Grosjean1982; Emmorey & Corina, Reference Emmorey and Corina1990). This is how we explained part of our results in the Ormel et al. (Reference Ormel, Hermans, Knoors and Verhoeven2009) study. Participants in the Ormel et al. study were shown signs and pictures that had overlapping phonological elements (i.e., neighbour status) but did not refer to the same concept (e.g., DOG–CHAIR in NGT). The participants then had to reject the sign–picture pair as not referring to the same concept (i.e., provide a “no” response) in a sign–picture verification task. Extended sign phonology overlap was expected to create an interference effect, as once the target sign has been activated, its sublexical sign phonology also becomes activated (i.e., movement, handshape, location and orientation information). Such sublexical sign information can simultaneously activate a cohort of sign neighbours (i.e., signs with overlapping sublexical elements). The sign for DOG, for example, can correctly activate DOG but also CHAIR – amongst other candidates – because the signs for DOG and CHAIR share a number of NGT elements but not the same handshape. As expected, those sign–picture pairs with considerable phonological overlap underlying them led to slower and less accurate responses on the part of the deaf children. Thus, the manipulation of sublexical sign phonology showed that lexical competition occurs during sign recognition, which is initiated by a match with active sublexical phonological units.
A second critical element for the recognition of sign language is sign iconicity, which refers to the mapping between the form of the sign and its meaning; the so-called level of transparency (see Ormel et al., Reference Ormel, Hermans, Knoors and Verhoeven2009, for effects of sign iconicity during sign recognition by deaf children). In spoken languages, onomatopoeia is a classic example of iconicity (Pietrandrea, Reference Pietrandrea2002). Just as in a spoken language, most of the relations between the form of a sign and its meaning are arbitrary for sign language. There are, however, signs that allow their meaning to be retrieved more or less directly from their form (van der Kooij, Reference Kooij2002), for example the NGT sign for “house”, which shows the prototypical shape of the roof of a house.
Sign–picture pairs were also used in the aforementioned study by Ormel et al. (Reference Ormel, Hermans, Knoors and Verhoeven2009) to examine whether sign iconicity plays a role in the overt processing of signs by deaf elementary school children. Extended sign iconicity was expected to create a facilitation effect and thus make it easier to respond “yes” to strongly iconic pairs of items. The results in the 2009 study showed strongly iconic signs to indeed elicit faster and more accurate responses than weakly iconic signs and thus a facilitatory effect of sign iconicity for the deaf children.
Bilingual deaf children present a unique opportunity to study bilingual language processing in children (see e.g., Bialystok, Luk and Kwan, Reference Bialystok, Luk and Kwan2005, for bilingual processing in hearing children). Given Ormel et al.'s (Reference Ormel, Hermans, Knoors and Verhoeven2009) finding that sign phonology and sign iconicity affect sign recognition among deaf children, the question we ask here is whether these aspects of lexical signs are also activated cross-linguistically during written word recognition.
The present study
In the Netherlands, deaf children usually grow up in a joint sign language/sign supported spoken Dutch language environment with mostly hearing parents. In the vast majority of cases, NGT is the more natural and accessible language for the children (Klatter-Folmer, van Hout, Kolen & Verhoeven, Reference Klatter–Folmer, van Hout, Kolen and Verhoeven2006; Knoors, Reference Knoors2007). The question is whether or not such knowledge of NGT possibly affects their processing of written Dutch. As mentioned before, the critical assumptions for the current study are based on the Sign–Picture Verification Task administered to deaf children by Ormel et al. (Reference Ormel, Hermans, Knoors and Verhoeven2009). In the present study, the question is whether these same sign elements (sign phonology and sign iconicity) are activated in a Word–Picture Verification Task administered to deaf children. Both deaf and hearing elementary school children were asked to make “yes/no” decisions with regard to whether the items in printed word–picture pairs presented to them on a computer screen referred to the same concept or not. Extended overlap between the underlying NGT translation equivalents of the items is expected to create an interference effect for (the rejection of) conceptually unrelated pairs. That is, the “no”-responses to simultaneously presented conceptually unrelated word–picture pairs, for which the underlying NGT (translation) equivalents are phonologically similar, could be expected to be slower and less accurate than the responses to simultaneously presented word–picture pairs for which the underlying NGT (translation) equivalents are not phonologically similar. Similarly, highly iconic NGT translation equivalents are expected to produce a facilitation effect (in the form of faster and more accurate responses) for (the acceptance of) conceptually identical pairs when compared to weakly iconic translation equivalents.
Method
Participants
A total of ninety-eight hearing children and forty deaf children participated in the present study. In the lower grades, fifty hearing and twenty deaf children were included. In the higher grades, forty-eight hearing and twenty deaf children were included (see Table 1). The hearing children were in grade 3 (mean = 8.9 years, SD = .5) and grade 5 (mean = 11.0 years, SD = .4). The hearing children attended one of two elementary schools in the Netherlands, and none of the hearing children were familiar with NGT. The age of the deaf children ranged from 9.1 to 13.1 years. The younger deaf children were in either grade 3 or 4Footnote 2 and between 9.1 and 10.11 years (mean = 10.2 years, SD = .6). The older deaf children were in grades 5 or 6 and between 11.0 and 13.1 years (mean = 12.0, SD = .7). All deaf children had a hearing loss of more than 80 dB in the better ear. The deaf children attended one of three schools for deaf education in the Netherlands. All of these schools provided bilingual deaf education with a curriculum that consisted of a combination of NGT and Sign Supported Dutch (SSD). In SSD, Dutch word order is used with the support of signs. The deaf children had been taught NGT from the age of four years at school. Prior to that age, many of the children had already attended a preschool for deaf children and thus interacted with caregivers who used sign language from the age of two or three. Formal exposure to written Dutch started at the age of four years.
Participant characteristics.
Note: Given that group numbers were small in the schools for the deaf children, the deaf children in grades 3 and 4 as well as in grades 5 and 6 were combined. The hearing children were included only to verify the non-existence of sign effects. Matching for age was for that reason not crucial.
Stimuli
The pictures originated from the Dutch Leesladder [Reading Ladder] (Irausquin & Mommers, Reference Irausquin and Mommers2001), which is a computer program for children with reading disabilities. The pictures were coloured 6 cm × 6 cm line drawings representing nouns and presented on the right side of the computer screen; the words were presented on the left side of the computer screen.
Stimulus selection. The experimental stimuli were established on the basis of two instrument design studies. The first study involved a Phonological Similarity Judgment Task in which signs had to be judged for their degree of phonological similarity. The second study involved an Iconicity Judgment Task in which signs had to be judged for their degree of sign iconicity. The signs were all part of the standard NGT lexicon. The (translation) equivalents of the signs were used in the Word–Picture Verification Task.
The stimuli for the Phonology Similarity Judgment Task used in the first instrument design study were 121 pairs of signs with different degrees of formational parameter overlap (i.e., overlap in location, movement and/ or handshape). The Dutch translation equivalents for the sign pairs were largely unrelated with respect to orthography, phonology and meaning. The signs were presented on a computer screen to three deaf and six hearing bimodal bilingual adults. Three of the six hearing participants were employed as teachers of NGT. The other three of the hearing participants were in their final year of training to become sign language interpreters in The Netherlands. The participants were asked to judge, as rapidly as possible, the extent to which two consecutively presented signs were similar in form along a 1–7 point rating scale. The sign items were presented randomly and varied with regard to the degree of overlap in their form (i.e., phonology). Based upon the similarity judgments of the nine participants, twenty-four pairs of items with a large degree of phonological overlap (minimal pairs overlapping for at least two of the formational parameters) were selected for inclusion in the Word–Picture Verification Task. In the second instrument design study, which involved the Iconicity Judgment Task, we demonstrated that hearing children without any knowledge of sign language could indeed recognize the meanings of strongly iconic NGT signs (Condition 1) better than the meanings of weakly iconic NGT signs (Condition 2). Thirty-one hearing elementary school children from grades 4, 5 and 6 in the Netherlands with an age of ten to twelve years were administered the Iconicity Judgment Task. Twenty-four strongly iconic signs and twenty-four arbitrary signs (judged by the experimenters) were presented on a computer screen. The iconic properties of the signs resembled form features of the referent (e.g., signs for MOUNTAIN [berg], MOON [maan] and HOUSE [huis] have strongly iconic properties, whereas signs for FRUIT [ fruit], MEAT [vlees] and CLOCK [klok] do not have any strong iconic properties). One second after the sign offset, four words appeared on the screen simultaneously. The participants were to determine which of the four words referred to the presented NGT sign. Accuracy rates were computed separately for each participant for both test conditions. The expectation was that if the iconic signs were indeed iconic, the meaning of those signs could be derived more easily than for arbitrary signs for non-signers. This was confirmed by the hearing children. A significant effect was obtained for sign iconicity (t(46) = 4.76, p < .001: Strong iconicity condition, M = 76.34%; Weak iconicity condition, M = 44.13%). Based upon the outcomes of the Iconicity Judgment Task, twenty-four translation equivalents for strongly iconic signs were accepted for inclusion in the Word–Picture Verification Task. In the present study, “sign iconicity” refers to the iconicity of the sign translation equivalent for the Dutch word presented; “sign phonology” refers to the degree of phonological similarity between the sign translation equivalent for the Dutch word presented and the sign for the picture presented in each pair.
Stimulus conditions. In the Word–Picture Verification Task, 192 word–picture pairs were presented: 50% of the word–picture pairs were conceptual matches and thus required a “yes” response while 50% of the pairs were conceptual mismatches and thus required a “no” response. The stimulus items were distributed across six conditions reflecting the strengths of three factors: Sign Iconicity (Condition 1: Strong sign iconicity, conceptual match items; Condition 2: Weak sign iconicity, conceptual match items; see Figure 2), Sign Phonology (Condition 3: Strong sign phonological relation between two underlying signs, conceptual mismatch items; Condition 4: No sign phonological relation between two underlying signs, conceptual mismatch items, see Figure 3), and Semantics (Condition 5: Strong semantic relation, conceptual mismatch items; Condition 6: No semantic relation, conceptual mismatch items, see Figure 4). The Semantics conditions were included only to verify that the children were able to perform a word–picture verification paradigm, by showing expected semantic relatedness effects. Semantic relatedness was expected to create an interference effect (Damien & Bowers, Reference Damien and Bowers2003). The presentation of a spoken or written word (e.g., dog) leads to the activation of semantic features, and these features may automatically activate semantically related words (e.g., cat, horse). The latter words can then compete for selection during the word recognition process. In other words, deaf children are expected to be slower and less accurate in the rejection of semantically related as opposed to unrelated Dutch word–picture pairs. Strong semantically related items were pairs within the same semantic category such as “spoon” (word) and “fork” (picture) or “bean” (word) and “carrot” (picture).
Level of sign iconicity. The participants were shown a word (e.g., the word “house” or “fruit”) and a picture (e.g., of a house or of fruit).
Sign phonological relatedness. Participants were shown a word (e.g., the word “dog”) and a picture (e.g., of a chair or a comb).
Semantic relatedness. The participants were shown a word (e.g., the word “spoon”) and a picture (e.g., of a fork or a purse).
Each of the six conditions contained twenty-four unique word–picture combinations (see Table 2). In each of the three factors (Sign Iconicity, Sign Phonology, Semantics) different stimuli were included.
Experimental design of the Word–Picture Verification Task.
In the Sign Iconicity conditions, the pictures and therefore the twenty-four unique pairs were repeated once, which resulted in forty-eight items for Condition 1 and forty-eight items for Condition 2, or ninety-six pairs referring to the same concept (e.g., the word for “house” and a picture of a “house”). The rationale behind the repetition of the twenty-four items in conditions 1 and 2 was based upon the design of conditions 3 through 6, which also involved repetition of the words and pictures. The Sign Phonology and Semantics conditions 3 through 6 each involved twenty-four pairs that referred to different concepts (e.g., the word for “dog” and a picture of a “chair”). For example, Condition 3 contained mismatch pairs for which the underlying signs showed strong sign phonological relations. Condition 4 was constructed by recombining the words and pictures from Condition 3 in such a manner that the underlying NGT translation equivalents would be phonologically unrelated. In Condition 3, for example, the Dutch word for “dog” was combined with a picture of a “chair” (the sign equivalents were phonologically related); similarly, the Dutch word for “uncle” was combined with a picture of a “comb” (the sign equivalents were phonologically related). In Condition 4, sign phonologically unrelated pairs were created via the combination of the word for “dog” with a picture of a “comb”, for example, or the combination of the word “uncle” with a picture of a “chair”.
The same procedure of recombining words and pictures was used for Condition 5 and Condition 6. The total number of word–picture pairs that referred to different concepts was ninety-six.
Set of experimental items. The set of experimental pairs presented to the children consisted of four sets of stimulus pairs preceded by a practice set of ten pairs that were different from the stimulus pairs. Each of the four experimental sets contained four additional practice pairs, followed by twelve pairs from Condition 1 (strong sign iconicity: conceptual match) and twelve pairs from Condition 2 (weak sign iconicity: conceptual match) or twenty-four pairs requiring a “yes” response; six pairs from Condition 3 (strong sign phonological relation: conceptual mismatch), six pairs from Condition 4 (no sign phonological relation: conceptual mismatch), six pairs from Condition 5 (strong semantic relation: conceptual mismatch), and six pairs from Condition 6 (no semantic relation: conceptual mismatch) or twenty-four pairs requiring a “no” response. A total of four practice trials and forty-eight unique word–picture pairs thus constituted each experimental set. We programmed the task in such a manner that the same pictures or words did not occur within the same set of fifty-two pairs.
While sign frequency measures are not yet available for NGT, the Dutch word orthography frequency measures could be determined on the basis of CELEX counts (Baayen, Piepenbrock & van Rijn, Reference Baayen, Piepenbrock and van Rijn1993) and the child database by Schrooten and Vermeer (Reference Schrooten and Vermeer1994). We did not compare performance in the Sign Phonology condition directly to Semantics or Sign Iconicity conditions, which meant that word properties could differ for the three factors.
The words corresponding to the pictures in Condition 1 had an average of 1.59 log frequency per million based upon CELEX measures and an average frequency of 22.2 per 15,000 according to Schrooten and Vermeer. The mean length was 4.66 letters. In Condition 2, the words corresponding to the pictures had an average of 1.52 log frequency per million based upon CELEX measures and an average frequency of 22.7 per 15,000 according to Schrooten and Vermeer. The mean length was 5.21 letters. For both the word frequencies and word lengths, the differences were not significant: for the log frequencies (F < 1); for the length (F(1,48) = 1.6, p > .1).
The recombination of the words and pictures in Conditions 3 and 5 to create unrelated pairs in Conditions 4 and 6, respectively, produced identical word orthography frequencies and word lengths for Conditions 3 and 4, and Conditions 5 and 6, respectively. In Conditions 3 and 4, the words corresponding to the pictures had a combined average of 1.52 log frequency per million based upon CELEX measures and an average frequency of 18.9 per 15,000 according to the children's corpora of Schrooten and Vermeer.Footnote 3 The mean length was 5.46 letters. In Conditions 5 and 6, the words corresponding to the pictures had an average of 1.53 log frequency per million based upon CELEX measures and an average frequency of 38.8 per 15,000 according to Schrooten and Vermeer. The mean length was 4.31 letters (see Table 3). The words were presented using an Arial size 36 font.
Word characteristics measured by mean word length (number of letters), frequencies based on the CELEX database and frequencies based on the Schrooten and Vermeer database.
Notes: 1 Log frequency per million.
2: Frequency per 15,000 words.
Design
The order of presentation was constructed using a Latin Square design. Within each of the four stimulus sets, the forty-eight pairs were presented in a random order for each child.
Apparatus
In this experiment, a laptop, a Dell Latitude 640, was used. The test was constructed using the commercially available software program E-Prime, version 1.0 (Schneider, Eschman & Zuccolotto, Reference Schneider, Eschman and Zuccolotto2002).
Procedure
The children's teachers provided the instructions for the task on a class basis in NGT for the deaf children and spoken Dutch for the hearing children. After instruction, questions could be asked about the procedure. The experiment was then conducted with groups of six children in a separate, well-lighted room with two experimenters present. The distance between the children and the laptop screen was approximately 40 cm. During instruction, the participants were informed that a fixation point would appear on the screen for one second, followed by a word on the left side of the screen and a picture on the right side of the screen. The picture and the word appeared simultaneously and remained visible for the same amount of time; both stimuli disappeared after the participant responded or after a period of 10 seconds and the next item followed. When both the word and picture referred to the same concept, a match response was required and the respondent had to press the “Enter” button, which had a green mark on it. When the word and picture did not refer to the same concept, a mismatch response was required and the respondent had to press the “Caps Lock” button, which had a red mark on it. The deaf participants were instructed similarly to the hearing children, namely to provide a match response when the word and picture referred to the same concept, and without reference to sign translation. After the instructions were provided and understood by the participants, a practice set of ten items was presented. The Dutch word for “break” then appeared on the screen to indicate the start of a self-paced break; the participants could continue by pressing one of the response buttons. After every fifty-two items (i.e., four practice pairs plus forty-eight stimulus pairs), a self-paced break again occurred. Upon completion of the four sets of stimuli, the Dutch word for “End” appeared.
Results
The deaf children were our main interest, but an overall analysis of the deaf and hearing children's responses was preferred in order to show that deaf and hearing children indeed performed differently on Sign Phonology and Sign Iconicity conditions. The Levene's test of Equality of Error, which tests the null hypothesis that the error variance of the dependent variable is equal across groups, showed significantly different standard deviations for the deaf versus hearing children. Both Sign Iconicity conditions showed significant differences in the error variance for the number of errors produced by the deaf versus hearing children (Condition 1: F(1,136) = 7.80, p = .006; Condition 2: F(1,136) = 14.04, p = .000). Both Sign Phonology conditions also showed significant differences in the error variance for the number of errors produced by the deaf versus hearing children (Condition 3: F(1,136) = 5.88, p = .017; Condition 4: F(1,136) = 9.56, p = .002).
Keeping in mind the Levene's test results, the interactions between Hearing Status (deaf vs. hearing) and Sign Iconicity, Sign Phonology and Semantics, respectively, were nevertheless analyzed at this stage in the study in order to gain insight into whether the responses of the deaf versus hearing children differed. Significant interactions were indeed found between Hearing Status and all of the conditions. The results of an overall ANOVA (GLM) analysis of the response times showed a significant interaction between Hearing Status and Sign Iconicity (“yes” response) (F1(1,136) = 11.62, p < .01, η2p = .08; F2(1,46) = 24.34, p < .001, η2p = .35), between Hearing Status and Sign Phonology (“no” response) (F1(1,136) = 4.71, p < .05, η2p = .03; F2 < 1), and between Hearing Status and Semantics (“no” response) (F1(1,136) = 6.06, p < .05, η2p = .04; F2 < 1).
The results of an overall ANOVA analysis of the error data again showed a significant interaction between Hearing Status and Sign Iconicity (F1(1,136) = 10.33, p < .01, η2p = .07; F2(1,23) = 17.29, p < .001, η2p = .27) and between Hearing Status and Sign Phonology (F1(1,136) = 5.28, p < .05, η2p = .04; F2(1,23) = 1.29, p > .1, η2p = .05). No significant interaction occurred between Hearing Status and Semantics for the error data, however (F1 < 1; F2 < 1).
The responses of the deaf and the hearing children were further examined. Hearing children, who are not familiar with sign language, should not show any sign effects in the Word–Picture Verification Task.
The responses to the 192 word–picture pairs were analyzed in an ANOVA. The results for the hearing children are summarized in Table 4, and the results for the deaf children are summarized in Table 5. For each participant, the mean response time to the correct responses (RT) and error scores were computed for the three factors: Sign Iconicity (“yes” response), Sign Phonology (“no” response) and Semantics (“no” response). For the RT measures, inaccurate responses and RTs that were more than two standard deviations from the participant and item mean were excluded from further analysis. For the younger hearing participants, 1.72% of the data was excluded. For the older hearing participants, 1.92% of the data was excluded. For the younger deaf participants, 0.4% of the data was excluded. For the older deaf participants, 0.76% of the data was excluded.Footnote 4 In separate analyses for the deaf vs. the hearing children, Sign Iconicity was treated as a within-subjects and between-items factor; Sign Phonology was treated as a within-subjects and within-items factor; and Semantics was treated as a within-subjects and within-items factor; those three factors were the independent variables. Again, RT and error were the dependent variables. Grade was treated as a between-subjects factor and within-items factor, and main effects of Grade as well as interactions between Grade and Sign Iconicity, Grade and Sign Phonology, and Grade and Semantics were calculated.
Mean response times (of the correct responses only, in milliseconds) and correct responses (in percentages) on the word–picture verification of hearing children in third or fifth grade (with standard deviations presented in parentheses).
Mean response times (of the correct responses only, in milliseconds) and correct responses (in percentages) on the word–picture verification of deaf children in third/fourth or fifth/sixth grade (with standard deviations presented in parentheses).
Response time data for Sign Iconicity (“yes” responses)
Hearing children. For the hearing children, no main effect of Sign Iconicity was obtained; word–picture pairs with strongly versus weakly iconic sign (translation) equivalents did not influence the response times of the hearing children (F1(1,96) = 2.07, p > .1, η2p = .02; F2 < 1). A main effect of Grade was detected (F1(1,96) = 25.37, p < .001, η2p = .21; F2(1,46) = 612.93, p < .001, η2p = .93) with the fifth graders responding faster than the third graders (see Table 4). There was no interaction between Grade and Sign Iconicity (F1 < 1; F2 < 1).
Deaf children. For the deaf children, a significant main effect of Sign Iconicity was found (F1(1,38) = 24.97, p < .001, η2p = .40; F2(1,46) = 10.84, p < .01, η2p = .19). Word–picture pairs with strongly iconic sign (translation) equivalents were responded to faster than word–picture pairs with weakly iconic sign (translation) equivalents by the deaf children. Neither a main effect of Grade (F1 < 1; F2(1,46) = 1.15, p > .1, η2p = .02) nor an interaction between Grade and Sign Iconicity (F1 < 1; F2 < 1) was found for the deaf children (see Table 5).
Response time data for Sign Phonology (“no” responses)
Hearing children. For the hearing children, no main effect of Sign Phonology was found; word–picture pairs with strongly overlapping sign (translation) equivalents versus non-overlapping sign (translation) equivalents did not affect the response times (F1 < 1; F2 < 1). A significant main effect of Grade was found with the fifth grade children responding faster than the third grade children (F1(1,96) = 22.99, p < .001, η2p = .19; F2(1,23) = 147.12, p < .001, η2p = .87). No significant interaction between Grade and Sign Phonology was found for the hearing children (F1 < 1; F2(1,23) = 1.04, p > .1, η2p = .04).
Deaf children. A significant main effect of Sign Phonology was found for the deaf children (F1(1,38) = 6.97, p < .05, η2p = .16; F2(1,23) = 1.26, p > .1, η2p = .05). The deaf children were slower to respond to conceptually unrelated Dutch word–picture pairs when the NGT signs underlying the word and the picture phonologically overlapped strongly as opposed to when they did not overlap. Once again, neither a main effect of Grade (F1 < 1; F2 < 1) nor an interaction between Grade and Sign Phonology (F1 < 1; F2 < 1) was found for the deaf children.
Response time data for Semantics (“no” responses)
Hearing children. A main effect of Semantics on the hearing children's response times was not found (F1 < 1; F2 < 1). However, a significant main effect of Grade was found with the fifth graders responding faster to both semantically related and semantically unrelated pairs than the third graders (F1(1,96) = 21.87, p < .001, η2p = .19; F2(1,23) = 307.85, p < .001, η2p = .93). No significant interaction between Grade and Semantics was found for the hearing children (F1(1,96) = 1.48, p > .1, η2p = .02; F2(1,23) = 1.52, p > .1, η2p = .06).
Deaf children. An inhibitory main effect of Semantics on the response times of the deaf children was found in the participant analyses (F1(1,38) = 12.03, p < .001, η2p = .24; F2 < 1). Semantically related word–picture pairs were responded to slower on average than semantically unrelated word–picture pairs. A marginally significant main effect of Grade was observed in the item analyses (F1 < 1; F2(1,23) = 3.52, p < .1, η2p = .13). No significant interaction between Grade and Semantics was found (F1(1,38) = 1.24, p > .1, η2p = .03; F2 < 1).
Error data for Sign Iconicity (“yes” responses)
Hearing children. A main effect of Sign Iconicity was not found in the error analyses for the hearing children (F1(1,96) = 1.70, p > .1 η2p = .02; F2 < 1). The error analyses also showed no main effect of Grade for the hearing children (F1 < 1; F2 < 1). There was also no significant interaction between Grade and Sign Iconicity (F1 < 1; F2 < 1).
Deaf children. For the deaf children, Sign Iconicity produced a significant main effect in the analyses of the error rates (F1(1,38) = 6.35, p < .05, η2p = .14; F2(1,46) = 18.46, p < .001, η2p = .29). The deaf children made fewer errors on those word–picture pairs with underlying strong sign iconicity. There was no main effect of Grade for the deaf children (F1 < 1; F2 < 1). No significant interaction between Grade and Sign Iconicity was found (F1 < 1; F2 < 1).
Error data for Sign Phonology (“no” responses)
Hearing children. There was no main effect of Sign Phonology (F1 < 1; F2 < 1). A marginal effect of Grade was found for error data but only in the item analyses (F1(1,96) = 2.03, p > .1, η2p = .02; F2(1,23) = 3.06, p < .1, η2p = .12). No significant interaction between Grade and Sign Phonology was found for the error data from the hearing children (F1 < 1; F2 < 1).
Deaf children. The deaf children made more errors in response to conceptually unrelated Dutch word–picture pairs when the NGT signs underlying the word and the picture were strongly overlapping as opposed to not overlapping (F1(1,38) = 9.28, p < .01, η2p = .19; F2(1,23) = 2.79, p < .1, η2p = .11). No main effect of Grade was found for the deaf children in the participant analyses (F1 < 1). However, the item analyses showed a significant effect (F2(1,23) = 5.38, p <.05, η2p = .19). No significant interaction between Grade and Sign Phonology was found in the participant analyses (F1 < 1). The item analyses showed a significant interaction between Grade and Sign Phonology for the deaf children's error data (F2(1,23) = 4.86, p < .05, η2p = .18). Sign Phonology uniformly affected the deaf children's error data irrespective of grade in the participant analyses (F1(1,38) = 6.35, p < .05, η2p = .14). However, in the item analysis, sign phonology only affected the error data of the older deaf children (F2(1,23) = 6.34, p < .05, η2p = .22), not the younger deaf children (F2 < 1).
Error data for Semantics (“no” responses)
Hearing children. A main effect of Semantics was found (F1(1,96) = 20.82, p < .01, η2p = .18; F2(1,23) = 6.69, p < .01, η2p = .23). More errors were made on word–picture pairs with a strong semantic relation than on pairs with no semantic relation. A significant main effect of Grade was not found for the error data from the hearing children, which means that the two grades showed similar error levels (F1 < 1; F2 < 1). No significant interaction between Grade and Semantics was found either (F1 < 1; F2 < 1), which means that the errors produced by the two grades were similarly affected by Semantics.
Deaf children. For the deaf children, the significant main effect of Semantics showed more errors to be made on semantically related word–picture pairs than on semantically unrelated pairs (F1(1,38) = 9.44, p < .01, η2p = .20; F2(1,23) = 3.78, p < .1, η2p = .14). In the item analyses, a marginally significant main effect of Grade was detected (F1 < 1; F2(1,23) = 3.23, p < .1, η2p = .12), with the older deaf children appearing to make fewer errors than the younger children on the Semantics items. No significant interaction was found between Grade and Semantics for the error data from the deaf children (F1 < 1; F2 < 1).
Discussion and conclusion
In this study, evidence was found for bilingual cross-language processing on the part of deaf children. During the word–picture verification task, the non-target (i.e., sign) language underwent activation as indicated by the sign phonology inhibition effects and the sign iconicity facilitation effects. When the phonology of the sign (translation) equivalents underlying the mismatching word–picture pairs overlapped partly, inhibition occurred; that is, the activation of the phonologically related signs during the word–picture verification task resulted in slower and less accurate responses on the part of the deaf children. Conversely, when the underlying sign (translation) equivalents for the target word–picture pairs were highly iconic (i.e., the meaning of the sign resembled the form of the sign to a considerable extent), facilitation occurred; that is, matching pairs with strongly iconic sign translation equivalents were responded to more quickly and accurately than those words with only weakly iconic sign translation equivalents. We can thus conclude that aspects of both the sign phonology and sign iconicity of the sign (translation) equivalents underlying the word–picture pairs presented to the deaf children were activated during their processing of the stimuli. Significant semantic effects were also found and thus reflect a general disposition to activate semantically related items when reading single words. This condition was added to verify that the children were actually performing the critical experimental tasks (Sign Phonology and Sign Iconicity) appropriately.
For the sake of thoroughness, it was further demonstrated that sign effects do not occur for hearing children, which means that the inhibitory and facilitatory effects found to occur for the deaf children during the word–picture verification task can only be attributed to the bilingual activation of their sign language knowledge. As expected, Semantics produced a main effect in the error data, which showed the hearing children to activate semantically related words when reading single Dutch words. Significant grade effects in the response times showed that the older hearing children were faster and, in some cases, made fewer errors than the younger children in all conditions.
The word–picture verification results reported here for bilingual deaf children are in line with the results of an increasing number of studies of bilingual people (e.g., Wang et al., Reference Wang, Koda and Perfetti2003; van Wijnendaele & Brysbaert, Reference Wijnendaele and Brysbaert2002). Bilingual language processing has been demonstrated not only for languages with similar scripts but also for languages with highly different scripts, and for languages with no script overlap. This last is the case for studies of cross-language activation for English and Chinese (see, e.g., Thierry & Wu, Reference Thierry and Wu2004; Wu & Thierry, Reference Wu and Thierry2010), and also for a signed language and a written language. One innovative aspect of the present study is that the two languages are produced in two different modalities without sharing orthography and only minimal phonology. The results of the present study extend the findings of cross-language activation for a signed and a spoken/written language in deaf adults and hearing adults by showing that cross-language effects already influence lexical processing before deaf children have attained full proficiency in either language. None of the studies showing cross-language activation looked into the time course of the activation.
In Figure 5 (see also Ormel, Reference Ormel2008), the processes that may be involved in the written word recognition of deaf children and in line with the present results are presented schematically (see the BIA+ model by Dijkstra & van Heuven, Reference Dijkstra and van Heuven2002, for the source of the current model). According to this view, once lexical orthography has been activated for the purposes of written word recognition of the word in a word–picture pair (1), the sign translation equivalent for the target word becomes activated (2). Sign phonology (i.e., sublexical sign elements) is also activated in the form of sign movement information, handshape, location of the sign and orientation of the hands (3). When this occurs, not only the correct combination of sublexical sign elements is activated but also other combinations involving one or more of the four sign elements. For example, in the DOG–CHAIR pair, the word dog may activate the sign for DOG as well as its phonological neighbour CHAIR (Figure 1). Arguably, the picture in this pair also activates the sign for CHAIR. This overlapping activation creates a conflict for the required “no” answer to this conceptually non-matching pair. The inhibition effect of sign phonology thus occurs as a result of underlying competing lexical items that contain overlapping combinations of sublexical sign elements (see also Thierry & Wu, Reference Thierry and Wu2004, for similar results with bilingual Chinese–English hearing students).
Sign activation during written word recognition in bilingual deaf children: Deaf Bilingual Interactive Activation model.
Whether or not sign phonology or, for that matter, sign iconicity actually mediates the retrieval of the meanings of written words (pre-conceptual activation) or is activated after access to the meaning (post-conceptual activation) remains an open question for further investigation. One way to explore the exact nature of contact involves a study in which words and pictures (or word pairs) are presented sequentially, varying the stimulus onset asynchrony (SOA) of the stimuli. Such a paradigm would provide further insight into the time course of bilingual lexical activation.
The present results showed sign iconicity to play a facilitating role in the written word recognition of deaf elementary school children (i.e., their provision of “yes” responses in a word–picture verification task). Past studies of the role of sign iconicity in early sign language acquisition have produced mixed results (e.g., Markham and Justice, Reference Markham and Justice2004; Meier, Reference Meier, Meier, Cormier and Quinto-Pozos2002; Orlansky & Bonvillian, Reference Orlansky and Bonvillian1984; Tolar, Lederberg, Gokhale and Tomasello, Reference Tolar, Lederberg, Gokhale and Tomasello2007). The present effects for sign iconicity can be explained by an on-line processing advantage. The iconic features of the word's sign translation equivalent may facilitate recognition of the printed word, leading to faster and more accurate judgments on the picture verification task (see also Thompson, Vinson and Vigliocco, Reference Thompson, Vinson and Vigliocco2009, for an on-line sign–picture study with deaf adults). However, it is possible that the isomorphism between the picture and the underlying iconic sign is what is actually producing the faster responses.
Directions for further research
The present findings provide much-needed insight into the word reading of young bilingual deaf children and thereby help us better understand the initial stages in bilingual language architecture. As described by Kroll and Stewart in 1994, more proficient users of their second language can access the meaning of a word in their second language directly whereas less proficient users access the meaning of a word in their second language indirectly, via the first language. Perhaps deaf children can be compared to less proficient users of their second language. Research with deaf adults who are fluent or near fluent readers could provide insight into the more proficient stages of bilingual processing of a signed language in combination with a written language. Some initial insight into the role of reading fluency has been provided by the research of Morford et al. (Reference Morford, Wilkinson, Villwock, Piñar and Kroll2011) who studied highly proficient deaf readers and by Grote and Linz (Reference Grote, Linz, Muller and Fisher2003) who studied highly proficient hearing readers who had deaf parents (so-called CODAs: Children of Deaf Adults). Future research is required to gain further insight into the distinction between bilingual signers who have different levels of proficiency in their (first and) second language. In this regard, it would be informative to obtain word and sign frequency norms for deaf children and adults. Deaf signer's lexical knowledge may differ from that of hearing controls. Related to the issue of proficiency, it would, moreover, be valuable to assess bilingual deaf and hearing children and adults for language dominance. In contrast to most of the studies using similar – alphabetic – scripts, priming effects in past studies using different scripts occurred for only the dominant language and not the other language (i.e., the dominant language primed recognition of words in the non-dominant language but not vice versa; see, for example, Keatley, Spinks & de Gelder, Reference Keatley, Spinks and de Gelder1994, and Thierry & Wu, Reference Thierry and Wu2004). Given the seemingly large language differences for bilinguals who know a sign language and a spoken/written language, the same unidirectional (priming) effects can be expected, whereby the dominant language can be used during processing of the non-dominant language, but not vice versa.
