1 Introduction
Intonation, rhythm or timing, and prominence interact to comprise a coherent component of the grammar, often referred to as the prosodic component (Selkirk Reference Selkirk and Goldsmith1995a, Fletcher Reference Fletcher, Hardcastle, Laver and Gibbon2010). For example, prosodic signals enter into a hierarchy of constituents that are closely related to morphosyntactic constituents (Selkirk Reference Selkirk1984, Nespor & Vogel Reference Nespor and Vogel1986), but are not isomorphic with them (Nespor & Vogel Reference Nespor and Vogel1986). The intonational system that is part of prosody has its own internal grammar, in the sense that it exhibits the linguistic properties of discreteness and compositionality (Hayes & Lahiri Reference Hayes and Lahiri1991, Gussenhoven Reference Gussenhoven2004). And while certain aspects appear to be universally shared across spoken languages (Bolinger Reference Bolinger1989), specific details of the system often differ across languages, and even across different dialects of the same language (Ladd Reference Ladd1996, Hirst & Di Cristo Reference Hirst, Cristo, Hirst and Cristo1998, Gussenhoven Reference Gussenhoven2012).
Sign languages, the natural languages of deaf communities, are conveyed in an entirely different physical modality, exploiting the hands, head, face and body as articulators, and the eyes for perception. Yet more than a half-century of research has shown that these languages share many central grammatical properties with spoken languages (see Sandler & Lillo-Martin Reference Sandler and Lillo-Martin2006 for a detailed synthesis of this research).
While the hands convey the lexical items of sign languages, researchers noticed early on that certain configurations of non-manual signals in American Sign Language (ASL), such as eyebrow and head position, have been grammaticalised, and systematically characterise certain types of clauses and sentences (Baker & Padden Reference Baker, Padden and Siple1978, Liddell Reference Liddell1980, Reilly et al. Reference Reilly and Bellugi1990). Other researchers on ASL and other sign languages followed suit.
There does not appear to be disagreement over whether or not the system incorporating non-manual signals is grammatical. However, not all researchers agree about whether the signals in question belong to an autonomous prosodic system or are inextricable from syntax, and few studies have explicitly argued for prosody as a coherent grammatical component in sign languages.
One approach holds that the non-manual signals in question, such as the furrowed brow for wh- questions, are an integral part of the syntax (Liddell Reference Liddell1980), directly manifesting the position and distribution of constituents in syntactic representations (Neidle et al. Reference Neidle, Kegl, Bahan and Lee2000). Neidle et al. use non-manual signals of this kind as a diagnostic for underlying syntactic structure. Such an analysis, if it could be supported, would have the advantages of simplifying the grammar by doing away with a prosodic component, and of exploiting the non-manual signals in question to identify overt representations of syntactic form.
Other researchers concede that the relevant facial expressions and other markers are intonational in character, but hold that they are intertwined with the syntax, either in the sense that their distribution is determined by syntax (Wilbur & Patchke Reference Wilbur and Patschke1999, Wilbur Reference Wilbur, Emmorey and Lane2000) or in the sense that their scope across constituents directly reflects syntactic structure, making them more intimately related to syntax than is the case in spoken language (Cecchetto et al. Reference Cecchetto, Geraci and Zucchi2009).
A different approach analyses these signals and the timing cues that accompany them as belonging to an independent prosodic/intonational component of the grammar that is related to syntax less directly. Lack of space precludes a more detailed comparison of the ‘direct syntax’ approach and the intonation approach, which can be found in Sandler & Lillo-Martin (Reference Sandler and Lillo-Martin2006) and Sandler (Reference Sandler2010). Here we adopt and further support the intonation approach.
In order to qualify as a linguistic system which is explicitly intonational in more than a metaphorical sense, the sign language system should meet three criteria: (i) it should be part of a prosodic system which is dissociable from other components of the grammar such as syntax, (ii) it should be systematic and conventionalised and (iii) it should perform similar linguistic functions to those of more familiar intonational systems found in spoken languages. In §2.1.1, we briefly summarise evidence provided to date suggesting that the system in question meets all three criteria.
The present study, the first to systematically compare the prosodic systems of two unrelated sign languages, brings novel evidence for this view. Paying particular attention to intonation, manifested primarily in facial expression, our study is the first to systematically compare prosodic systems across sign languages, specifically Israeli Sign Language (ISL) and American Sign Language. By studying the same sentences signed by several signers in two sign languages and using the same methodology, we are able to confirm the degree of conventionalisation of the non-manual markers more rigorously than has been possible in the past, both by rating the frequency of occurrence and by identifying language-specific patterns. We find similarities and also systematic differences in the intonation of the two languages studied, providing evidence that the facial configurations do indeed constitute a grammatical system. We argue that where the distribution of intonational signals in the two languages is different, this difference is best explained in terms of the semantics and pragmatics of information structure. This study thus provides additional evidence for the claim that the system in question is intonational, while empirically contributing to a more nuanced model of intonation in sign language.
Naive signers of either language can immediately sense that the other has a different ‘flavour’ to it. However, as with intonation in any two languages, it is very difficult to say why, based on intuition alone. Using a detailed coding system for the same set of sentences in each language, we are able to identify specific intonational differences.
We begin in §2 with some background about sign language prosody and a description of the model of sign language adopted here. We review earlier work supporting the view that systematic facial expressions and head positions which align with temporally demarcated constituents in signed languages are explicitly part of a prosodic system.
As in spoken language, it is difficult to define, categorise and compare the form and function of prosody in sign language. In a collection of studies of intonation in different spoken languages, Hirst & Di Cristo (Reference Hirst, Cristo, Hirst and Cristo1998: 43) recommend the following requirements for comparative research: (i) consistent transcription, (ii) consistent parameters, (iii) comparable corpus and (iv) cooperative research. We adopt all four requirements, spelled out in §3.1.
Using these techniques, we identify similarities and differences between the two sign languages, by recording and analysing data from six ASL signers and five ISL signers. The results are presented in §3.2, where we demonstrate in detail why these two languages ‘look different’. In the analysis and discussion in §4, we pay special attention to the meaning of intonational patterns. In the process, we posit possible candidates for sign language universals in intonation, and identify language-particular elements of the grammar of intonation in each language. We argue that some of the language-particular differences can be explained by different ways of organising information, a function explicitly attributed to intonation and only indirectly related to syntactic organisation (e.g. Pierrehumbert & Hirschberg Reference Pierrehumbert, Hirschberg, Cohen, Morgan and Pollack1990, Hirschberg Reference Hirschberg, Horn and Ward2004, Baumann Reference Baumann2006).
The multiplicity of articulatory resources available to sign languages gives them the potential for more complex simultaneous intonational arrays than is the case in spoken languages. The extent to which they exploit this potential, and the extent to which the physical modality itself determines the intonational grammar, are questions raised for future research in the conclusion (§5). There we provide a brief summary of our findings and the theoretical implications of this study, and of the study of sign language intonation more generally.
2 Sign language prosody
A great deal of research has shown that the spontaneously arising visual languages of deaf communities are characterised by many of the same grammatical properties found in spoken languages, prosody among them. Work on ISL suggests that signed languages, like their spoken counterparts, have hierarchically organised prosodic constituents, marked systematically by patterns of intonation and timing (Nespor & Sandler Reference Nespor and Sandler1999, Sandler Reference Sandler2010). We describe other properties of sign language prosody in §2.1, relying primarily on the Nespor & Sandler model and its refinements. We then go on to survey previous comparisons of prosody across sign languages in §2.2, and turn to our study in §3.
2.1 The prosodic component of sign language grammar
The lexical items of sign languages are conveyed by the hands, and these are accompanied by non-manual signals of the face and head. While other analyses of the system under investigation focus exclusively on non-manual markers, the Nespor & Sandler model also explicitly analyses manual signals superimposed on the signed words as part of the prosodic system. Specifically, the timing of manual signs, and sometimes their larger size as well, marks final prosodic constituent boundaries. These manual manipulations are comparable to phenomena in spoken language such as phrase-final lengthening and amplitude. Aligned with these manual markings at constituent boundaries are particular non-manual signals of facial expression and head position. These signals convey the illocutionary force of utterances, such as yes/no and wh- questions, and other pragmatic properties such as shared information.
Both the temporal distribution and the meaning of these non-manual signals bear key similarities to those of intonation of spoken language. This is true despite dramatic differences in their physical realisation: intonation in spoken language is conveyed by pitch changes controlled by the vocal cords, while intonation in sign language is conveyed primarily by facial configuration changes controlled by muscles of the face (e.g. Reilly et al. Reference Reilly and Bellugi1990, Wilbur Reference Wilbur1996, Nespor & Sandler Reference Nespor and Sandler1999, Dachkovsky & Sandler Reference Dachkovsky and Sandler2009). In sign language, two or more intonational articulations can simultaneously co-occur. This is possible because, unlike the vocal cords in spoken language, which can only vibrate at one frequency at a time, the facial articulations of sign language – eyebrows, upper and lower eyelids, nose, cheeks, mouth, head – are independent of one another, and can be activated individually or together, to create ‘tunes’ whose ‘tones’ occur simultaneously rather than sequentially. Here we adopt the terms ‘array’ instead of ‘tune’, and ‘action unit’ instead of ‘tone’.Footnote 1 Sign language intonation, then, is manifested by the simultaneous action of a number of non-manual articulators which coordinate with temporal structure conveyed by the hands.Footnote 2
Certain intonational markers seem very common across sign languages, as noted above. For example, brow raise is a marker of yes/no questions in American Sign Language (Liddell Reference Liddell1980, Baker-Shenk Reference Baker-Shenk1983), British Sign Language (Woll Reference Woll, Woll, Kyle and Deuchar1981), Swedish Sign Language (Bergman Reference Bergman, Loncke, Braem and Lebrun1984), Sign Language of the Netherlands (Coerts Reference Coerts1992), Norwegian Sign Language (Vogt-Svendsen Reference Vogt-Svendsen1990), German Sign Language (Herrmann Reference Herrmann2010) and others. Despite these similarities, even a cursory glance at facial configurations and head movements used by people communicating in different sign languages gives an impression of different intonational systems, an impression that we will substantiate here for two languages.
Along with facial configurations, head movements play an important role in sign language grammar. Head position was included in the non-manual displays associated with certain sentence or clause types in early linguistic studies on sign languages (Liddell Reference Liddell1980). Reilly et al.'s (Reference Reilly and Bellugi1990) account of conditionals in ASL, which included head position, compared all of these articulations to intonation in spoken language. Yet most work on prosody has not explicitly included head position as intonational. In the present study we support Reilly and colleagues' intonational view of certain head movements, since these articulations are not only aligned to prosodic constituents but, like facial expression, also contribute general meanings to the interpretation of the whole utterance.
In the subsections that follow, we address two aspects of sign language prosody that are central to the comparative study reported here: the intonational phrase constituent and the compositional view of intonation structure.
2.1.1 Prosody of the intonational phrase in sign language
Research on spoken language prosody has demonstrated that the language stream is not continuous, but consists of distinct prosodic constituents organised in a hierarchy (see e.g. Selkirk Reference Selkirk1984, Reference Selkirk, Beckman, Dickey and Urbanczyk1995b, Ladd Reference Ladd1986, Reference Ladd1996, Nespor & Vogel Reference Nespor and Vogel1986), and evidence from ISL indicates that sign language prosody is similarly organised (Nespor & Sandler Reference Nespor and Sandler1999, Sandler Reference Sandler, Hall and Kleinhenz1999a, 2011). In the present paper we discuss only the intonational phrase constituent – the major domain for the alignment of intonational arrays, the focus of our study.
In the Nespor & Sandler corpus of elicited sentences, the final boundary of the intonational phrase (IP) in ISL is consistently marked either by a pause (relaxation of the hands) or by lengthening the articulation of the last sign in the phrase. This lengthening is manifested by a hold (keeping the hands in the final configuration and location of the last sign) or by a reiteration of the sign. The last sign is also typically larger and slower than those in non-final position, making the end of the IP salient or prominent. These signals are comparable to phrase-final lengthening realised at the edges of intonational phrases in spoken languages (e.g. Klatt Reference Klatt1976). Earlier research noted phrase-final lengthening in ASL as well (Sandler Reference Sandler1986, Perlmutter Reference Perlmutter1992). However, those studies were based primarily on observation of individual signers rather than across signers, and they did not specify the character of the phrase being lengthened. The Nespor & Sandler (Reference Nespor and Sandler1999) ISL study relied on detailed coding of elicited sentences signed by three signers and explicitly motivated the IP. Crucially, the boundary between intonational phrases is also marked consistently by an across-the-board change of facial articulations, together with a change of head position.Footnote 3
Intonational phrase boundaries commonly demarcate constituents such as topics, parentheticals and if clauses in conditionals, in both spoken and sign languages. The division of an ISL utterance into two distinct intonational phrases is exemplified in Fig. 1 with the counterfactual conditional sentence meaning ‘If the goalkeeper had caught the ball, (the team) would have won the game’.Footnote 4 Although it is difficult to illustrate the manual prosodic cues in static pictures, the distance of the hands from the body in the last sign of the first IP, catch-ball, indicates its larger size. The last sign in the first intonational phrase and the first sign in the second are shown again in Fig. 2, and reflect the effect of the global change in head position and facial expression. The if clause is accompanied by a simultaneous array of raised brows, squinted eyes and a forward head lean. At the boundary, the brows and the eye aperture change to a neutral position, while the head moves into a backward lean. The change of intonational facial expression at IP boundaries is comparable to the reset of pitch excursions found at spoken language IP boundaries, and this change, together with changes in timing and prominence, comprise the phonetic signals at the intonational phrase boundary in ISL.
The alignment of facial expressions, head movements and rhythmic cues with the prosodic boundary in the ISL counterfactual conditional sentence ‘If the goalkeeper had caught the ball, (the team) would have won the game’. (Reprinted with permission from Dachkovsky & Sandler Reference Dachkovsky and Sandler2009.)
The juncture of the two intonational phrases in the sentence in Fig. 1.
Intonational phrases in ISL typically correspond to certain syntactic constructions noted in Nespor & Vogel (Reference Nespor and Vogel1986) for spoken language, such as parentheticals, non-restrictive relative clauses, topicalisations and other types of extraposed elements (Nespor & Sandler Reference Nespor and Sandler1999). However, Nespor & Sandler demonstrate that the two are not strictly isomorphic. For example, if a constituent is very short, it can be restructured into a neighbouring phrase, as is the case in spoken language (Nespor & Vogel Reference Nespor and Vogel1986). And just as prosodic constituency and syntactic constituency are not isomorphic, neither is the intonational array isomorphic with syntactic sentence type. For example, utterances that are syntactically yes/no questions have different intonation in choice questions, and ironic wh- questions that have wh- question syntax do not have standard wh- intonation (Sandler & Lillo-Martin Reference Sandler and Lillo-Martin2006, Meir & Sandler Reference Meir and Sandler2008). Non-manual configurations do not necessarily correspond to particular syntactic constituents, then; instead they convey pragmatic notions such as illocutionary force, information status and other discourse functions.
2.1.2 Sign language intonation is compositionally structured
It has long been known that constellations of two or more facial and head movements often mark particular kinds of constituents. For example, topics in American Sign Language are typically marked by raised brows and backward head tilt (Liddell Reference Liddell1980). However, subsequent closer analysis by other researchers has singled out individual components and associated meanings within such intonational clusters. For example, Coulter (Reference Coulter1979) proposes that the general function of ASL brow raise is to signal background information, and he argues that this is the reason that it is found consistently on various types of topics. Wilbur & Patschke (Reference Wilbur and Patschke1998) propose that in ASL, forward and backward upper body leans convey the notion of ‘contrast’ in several ways. For example, forward body lean signals inclusion, while backward lean signals exclusion. By implication, forward and backward head position may also combine with brow position to convey complex meanings.
Research on Israeli Sign Language has identified individual semantic/pragmatic contributions of different facial components in that language such as brow raise, brow furrow and squint, which can be combined for more complex meanings (Nespor & Sandler Reference Nespor and Sandler1999, Sandler Reference Sandler1999b). A more detailed semantic analysis of the distribution of brow raise and squint accounts for their occurrence and co-occurrence in a range of language structures (Dachkovsky Reference Dachkovsky2005, Dachkovsky & Sandler Reference Dachkovsky and Sandler2009). The appeal to information structure inherent in the compositional analysis of these two components plays a central role in the present comparative study, and we therefore describe it in some detail here.
According to this analysis, brow raise signals continuation in the sense that the constituent marked by it is to be interpreted in light of subsequent information. On this interpretation, the function of brow raise is analogous to the meaning of high tone in many spoken languages. For example, Ford (Reference Ford1993) demonstrates that a high phrasal tone often co-occurs with initial adverbial clauses in English, signalling that the information of the subordinate clause is to be completed by the subsequent main clause. Similarly, ISL adverbial clauses are very often marked by raised brows (Dachkovsky Reference Dachkovsky2005), as the conditional clause (a type of adverbial clause) in Figs 1 and 2 above demonstrates. It is not brow raise alone that conveys the meaning of a constituent and its relation to the subsequent one; rather, this meaning relation is derived via implicature from interaction with a whole ensemble of features, such as context, lexical semantics and word order. In the same vein, Bartels (Reference Bartels1999) shows that the general continuation semantics of high boundary tones in English have more concrete interpretations via implicature, depending on other semantic and pragmatic properties of the utterance. For example, consider the English conditional sentence in (1), where the if clause boundary is associated with a high boundary tone (H%) (Pierrehumbert & Hirschberg Reference Pierrehumbert, Hirschberg, Cohen, Morgan and Pollack1990).
(1)
In this example the high boundary tone conveys continuation or incompleteness, indicating that the current phrase is to be interpreted with respect to a succeeding phrase. It contributes to the contingency relations in the utterance, while the specific conditionality relationship between the clauses is signalled by the conjunction if and verbal morphology. This example demonstrates an intonational tune reflecting a particular aspect of meaning, interpreted in the context of the utterance.
A similarly complex relation between a facial articulation and the meaning or meaning relation it conveys applies to a frequent facial component in ISL: squint. This cue is articulated by tightening and pulling up the lower eyelid, thereby narrowing the eye aperture. Squint functions as a signal to the interlocutor to retrieve or infer information that is mutually accessible, but not salient from the preceding discourse context. The notion of accessibility and its prosodic marking will be addressed in §4.3.2 in relation to our findings.
As we have said, intonational features combine to produce complex meaning relations which are derivable from the meanings of individual components (Nespor & Sandler Reference Nespor and Sandler1999, Sandler Reference Sandler1999b). Note that the facial expression in the first IP in Figs 1 and 2 is marked by both brow raise and squint. Here, brow raise signals continuation, implying the conditional contingency, and squint signals that the information in the if clause is not readily accessible, since it contradicts the current situation. Together, these intonational articulations convey a conditional that is counterfactual (Dachkovsky Reference Dachkovsky2005, Dachkovsky & Sandler Reference Dachkovsky and Sandler2009). Compositional analyses of intonation have been proposed for spoken language intonation, where the intonational components are sequentially occurring high and low tones (e.g. Pierrehumbert & Hirschberg Reference Pierrehumbert, Hirschberg, Cohen, Morgan and Pollack1990, Hayes & Lahiri Reference Hayes and Lahiri1991, Steedman Reference Steedman2000). For example, in Bengali, the L*HL focus contour combines with an H continuation rise in continuation contexts to create a L*HLH sequence (Hayes & Lahiri Reference Hayes and Lahiri1991). In our comparison of the ISL and ASL data, this kind of compositional analysis of intonational meaning will help to explain certain differences between the two languages.
2.2 Previous comparative studies of sign language intonation
In their survey, Hirst & Di Cristo note that in the intonation literature it is difficult to find a precise statement of specific characteristics which make one language sound prosodically different from another (1998: 2). Descriptions of specific distinguishing characteristics are even rarer with respect to sign language prosody. In fact, no previous studies have systematically compared the prosodic structure of different sign languages.
Many studies have described the occurrence and function of non-manual signals in individual sign languages, however. Since the advent of research on non-manual signals in American Sign Language in the late 1970s, research on several other languages has shown that certain facial expressions and head positions are commonly recruited for similar functions (American Sign Language: Baker & Cokely Reference Baker and Cokely1980, Liddell Reference Liddell1980, Baker-Shenk Reference Baker-Shenk1983; British Sign Language: Woll Reference Woll, Woll, Kyle and Deuchar1981, Deuchar Reference Deuchar1984, Fenlon et al. Reference Fenlon, Denmark, Campbell and Woll2008; Swedish Sign Language: Bergman Reference Bergman, Loncke, Braem and Lebrun1984; Sign Language of the Netherlands: Coerts Reference Coerts1992; Norwegian Sign Language: Vogt-Svendsen Reference Vogt-Svendsen1990; German Sign Language: Herrmann Reference Herrmann2010; Danish Sign Language: Engberg-Pedersen Reference Engberg-Pedersen, Edmondson and Karlsson1990; Hong Kong Sign Language: Tang Reference Tang and Zeshan2006, Sze Reference Sze2009; see Pfau & Quer Reference Pfau, Quer and Brentari2010 for an overview). Yet none of these studies are comparative.
A cross-linguistic comparison of some features corresponding to prosody in sign languages appears in Zeshan's (Reference Zeshan2004) survey of interrogatives across 35 sign languages. The study summarises descriptions from three sources: questionnaires, publications and the author's own fieldwork. Zeshan found that while the prosodic marking for questions in each language varies, especially in the spreading of cues and the degree of obligatoriness for each cue, some similarities were found. For example, cross-linguistically, yes/no questions are typically marked with the head forward and down, the eyebrows raised, raised upper eyelids and intense eye contact with the addressee. For content (wh-) questions, Zeshan's discussion focused mostly on lexical markers, but noted that facial and head movements varied considerably across languages, e.g. lowered brows for some languages and raised brows for others. Following up on this study, Šarac et al. (Reference Šarac, Schalber, Alibašić, Wilbur, Perniss, Pfau and Steinbach2007) report on interrogatives in Croatian Sign Language (HZJ) and Austrian Sign Language (OGS), contrasting them with ASL using a variety of data sets and published sources. They describe certain similarities between HZJ and OGS and differences between these languages and ASL.
Numerous studies have demonstrated that topic–comment is a common organising principle for sentences in sign languages, and that topics are typically marked with particular facial expressions and head positions. Early work on such marking in ASL identified specific cues for topics (Fischer Reference Fischer and Li1975, Friedman Reference Friedman and Li1976, Liddell Reference Liddell1980). Aarons (Reference Aarons1994) adopts a syntactic definition of topics as constituents left-adjoined to CP. Working with a native ASL consultant, she distinguishes three syntactico-semantic types of topics, each correlated with particular facial expressions and head movements.Footnote 5
Topic–comment constructions, set off by specific and systematic non-manual marking, have also been reported in many European sign languages, such as Swedish Sign Language (Bergman Reference Bergman, Loncke, Braem and Lebrun1984), British Sign Language (Deuchar Reference Deuchar, Kyle and Woll1983), Danish Sign Language (Engberg-Pedersen Reference Engberg-Pedersen, Edmondson and Karlsson1990) and Sign Language of the Netherlands (Coerts Reference Coerts1992, Crasborn et al. Reference Crasborn, van der Kooij, Ros and Hoop2009). However, a study of Hong Kong Sign Language showed a good deal of variation both in the position of topics in sentences and in the non-manual marking (Sze Reference Sze2009). Rosenstein (Reference Rosenstein2001) investigated spontaneous discourse in ISL, and concluded that it is a topic-prominent language in which the topic constituent is typically sentence-initial. The author found that topics in that language are not marked consistently by one particular facial cue, but rather by a variety of signals.
Prosodic marking of conditional sentences has been reported in a handful of sign languages, though no comparisons appear in the literature. In those languages that have been studied, the protasis (the if clause) is marked with raised brows and a non-neutral head position, followed by neutral brows and shifted head position on the apodosis (the then clause) (Baker & Padden Reference Baker, Padden and Siple1978, Liddell Reference Liddell1986, Reilly et al. Reference Reilly and Bellugi1990 for ASL; Bergman Reference Bergman, Loncke, Braem and Lebrun1984 for Swedish Sign Language; Sutton-Spence & Woll Reference Sutton-Spence and Woll1999 for British Sign Language; Engberg-Pedersen Reference Engberg-Pedersen, Edmondson and Karlsson1990 for Danish Sign Language; Dachkovsky Reference Dachkovsky2005, Dachkovsky & Sandler Reference Dachkovsky and Sandler2009 for ISL). While the specific head position for the protasis seems to vary across sign languages, both brow raise on if clauses, similar to the rising intonation that commonly occurs on protases in spoken languages, and a change of non-manual markers at the clause boundary are commonly found.
3 Prosody and intonation in ASL and ISL: a comparative study
The present study comparing the prosody of ASL and ISL departs from previous work in a number of ways. First, it is an explicit comparison of the same phenomena across sign languages. Second, unlike all earlier work, the data collection and methodology for the present analysis were the same for the two sign languages, and the same researchers worked together to code and compare data. In addition, the same sentences signed by several signers in each language were recorded, coded and analysed. In this way, the study yields a parallel data sample and a consistent and detailed description, contributing to a more rigorous comparison and analysis than have previously been available. We attempt to go beyond description by suggesting a semantic-pragmatic basis for understanding aspects of prosody, and especially intonation, in sign languages. In so doing, we add to the body of evidence that the signals under discussion are part and parcel of an explicitly intonational system, bearing central functional similarities to that of spoken languages.
3.1 Methodology
We sought to collect data sets that were as comparable to each other as possible. The elicitation materials were originally developed for a study of Israeli Sign Language (Dachkovsky Reference Dachkovsky2005, Dachkovsky & Sandler Reference Dachkovsky and Sandler2009), which further refined the analysis of prosodic constituents and intonational arrays identified in an earlier analysis (Nespor & Sandler Reference Nespor and Sandler1999). Based on earlier results, particular attention was paid here to brow raise and squint, and the combination of the two. Fifty-two sentences were elicited (see the Appendix). The sentences were presented in written Hebrew for the ISL study and in written English for ASL.Footnote 6 Most of the sentences were preceded by some context, given in italics.
Five native signers participated in the ISL study and six in the ASL study. Subjects were asked to read the sentences, internalise the meaning, put the written sentence aside and sign in natural ISL or ASL to another fluent signer, seated by the camera. Seven practice sentences were included for the purpose of training subjects to avoid interference from the written language. Two cameras videotaped the participants, one zoomed to a close-up on the face to capture details of muscle movements, and a second zoomed out to record movements of the hands, head and torso.
Hand movements were coded for the size and duration of a sign, reiterations, manual holds at the end of a sign, and pauses. Each facial and head articulation was coded for action units, using the Facial Action Coding System (FACS) of Ekman & Friesen (Reference Ekman and Friesen1978) and Ekman et al. (Reference Ekman, Friesen and Hager2002), and for scope. FACS is an anatomically based descriptive system which specifies a set of 44 action units to code for individual muscle movements made by the eyebrows, eyelids, cheeks, nose, mouth, head, eyes and neck. Two of the authors (Dachkovsky for ISL and Healy for ASL) are certified FACS coders. Inter-coder reliability was checked by cross-coding a representative subset of ten sentences of different types, and reached 87%. The ISL data were coded by hand on coding sheets created specifically for the purpose. The ASL data were coded in a similar way, making use of ELAN computer software.Footnote 7 An example of each is shown in Fig. 3 for the first clause of the sentence ‘If you go with me to Hawaii, I'll be the happiest person in the world’.
Example of (a) ISL coding from Dachkovsky (Reference Dachkovsky2005) and (b) ASL coding in ELAN. Translations of ‘If you go with me to Hawaii, I'll be the happiest person in the world’. The numbers indicate the action unit coding.
For the ISL coding sheets, the videotaped sentences were glossed in Hebrew by a linguistically trained deaf consultant, bilingually proficient in ISL and Hebrew. Responses were discarded if the consultant judged them to be either ungrammatical due to performance problems, or incorrectly translated. Table I below shows the number of tokens analysed for each targeted structure in each language.Footnote 8 Two deaf consultants, proficient in ASL and English, reviewed the English glosses of each ASL sentence to ensure accurate transcription of the signs. The timing of signs was recorded for the ISL data by counting video frames, while for the ASL data, timing was retrieved from the time code available in the ELAN software. We focused on the relative alignment of articulations within a language, and the comparison was frame-accurate.
Number of elicited and analysed tokens for each type of targeted structure.
After coding of the data, articulatory patterns were tallied across signers of the same language, and the patterns were then compared cross-linguistically for each constituent type. Specifically, we investigated the sentence types listed and exemplified in (2), in which italicised text is given as context. It is important to emphasise that these examples show the rough division of sentence types used in formulating the stimuli. However, the analysis was performed on the sign language sentences, which often took different forms, many of which exemplified more than one type of structure or information. For example, yes/no and wh- questions might contain topics, NPs containing relative clauses can be topics, and topics occurring sentence-finally in the Hebrew or English stimuli are sometimes fronted in the sign language data. Also, in most sentences, the subject is also the topic.
(2)
While the nature of the rest of the structures is relatively clear, the notion ‘topic’ can have more than one interpretation, and we clarify ours here. We adopt Krifka's (Reference Krifka, Féry, Fanselow and Krifka2007) functional definition of topics as entities intended to be stored in the common ground, i.e. intended to be shared by the interlocutors for the purpose of the discourse. We further elaborate the definition and discussion of topics in §4.3.1. In our analysis, we considered only topics of this kind that appeared sentence-initially. Similarly, many different sentence types (e.g. relative clauses, topics) may contain ‘shared information’, analysed in §4.3 in terms of accessibility.
In the following section, we report the results of the study. The commonalities and differences displayed by the two languages demonstrate how each exploits facial and head movements, creating systematic combinations that serve grammatical functions. In §4 we analyse the data, accounting for both similarities and differences.
3.2 Results
We begin with features that the two languages have in common, in particular timing cues at the ends of intonational phrases and intonational marking for yes/no questions and for most features of wh- questions. We then describe differences between ASL and ISL, particularly with respect to topics and conditionals.
3.2.1 Manual timing at final constituent boundaries
Typically, constituents such as sentence-initial topics, the if clauses of conditionals, and sentence-initial relative clauses form their own intonational phrases, usually marked in both the ISL and the ASL data by a hold, reduplication or a pause at the final boundary. That is, the timing of the manual signals delimits the scope of the constituent, with which intonational facial expressions and head movements align. These non-manual articulations tend to change dramatically at intonational phrase boundaries. Specific facial expressions and head positions associated with different types of grammatical structures are described below.
One of the characteristics that make the two languages look different is the overall temporal structure. When topics are utterance-initial, both languages tend to separate them from the comment through timing, but the tendency is stronger in ISL (98%) than in ASL (76%). For example, the parallel ASL and ISL instantiations of the sentence ‘Joe's brother was killed in a car accident’ are divided prosodically into a different number of constituents in each language. The ISL sentence is divided into the two prosodic constituents: [brother possessive joe] and [die car-accident]. In this case, the subject is also the topic and the comment the predicate. At the end of the first IP, there is a hold and an across-the-board change in facial expression and head position at the boundary between the two IPs. In producing the ASL sentence, none of the signers included a timing break between the topic (subject) and the comment (predicate). For some individuals, rate of signing seemed to influence whether or not the topic was set off through timing. However, we were unable to find a reliable correlation across the corpus between timing breaks and either the rate of signing or the length or complexity of a constituent (whether prosodic or syntactic). Regardless, with or without a timing break, there is typically a change in non-manual signals at IP boundaries in ASL, as in ISL. We return to this issue in §4.
The temporal difference can give the impression that one is looking at two different languages. However, the intonational differences are more striking. In what follows, we start with sentence types that have similar intonation, and then go on to the differences.
3.2.2 Yes/no questions
The constellation of facial and head cues that typically accompany yes/no (polar) questions is the same for ISL and ASL. It is the only linguistic construction where no differences in the intonational marking between the two languages were found. Yes/no questions are systematically marked by brow raise (95% for ISL and 100% for ASL). Raised brows (action units (AUs) 1 and 2) are usually accompanied by upper lid raise (AU 5), which widens the eye aperture. In a majority of yes/no questions the head moves forward (AU 57) in both ISL (90%) and ASL (70%). In many cases this forward movement combines with downward movement (AU 54). A prototypical intonational array characterising yes/no questions is presented in Fig. 4. The ISL image (Fig. 4a) is extracted from a sentence meaning ‘Was it hot (at the beach)?’, and the ASL image (Fig. 4b) from ‘Did he buy a car?’. Different examples are chosen for the two languages because they most clearly demonstrate the intonational array even to the untrained eye.
Yes/no questions in (a) ISL and (b) ASL.
3.2.3 Wh- questions
Wh- questions include those with question words such as English who, what, where, when, why, how and how much. They are interrogatives requesting an answer of content from the recipient rather than just a ‘yes’ or ‘no’ response. In our data, the hallmark of wh- questions in both languages is brow furrow (AU 4), which accompanies all wh- questions in both ISL and ASL. Another typical non-manual component associated with wh- questions in both languages is a forward head position (AU 57), which intensifies toward the end of the intonational phrase by moving further forward. Unlike yes/no questions, the head is held neutral along the vertical axis, or raised slightly in wh- questions. So ISL and ASL wh- questions yield a similar overall impression, with the brows furrowed and the head forward.
Nevertheless, some features of wh- questions differ in the two languages. While upper lid raise (AU 5) is usually present in ISL (75%), this action is very rare in the ASL responses (3%). Because of the dominance of brow furrow, upper lid raise is not noticeable to the untrained observer, but gives the appearance of an intense look or stare (Ekman & Friesen Reference Ekman and Friesen1978: 32–33).Footnote 9 In ASL, on the other hand, cheek raise (AU 6) is common in wh- questions (79%). The ISL correlate of ASL cheek raise is lower lid tighten (AU 7), a phonetic difference to which we return in §3.2.6. This articulation is less common (40%) in ISL. Figure 5 shows typical wh- question faces from our data, and Fig. 6 charts the facial action units that are typical of ISL and ASL wh-questions.
Wh- questions in (a) ISL and (b) ASL.
Frequency of facial action units in ISL and ASL wh- questions.
3.2.4 Topics
Structurally, topics in both languages are often found in sentence-initial position. However, topics were not found in sentence-initial position as frequently in ASL as in ISL. For example, the question ‘Where is the book?’ was signed book, where? by three out of the five ISL signers, while in ASL all six signers signed where book?
In sign languages, as in spoken languages, topics may constitute separate prosodic constituents. They may range in length from one sign to complex topics composed of a long noun phrase or a noun and modifying relative clause. In some cases there may be more than one topic in an utterance. In our data, ISL signers were more likely to create separate prosodic constituents for topics regardless of the prosodic length or syntactic complexity of the utterance, while ASL signers more often set off a topic by timing if the utterance was long and complex, as in the sentence ‘My friend, the head of the English Department at Gallaudet University [break] called me yesterday’. We return to this in §4.
Different frequency of temporal marking for boundaries is not the only way in which the languages differ with respect to topic marking. The intonational markers of prosodic constituents containing sentence-initial topics in ISL and ASL are distinct as well, as Fig. 7 illustrates.
Topics in (a) ISL and (b) ASL.
In the ASL data, the vast majority of sentence-initial topics (95%) are marked with raised brows, while in ISL almost 90% are marked by squint. Another salient difference between the two languages is in the head position. Whereas more than 80% of ASL topics were signed with a raised head position over the entire topic, 90% of ISL topics were characterised by gradual movement of the head from a neutral or slightly raised position at the beginning of the topic constituent to a lean forward (and sometimes down) toward the end. The frequency of use of these markers is shown in Fig. 8. The findings for ASL are consistent with earlier treatments of topic marking (e.g. Liddell Reference Liddell1980).
Frequency of facial action units in ISL and ASL topics.
While prototypical ASL topics have raised brows and prototypical ISL topics have narrowed eye aperture (squint), each language also makes use of the other articulation, often in combination with the prototypical one. That is, raised brows occur in 40% of ISL topics, and squint occurs in about 45% of ASL topics. We see then that two facial articulations, raised brows and narrowed eye aperture, are associated with topics in both sign languages, while frequency of occurrence is the opposite in each language. We return to this point in §4.
3.2.5 Conditionals
In all conditional utterances in ISL and ASL, the if clauses were accompanied by raised brows. However, the head positions are different in the two languages. In 85% of ASL if clauses, a raised head position is maintained across the whole length of the clause. The ISL if clauses, however, are associated with gradual head position change from neutral to a forward movement (95%). The head positions for the if clause of conditionals in each language are the same as for topics in that language, and therefore different in each language. Figure 9 shows a typical constellation of intonational cues characteristic of a conditional clause in ASL and ISL. Another difference between the facial configurations for conditionals in the two languages is that in ISL, the if clause tends to be accompanied by upper lid raise (AU 5), while this is less common in ASL, similarly to what we found for wh- questions. The widened eyes are more salient in conditionals than in wh- questions, since the brows are raised rather than lowered. Impressionistically, then, the articulation with raised upper eyelids is more characteristic of ISL than of ASL. The examples from both languages in Fig. 9 were extracted from the first clause of the sentence ‘If you come with me to Hawaii, I'll be the happiest person in the world’. The graph in Fig. 10 shows the occurrences of intonational signals on conditionals in the two languages.
Conditionals in (a) ISL and (b) ASL.
Frequency of facial action units in ISL and ASL conditionals.
3.2.6 Realisational differences
We have said that both ASL and ISL narrow the eye aperture on certain types of constituents, and we refer to this articulation as ‘squint’. From the contexts in which they occur, we interpret this articulation in both languages as a signal that certain information is known to the addressee, but not automatically accessed from the discourse context. In ASL, squint is also often used affectively (e.g. to evoke empathy, as in the sentence ‘She is sick’), so that overall our impression is that squint is not as systematic or grammaticalised in ASL as it is in ISL. Closer inspection of squint in both languages also reveals a phonetic difference in its articulation cross-linguistically. What we are informally calling ‘squint’ is typically achieved by different muscle actions for each language: in ISL by lower lid tightening (AU 7), and in ASL by raising the cheek (AU 6).
In addition, the narrowing of the eye aperture in the two languages is usually accompanied by the activation of other muscles outlining the infraorbital triangle – the muscles at the upper part of the mouth and around the nose. We refer to the visual impression as ‘nasolabial prominence’ in Fig. 11. Yet these physical characteristics also vary in the two languages. Lower lid tightening in ISL is accompanied by deepening of the nasolabial furrow (AU 11), while in ASL cheek raise usually co-occurs with upper lip raise (AU 10). Thus these phonetic differences lead to similar but not identical facial appearances in the two languages.
‘Squint’ and ‘nasolabial prominence’ in the two sign languages: (a) ‘squint’ in ISL involves lower lid tightening (AU 7) and nasolabial fold deepening (AU 11); (b) ‘squint’ in ASL involves cheek raise (AU 6) and upper lip raise (AU 10).
4 Analysis and discussion
The preceding section elaborated similarities and, perhaps more interestingly, differences in the prosodic systems of American Sign Language and Israeli Sign Language. We summarise these here, and provide an analysis of certain intonational differences that we found.
4.1 Similarities in the two languages
Each language uses similar manual timing cues (hold, repetition or pause) to delineate prosodic constituents, specifically intonational phrases. However, ASL does so less consistently, as we discuss below. Certain non-manual signals are associated with meanings or pragmatic functions similar to those found in spoken language intonation, and, like intonational boundary tones, align themselves with manually cued prosodic constituent boundaries. On the basis of function and distribution, then, these findings provide additional evidence that the specific non-manual signals in the contexts investigated correspond to intonation. Additionally, we found that some intonational arrays are marked similarly for similar functions in the two languages. Yes/no questions have the same intonational character in ASL and ISL, and wh- questions are similarly (though not identically) marked as well. Finally, both languages mark if clauses in conditionals with brow raise, although other markings differ on this construction. Just as high tone often marks yes/no questions as well as continuation (as in if clauses) in spoken languages, so raised brows have a similar function in sign languages, at least in the two investigated here. Similarly, furrowed brow, already reported for wh- questions in many sign languages, was found to characterise these constructions in the two languages. These findings provide additional confirmation that this intonational marking is widespread in sign languages. We now turn to the differences found in this study.
4.2 Differences in prosodic constituency marking
First, there are what might be called ‘chunking’ differences. As we have said, while both ISL and ASL tend to separate sentence-initial topics through timing, i.e. phrase-final lengthening, the tendency to do so in our ISL data is greater than in the ASL data: 98% vs. 76%. In spoken language, different factors, such as syntactic complexity, prosodic length, rate of speech and relative length of adjacent constituents, have been shown to explain variable behaviour of marking breaks between constituents in the same language and across languages (Bing Reference Bing1979, Selkirk Reference Selkirk1984, Nespor & Vogel Reference Nespor and Vogel1986, Jun Reference Jun2003). After careful analysis, we were unable to find reliable criteria for determining which ASL topics were likely to be demarcated by manual timing in our data. For some signers, speed was a consistent factor (only slower productions triggered demarcation of topics by lengthening), but for others, neither speed, prosodic length of a constituent nor syntactic complexity were reliable predictors. As in spoken language, phrase boundaries are remarkably difficult to define and identify consistently, due to their numerous and various manifestations, from a clear pause accompanied by a drastic pitch reset, to a subtle local change of tempo or pitch (Ladd Reference Ladd1996: 235). A study that intentionally manipulates the various possible factors is needed to account for the behaviour of ASL in this regard, and we leave this for future research. In any case, we find that topics – typically syntactic phrases or clauses – were not necessarily marked through timing in ASL. They were, however, marked intonationally.
4.3 Distribution of facial cues in ASL and ISL topic constituents: an analysis
A second difference is found in the particular facial intonational arrays that characterise topic constituents. We analyse these differences in terms of different preferences for marking information structure in the two languages.
The role of intonation in the interpretation of numerous information-structure phenomena has been much studied in spoken language, showing that the basic categories of information structure – topic–comment distinctions and given–new status – can be marked intonationally (Terken & Hirschberg Reference Terken and Hirschberg1994, Prevost Reference Prevost1995, Steedman Reference Steedman2000, Baumann Reference Baumann2006). For example, in German and English, the prosodic marking of these dimensions is mainly achieved by the speaker's choice of the position and the type of pitch accent, and by the division of utterances into intonational phrases. In their overview of sign language information structure, Kimmelman & Pfau (forthcoming) demonstrate that information-structure concepts such as topic–comment and given–new have been overlooked, and are crucial for the analysis of sign language grammatical structure.
Prototypical ASL topics are associated with raised brows (98%), whereas in ISL topic constituents are usually marked by squint (88%), though each language also sometimes exhibits the other articulation (often in combination with the prototypical one) as well, with raised brows occurring on 40% of ISL topics, and squint on about 45% of ASL topics.
We hypothesise that the two intonational elements, brow raise and squint, have similar meanings in the two sign languages, based on the types of structures on which they occur. We propose that each language has a different principled preference in the distribution of facial components with respect to different dimensions of topichood. In ISL intonational marking (with squint) of information that is shared but not immediately accessible is paramount, while in ASL it is the continuation from one constituent to the next that is paramount and obligatorily marked (with brow raise). This implies that the two sign languages might prioritise marking different information structure dimensions with intonation – either topic–comment or given–new. Since here we are interested in the interaction between intonation and information structure, we first give a short account of the relevant dimensions and their prosodic marking in other (spoken) languages.
4.3.1 The topic–comment dimension
Structurally, the topic is usually the initial element of the clause, and the rest is the comment. Functionally, the topic constituent ‘identifies the entity or a set of entities under which the information expressed in the comment constituent should be stored in the C[ommon] G[round] content’ (Krifka Reference Krifka, Féry, Fanselow and Krifka2007). Thus, while the two sentences (3a, b) express the same proposition, they structure it differently: (a) is meant to be stored as information about Jack, whereas (b) is meant to be stored as information about Mary.
(3)
In general, phrases beginning new topics are produced with a wider pitch range than other phrases. Contour type has also been correlated with topic constituents (Brown et al. Reference Brown, Currie and Kenworthy1980, Hirschberg & Pierrehumbert Reference Hirschberg and Pierrehumbert1986, Swerts et al. Reference Swerts, Geluykens, Terken, Ohala, Nearey, Derwing, Hodge and Wiebe1992). In particular, Hirschberg & Pierrehumbert (Reference Hirschberg and Pierrehumbert1986) suggests that so-called downstepped contours in English are commonly associated with topics. Empirical studies have shown that cross-linguistically ‘not-low’ boundary tones are good predictors of topic continuations (Swerts et al. Reference Swerts, Geluykens, Terken, Ohala, Nearey, Derwing, Hodge and Wiebe1992, Hirschberg Reference Hirschberg, Horn and Ward2004). Steedman (Reference Steedman1991) claims that the pitch contour described as L+H*LH% is one of the topic tunes of English. We now turn to the interaction of intonation with given and new information.
4.3.2 The given–new dimension
Another factor in information structure is the level of accessibility of referents or propositions in the interlocutors' minds, often expressed by the terms ‘given’ and ‘new’. In some languages, speakers deaccent items that represent old or given information in the discourse (Prince Reference Prince and Cole1981). Yet there is no simple one-to-one mapping between givenness and deaccenting: both given and new information can be accented, with the type of pitch accent or contour distinguishing them (Halliday Reference Halliday1967, Brazil et al. Reference Brazil, Coulthard and Johns1980, Pierrehumbert & Hirschberg Reference Pierrehumbert, Hirschberg, Cohen, Morgan and Pollack1990).
A number of recent studies have supported Chafe's (Reference Chafe1973) proposal that different types of intonational features are used to distinguish various degrees of referent accessibility, rather than a binary relation. Some accounts have shown that different accent types might be used for marking different accessibility states (Pierrehumbert & Hirschberg Reference Pierrehumbert, Hirschberg, Cohen, Morgan and Pollack1990, Kohler Reference Kohler1991, Baumann Reference Baumann2005, Reference Baumann2006). Here we follow Chafe in interpreting information that falls on the continuum from ‘given’ to ‘new’ in terms of a scale of accessibility.
We further rely on Ariel's (Reference Ariel1990, Reference Ariel1991, Reference Ariel, Sanders, Schilperoord and Spooren2001) theory of accessibility marking, which provides a detailed and nuanced scale of referent accessibility, determined by a range of contextual factors. For example, a referent is considered not automatically accessible if the information it conveys is geographically distant or if it is conceptually distant because it has not been a topic of recent discourse. (4) shows some typical correlations between relative accessibility and its grammatical expression according to Ariel.
(4)
The lower a referent is on the accessibility scale, the more likely it is to be marked by various language devices, among them prosodic cues. Ariel suggests a three-way split in degree of accessibility, where the structures mapped on the lower end of the accessibility scale typically include proper names and definite descriptions, as well as their combinations, constituents such as first names, demonstratives and stressed pronouns are distributed along the intermediate part of the accessibility scale, and the structures at the higher end of accessibility typically include personal pronouns and zero anaphora. (5), from the Wikipedia entry for ‘Dostoyevsky’, demonstrates a typical use of the accessibility scale, using progressively higher accessibility marking throughout the paragraph as the referent is established in the discourse.
(5)
Thus a typical paragraph starts with a full name, sometimes in combination with a definite description (a low accessibility marker), later mentions the same referent with only a first or last name (a higher accessibility marker) and finally uses only a pronoun (a high accessibility marker).
4.3.3 Prosodic marking of information status in different spoken languages
Perception experiments on German (Baumann & Hadelich Reference Baumann, Hadelich and Solé2003, Baumann & Grice Reference Baumann and Grice2006, Schumacher & Baumann Reference Schumacher and Baumann2010) have shown that, from the perspective of prosodic marking, information between the poles given and new cannot be treated as a uniform category, and that different types of more or less accessible information demand different accent types. Baumann and his co-authors have demonstrated that one particular type of pitch accent, H+L*, is significantly preferred over H* and deaccentuation as a marker of accessible information in German. In English, various degrees of accessibility are also realised through a particular choice of pitch accents, as summarised in Table II.
Relationship between pitch accent and givenness status for American English, as proposed by Pierrehumbert & Hirschberg (1990).
4.3.4 Different prosodic marking of two information dimensions with squint and brow raise in ASL and ISL
We begin with squint. Figure 7 above shows the prototypical facial array for topics in each of the two sign languages studied here. We suggest that the ISL intonational system is more sensitive to the accessibility status of a constituent than that of ASL. Specifically, in ISL, squint often co-occurs with such structures as restrictive relative clauses, counterfactual conditionals, parentheticals and temporal clauses referring to the remote past, structures which are associated with mid to low accessibility in Ariel's theory.Footnote 10 ASL tends to reserve squint for constituents with very low accessibility only, as evidenced by their common occurrence in relative clauses and long noun phrases with multiple modifiers.
For example, the topic constituent ‘Joe's brother’ in the sentence ‘Joe's brother died in a car accident’ is marked with squint in ISL, whereas in ASL it is accompanied by the continuation marker, brow raise, as in (6) and Fig. 12.
(6)
Different intonational marking of the topic ‘Joe's brother’ in the sentence ‘Joe's brother died in a car accident’ in (a) ISL and (b) ASL.
In this example the referent of the ISL topic, ‘Joe’ (‘Yossi’ in Hebrew), is not automatically accessible from the immediate discourse context, although use of this name without additional information about his identity assumes some acquaintance on the part of the addressee, and therefore the constituent in ISL is marked by squint as an instruction to the addressee to retrieve this information from memory or inference. While we interpret the function of squint in ASL as an instruction to retrieve information as well, its use is restricted to contexts with very low accessibility. The ASL topic in (6b) is not of the lowest retrieval status, but rather of low-mid level accessibility, as reflected by the use of only two words, one of them the first name, ‘Joe’. It is therefore less likely to be marked with squint.
It is only when topics are ranked very low on the accessibility scale that they are marked by narrowed eyes in ASL. One such topic is presented in (7). Here, the referent of the relative clause, the apartment, is not a current topic of conversation, but is assumed to be known from some prior experience. The relative clause, by providing extensive lexical information, fulfils its function of moving a referent that the addressee can identify or recognise into the centre of attention of communicative interaction between the interlocutors. The use of the squint signals that the set of competing referents (other apartments) should be restricted to one, about which the predication holds. Thus, although the referent of the topic expression is mutually retrievable, it is ranked low on the accessibility scale, because it is situated in a non-immediate spatial and temporal frame and is competing with other referents. In sum, the frequency of squint on topics is much higher in ISL than in ASL.
(7)
We used Ariel's scale to rank the topics in all sentence productions in each language in order to quantify the different intonation patterns in terms of accessibility. Although Ariel (Reference Ariel1990: 29) acknowledges that actual accessibility marking systems are to some extent language-specific, for the most part they are based on a principled connection between marker form and degree of accessibility. That is, the more informative (e.g. including more lexical information), unambiguous and unattenuated the marker, the lower the accessibility it is specified for, and vice versa. In the present study, we assigned accessibility ratings to all of the sentence-initial topics in our corpus in accordance with these principles. Figure 13 illustrates the distributions of squints on topics with different degrees of accessibility, and the differences between the two languages in this respect. It demonstrates that although both languages tend to signal low accessibility by squint, the percentage of such topics is higher for ISL, particularly for topics of mid accessibility.
The distribution of squint in ISL and ASL topics along different degrees of referent accessibility. The figures represent the number of topics with squint out of the total number of topics.
The y-axis of the graph shows the percentage yielded by the relation between the number of exemplars with squint out of the total number of possible targets for that action unit according to our accessibility analysis. Both languages tend to mark topics of low accessibility with squint, though this tendency is stronger in ISL (92%; 48 out of 52 tokens) than in ASL (67%; 37 out of 55 tokens). However, there is a striking difference between ASL and ISL at the intermediate (mid) degree of accessibility, with 77% of mid accessibility topics marked by squint in ISL (28 out of 36), and only 18% in ASL (7 out of 40).
We now turn to brow raise. As we noted above, brow raise seems to correspond to the meaning denoted by high edge tones in many spoken languages. It signals continuation and forward directionality (Bolinger Reference Bolinger, Greenberg, Ferguson and Moravcsik1978, Pierrehumbert & Hirschberg Reference Pierrehumbert, Hirschberg, Cohen, Morgan and Pollack1990, Bartels Reference Bartels1999), indicating that the prosodic constituent which it marks is to be followed by another phrase, produced either by the same interlocutor or, in the case of polar questions, by another. This general continuation meaning can, by implicature, have different, relatively more concrete interpretations, depending on other properties of the utterance (Bartels Reference Bartels1999). For example, by signalling continuation and forward directionality, brow raise, like high tone, can contribute to the contingency relations between phrases and clauses.
However, the two languages do not use this signal with equal frequency. ISL does not select brow raise with the majority of topics as ASL does, but rather with a subset: in contexts where the continuation from one constituent to another implies contingency. The contingency relationship always involves an asymmetrical relationship of precedence and consequence, and often (but not always) cause and effect (Jacobsen Reference Jacobsen, Kamio and Takami1999). The following types of topics are associated with the meaning of contingency: stage topics with temporal and spatial information (see e.g. Krifka Reference Krifka, Féry, Fanselow and Krifka2007) and relative clauses with an implied enabling condition (see e.g. Ziv Reference Ziv, Athanasiadou and Dirven1997: 229–230). A relative clause with an implied enabling condition is exemplified in (8). As with most sentence-initial topics, this topic also has brow raise in ASL.
(8)
Brow raise in (8) signals the contingency between the clauses, where the relative clause provides the enabling condition (getting a grade above 80) for the felicitous predication of the main clause (going to the next course). Informally, when the topic can be understood as a condition, it gets the same intonational marking as the if clause of ISL conditionals, and although the Hebrew sentence eliciting this marking was not a conditional, either interpretation of the ISL sentence is possible. The correlation of contingency with brow-raise marking on topics in ISL and ASL is illustrated in Fig. 14. As can be seen in the graph, ASL topics associated with brow raise constitute 95% of all the topics in our data (94 out of 98 tokens). Of these, topics with contingency implication amount to 30% of the total number of ASL topics (32 tokens). In contrast, the number of ISL topics marked by raised brows is much lower – 40% (35 out of 88), and contingency topics make up the majority of such cases (29 tokens).
The number of contingency topics out of all topics marked by raised brows in ISL and ASL.
In sum, the two sign languages seem to have different preferences with regard to intonational topic marking: ASL topics are systematically marked with brow raise highlighting the continuation relation between the topic and the comment, whereas ISL topics are consistently accompanied by squint, as constituents that typically contain retrievable or shared referents.Footnote 12 The results of the analysis of the two sign languages support the assumption that the given–new dimension of information structure is not a binary distinction, but a continuum, since the two languages are sensitive to different degrees of accessibility in their intonational marking. In this way, we find that information structure predicts intonational marking, providing further evidence that the signals in question are part of an intonational system. The claim that the markers are intonational is strengthened by the fact that their occurrence can be accounted for in terms of information structure, while an appeal to syntactic structure would not help us. Our analysis thus further supports the intonation approach to the structure and distribution of these signals in sign languages.
4.4 Phonetic differences in the realisations of grammatical facial expressions
Another difference between ASL and ISL that gives a different ‘flavour’ to the intonation of each language is in the different phonetic realisation of what appears to be the same facial ‘tone’ – what we have been calling squint.
In spoken languages, the universal tendency of using high pitch for polar questions is implemented differently in different languages. For example, for polar questions, English uses a rising tune, while in Jita, high pitch is used as well, but is implemented by raising the register rather than the local tune (Gussenhoven Reference Gussenhoven2004). Similarly, Pierrehumbert & Beckman (Reference Pierrehumbert and Beckman1988) demonstrate that tunes that are phonologically the same can have different phonetic implementations. An example is the phrase-final declarative L% boundary tone in English and Japanese. When asked to produce declarative patterns in many different voice levels, each English-speaking subject displayed a fixed value of L%, irrespective of the preceding environment. The implementation rule for Tokyo Japanese L% yielded contrasting results: the F0 value of L% was not fixed for each speaker, but was rather derived as a cumulative function of the overall pitch range and the preceding intonational events. Similarly, addressing one of the main issues of our paper – prosodic markers of accessibility – Baumann observed that German marks information of intermediate ‘newness’ status with H+L* and an early peak, while English marks the same information with downstep, either !H* or H+H! (Baumann Reference Baumann2006).
Fine-tuned coding made possible by FACS reveals that the low-accessibility marker we refer to as ‘squint’ is articulated differently in each of the sign languages we studied. ISL most often produces narrowed eyes through the interaction of lower lid tighten (AU 7) and nasolabial deepen (AU 11), while ASL typically employs a combination of cheek raise (AU 6) and upper lip raise (AU 10) for a similar effect. AU 6 and AU 7 both narrow the eye aperture to produce the impression of a squint, and AU 10 and AU11 both make the nasolabial fold salient to the viewer, though in the two cases, different muscles are used. The difference is demonstrated in Fig. 11 above.
The grammatical function of the narrowed eye aperture in the squint might be traced back to its non-linguistic use. Ekman (Reference Ekman2003: 139–140, 238) notes that tensed lower eyelids can indicate ‘having trouble focusing on something (literally or figuratively)’ or intense concentration on something. In his analysis of this facial action he follows Darwin (Reference Darwin1872: 223–224), who identified three main functional aspects of tensed lower eyelids: concentrating on something specific, an object that is distant or difficult to see, and excluding the rest of the environment. As Gussenhoven (Reference Gussenhoven2004: 50) notes, ‘a crucial difference between meaning expressed in the phonetic implementation and meaning expressed by the intonational morphology is that the latter can – but need not! – be arbitrary’. The difference between ASL and ISL intonational inventories illustrates this generalisation: once the natural form–meaning relation of the narrowed eye opening became phonologically encoded, it acquired a language-specific, and therefore partially arbitrary, form.
4.5 Head movements as part of the intonational inventory
Reilly et al.'s (Reference Reilly and Bellugi1990) account of ASL conditionals attributed non-manual markers of the protases, including head position, to the intonational system. However, in more general investigations of sign language prosody, head position has a more ambiguous status. The results of the present study suggest that the role of head position is indeed intonational. Initial topics and adverbial clauses are characterised by consistent configurations of head postures in each sign language. In particular, in ISL these constructions are marked by an intonational contour across the two constituents: the first constituent is marked by head movement that starts in a neutral or slightly up position and gradually proceeds to a forward position, whereas the left boundary of the second constituent is associated with a head position which is backward and up. The same constituents are marked differently in ASL: a static upward head position marks the scope of either a topic or an introductory adverbial clause, which is relaxed to neutral on the next constituent. These differences are shown in Figs 7 and 9 above, and exemplified in (9).
(9)
The head positions and movements in our study, like the facial expressions, meet all the defining criteria for intonation that we established in §1 of this article. They also meet the somewhat overlapping criteria established by Ladd (Reference Ladd1996): they are suprasegmental, postlexical and ‘linguistically structured’. Evidence that the head positions are suprasegmental is that they are often articulated across several signs. They are also postlexical, occurring across whole constituents and contributing to their interpretation. Finally, the signals are linguistically structured: their arrays and alignments differ in the two languages, they consistently correspond to particular types of structures and their timing is tightly adjusted to the timing of the prosodic constituents, as indicated by the scope of the lines in the examples above. We conclude then that the actions of the head, in addition to those of articulators of the face, are intonational in sign languages. This multi-articulator characterisation raises the questions of whether intonation in spoken and signed language can meaningfully be considered ‘the same’, and what we can learn about intonation from sign language. We turn to these in the conclusion.
5 Summary and conclusion
It is not obvious a priori that sign languages should have prosodic systems at all. In fact, a number of researchers have claimed that some of the signals investigated here belong exclusively to the syntactic component, while others attribute their distribution to syntactic structure, both implying that the system under investigation is quite different from any in spoken languages. It is also fully possible that the non-manual markers in question could be paralinguistic ‘visual prosody’ of the kind that some researchers claim accompanies speech (see Krahmer & Swerts Reference Krahmer and Swerts2009). Without a systematic demonstration that sign languages can differ from one another in their intonational systems, the possibility that these systems are not fully grammaticalised could still be entertained. By investigating and comparing two different sign languages, and analysing the differences between their intonational systems, we have found strong confirmation both for their intonational character and for the claim that they are conventionalised grammatical systems. Yet the broader theoretical question behind any investigation of sign language intonation (and prosody) is: what can it teach us about intonation in language generally? We summarise the implications of our findings in §5.1, note the empirical differences between the two languages in §5.2 and turn to the broader question in §5.3.
5.1 Intonation and prosody in sign languages
A number of earlier studies have argued for or assumed an intonational system in sign languages, cited throughout this article. Here we have provided additional evidence for this claim by showing systematicity in two sign languages, using the same sentences signed by several signers in each language, and the same methods of coding and analysis. Specific non-manual components predictably mark constituents for particular pragmatic properties in each language. That their distribution is systematic and conventionalised, and differs between the two languages, provides further support for the view that they are grammatical. In sum, we have provided evidence in support of all the criteria we adopted at the outset: (i) participation in a prosodic system which is dissociable from other components of the grammar, (ii) systematicity and conventionalisation and (iii) linguistic functions common to those of more familiar intonational systems found in spoken languages.
An important issue in supporting the existence of a prosodic component is determining whether certain systematic non-manual signals are integral to the syntax or to the intonation of these languages. First, our findings show that the syntactic nature of a constituent could not fully determine the marking it received. For example, topics tend to have characteristic intonational marking regardless of whether they are adverbial phrases, object noun phrases, subject noun phrases or discourse topics with no syntactic role in the matrix sentence.Footnote 13 At the same time, there is an interaction between intonation and syntax, as reflected for example in the types of syntactic constituents that typically occur at different intonationally marked levels of the accessibility hierarchy. Second, we argue that differences in the distribution of intonational elements are best predicted by pragmatic (not syntactic) categories and relations, such as language-specific sensitivity to the degree of accessibility and types of continuation. The analysis further supports the theoretical position that intonation is compositionally structured in sign languages, as has been argued for spoken language by some researchers.
5.2 Different intonational grammars in different sign languages
On the empirical level, the comparison results in a more detailed and nuanced analysis of the intonation of each language than has previously been available, and the first comparative description. We found similarities in the way the languages mark intonational phrase boundaries and polar questions. This finding is reminiscent of the strong cross-linguistic tendency in spoken languages to mark constituents such as non-final clauses and interrogatives with high phrase tones. However, we found language-specific prosodic marking on topic–comment constructions, conditional sentences and content questions, which we accounted for by appealing to pragmatic principles. Finally, we described a perceptually similar, yet phonetically distinct marker used in the two languages to signal that a referent is not automatically accessible from the discourse context, a marker which we can informally call the ‘shared information squint’.
5.3 Sign language intonation and linguistic theory
Similarities with spoken language intonation notwithstanding, it is too early, and probably wrong, to expect that sign languages are just like spoken languages, but just ‘happen’ to be transmitted by different articulators. We should therefore not be addressing questions about whether sign language intonation can resolve particular disputes within intonation theory. What, then, are the theoretical questions that can be fruitfully addressed through the study of sign language intonation?
Since sign languages seem to function just like spoken languages, are acquired by children just as automatically and on the same timetable as spoken languages (Newport & Meier Reference Newport, Meier and Slobin1985), have much neurological overlap (Emmorey Reference Emmorey2002) and emerge spontaneously whenever a community of deaf people has an opportunity to form (Senghas Reference Senghas2003, Sandler et al. Reference Sandler, Meir, Padden and Aronoff2005), linguists are justified in considering them to be full-fledged human languages. At the same time sign languages are not ‘just’ another bunch of languages. They are different from spoken languages in two important ways. First, they are not the dominant system selected by evolution, as they serve as primary communication systems among deaf people only, and second, in the creation of linguistic systems, they recruit all and only those articulators that produce signals which are perceived by sight and not sound. These differences are precisely why they are so valuable for linguistic research. They lead us to expect to learn two counterbalanced kinds of things from them: (i) what is basic and essential to any human language, regardless of modality, and (ii) which characteristics of language structure – both signed and spoken – can be accounted for by the physical transmission system.
Demonstrating that sign languages have conventionalised prosodic systems that can be isolated from other components of grammar represents a strong theoretical claim about a basic and essential property of human language. And showing that two sign languages have somewhat different though regular systems of this kind reinforces the claim that we are indeed talking about grammar. Most of the work on non-manual signals in sign language does not explicitly argue for these claims, and we are pleased to have been able to do so here.
Non-trivial differences from spoken language intonation are directly related to the medium of transmission, and these are equally instructive. Consider, for example, the property of linearity and the inventory of intonational primitives. While intonation in spoken language is suprasegmental and thus simultaneous with the text, the tones that comprise tunes are linearly organised. Furthermore, intonation is conveyed by pitch, using a single articulator, the vocal folds. Because of the availability of only one articulator for intonation, the number of possible phonological pitch features is small, and intonational tones must be produced sequentially.Footnote 14 This is one of many more obvious reasons that linguists have perceived linearity as an essential feature of language organisation.
Sign languages also have identifiable linear prosodic structure, such as sequences of temporally separated constituents, which are signalled mostly by the hands. The present study has even demonstrated that intonational head movements on topics in ISL can be characterised as a sequence of positions, from neutral or slightly raised to forward.
Nevertheless, in sign languages, many articulators are simultaneously exploited in the service of language, and there is a good deal of simultaneously organised structure at all levels of organisation (Vermeerbergen et al. Reference Vermeerbergen, Leeson, Crasborn, Vermeerbergen, Leeson and Crasborn2007).Footnote 15 In the present paper we have paid particular attention to the eyebrows, eyelids, cheeks and lips, as well as the head. In principle, this abundance of articulators with different degrees of freedom should allow for an enormous inventory of intonational arrays. In our compositional analysis, we have begun to study the independent and simultaneously executed contributions to the intonational system of each action of each articulator. It is not yet known how many actions and combinations distinctly and contrastively participate in the prosodic component of the grammars of sign languages and how they may combine, nor are we sure about the contribution of secondary features such as intensity. We can say, however, that several articulators contribute to intonational arrays simultaneously. The position of the head, the eyebrows and the eyelids can all vary independently and simultaneously to provide different intonational meanings, as we have seen. In this way, the intonational system has a much larger inventory of potentially distinct action units (tones) than is the case for spoken language. The intonational meaning characterised simultaneously by head forward, brow raise and wide eyes is changed if any of the articulators assumes a different position. This is quite different from spoken language intonation, which is argued to phonologically consist only of sequences of high and low tones, accented or unaccented.
The existence of the simultaneous combinatorial possibilities of sign languages leads us to ask the following theoretical question: How much of the prosodic and intonational system in spoken language is directly determined by the nature of the articulatory system? The extent to which intonation is phonetically grounded should have implications for our understanding of the universality of intonational primitives and structure and for the contribution of the physical system in the creation of grammar more generally.
Appendix
Some of these sentences were intended to elicit phenomena not dealt with in this article, such as emotions.
