1 Introduction
Sarcasm is a common phenomenon in interactive communication. It refers to the processes in which speakers use words to express an intention opposite to the literal meaning of those words (Kreuz & Glucksberg Reference Kreuz and Glucksberg1989, Capelli, Nakagawa & Madden Reference Capelli, Nakagawa and Madden1990, Anolli, Ciceri & Infantino Reference Anolli, Ciceri and Infantino2002).Footnote 1 Speakers can use sarcasm to either emphasise a criticism or soften a critical intent, and to harmonise the effect of a praise (Knox Reference Knox1961; Brown & Levinson Reference Brown and Levinson1987; Kreuz & Glucksberg Reference Kreuz and Glucksberg1989; Oring Reference Oring1994; Jorgensen Reference Jorgensen1996; Gibbs Reference Gibbs1999; Clift Reference Clift1999; Anolli, Ciceri & Infantino Reference Anolli, Ciceri and Infantino2000, Reference Anolli, Ciceri and Infantino2002; Kreuz Reference Kreuz2000; Rockwell Reference Rockwell2007). The use of sarcasm presents itself as an interesting challenge for developing spoken dialogue systems due to the complexity in its semantic nature and its expression (e.g. Tepperman, Traum & Narayanan Reference Tepperman, Traum and Narayanan2006). A thorough understanding of how speakers express sarcasm is thus not only desirable from a theoretical perspective but also necessary for achieving accurate recognition of speaker intention and generating adequate responses in real time in highly-interactive dialogue systems (Rakov & Rosenberg Reference Rakov and Rosenberg2013, Ward & DeVault Reference Ward and DeVault2017).
Speakers use both verbal means (e.g. syntactic, lexical, and vocal cues) and non-verbal means (e.g. facial expressions) to convey sarcasm (Rockwell Reference Rockwell2001, Attardo et al. Reference Attardo, Eisterhold, Hay and Poggi2003). The importance of vocal cues for the expression of sarcasm has long been recognised in theories of verbal irony (e.g. Clark & Gerrig Reference Clark and Gerrig1984, Sperber Reference Sperber1984), but how speakers use vocal cues has been empirically investigated only in recent years. Some researchers have argued that there are universal vocal cues for sarcasm, including a slower speaking rate, greater intensity, nasalisation, monotony, and extreme pitch levels (Rockwell Reference Rockwell2000, Reference Rockwell2007; Attardo et al. Reference Attardo, Eisterhold, Hay and Poggi2003). However, production studies examining a range of prosodic parameters in different languages have not only shown similarities but also differences in the use of prosody in expressing sarcasm across languages (e.g. Anolli et al. Reference Anolli, Ciceri and Infantino2002 on Italian; Cheang & Pell Reference Cheang and Pell2009 on Cantonese; Loevenbruck et al. Reference Loevenbruck, Ben Jannet, D'Imperio, Spini and Champagne-Lavau2013 and González-Fuente, Prieto & Noveck Reference González-Fuente, Prieto and Noveck2016 on French; Rao Reference Rao2013 on Mexican Spanish; Niebuhr Reference Niebuhr2014 on German; González-Fuente, Escandell-Vidal & Prieto Reference González-Fuente, Escandell-Vidal and Prieto2015 on Catalan). For example, sarcastic utterances are realised with a larger pitch span (difference between the highest and lowest pitch) in Italian (Anolli et al. Reference Anolli, Ciceri and Infantino2002) but a smaller pitch span in German and Mexican Spanish, compared to neutral or sincere utterances.
In the current study, we aim to obtain a deeper understanding of prosodic realisation of sarcasm in English by studying both what the prosodic differences are between sarcasm and sincerity and which prosodic and non-prosodic factors can predict the presence (or absence) of sarcasm. To this end, we take a novel linguistically-motivated approach to the locus of prosodic characteristics of sarcasm and address major methodological limitations in previous analysis. In the remaining part of this section, we first briefly review main findings from earlier research on prosodic realisation of sarcasm in English, then present our linguistically-motivated approach, and finally describe the methodological limitations to be tackled.
1.1 Past work on prosodic realisation of sarcasm in English
Previous production studies of prosodic realisation of sarcasm in English were exclusively concerned with North American English. For example, Rockwell (Reference Rockwell2000) examined utterances read out by professional speakers (e.g. radio announcers, actors) and experienced non-professional speakers in different contexts in American English and found that the speakers realised the utterances in a sarcastic context with a slower speech rate, lower mean pitch and greater intensity than in a neutral context. Rakov & Rosenberg (Reference Rakov and Rosenberg2013) were concerned with another type of read speech, i.e. acted speech by voice actors reading from a script. They analysed pitch span of each utterance and modelled the pitch and intensity contours both across the whole utterance and in each word of an utterance using Legendre polynomials (Dumouchel et al. Reference Dumouchel, Dehak, Attabi, Dehak and Boufaden2009). They found that sarcasm and sincerity in acted speech could be automatically detected with about 82% accuracy on the basis of combined information on pitch span and shape of pitch and intensity contours across the utterance and in each word. This finding suggests that sarcastic speech might differ from sincere speech in the shape of pitch and intensity contours as well as in pitch span. The authors observed that sarcastic speech had a smaller pitch span, fewer instances of falling pitch contours and shallowly rising pitch contours but more instances of shallowly rising intensity contours than sincere speech did.
Different from Rockwell (Reference Rockwell2000) and Rakov & Rosenberg (Reference Rakov and Rosenberg2013), Rockwell (Reference Rockwell2007) studied sarcastic prosody in spontaneous dialogues about pre-selected questions between familiar speakers in American English. Analysing only the utterances that were identified to convey sarcastic messages by the speakers themselves, Rockwell found that the sarcastic utterances were spoken with a higher mean pitch, larger pitch span, and longer duration, compared to the non-sarcastic utterances. The longer duration suggests a slower speech rate in sarcastic utterances in spontaneous speech, similar to what was found on read speech, whereas the use of a higher mean pitch and a larger pitch span was not found in sarcastic utterances in read speech (Rockwell Reference Rockwell2000, Rakov & Rosenberg Reference Rakov and Rosenberg2013). However, because the sarcastic utterances in Rockwell (Reference Rockwell2007) were lexically different from the non-sarcastic utterances, it casts doubt on the validity of the reported results based on direct comparisons between the sarcastic and non-sarcastic utterances. The differences in the results between these studies thus do not necessarily suggest different uses of prosody in sarcastic expression between read speech and spontaneous speech in American English. Bryant (Reference Bryant2010) also examined the prosody in sarcastic utterances produced in spontaneous speech in American English. He analysed utterance-level prosodic changes from two utterances preceding a sarcastic utterance to the sarcastic utterance and found that only speech rate differed significantly between the sarcastic utterance and its preceding utterances. That is, speech rate was lower in the sarcastic utterance than in the preceding utterances, in line with the result on speech rate reported by Rockwell (Reference Rockwell2000, Reference Rockwell2007).
Finally, Cheang & Pell (Reference Cheang and Pell2008) examined utterances read out in different contexts by native speakers of Canadian English. Analysing only the utterances that were perceived to sound sarcastic by other native speakers of Canadian English, they found that the utterances were consistently realised with a lower mean pitch, less within-utterance pitch variation, and a slower speech rate (albeit only in short utterances like ‘Is that so’) in the sarcastic context than in the sincere and humour contexts. The result on speech rate was compatible with the finding reported for read speech (Rockwell Reference Rockwell2000, Reference Rockwell2007) and spontaneous speech (Bryant Reference Bryant2010) in American English; the result on mean pitch was similar to the finding reported in read speech in American English (Rockwell Reference Rockwell2000). There thus appears to be more similarities than differences in prosodic realisation of sarcasm in the two varieties of North American English.
1.2 The current approach
Notably, the studies reviewed above have all taken the whole utterance as the target of analysis, i.e. analysing prosodic variation across the utterance or word-by-word, similar to the studies on sarcastic prosody in other languages (see González-Fuente et al. Reference González-Fuente, Prieto and Noveck2016 for an exception). Typical prosodic measurements include minimum pitch, maximum pitch, mean pitch, pitch span and speech rate of the utterance. While such a holistic approach has proved to be useful in this line of research, it is not entirely clear what the underlying linguistic or methodological motivation is. In this study, we focus on the word that is semantically critical to the expression of sarcasm in an utterance (hereafter key word) and examine whether sarcasm is prosodically distinguishable from sincerity in the key words alone. For example, in the utterance ‘She's a healthy lady’ said as a sarcastic response to a friend's remark about his aunt (‘My aunt smokes a pack a day’), the real message of the speaker is that the friend's aunt is an unhealthy lady. The word healthy thus holds the key to the sarcastic interpretation of the utterance, different from the other words in the same utterance. The key word is identifiable even when multiple words can play a role in the expression of sarcasm in an utterance. Take for example the utterance ‘The service is really good here’ said as a sarcastic response to the poor service in a restaurant. Although both really and good contribute to the expression of sarcasm, good is the key word, because the word really is dispensable but the opposite meeting of the word good is the real message.
The significance of the key word in a sarcastic utterance is comparable to that of the word in focus in an utterance. Focus is an information structural category, refers to the predication on a topic in an utterance, and typically contains new information to the receiver (Lambrecht Reference Lambrecht1994, Vallduví & Engdahl Reference Vallduví and Engdahl1996). The same utterance can be a felicitous response in different contexts, which subject the same word to different focus types or different words to the scope of focus. For example, the utterance ‘She's a healthy lady’ can be said as a response to the question What do you know about her?, the question Is she weak?, or the question She's a healthy what?. In the first two cases, the word healthy conveys new information sought by the questioner. But the status of the word healthy is different. In the first case, the whole verb phrase (is a healthy lady) is in focus (i.e. broad focus); the word healthy is part of the broad focus. In the second case, only the word healthy conveys new and contrastive information; the word healthy is thus in narrow contrastive focus. In the third case, the word lady is focal but the word healthy is not. The information structural difference between the three renditions of the utterance is primarily expressed in the prosody of the word healthy, although the post-focus word lady also undergoes some prosodic changes in the second case (e.g. Chen Reference Chen, Krifka and Musan2012). In the light of the parallel between the key word in a sarcastic utterance and the focal word in an utterance, we hypothesise that prosodic differences between sarcasm and sincerity can be found in the key words alone in British English. Preliminary evidence has been reported in a recent study on vocal cues to sarcasm in French. González-Fuente et al. (Reference González-Fuente, Prieto and Noveck2016) examined prosodic differences between the sarcastic and sincere renditions of six declaratives with the key word in utterance-final position, which were read aloud by ten female natives speakers of French. They found similar prosodic differences between the two renditions of utterances at the utterance- and word-levels.
1.3 Open methodological issues
Previous studies of sarcastic prosody in English (and other languages) have frequently represented pitch as static values (e.g. mean pitch, pitch-maximum, pitch-minimum, pitch span) in statistical analyses. This method stands in sharp contrast to the fact that pitch is a continuous variable and changes over time in an utterance, resulting in variation in the shape of the pitch contour. Our understanding of the role of contour shape in sarcastic expression is consequently still very limited. Traditionally, researchers transcribe the shape of the pitch contour in individual words of an utterance using two discrete tones (i.e. high and low tones) and their modifications (i.e. downstep, upstep), following notations proposed within the autosegmental–metrical framework (Pierrehumbert Reference Pierrehumbert1980, Ladd Reference Ladd1996). Such an approach can generate valuable insights into the role of contour shape in the expression of sarcasm (González-Fuente et al. Reference González-Fuente, Prieto and Noveck2016) and other attributes. But it requires substantial knowledge of intonational phonology and training in the annotators and labour-intensive inter-rater agreement experiments to assess the reliability of the resulting annotation (Grabe, Kochanski & Coleman Reference Grabe, Kochanski and Coleman2007). Also, it is questionable whether the discrete representation of the pitch contour in individual words can sufficiently capture subtle variation in the shape of pitch contour involved in the expression of affect and attitude such as sarcasm (Scherer & Bänziger Reference Scherer and Bänziger2004).
Moreover, the use of prosody in sarcasm has not been studied systematically in different utterance types. Certain utterance types appear to be more readily used for sarcasm (e.g. positive declaratives) than other utterance types (e.g. negative declaratives, tag questions) (Kreuz & Glucksberg Reference Kreuz and Glucksberg1989, Kreuz & Caucci Reference Kreuz and Caucci2007). This raises the question of whether there may be a functional trade-off between prosodic and syntactic cues, as postulated in the Functional Hypothesis (Haan Reference Haan2002). Stemming primarily from research on question intonation in Dutch, this hypothesis argues that speakers use prosody to a lesser degree to convey a meaning in utterances that contain more syntactic and lexical cues to the same meaning than in utterances that contain fewer such cues. For example, wh-questions contain more lexical and syntactic markers of questions than yes-no questions and declarative questions and thus have fewer prosodic markers of questions that the latter in Dutch.
Finally, gender-related differences have been reported in the use of prosody to express affect and attitude (e.g. Van Leeuwen Reference Van Leeuwen1999). It remains to be investigated whether male and female speakers use prosody differently to express sarcasm.
To circumvent the limitations in previous analysis on sarcastic prosody, we adopt Functional Data Analysis (hereafter FDA; Ramsay & Silverman Reference Ramsay and Silverman2005), a new method for analysing continuous phonetic parameters, to model pitch variation in the key words in sarcastic utterances and their counterparts in sincere utterances as contours and represent the contours as continuous functions in statistical analysis (Gubian, Boves & Cangemi Reference Gubian, Boves and Cangemi2011, Jokisch, Langenberg & Pintér Reference Jokisch, Langenberg and Pintér2014, Gubian, Torreira & Boves Reference Gubian, Torreira and Boves2015). Furthermore, to elucidate the use of prosody in different utterance types and gender-related differences, we take utterance type and gender into account in our experimental design.
2 The production experiment
2.1 Participants
Seventeen monolingual native speakers of the southern variety of British English (12 women, five men) participated in the production experiment. They were recruited from the student population of the University of Leeds, and between 18 and 23 years old (mean age = 21.6 years). They were paid a small fee for participation.
2.2 Tasks
We adapted the simulated telephone conversation task used in two studies of sarcastic prosody in second language learners of English (Chen & de Jong Reference Chen, de Jong and Chini2015, Smorenburg, Rodd & Chen Reference Smorenburg, Rodd and Chen2015) to elicit sarcastic production in a dialogue setting. In this task, the participants were asked to imagine themselves on the phone chatting with a good friend. This was done to make the participants feel comfortable about using sarcastic speech, as previous studies have found that friends and familiar speakers are more likely to produce sarcasm (Gibbs Reference Gibbs2000, Rockwell Reference Rockwell2007, Bryant Reference Bryant2010). The friend, who was a native speaker of the southern variety of British English, made a number of remarks about fictional people and situations. The participants’ task was to respond to the friend's remarks in a sarcastic manner. The utterances which they were supposed to say as the responses to the friend were displayed on PowerPoint slides (one slide per trial) to ensure consistency in choice of word and syntactic structure across participants.
Following the simulated telephone conversation task, the participants were asked to produce the response utterances in a sincere manner as responses to some imaginary remarks of the friend that would render a sincere manner appropriate. This time, only the response utterances were displayed on the PowerPoint slides and no context-setting remarks were played.
The participants were told in written instructions prior to the experiment that they could give an utterance a sarcastic or sincere sounding via prosody or the melody of speech. But they were not given definitions of ‘sarcastic’ and ‘sincere’. The manner in which they were supposed to respond was displayed on the PowerPoint slides throughout the experiment. This was done because previous studies of verbal behaviour have shown that moderate emotions, such as interest, distress and sarcasm, are encoded more accurately when speakers are given explicit instructions to produce the intended emotion than otherwise (Rockwell Reference Rockwell2000 and references therein).
2.3 Materials
The response utterances (to be produced by the participants) consisted of 48 utterances, representing three utterance types (16 utterances per type): simple (positive) declaratives (hereafter declaratives), utterances with negative question tags (hereafter tag questions), and wh-exclamatives. An example of each utterance type is given in (1)–(3), together with the context-setting remarks (only applicable in the ‘sarcastic’ condition).
(1) Declaratives
Remark: My aunt smokes a pack a day.
Response: She's a healthy lady.
(2) Tag questions
Remark: Kim turned up at my party even though she wasn't invited.
Response: You were pleased to see her, weren't you?
(3) Wh-exclamatives
Remark: My mother-in-law always smirks and snorts loudly when I misspeak.
Response: What a respectful gesture!
The context-setting remarks and response utterances (see the Appendix) were identical to the stimuli used in Smorenburg et al. (Reference Smorenburg, Rodd and Chen2015), which were partially derived from previous studies (Ackerman Reference Ackerman1983, Kreuz & Glucksberg Reference Kreuz and Glucksberg1989, Capelli et al. Reference Capelli, Nakagawa and Madden1990, Cheang & Pell. Reference Cheang and Pell2008, Chen & de Jong Reference Chen, de Jong and Chini2015). The response utterances were controlled for comparability in length and syntactic complexity. Three native speakers of British English, who had no connection to this study, were consulted to ensure equal acceptability of the response utterances being uttered sarcastically or sincerely on lexical and syntactic levels. The context-setting remarks were constructed carefully such that they created a convincing situation for the participants to respond in a sarcastic manner. However, in 19 of the cases, it was felt that the context-setting remarks alone might not sufficiently set the common ground between the participants and the English ‘friend’, which is supposed to facilitate the production of sarcasm (Kreuz et al. Reference Kreuz, Kassler, Coppenrath and Allen1999). Additional written background information was thus provided in these cases.
The context-setting remarks in the ‘sarcasm’ condition were recorded by a male native speaker of the southern variety of British English. The speaker was provided with the context-setting remark–response sequences in the form of a recording script, and was asked to familiarize himself with the remarks. He was then asked to produce the context-setting remark-response sequences together with a female research assistant, a highly proficient learner of British English, who acted as the interlocutor and produced the responses. The male speaker was instructed to say the context-setting remarks as if he was having real conversations with the interlocutor and expecting responses from her on his remarks. The speakers produced the 48 context-setting remark–response sequences twice and recorded at a sampling frequency of 44.1 kHz (16 bits accuracy) in a sound-attenuated booth at the Linguistics Laboratory of the Utrecht Institute of Linguistics. During both rounds of recordings, the male speaker was encouraged to make additional attempts if he was not satisfied with his first attempt or if the first author, who was present during the recording, noticed inconsistency in the male speaker's prosody. The context-setting remarks from the second round of recording were selected and saved as individual .wav files using Praat (Boersma & Weenink Reference Boersma and Weenink2014). In the case of multiple attempts, the last one was selected.
2.4 Procedure
The participants did the experiment individually in a sound-attenuated booth at the Phonetics Laboratory at the University of Leeds. Each participant was first presented with written instructions. In the instructions, the tasks and procedure of the experiment were explained. The participant was also informed that he/she was allowed to make multiple attempts on each trial until he/she was satisfied with the response. The stimulus order was semi-randomised such that responses of the same utterance type did not occur twice in a row.
The experiment began with six practice trials (two per utterance type), comparable to the experimental trials, and proceeded first with the 48 trials in the sarcasm condition and then the 48 trials in the sincerity condition. It was self-paced and conducted using a laptop. Each context-setting remark (only applicable on the ‘sarcastic’ trials), the corresponding response, additional background information (only applicable on 19 of the ‘sarcastic’ trials) and the manner of response (sarcastic or sincere) were displayed in a PowerPoint slide on the laptop screen on each trial. The participants could hear the context-setting remark from the fictional friend via a headphone set by clicking on a sound icon placed next to the context-setting remark in the slide and respond to it accordingly. They could move on to the next trial by pressing the ‘enter’ key on the keyboard and took a short break during the experiment. Their responses were recorded using a ZOOM 1 digital recorder. It took on the average about 15 minutes for the participants to finish the experiment.
3 Data annotation and pre-processing
A total of 1529 utterances were obtained from 16 speakers (11 female, five male). The data of one female speaker was lost due to technical failure. Another seven utterances were judged unusable due to unexpected noise during the recording. To check whether the speakers produced the utterances as required in their respective condition, we conducted a small perception experiment. In this experiment, eight native speakers of English (four British English, three American English, one British English and American English bilingual) listened to 243 utterances from six speakers (three female, three male) and judged for each utterance whether the speaker intended to be sarcastic or sincere in a two-alternative forced choice task. We conducted mixed-effect binary logistic regression in SPSS (IBM SPSS version 22) to statistically assess which variables were predictive of the listeners’ judgements (i.e. message_type_perceived). The predictors included three main effects, message_type_required (sarcastic vs. sincere), i.e. the message type that the speakers were supposed to produce in the production experiment, utterance_type (declarative, tag question, wh-exclamative) and variety_of_English spoken by the listeners (British English vs. American English), three two-way interactions between these predictors and one three-way interaction of these predictors. The model could predict the intended message type of an utterance correctly in 73.3% of the sarcastic cases and 73.9% of the sincere cases. Only one of the predictors made a significant contribution to the model: message_type_required (β = –1.854, SE = 0.207, t = –8.964, p < .01).
This result showed that the utterances were uttered by and large as they were supposed to in their respective conditions. Some utterances were perceived to sound opposite to what they were supposed to by some of the listeners. But the misperception of these utterances could be due to various reasons and itself did not suggest that the speakers did not produce the utterances according to the instructions in the production experiment. It is possible that the prosodic cues to sincerity and sarcasm might be weaker in these utterances than in other utterance. Such variation is, however, characteristic of prosody in natural speech. We therefore decided to include all the usable utterances into our analysis. These utterances were subsequently annotated and processed for further analysis in the following order.
First, the key word (see the underlined words in the Appendix) was identified for each sarcastic utterance in the light of the context-setting remarks and available background information. The key words in the sarcastic utterances and their counterparts in the sincere utterances were the words of interest (hereafter target words) in our prosodic and statistical analysis.
Second, each utterance was annotated for the begin and end of the target word by two annotators using Praat (Boersma & Weenink Reference Boersma and Weenink2014). Each annotator annotated a subset of the speakers, two of which were annotated by both of them. An intra-class correlation coefficient (ICC) test was conducted on the annotation in the target words of the two speakers annotated by both annotators (N = 192). The ICC for the duration annotation was .993 (F(191) = 142.451, p < .01), indicating that the annotation done by the two annotator was highly consistent. The annotation of all the utterances was subsequently checked by one of the annotators and was adjusted when necessary.
Third, the tool ProsodyPro (Xu Reference Xu2013) was used to extract from the target words the pitch and duration values at a sampling rate of 100 Hz (i.e. 10 ms spacing between successive points), including mean pitch, maximum pitch (pitch-max), minimum pitch (pitch-min), pitch span and word duration (duration). ProsodyPro exports continuous pitch values, even if the raw contour contain gaps due to the presence of voiceless sounds. Voiceless intervals are bridged by linear interpolation between the last pitch measurement before and the first pitch measurement after the gap.
Fourth, the pitch contours of the target words, which were of variable lengths and phonetic make-up, were normalised to have an equal number of samples, in preparation for FDA. The pitch contour of the shortest target word consisted of 12 samples of pitch values, while the pitch contour of the longest target word comprised 157 samples of pitch values. To create length-normalised contours, we re-sampled and smoothed the curves to obtain contours consisting of 50 samples each using spline interpolation (Gubian et al. Reference Gubian, Torreira and Boves2015). Spline interpolation transforms sampled data representations to a continuous mathematical function that is guaranteed to be smooth and differentiable. For example, spline interpolation removes the sharp edges that may be caused by linear interpolation to close gaps related to voiceless intervals. Time normalisation by means of spline interpolation is assumed to capture the linguistically relevant aspects of the contour shapes, which are supposed to be independent of the length and phonetic make-up of the words.
Finally, we performed Functional Principal Component Analysis, a tool available in FDA, on the pitch contours of all target words, and extracted functional principal components (FPC) for each pitch contour for further analysis, following the procedure described in Gubian et al. (Reference Gubian, Torreira and Boves2015). Conventional Principal Component Analysis (PCA) identifies a small number of orthogonal dimensions on which the original observations can be represented with minimal loss of detail. Similarly, Functional PCA returns a number of continuous functions that can be used to reconstruct the original functions, with minimal loss of accuracy. In our case, it represents pitch contours by continuous spline functions. The FPCs are orthogonal, meaning that the integral of the product of any pair of FPCs is zero, reminiscent of Fourier decomposition. Individual observations are reconstructed by summing the mean function (common to all observations) and weighted versions of the FPCs. The result of the functional PCA analysis consists of the weights of the FPCs for each individual observation. This makes continuous time functions suitable for statistical analysis that operates on discrete numbers. At the same time, the weights represent the complete pitch contour, rather than a subset of hand-picked points on a pitch contour.
In our analysis, we decided to extract the first five FPCs for the pitch contour of each target word. These FPCs accounted for 94% of the total variance in the pitch contours of the target words (FPC1: 54.1%, FPC2: 23.6%, FPC3: 8.4%, FPC4: 4.8%, FPC5: 2.8% of the variance). We could have extracted additional FPCs, but these would have probably not been informative. It is already doubtful whether FPC5 is relevant. Figure 1 shows the shape of each FPC together with the average normalized pitch contour of the target words. To illustrate how the FPCs can modify a pitch contour, we show in Figure 2 how adding each FPC by a positive weight and a negative weight can change the shape of the average normalized pitch contour of the target words. More specifically, FPC1 was negative at the left end and positive at the right end of the pitch contour. A positive weight of FPC1 would lower the pitch at the left end of the average normalised pitch contour, and raise the pitch towards the right end; a negative weight of FPC1 would raise the pitch at the left end of the average normalized pitch contour and lower the pitch towards the right end. Since the average normalised contour was characterised by a fall in the second half of the contour, a positive weight of FPC1 would make the contour overall flatter and a negative weight of FPC1 would make the contour overall steeper (Figure 2). FPC2 was positive at the left end and the right end, and negative in the middle of the pitch contour. A positive weight of FPC2 would shift the onset of the pitch fall towards the left end of the contour and make the fall less steep; a negative weight of FPC2 would shift the onset of the fall towards the right end of the contour, and make the fall steeper (Figure 2). FPC3 strengthens the effect of FPC2 in alignment of the fall: a positive weight would result in an earlier fall, while a negative weight would result in a later fall. FPCs 4 and 5 add rapid pitch fluctuations to the average pitch contour. Their effect may have more to do with the perceived naturalness of the speech than with linguistic or paralinguistic information. It should be noted that the FPCs do not have an intrinsic scale. Their contribution to the reconstruction of a specific pitch contour is completely determined by the weight assigned by the Functional PCA to the FPCs for that contour.
The average normalized pitch contour and the first five functional Principal Components of the target words (i.e. the key words in the sarcastic utterances and their counterparts in the sincere utterances). The horizontal axis corresponds to normalized time: t = 0 corresponds to the start of the target word, and t = 1 to the offset of the target word. The vertical axis corresponds to the normalized pitch in semitones, centered on zero.
Modifications of the first five functional Principal Components to the normalized average pitch contour of the target words (i.e. the key words in the sarcastic utterances and their counterparts in the sincere utterances). In each panel, the solid line represents the normalized average pitch contour; the broken line and the plus-signed line represent the contours that would be obtained if the FPC would be added to the average with a weight of minus or plus 4 respectively. The horizontal axis corresponds to normalized time: t = 0 corresponds to the start of the target word, and t = 1 to the offset of the target word. The vertical axis corresponds to the normalized pitch in semitones, centered on zero.
The procedure described above resulted in ten measurements for statistical analysis: mean pitch, pitch-min, pitch-max, pitch span, duration, FPC1, FPC2, FPC3, FPC4, and FPC5.
4 Statistical analysis and results
4.1 Prosodic differences between sarcasm and sincerity
We first analysed effects of the experimental variables on each of the ten measurements (i.e. outcome variables) with linear mixed-effect models, using the lme4 package (Bates & DebRoy Reference Bates and DebRoy2006) in the R environment (R Development Core Team Reference Development Core Team2016). We included message_type (sarcastic vs. sincere), utterance_type (declarative, tag question, wh-exclamative) and gender (male, female) as the fixed factors and speaker and utterance as the random factors. For each outcome variable, six models were built, starting with a model with only the constant and random factors and adding one more term (i.e. a main effect or an interaction) to each subsequent model. The anova function in R was used to compare model fit of different models. The model with the best-fit was determined by comparing each model with the winning model in the preceding model comparisons. The summary of the best-fit model indicated which terms reached statistical significance. As we are primarily interested in the effect of message_type in interaction with the other fixed factors, we will only report significant interactions between message_type and the other fixed factors, and in the absence of such interactions, significant main effect of message_type. In the case of interactions involving the factor message_type, we used mixed-effect models to analyse the effect of message_type in each utterance type or each gender to determine where the interactions came from.
4.1.1 Mean pitch
The best-fit model retained a significant interaction of message_type by gender (p < .01). Examining the effect of message_type in the production of male and female speakers separately, we found a significant main effect of message_type only in the female speakers’ production (β = 184.70, SE = 16.13, t = 11.45, p < .01). The female speakers used a significantly lower mean pitch in the sarcastic condition (201 Hz) than in the sincere condition (229 Hz), whereas the male speakers used similar pitch means in these conditions (124 Hz in the sarcastic condition vs. 131 Hz in the sincere condition). As can be seen in Figure 3, the pitch contours of the target words (biting, smart and brilliant) were positioned clearly lower in the pitch plot in the sarcastic condition than in the sincere condition in the female speakers (middle and bottom panels) but this was not the case in the male speaker (top panel).
Examples of the pitch contours of the target words produced in three utterance types (top panels: tag question, middle panels: wh-exclamative; bottom panels: declarative) in the sarcastic (left panels) and sincere (right panels) conditions by one male speaker (L34) (top panel) and one female speaker (L05) (middle and lower panels). The target word biting was from the utterance ‘They're biting this season, aren't’ they?’. The target word brilliant was from the utterance ‘What a brilliant header!’. The target word smart was from the utterance ‘You're a smart gamer’. ‘H’ stands for pitch-max, ‘L’ stands for pitch-min’. The horizontal axis shows the duration of each word in seconds; the vertical axis shows the amplitude in Pascals in the waveform and pitch in Hz in the pitch contour.
4.1.2 Pitch-min
The best-fit model retained a significant interaction of message_type by utterance_type (p < .01). Subsequent analyses revealed a significant main effect of message_type in each utterance type (β = 12.67, SE = 3.19, t = 3.97, p < .01 in declaratives; β = 14.68, SE = 3.05, t = 4.85, p < .01 in tag questions; β = 29.29, SE = 4.16, t = 7.05, p < .01 in wh-exclamatives). The speakers used a significantly lower pitch-min in the sarcastic condition than in the sincere condition across utterance types. Importantly, the difference in pitch-min was more pronounced in the wh-exclamatives (144 Hz in the sarcastic condition vs. 173 Hz in the sincere condition) than in the tag questions (133 Hz in the sarcastic condition vs. 148 Hz in the sincere condition) and declaratives (126 Hz in the sarcastic condition vs. 136 Hz in the sincere condition), as illustrated by examples in Figure 3.
4.1.3 Pitch-max
The best-fit model retained a significant interaction of message_type by utterance_type (p < .05). Subsequent analyses revealed a significant main effect of message_type in the tag questions (β = 28.21, SE = 4.51, t = 6.26, p < .01) and wh-exclamatives (β = 24.53, SE = 5.68, t = 4.32, p < .01). The speakers used a significantly lower pitch-max in the sarcastic condition than in the sincere condition in these two utterance types (203 Hz in the sarcastic condition vs. 231 Hz in the sincere condition in the tag questions; 239 Hz in the sarcastic condition vs. 262 Hz in the sincere condition in the wh-exclamatives) but not in the declaratives (208 Hz in the sarcastic condition vs. 216 Hz in the sincere condition), as illustrated by examples in Figure 3.
4.1.4 Pitch span
The best-fit model did not retain any significant main effects and interactions for any of the fixed factors for variation in pitch span. The speakers thus did not vary pitch span systematically to express sarcasm.
4.1.5 Duration
The best-fit model retained a significant interaction of message_type by gender (p < .01). Subsequent analyses revealed a significant main effect of message_type in both the male (β = –97.23, SE = 8.85, t = –10.99, p < .01) and female speakers (β = –58.68, SE = 5.51, t = –10.65, p < .01). Both groups of speakers used a significantly longer duration in the sarcastic condition than in the sincere condition, but the male speakers exhibited a larger duration difference (507 ms in the sarcastic condition vs. 407 ms in the sincere condition) than the female speakers (521 ms in the sarcastic condition vs. 463 ms in the sincere condition), as illustrated in the examples in Figure 4.
Renditions of the target word biting in the utterance ‘They're biting this season, aren't they?’ produced by a male speaker (L34) (top panels) and a female speaker (L03) (bottom panels) in the ‘sarcastic’ condition (left panels) and the ‘sincere’ condition’ (right panels). The duration difference in biting between the two conditions was 56 ms in the male speaker but 26 ms in the female speaker. The horizontal axis shows the duration of each word in seconds; the vertical axis shows the amplitude in Pascals.
4.1.6 FPC1
The best-fit model retained a significant main effect of message_type (β = –4.85, SE = 1.26, t = –3.84, p < .01). FPC1 was positive (2.19) in the sarcastic condition but negative in the sincere condition (–2.19). That is, the pitch contours of the target words had a flatter fall in the sarcastic condition than in the sincere condition.
4.1.7 FPC3
The best-fit model retained a significant interaction of message_type by utterance_type (p < .01). Subsequent analyses revealed a significant main effect of message_type only in the tag questions (β = 1.42, SE = 0.48, t = 2.93, p < .01). FPC3 was negative in the sarcastic condition (–0.34) and positive in the sincere condition (1.09) in these utterances. That is, the contours of the target words in the tag questions had a later fall in the sarcastic condition than in the sincere condition
4.1.8 FPCs 2, 4 and 5
The best-fit model did not retain any significant main effects and interactions for any of the fixed factors regarding FPCs 2, 4 and 5.
4.1.9 Interim summary
Our linear mixed-effect models show that six of the ten measurements (mean pitch, pitch-min, pitch-max, duration, FPC1, FPC3) were significantly different between the sarcastic condition and the sincere condition. However, the measurements mean pitch and duration were not varied to the same degree in both groups of speakers. Specifically, only the female speakers used a significantly lower mean pitch in the sarcastic condition than in the sincere condition; the male speakers produced a significantly larger difference in duration between the sarcastic and sincere conditions than the female speakers. Further, the measurements pitch-min, pitch-max and FPC3 were not varied to the same degree in all utterance types. Specifically, the speakers lowered pitch-min in the sarcastic condition to a larger degree in the wh-exclamatives than in tag questions and declaratives, used a significantly lower pitch-max in the sarcastic condition than in the sincere condition only in wh-exclamatives and tag questions, and a later fall in the sarcastic condition than in the sincere condition only in the tag questions.
4.2 Predicting sarcasm vs. sincerity
Although most of the measurements have turned out to differ between the sarcastic condition and sincere condition, this does not necessarily mean that these measurements would be similarly useful to predict whether an utterance was uttered with sarcasm or not. We assessed the predictive power of these measurements using the mixed-effect binary logistic regression model in SPSS (IBM SPSS version 22). The outcome variable of the model was message_type (sarcastic vs. not sarcastic). The predictor variables included 12 main effects (i.e. gender, utterance_type, mean pitch, pitch-min, pitch-max, pitch span, duration, FPC1, FPC2, FPC3, FPC4 and FPC5), 21 two-way interactions (i.e. gender by utterance_type, and the interaction of each continuous predictor by each categorical predictor), and ten three-way interactions between the two categorical predictors (gender, utterance_type) and each continuous predictor. In addition, two random variables were added to the model, i.e. speaker and utterance.
The model could predict the message type of an utterance correctly in 73.7% of the cases. Seven of the predictor variables made a significant contribution to the model: gender (p < .05), FPC3 (p < .05), FPC5 (p < .05), duration (p < .01), FPC3 by gender (p < .05), duration by gender (p < .05) and duration by utterance_type (p < .05). This result shows that not all the measurements in which sarcastic and sincere conditions differ are equally useful in predicting the presence of sarcasm. Since the predictors gender, FPC3 and duration were involved in the interactions, we focus on the effects of FPC5 and the three interactions here (Seltman Reference Seltman2015). First, as shown by the coefficient estimates (Table 1), every unit of increase in FPC5 led to an increase by 0.17 in the odds of an utterance being interpreted to sound sarcastic (hereafter the odds of sarcasm). Second, although a unit of increase in FPC3 led to an increase in the odds of sarcasm in general (main effect of FPC3), this increase was smaller for the female speakers than for the male speakers by 0.122. Similarly, although a unit of increase in duration led to an increase in the odds of sarcasm in general (main effect of duration), the increase the odds of sarcasm was smaller in the declaratives than in the tag questions and wh-exclamatives.
Summary of the results of the mixed-effect binary logistic regression model on the prediction that an utterance was uttered with sarcasm. The reference category was ‘not sarcastic’ for the outcome variable, male for the fixed factor gender, and wh-exclamative for the fixed factor utterance_type.
5 Discussion and conclusions
Taking a linguistically-motivated approach, we have examined prosodic expression of sarcasm in the key word of an utterance (i.e. the word that is semantically critical word to the expression of sarcasm) with respect to both static and dynamic prosodic measurements, and the predictive power of prosodic and non-prosodic factors for the presence (or absence) of sarcasm in the southern variety of British English. We have found that the target words (i.e. the key words in sarcastic utterances and their counterparts in the sincere utterances) are realised with a longer duration and a flatter pitch contour in the sarcastic condition than in the sincere condition regardless of utterance type and speaker gender. The longer duration of the target words in sarcastic utterances may suggest a slower speaking rate at the utterance level, as reported for American and Canadian English (Rockwell Reference Rockwell2000, Cheang & Pell Reference Cheang and Pell2008), Cantonese (Cheang & Pell Reference Cheang and Pell2009), Mexican Spanish (Rao Reference Rao2013) and French (Loevenbruck et al. Reference Loevenbruck, Ben Jannet, D'Imperio, Spini and Champagne-Lavau2013, González-Fuente et al. Reference González-Fuente, Prieto and Noveck2016). The flatter pitch contour appears to be compatible with a smaller pitch span at the utterance level in sarcastic utterance, as reported for Cantonese (Cheang & Pell Reference Cheang and Pell2009), American English (Rakov & Rosenberg Reference Rakov and Rosenberg2013), Mexican Spanish (Rao Reference Rao2013) and German (Niebuhr Reference Niebuhr2014), but different from the reported use of a larger pitch span in Italian (Anolli et al. Reference Anolli, Ciceri and Infantino2002) and French (González-Fuente et al. Reference González-Fuente, Prieto and Noveck2016, see also Loevenbruck et al. Reference Loevenbruck, Ben Jannet, D'Imperio, Spini and Champagne-Lavau2013) and more dynamic pitch contours in Catalan (González-Fuente et al. Reference González-Fuente, Escandell-Vidal and Prieto2015). Sarcastic speech in British English thus appears to be largely similar to North American English and differs from other languages in pitch span.
More importantly, our study has yielded insights into aspects of prosodic expression of sarcasm that have not been studied in previous work on English and other languages. First, we have found notable differences in the use of prosody in different utterance types. Specifically, the target words are realised with a lower pitch-min in the sarcastic condition than in the sincere condition across utterance types but the difference in pitch-min is more pronounced in wh-exclamatives than in tag questions and declaratives. Further, they are realised with a lower pitch-max in the sarcastic condition than in the sincere condition in wh-exclamatives and tag questions but not in declaratives. The target words are also realised with a later fall in the sarcastic condition than in the sincere condition in tag questions. Taking account of the findings on duration and flatness of pitch contour mentioned in the preceding paragraph, we may suggest that sarcasm and sincerity differ in five of the prosodic parameters under examination in tag questions (i.e. duration, pitch-min, pitch-max, FPC1, FPC3), in four of the prosodic parameters in wh-exclamatives (i.e. duration, pitch-min, pitch-max, FPC1), and three of the prosodic parameters in declaratives (i.e. duration, pitch-min, FPC1). It has been claimed that positive declaratives may be more readily used sarcastically than tag questions (Kreuz & Glucksberg Reference Kreuz and Glucksberg1989, Kreuz & Caucci Reference Kreuz and Caucci2007). Our results may thus suggest a functional trade-off between the readiness of an utterance type being used sarcastically and the presence of prosodic cues to sarcasm, in line with the Functional Hypothesis (Haan Reference Haan2002). However, as independent empirical evidence for differences in how readily different utterance types can be used sarcastically is still lacking, future research is needed to validate our interpretation of the results on the presence of prosodic cues in different utterance types.
Interestingly, we have also observed gender-related differences in prosodic expression of sarcasm. Specifically, the target words in the sarcastic condition are realised with a lower mean pitch than in the sincere condition by female speakers, not by male speakers. This finding raises the question as to whether the reported use of a lower mean pitch in sarcastic utterances at the utterance level in North American English is primarily applicable to female speakers. The target words are also realised with a longer duration in the sarcastic condition than in the sincere condition by both male and female speakers but the duration difference between the two conditions is much larger in male speakers’ production. The more extensive use of duration in male speakers may serve as a compensation for a lack of use of mean pitch.
Additionally, we have found that the prosody of the target words has yielded useful predictors for the presence (or absence) or sarcasm, i.e. FPC3, FPC5 and duration. Together with the non-prosodic factors utterance_type and gender, they can accurately predict the presence (or absence) of sarcasm in nearly 74% of the time. To get an idea of the predictive power of the prosodic measurements alone, we conducted the mixed-effect binary logistic regression analysis excluding the non-prosodic predictors utterance_type and gender. The new model (containing mean pitch, pitch-min, pitch-max, pitch span, duration, FPC1, FPC2, FPC3, FPC4 and FPC5) could make correct predictions in 70.4% of the cases, suggesting that the prosodic predictors are primarily responsible for the improved performance of the best-fit model, compared to the intercept-only model.
Together, our results show that sarcasm and sincerity can be prosodically distinguishable in the key words alone and the presence of sarcasm can be predicted with a reasonable accuracy with prosodic information from the key words in British English, lending support to our key-word–based approach. Arguably, the key words may also be analysed as the focus in the sarcastic utterances (narrow focus). Its counterpart in the sincere utterances may be considered part of the whole-utterance focus (or broad focus), as the sincere utterances were produced without explicit contexts. It can thus not be ruled out that the prosodic differences found between the sarcastic and sincere conditions might in part be attributed to the differences between narrow focus and broad focus at least on some trials and in some speakers. Future studies in which focus conditions of sarcasm and sincere utterances are systematically varied are needed to shed light on the interface between sarcasm and focus in prosody.
Our results also imply that the key words do not contain all the prosodic variation relevant to the expression of sarcasm, and that not all relevant prosodic variation is encoded in conventional prosodic parameters. Informal listening to the utterances suggested that sarcasm may also be signalled by subtle changes in the overall pitch contour of the complete utterance in British English. In follow-up research we will investigate this issue more formally. However, it is not straightforward to apply functional PCA to complete utterances in the current data set, mainly because of the presence of variable numbers and lengths of voiceless intervals. While we could apply functional PCA to the target words, despite the fact that many of the target words contained one or more unvoiced intervals, we will explore whether more advanced interpolation methods for handling voiceless intervals can improve the results of FDA, for example, by taking the phonetic make-up of the words and utterances into account. Informal listening also suggested that voice quality may play a role in the expression of sarcasm in British English, as found for German (Niebhur Reference Niebuhr2014). Many voice quality features (Laver Reference Laver1980, Boves Reference Boves1984, Van Bezooijen Reference Van Bezooijen1984) can be expressed as continuous functions, and made accessible to functional PCA processing. In future research, we will extend the use of FDA from pitch to voice quality and examine the use of different voice quality features at both the key-word level and the utterance level in the expression of sarcasm in British English.
Acknowledgements
We are grateful to Laura Smorenburg and Joe Rodd for their assistance in recording the context-setting remarks used in the production experiment, and preparing and administering the production experiment in the UK, Laura Smorenburg for preparing and administering the perception experiment, Cécile de Cat for allowing us to use the Phonetics Lab at the University of Leeds to conduct the production experiment, Karlijn Blommers and Rianne Kamerbeek for data annotation, and Karlijn Blommers for editing the references. We also wish to thank Santiago González-Fuente and four anonymous reviewers for their thorough reviews and constructive feedback. This research was conducted with the support of an Aspasia grant (grant number 015.007.013) awarded to the first author by the Netherlands Organisation for Scientific Research (NWO).
Appendix. Context-setting remarks and response utterances
The context-setting remarks and the response utterances in the sarcastic condition with the semantically crucial word or the key word underlined (Decl: declarative, Tag: tag question, Wh: wh-exclamative
