INTRODUCTION
Research documenting sequelae of traumatic brain injury (TBI) consistently show that changes in socio-emotional functioning are more common than neuropsychological deficits. Such deficits increase with severity of the TBI, occurring in up to 80% of the severely injured population (e.g., Brooks, Campsie, Symington, Beattie, & McKinlay, Reference Brooks, Campsie, Symington, Beattie and McKinlay1986; Thomsen, Reference Thomsen1984). Such changes reflect, in part, an inability to recognize and regulate responses to emotional cues during social interactions (Croker & McDonald, Reference Croker and McDonald2005; Hornak, Rolls, & Wade, Reference Hornak, Rolls and Wade1996; Ietswaart, Milders, Crawford, Currie, & Scott, Reference Ietswaart, Milders, Crawford, Currie and Scott2008). Individuals with TBI have difficulty identifying emotions in face (Green, Turner, & Thompson, Reference Green, Turner and Thompson2004; Jackson & Moffatt, Reference Jackson and Moffatt1987; Milders, Fuchs, & Crawford, Reference Milders, Fuchs and Crawford2003; Spell & Frank, Reference Spell and Frank2000) and voice (Marquardt, Rios-Brown, Richburg, Seibert, & Cannito, Reference Marquardt, Rios-Brown, Richburg, Seibert and Cannito2001; McDonald & Pearce, Reference McDonald and Pearce1996; Milders et al., Reference Milders, Fuchs and Crawford2003). However, impairment is greater for vocal emotion (Spell & Frank, Reference Spell and Frank2000) even when paired with visual emotional cues (McDonald & Saunders, Reference McDonald and Saunders2005), and improves less over time, compared with facial emotion perception (Ietswaart et al., Reference Ietswaart, Milders, Crawford, Currie and Scott2008). Yet relatively few studies have examined vocal affect deficits in TBI (Zupan, Neumann, Babbage, & Willer, Reference Zupan, Neumann, Babbage and Willer2009).
Most research examining emotional prosody has focused on patients with focal brain lesions. There is some evidence for right hemispheric specialization in processing emotional prosody (Bowers, Coslett, Bauer, Speedie, & Heilman, Reference Bowers, Coslett, Bauer, Speedie and Heilman1987; Heilman, Bowers, Speedie, & Coslett, Reference Heilman, Bowers, Speedie and Coslett1984; Ross & Monnot, Reference Ross and Monnot2008), although others find no distinction (Baum & Pell, Reference Baum and Pell1999; Pell & Baum, Reference Pell and Baum1997; Pell, Reference Pell2006; Schlanger, Schlanger, & Gerstmann, Reference Schlanger, Schlanger and Gerstmann1976). Processing of both emotional and grammatical prosody involves the superior temporal gyrus bilaterally, although evidence continues to favor right hemisphere specialization for emotion (Adolphs, Reference Adolphs2002; Mitchell, Elliott, Barry, Cruttenden, & Woodruff, Reference Mitchell, Elliott, Barry, Cruttenden and Woodruff2003). In addition, overlapping frontal–subcortical circuits become selectively activated depending on task demands. The orbito and ventral medial frontal regions, amygdale, and basal ganglia are engaged, especially during encoding and maintenance of meaningful representations of emotional prosody (Buchanan et al., Reference Buchanan, Lutz, Mirzazade, Specht, Shah and Zilles2000; Johnstone, van Reekum, Oakes, & Davidson, Reference Johnstone, van Reekum, Oakes and Davidson2006; Kotz, Meyer, Alter, Besson, von Cramon, & Friederici, Reference Kotz, Meyer, Alter, Besson, von Cramon and Friederici2003; Pell & Leonard, Reference Pell and Leonard2003). The dorsolateral prefrontal cortex is also engaged during effortful access of semantic memory for labeling an emotional cue (Hariri, Bookheiner, & Mazziotta, Reference Hariri, Bookheiner and Mazziotta2000; Oschner & Barrett, Reference Oschner, Barrett, Mayne and Bonanno2001; Phillips, Drevets, Rauch, & Lane, Reference Phillips, Drevets, Rauch and Lane2003). Ventral frontal and temporal regions implicated in emotion processing are commonly compromised following TBI (Fujiwara, Schwartz, Gao, Black, & Levine, Reference Fujiwara, Schwartz, Gao, Black and Levine2008; MacKenzie et al., Reference MacKenzie, Siddiqi, Babb, Bagley, Mannon and Sinson2002). Impairment is not, however, contingent on the presence of particular focal injuries (Ietswaart et al., Reference Ietswaart, Milders, Crawford, Currie and Scott2008) but may also be due to diffuse axonal injury causing a shearing of connections between critical areas mediating emotion perception (Adolphs, Damasio, Tranel, Cooper, & Damasio, Reference Adolphs, Damasio, Tranel, Cooper and Damasio2000; Green et al., Reference Green, Turner and Thompson2004).
In part, judgments of vocal emotion are driven by the distinctiveness of prosodic elements including pitch, intonation, loudness, and tempo. Patterns of errors suggest they are caused by similarities in prosodic parameters between emotions (Leinonen, Hiltunen, Linnankoski, & Laakso, Reference Leinonen, Hiltunen, Linnankoski and Laakso1997) with some emotions easier to identify than others. Anger is characterized by increased speech rate, wider range in pitch variations and greater intensity, making it one of the most identifiable emotions in voice (Scherer, Reference Scherer2003). Fear is also relatively easy to identify, while happiness is more difficult to recognize in vocal than facial expressions (Scherer, Reference Scherer2003; Zupan et al., Reference Zupan, Neumann, Babbage and Willer2009) and surprise is the most difficult to distinguish (Pell, Reference Pell2006). Although some have reported that TBI causes impairment across all emotions (Ietswaart et al., Reference Ietswaart, Milders, Crawford, Currie and Scott2008; McDonald & Saunders, Reference McDonald and Saunders2005), Spell and Frank (Reference Spell and Frank2000) found adults with TBI were more accurate identifying anger and happiness compared with the fearful voice. This pattern is difficult to reconcile with an emotion perception problem based on acoustic features alone and suggests other factors underpin deficits following TBI.
An important consideration is the interplay between speech prosody (the how) and the content of what is being said (the what). Speech prosody delivers the emotion behind the words in a spoken message (Mitchell & Ross, Reference Mitchell and Ross2008). While prosodic information is available and used early in comprehension (Kaganovich, Francis, & Melara, Reference Kaganovich, Francis and Melara2006; Steinhauer, Alter, & Friederici, Reference Steinhauer, Alter and Friederici1999), there is a bias for processing semantic content over prosody (Lew, Chmiel, Jerger, Pomerantz & Jerger, Reference Lew, Chmiel, Jerger, Pomerantz and Jerger1997). Semantic information is less open to interference than prosody (Jerger et al., Reference Jerger, Pirozzolo, Jerger, Elizondo, Desai and Wright1993; Morgan & Brandt, Reference Morgan and Brandt1989). Furthermore, in a modified Stroop task, incongruent semantic information interferes more with the identification of prosody than vice versa (Grimshaw, Reference Grimshaw1998) and while directions to ignore prosody are easily complied with, semantic information is difficult to completely ignore when focusing on prosody (Besson, Magne, & Schön, Reference Besson, Magne and Schön2002; Wambacq & Jerger, Reference Wambacq and Jerger2004).
It is possible that some of the difficulties people with TBI experience with vocal emotion stem from a failure to manage dual processing demands. Many people with TBI show an overly literal interpretation of indirect conversational remarks such as sarcasm (McDonald & Flanagan, Reference McDonald and Flanagan2004). Although various explanations have been considered (e.g., McDonald & Flanagan, Reference McDonald and Flanagan2004; Shamay-Tsoory et al., Reference Shamay-Tsoory, Tomer, Berger, Goldsher and Aharon-Peretz2005) a consistent finding is that deficits in information processing speed, working memory, and reasoning are commonly associated (McDonald et al., Reference McDonald, Bornhofen, Shum, Long, Saunders and Neulinger2006; McDonald & Pearce, Reference McDonald and Pearce1996, Martin & McDonald, Reference Martin and McDonald2004). These cognitive deficits appear to lead to an over-reliance upon the salient semantic information at the expense of contextual cues, some of which are prosodic. If people with TBI have difficulty processing both prosody and semantic content, reducing semantic content should lead to a shift in processing resources toward prosody (Creusere, Alt, & Plante, Reference Creusere, Alt and Plante2004) improving accuracy. This is unless TBI causes impaired processing of emotional prosody itself (Zupan et al., Reference Zupan, Neumann, Babbage and Willer2009). Semantic information can be reduced by using a “nonsense” language that conveys little semantic meaning (Grandjean et al., Reference Grandjean, Sander, Pourtois, Schwartz, Seghier and Scherer2005) or “filtering” out high frequencies in speech making the semantic content inaudible whilst preserving the prosodic elements (Kotz et al., Reference Kotz, Meyer, Alter, Besson, von Cramon and Friederici2003).
Although studies of patients with focal unilateral lesions have largely found a specific impairment in processing emotional prosody (Bowers et al., Reference Bowers, Coslett, Bauer, Speedie and Heilman1987; Pell & Baum, Reference Pell and Baum1997; Pell, Reference Pell2006), some have found that perception of prosody improves with removal of semantic information in patients with left-hemisphere lesions (Behrens, Reference Behrens1985; Heilman et al., Reference Heilman, Bowers, Speedie and Coslett1984; Lalande, Braun, Charlebois, & Whitaker, Reference Lalande, Braun, Charlebois and Whitaker1992; Ross & Monnot, Reference Ross and Monnot2008; Ross, Thompson, & Yenkowsky, Reference Ross, Thompson and Yenkowsky1997). TBI is characterized by diffuse, multifocal damage that is often bilateral affecting frontal and temporal systems (Levin, Williams, Eisenberg, High, & Guinto, Reference Levin, Williams, Eisenberg, High and Guinto1992). Consequently, it is unclear whether removing semantic information will ameliorate deficits in recognition of vocal emotion.
The following study aimed to replicate findings of vocal emotion perception problems in a sample of adults with TBI presenting with postmorbid changes in social skills. We hypothesized that if an inability to manage the dual processing requirements of understanding the how and the what of spoken speech contributes to impaired emotional prosody recognition, then this impairment should be ameliorated when little (nonsense sentences) or no semantic information (filtered sentences) is provided. Alternatively, if TBI causes impairment in the processing of emotional prosody itself, then we anticipated performance would become more degraded with reduced semantic information.
Lesions in the orbito and ventral medial frontal cortices cause perseveration of prepotent/automatic responses (Bechara et al., Reference Bechara, Tranel, Damasio and Damasio1996; Freedman et al., Reference Freedman, Black, Ebert and Binns1998) and disinhibited, impulsive behavior (Starkstein, Reference Starkstein1997) concurrently with deficits in emotion recognition (Damasio, Reference Damasio1994; Hornak et al., Reference Hornak, Rolls and Wade1996; Pettersen, Reference Pettersen1991). These regions, with connections to subcortical and brain stem nuclei (Vogt, Reference Vogt, Peters and Jones1986) may mediate a reciprocal function where effective recognition of emotional cues assists with regulation of behavioral responses (Oschner & Barrett, Reference Oschner, Barrett, Mayne and Bonanno2001). Emotion recognition has been linked to the display of socially appropriate behavior in brain-injured patients (Hornak et al., Reference Hornak, Rolls and Wade1996; Pettersen, Reference Pettersen1991) and cognitive measures of inhibitory control (Dujardin et al., Reference Dujardin, Blairy, Defebvre, Duhem, Noël and Hess2004; Uekermann, Daum, Schlebusch, & Trenckmann, Reference Uekermann, Daum, Schlebusch and Trenckmann2005) although not always (Langenecker, Bieliauskas, Rapport, Zubieta, Wilde, & Berent, Reference Langenecker, Bieliauskas, Rapport, Zubieta, Wilde and Berent2005; Milders et al., Reference Milders, Fuchs and Crawford2003). The second aim of this study was to examine the relation between emotion perception, inhibitory control, and behavioral impulsivity. We hypothesized that there would be a positive association among these three constructs.
METHODS
Participants
Nineteen adults with moderate–severe TBI were recruited from metropolitan Brain Injury Units in NSW, Australia. All participants had suffered brain injuries of sufficient severity to warrant inpatient rehabilitation, and were recruited on the basis of clinical judgment that they (1) were experiencing social difficulties as a result of the TBI and (2) had sufficient cognitive and motor capacity to understand and comply with instructions. Exclusion criteria included premorbid neurological or psychiatric conditions; current aphasia or agnosia; current psychosis. One participant was subsequently excluded because of an insufficient number of attempted responses to allow meaningful statistical analysis. The remaining 18 participants (13 males, 5 females) were aged 22 to 63 years (Mean 45.2; SD 11.7) (Table 1).
Demographic and clinical characteristics of participants with TBI
Note
Severity classification as recommended by Jennett (Reference Jennett1976, Reference Jennett1996); Ext = extremely; GCS = Glasgow Coma Scale; MVA = motor vehicle accident; MVA Ped = motor vehicle accident as a pedestrian; R = right; L = left;
1 Self-report.
2 Clinician report.
3 Maternal report.
Posttraumatic amnesia (PTA) was established from medical records. Mean duration was 79.8 days (range, 1–270 days), no different (one sample t-test; p = .94) to that reported in a consecutive series of TBI inpatients (e.g., Tate, Broe & Lulham, Reference Tate, Lulham, Broe, Strettles and Pfaff1989: PTA mean = 81.2 days; range 4 to >168 days). That is, this sample was representative of the severity of injury typically seen in this population. Our sample did include two participants with PTA of 1 day or slightly less (Pts 15 and 21) which is identified as a mild TBI by some grading classifications (Mild Traumatic Brain Injury Committee of the Head Injury Interdisciplinary Special Interest Group of the American Congress of Rehabilitation Medicine, 1993) and moderate by others (Jennett, Reference Jennett1976), although the presence of radiological abnormalities suggests that these injuries were more complicated. Both participants experienced social difficulties as a consequence of their brain injuries and had undergone inpatient rehabilitation. Mean time post injury for the group was 15.0 years (SD 9.5 months). Before the injury, all participants had been in full-time work (four professional, eight skilled, four unskilled) or studying (2). Following their injuries, two participants were in full-time employment while six worked on a part-time basis and 10 were unemployed. On average, TBI participants had 12 years (SD 2.2 years) of education. One patient with TBI was deaf in his right ear due to peripheral damage. While unilateral deafness affects speech localization and reduces ability to screen background noise, speech discrimination in the good ear remains intact. Thus, the participant’s ability to perform the tasks was unaffected, and this was confirmed with the participant during practice trials.
Eighteen control participants (12 males) aged 23–62 years (Mean 44.4; SD 12.1), with 11.3 years education (SD 1.6) and without neurological impairment, were also tested. These were volunteers recruited through community newspaper advertisements and flyers in local hospitals. They were screened for neurological, psychiatric, and motor–sensory impairments before acceptance into the study. The groups did not differ with respect to age (p = .837) or education (p = .317). All participants spoke fluent English and had normal or corrected-to-normal vision.
Stimuli
Three sets of prosody stimuli were presented over two emotion recognition tasks. Two sets (Pell, Paulmann, Dara, Alasseri, & Kotz, Reference Pell, Paulmann, Dara, Alasseri and Kotz2009) consisted of short, digitally recorded declarative sentences (Mean Length: 1.7 s; Range: 0.9 to 2.5 s) produced by two male (Mean Pitch: 192 Hz; Range: 114–388 Hz) and two female (Mean Pitch: 237 Hz; Range: 112–388 Hz) actors. Sentences were sampled at a rate of 48 kHz, at an intensity of 75dB, and were spoken in either (1) semantically well-formed American English (e.g., That car just splashed me!) or (2) a nonsense language with appropriate phonetic and prosodic structure for English (e.g., Someone migged the pazing). Sentences were spoken as: happy, pleasantly surprised, angry, and afraid (see Table 2 for acoustic characteristics). The semantically well formed stimuli provided emotionally biased semantic content that facilitated discrimination for happy, angry, and afraid. (The pleasantly surprised content was more ambiguous (e.g., You cleaned the entire garage!). In contrast, nonsense words provided little linguistic information while leaving emotion prosody intact. Each actor gave two exemplars of each emotion resulting in a set of 32 stimuli for the semantic stimuli and a different set of 32 stimuli for the nonsense stimuli.
Acoustic characteristics of emotional prosody stimuli
Note
“Voice breaks” was estimated as the number of distances between consecutive pulses that are longer than 1.25 divided by the pitch floor (Boersma & Weenink, Reference Boersma and Weenink2007).
A third set of 32 stimuli was created by low-pass filtering the semantically-well formed stimuli using a Hann band filter in Praat (v 5.0.2; www.praat.org) with a cutoff at 360 Hz for male actors and 400 Hz for female actors (Koff et al., Reference Koff, Zaitchik, Montepare and Albert1999), and a smoothing width of 100 Hz. Cutoff points were selected following analysis of the fundamental frequency (f0) maximum across emotions and incrementally adjusting the cutoff by 30 Hz around this point until linguistic information was subjectively inaudible. The resultant filtered stimuli retained pitch and contour variations over time whilst offering no linguistic information, sounding like “muffled” speech. Five postgraduate students listened to a subset of the stimuli (n = 10), and all reported that no linguistic information was identifiable.
Tasks
The design of the emotion discrimination and emotion labeling tasks is depicted in Figure 1.
In this experimental design, each participant completed an emotion labeling and an emotion discrimination task, each consisting of two stimulus sets (32 trials each), resulting in a total of 128 emotional prosody trials.
Emotion discrimination
In the emotion discrimination task 32 pairs of nonsense stimuli and 32 pairs of filtered stimuli were presented and participants made same/different judgments regarding the emotion in the voice pairs. Response options were presented on computer screen and participants responded with either a left (different) or right (same) button press on a keyboard. Each trial began with a 500-ms warning stimulus followed by the first stimulus, a 500-ms inter-stimulus-interval, and then the second stimulus. Participants had 10 seconds to respond after the second speaker. The task consisted of eight blocks of eight stimuli pairs, alternating between blocks of nonsense and filtered stimuli. Pairings of “same” and “different” emotional prosody were equiprobable (50%) across the task. All pairings involved sentences of different semantic content spoken by different speakers of the same gender. The first block of each nonsense and filtered stimulus set began with three practice trials.
Emotion labeling
In the emotion labeling task participants were required to identify emotional tone in speech using either prosodic features alone or with semantic information. Participants listened to a single stimulus and identified whether the emotion in the voice was happy, surprised, angry, or afraid in a forced-choice task. Use of only four options reduced working memory demands (Pell, Reference Pell2006) as did presentation of the response options on both computer screen and keyboard buttons. The task consisted of eight blocks (eight trials/block) of alternating semantic and nonsense stimuli (i.e., four blocks for each stimulus type).Footnote 1 Each trial began with a warning stimulus for 500 ms, followed by the speech stimulus and then a request to match the speech stimulus to the appropriate emotion (within 10 s). In the first four blocks, the response options on screen consisted of written words, with participants required to press the corresponding labeled keyboard button. In the second set of four blocks, the response options were four black and white photographs (two male, two female) (Ekman & Friesen, Reference Ekman and Friesen1976) representing the four emotions. Participants again were required to press the corresponding labeled keyboard button (in the same order as above). The first block of each semantic and nonsense stimulus set began with three practice trials.
Measures of Impulsivity and Inhibition
Trait impulsiveness
Participants completed Barratt’s Impulsiveness Scale (BIS-11), a 30-item self-report survey measuring Motor Impulsiveness (i.e., acting without thinking), Attentional Impulsiveness (i.e., easily bored when sustained attention required) and Nonplanning Impulsiveness (i.e., a lack of concern for future consequences) (Patton, Stanford, & Barratt, Reference Patton, Stanford and Barratt1995). Internal consistency reliability ranges from 0.79 to 0.83 for healthy adults, substance-abuse patients and prison inmates (Patton et al., Reference Patton, Stanford and Barratt1995) and is approximately 0.78 (Votruba et al., Reference Votruba, Rapport, Vangel, Hanks, Lequerica and Whitman2008) for adults with TBI, who show greater total impulsiveness scores than controls (Dyer, Bell, McCann, & Rauch, Reference Dyer, Bell, McCann and Rauch2006; McHugh & Wood, Reference McHugh and Wood2008). Scores for each sub-factor and an overall Impulsiveness score were calculated
Inhibitory control
Inhibition of a prepotent, automatic response was evaluated using the Haylings Sentence Completion Test (Burgess & Shallice, Reference Burgess and Shallice1997) whereby participants are asked to provide a word to finish a sentence that either completes the sentence (“initiation” task) or is completely unconnected (“inhibition” task). TBI patients (Draper & Ponsford, Reference Draper and Ponsford2008) and frontal-injured patients (Burgess & Shallice, Reference Burgess and Shallice1996) show poor performance on this test. A response time score and an error score for the inhibition task as well as an overall response score across tasks were calculated and scaled using a scoring system in Burgess and Shallice (Reference Burgess and Shallice1997) and converted to Sten scaled scores (differences between Sten scores = 0.5 of an SD). Scores ranged from 1 (outside normal range) through to 10 (99th percentile or very superior). A scaled score of 2 (1%) or below is considered impaired (Burgess & Shallice, Reference Burgess and Shallice1997). Test–retest reliability of the inhibition error score ranges from 0.41 for healthy controls to 0.72 for brain injured (lesion) patients (Burgess & Shallice, Reference Burgess and Shallice1997).
Procedure
The discrimination and labeling tasks were presented on a computer in counterbalanced order across participants. Participants listened to prosody stimuli over volume-adjustable computer headphones. In the labeling task, participants were told that they would hear a person speaking in either English or a “made-up” nonsense language and that they were to identify the emotion in each voice from the four alternatives offered. In the discrimination task, participants were instructed to listen to both speakers, who would either be speaking in a nonsense language or a muffled tone, before deciding whether the emotion in the two speakers was “same” or “different”. Across all tasks, participants were instructed that it was more important that they should listen to “how” the speaker was speaking rather than to “what” they were saying.
Following the experimental tasks, participants completed the BIS-11 and neuropsychological tests including the Wechsler Test of Adult Reading (WTAR), the Trail-Making Test, WMSIII subtests: Logical Memory 1 and Faces 1, WAISIII subtests: Similarities, Digit Span, Digit Symbol-Coding (WAIS-III), and the Haylings Sentence Completion Test. This study and its procedures were approved by the University of NSW Human Research Ethics Committee.
Statistical Analyses
Accuracy rates, converted to percentages of total attempted responses (i.e., omission errors excluded), and RT were calculated. A Repeated-measures 2 × 2 ANOVA was used in the discrimination task, with the within-subject factor CONDITION (nonsense vs. filtered) examining participants’ accuracy (%) and mean reaction time (MRT). A 2 × 2 × 2 design was used in the labeling task, with the within subjects factors CONDITION (original vs. nonsense) and TASK (match-to-words vs. match-to-faces). A separate analysis was performed in each task to examine differences between Emotion Types, that is, happy versus pleasantly surprised, angry versus afraid, and positive (happy + surprised) versus negative (angry + afraid). Significant main effects were followed with separate analyses and alpha adjusted for the number of comparisons.
For the Impulsiveness Scale, where data were missing for items (2% of all cells), meanperson was used to replace these (Roth, Switzer, & Switzer, Reference Roth, Switzer and Switzer1999), allowing a total score to be calculated for each individual. ANOVAs were used to examine between-group differences on the Impulsiveness subscales and on all neuropsychological measures except the Haylings test for which the Mann-Whitney U test was used. Pearson’s bivariate correlations were performed between experimental task measures, and neuropsychological and personality measures.
RESULTS
Table 3 presents neuropsychological test scores for individual participants with TBI as well as between-group differences.
Neuropsychological test scores for participants with TBI
Note
WTAR = Wechsler Test of Adult Reading; Trails = Trail-Making Test A and B. All scores represent standard scores based on Wechsler manuals except Trails A and B which are presented as seconds taken to complete. *TBI significantly impaired relative to control group (p < 0.05).
1 Haylings scaled scores whereby lower scores reflect poorer performance (see Burgess & Shallice, Reference Burgess and Shallice1997), and group scores are expressed as medians.
Discrimination Task
Participants with TBI showed overall poorer accuracy (69.3%) when making same/different judgments on pairs of emotional prosody stimuli compared with controls (76.6% accuracy rate)(F[1,34] = 7.3; p < .05) (Figure 2).
Mean accuracy (%) and standard deviation (SD) of making same/different judgements in the nonsense and filtered conditions for each group; y-axis = 50–100%, bar = 1 SD. TBI, traumatic brain injury.
Accuracy was higher in the nonsense (76.2%) then filtered stimuli (69.7%) (F[1,34] = 8.9; p < .01), although this effect did not differ between groups (p = .285). An analysis for each emotion type on “same” trials only revealed no significant differences between groups in the recognition of different emotion types (p > .10).
There were no differences in MRT (Controls: 1.6 vs. TBI: 1.9 s; p > .10) or the probability of omissions (Controls: 2.8 vs. TBI: 2.2 %; p > .5).
Labeling Task
Participants with TBI (70.7% accuracy rate) had overall greater difficulty labeling emotional prosody compared with controls (86.5% accuracy rate) (F[1,34] = 11.2; p < .01) (Figure 3).
Mean accuracy (%) and standard deviation (SD) when matching emotional prosody to words (left) and faces (right) in the semantic and nonsense conditions for each group. Significant group differences are indicated by asterisks; y-axis = 50–100%, bar = 1 SD. TBI, traumatic brain injury.
Across groups, participants were more accurate when judging semantic (84.9%) than nonsense stimuli (73.5%) (F[1,34] = 24.7; p < .001). A significant interaction with Group (F[1,34] = 6.1; p < .05) revealed that this effect occurred only in the TBI group. A significant Task effect revealed that participants were better at matching prosody to words (83.1%) than to faces (75.2%)(F[1,34] = 20.7; p < .001), and this difference was greater for nonsense than semantic stimuli (F[1,34] = 5.1; p < .05). A Group × Task interaction was not significant (p = .198), although difficulty matching prosody to faces appeared greater in the TBI group.
Subsidiary analyses in the matching-to-words condition, with alpha adjusted for the number of comparisons (α < .0125), revealed accuracy differed between groups for nonsense (t[34] = 1.7; p < .01) but not semantic stimuli (p > .0125). In the matching-to-faces condition, the groups differed for both nonsense (t[34] = 3.2; p < .01) and semantic stimuli (t[34] = 2.8; p < .01).
A significant main effect of Emotion-Type was found (F[3,32] = 6.1; p < .01), and this interacted with Group (F[3,32] = 4.7; p < .01) (Figure 4). Subsidiary analyses revealed group differences for happy (F[1,34] = 19.3; p < .001) and afraid (F[1,34] = 10.1; p < .01), but not angry (p = .053) or surprised (p = .110). Both groups performed most poorly when judging surprised. Anger was the easiest of the emotions for TBI participants to identify. There was also a tendency toward more accurate identification of negative than position valence emotions across groups (F[1,34] = 3.4; p = .072).
Mean accuracy (%) and standard deviation (SD) of labeling emotional prosody for each emotion type in the control and traumatic brain injury (TBI) groups. Significant group differences for happy and afraid are indicated by asterisks; y-axis = 50–100%, bar = 1 SD.
Mean reaction time was overall slower in the TBI (4.514 s) than control group (3.457 s)(F[1,33] = 13.7; p = .001), with a tendency toward faster responses when matching emotional prosody with faces (3.9 s) rather than words (4.1 s) (p = .116). However, there were no interactions between task, condition, and group. There were also no differences in the probability of omissions committed between TBI (4.5%) and control (4.9%) groups (p > .05).
Discrimination versus Labeling
Nonsense stimuli were presented in both emotion tasks, therefore, a direct comparison could be made. TBI participants showed greater accuracy in the discrimination (71.3%) task compared with labeling (61.3%), while controls performed similarly across tasks (81.1% vs. 83.8%) (F[1,34] = 10.6; p < .01). Across all conditions, overall performance in the discrimination and labeling tasks was highly correlated (r = .605; p < .001), as were matching-to-labels and matching-to-faces within the labeling task (r = .759; p < .001).
Haylings Test
The TBI group scored lower than the control group on Haylings overall (Median score TBI: 3 vs. Controls: 6, U = 22.5; p < .001), inhibition response time (Median score TBI: 4 vs. Controls: 6, U = 49.0; p < .001) and inhibition error scores (Median score TBI: 2 vs. Controls: 7, U = 58.5; p = .001). Eleven TBI participants (and one control participant) obtained an abnormal score (i.e., 2 or less) for Haylings inhibition errors (Table 3)
Impulsivity
The TBI group self-reported greater impulsivity than the control group (73 vs. 63, F[1,36] = 8.7; p < .01). Sub-test scores revealed a significant difference in Nonplanning Impulsiveness (20 vs. 25, F[1,36] = 11.2; p < .01) and almost significant differences for Motor (p = .083) and Attentional Impulsiveness (p = .084).
Correlations Between Measures
Across groups, Haylings inhibition error rate was positively related with performance on both the discrimination (r = 0.693; p < .001) and labeling tasks (r = 0.572; p < .001). Inhibition error rate was also negatively related with Barratt’s Nonplanning Impulsiveness (r = −0.351; p < .05). Impulsiveness scores did not correlate with accuracy scores on the emotion recognition tasks.
There was a positive relationship between two neuropsychological constructs, that is, working memory (Digit Span) and verbal comprehension (Similarities), and emotion labeling (r = 0.520; p < .01; r = 0.413; p < .05, respectively), but not discrimination (both p > .05). These significant effects remained after controlling for education. No other neuropsychological measures showed significant relationships with emotion perception accuracy. Notably, correlations between measures of RT and Symbol Digit-Coding or Trail Making Tests A and B were nonsignificant. Due to the relationship observed between working memory and the emotion recognition tasks, analyses were conducted with working memory as a covariate. Findings remained significant (p < .05).
Individual TBI Profiles
Despite a significant group difference in discrimination accuracy, only one TBI participant was impaired when judging nonsense stimuli and one participant was impaired when judging filtered stimuli; no participants were impaired on overall discrimination accuracy (Table 4). In the labeling task, nine participants were impaired in overall labeling accuracy. When matching emotional prosody to words, four were impaired in judging semantic stimuli and five when judging nonsense stimuli. When matching emotional prosody to faces, seven were impaired when judging semantic stimuli and seven when judging nonsense stimuli.
Impaired performance of individuals with TBI in emotion discrimination, inhibition, and impulsivity measures
Note
Non-Plan = Non-Planning Sub-scale, F = Filtered stimuli, N = Nonsense stimuli, S = Semantic stimuli, ** = abnormal performance score falling below 2 SD of the control group’s mean.
CT scans were available for 16 of the 18 TBI participants. As our group was heterogeneous in their injuries, we could not examine left versus right brain injuries (contrast: Pell & Baum, Reference Pell and Baum1997; Pell, Reference Pell2006). However, it was interesting to note that participants who were impaired on labeling were those with intracerebral and subdural hemorrhages, cerebral edema, or extensive injuries to the left or right temporal lobes. In contrast, while four participants presented with focal lesions in the right frontal region (Pts: 1, 8, 9, 11) only one was impaired on the labeling task.
Four TBI participants (Pts: 1, 8, 27, 28) had received emotion recognition training three years before testing (Bornhofen & McDonald, Reference Bornhofen and McDonald2007), and three of these showed no impaired emotion recognition on the tasks. Thus, discrimination and labeling accuracy were re-analyzed excluding these four participants, as well as the partially deaf participant and five corresponding control participants. The significant group and condition effects remained.
Time since injury was unrelated to any performance measures (p > .05). The skewed distribution toward extremely severe injury (i.e., PTA > 28 days) meant a sub-group analysis for injury severity was not possible.
DISCUSSION
Participants with TBI were poorer at both discriminating and labeling emotional prosody, relative to non–brain-injured controls. Contradicting expectations, performance was not improved when there was little or no semantic information. Group differences were observed in both discrimination and labeling tasks where semantic information was reduced (nonsense) or completely removed (filtered). In contrast, groups were similar when judging stimuli containing semantically biased information that helped identify the emotional intent.
It is possible that removing semantic information did not have the intended effect of reducing semantic bias, but rather made individuals search harder for this content (Beaucousin, Lacheret, Turbelin, Morel, Mazoyer, & Tzourio-Mazoyer, Reference Beaucousin, Lacheret, Turbelin, Morel, Mazoyer and Tzourio-Mazoyer2007; Kotz et al., Reference Kotz, Meyer, Alter, Besson, von Cramon and Friederici2003; Meyer, Alter, Friederici, Lohmann, & Von Cramon, Reference MacKenzie, Siddiqi, Babb, Bagley, Mannon and Sinson2002). While we cannot determine this, our results highlight the importance in both healthy and TBI participants of semantic content over prosodic information (Pell & Baum, Reference Pell and Baum1997; Pell, Reference Pell2006), in line with a semantic processing bias (Besson et al., Reference Besson, Magne and Schön2002; Grimshaw, Reference Grimshaw1998; Jerger et al., Reference Jerger, Pirozzolo, Jerger, Elizondo, Desai and Wright1993; Wambacq & Jerger, Reference Wambacq and Jerger2004).
Although TBI performance in the discrimination task was poorer compared with controls, individual participants were not abnormally poor. Same/different discrimination is believed to involve automatic emotion recognition, mediated by the ventral frontal–subcortical circuit (Adolphs, Reference Adolphs2002; Kotz et al., Reference Kotz, Meyer, Alter, Besson, von Cramon and Friederici2003; Pell & Leonard, Reference Pell and Leonard2003). In our study this ability, while less effective in TBI participants, was not wholly impaired. In contrast, half the TBI group was impaired when making semantic judgments about emotional prosody stimuli. This is consistent with research showing that labeling emotions is more demanding for brain injured individuals than making same/different discriminations (Ietswaart et al., Reference Ietswaart, Milders, Crawford, Currie and Scott2008; Pell, Reference Pell2006; Tompkins & Flowers, Reference Tompkins and Flowers1985) because it requires effortful access of semantic memory, linked with the dorsolateral prefrontal cortex (Adolphs, Reference Adolphs2002). This interpretation fits with the association found between working memory and performance on the labeling but not discrimination task. Even so, working memory did not entirely account for group differences which remained when memory was controlled. Rather, a process common to discrimination and labeling (which were highly inter-correlated) appears to be responsible. Overall, our findings suggest that automatic emotion recognition is reduced in effectiveness following TBI, and that increased semantic emotion memory demands cause further impairment.
The TBI individuals were selectively impaired when labeling some emotions (i.e., happy, afraid) but not others (e.g., pleasantly surprised, angry). Pleasantly surprised was difficult for both groups, while anger was easiest for the TBI group. Spell and Frank (Reference Spell and Frank2000) also found TBI adults were impaired when judging fear and accurate when judging angry voices, but in contrast to us, they found that happy was well recognized (Spell & Frank, Reference Spell and Frank2000). No differences in accuracy for different emotions were found in the discrimination task. This is in line with the suggestion of Ietswaart et al. (Reference Ietswaart, Milders, Crawford, Currie and Scott2008) that TBI may selectively impair the semantic component of emotion identification, rather than automatic emotion discrimination per se. The salience and distinctiveness of prosodic cues may be essential for effortful-semantic categorization of emotion following TBI. Enhancing some prosodic parameters may, therefore, improve accuracy.
Our TBI participants showed a 9% drop in accuracy when matching semantic prosody to faces rather than words (compared with 2% for controls) and this effect tended toward significance. At an individual level, five participants were impaired when matching semantic prosody to faces but not when matching to words, despite the fact that semantic labels were always visible. This suggests that, for these participants, presence of facial information degraded evaluation of prosody. Neuroimaging has shown that the supratemporal auditory cortex, normally activated to audio-only stimuli, becomes suppressed during audiovisual presentations (Besle, Fort, Delpeuch, & Giard, Reference Besle, Fort, Delpeuch and Giard2004). McDonald and Saunders (Reference McDonald and Saunders2005) also found more individuals were impaired judging audiovisual than audio-only or visual-only displays of emotion. Clearly further research is needed to examine how bimodal emotion cues of emotion are integrated. This appears to be a unique deficit in some adults with TBI (McDonald & Saunders, Reference McDonald and Saunders2005).
The heterogeneous make-up of the TBI participants limits discussion of neural correlates. However, it is interesting to note that three of the four participants presenting with focal lesions in the right frontal region (Pts: 1, 8, 11) were not impaired on any emotion recognition tasks, supporting a view of bilateral contributions to emotion processing (Baum & Pell, Reference Baum and Pell1999; Pell & Baum, Reference Pell and Baum1997; Pell, Reference Pell2006; Schlanger et al., Reference Schlanger, Schlanger and Gerstmann1976). Given that emotional prosody recognition is a highly integrative process and past studies have not found a correlation between focal injuries in TBI and emotion perception impairment (Green et al., Reference Green, Turner and Thompson2004; Ietswaart et al., Reference Ietswaart, Milders, Crawford, Currie and Scott2008), future studies should examine the role of diffuse axonal injury.
Consistent with past studies (Dujardin et al., Reference Dujardin, Blairy, Defebvre, Duhem, Noël and Hess2004; Uekermann et al., Reference Uekermann, Daum, Schlebusch and Trenckmann2005), the TBI group had poor inhibitory control compared with controls and this was associated with poor emotion discrimination and labeling. This highlights the neuroantomical and functional inter-relatedness of these domains (Damasio, Reference Damasio1994; Oschner & Barrett, Reference Oschner, Barrett, Mayne and Bonanno2001; Phillips et al., Reference Phillips, Drevets, Rauch and Lane2003). This relationship did not extend to self-reported trait impulsiveness, which was related to inhibitory control but not emotion recognition. Trait impulsivity may measure a facet of impulsivity that is not relevant to emotion regulation.
In terms of limitations, we cannot exclude the possibility that acoustic deficits affected prosody perception, although differential performance across tasks makes this unlikely. Our sample of people with TBI was selected from those manifesting social difficulties during rehabilitation, so our conclusions do not apply to the general TBI population. Nonetheless, given that changes in social function are prevalent, seen in as many as 80% of adults years after their injuries (Brooks et al., Reference Brooks, Campsie, Symington, Beattie and McKinlay1986; Thomsen et al., Reference Thomsen1984), our findings are likely to apply to a substantial proportion of those with severe TBI. Indeed, the kinds of social problems described of our TBI participants are typical, reflecting withdrawn, inert behavior, rigid thinking, garrulous disinhibited social interactions, and/or general inefficiency processing conversation (Hartley and Jensen, Reference Hartley and Jensen1992; Lezak, Reference Lezak1978; Thomsen, Reference Thomsen1984). Self-reported deficits should be treated with caution as adults with TBI may have limited insight into their behavior, although this should lead to an underestimation of reported deficits. The extent to which difficulties with prosody contributes to these deficits awaits further examination. Our group included some who had received specific remediation of emotion perception which may have affected our findings although a re-analysis of the data without these four participants did not alter the pattern of results.
In conclusion, we found an impairment of recognizing emotional prosody may contribute to social skills deficits many years following a TBI. Automatic discrimination of emotional prosody was impaired, although not abnormally so, while additional effortful accessing of semantic memory led to significantly impaired performance in at least half the group. Our results did not support our hypothesis that the dual demands of processing the “what” and “how” in vocal speech underpins poor performance. Indeed, the participants in this study relied upon the “what” to recognize vocal emotion. Without this, they were significantly impaired, suggesting problems processing emotional prosody per se. Differences between emotion types suggest that manipulating salience of prosodic cues may be a way to improve vocal emotion recognition.
ACKNOWLEDGMENTS
This research was supported by a project grant from the National Health & Medical Research Council of Australia. We express our gratitude to people with traumatic brain injuries who participated in the studies reported here as well as to our community control participants who gave willingly of their time. The authors have no competing or conflicts of interest to report.
