INTRODUCTION
Emotion recognition is a complex process that is known to be impaired in some individuals with schizophrenia (Sz; Kohler, Walker, Martin, Healey, & Moberg, Reference Kohler, Walker, Martin, Healey and Moberg2010). Impairments are also evident in bipolar disorder (BD), albeit to a lesser extent (Rossell, Van Rheenen, Joshua, O’Regan, & Gogos, Reference Rossell, Van Rheenen, Joshua, O’Regan and Gogos2014; Van Rheenen & Rossell, Reference Van Rheenen and Rossell2013, Reference Van Rheenen and Rossell2014). Individuals with Sz show significant and consistent decrements in performance across a range of emotional expressions and task formats, whereas the current evidence in BD is more variable across studies; with the magnitude of observed impairments generally tending to be lesser than those found in Sz.
Discrepancies in profiles of emotion recognition between Sz and BD may be related, in part, to different face processing strategies engaged during emotion perception. Healthy face processing requires both part-based (featural) and holistic (configural) face processing strategies; the former refers to use of local featural information and the latter to holistic processing of spatial relationships between features (Maurer, Reference Maurer2002). Recently, using a non-emotional face processing paradigm, we showed that patients with Sz had impairments in both strategies, whereas BD patients were impaired only in featural processing (Joshua, Van Rheenen, Castle, & Rossell, Reference Joshua, Van Rheenen, Castle and Rossell2016). The influence on facial emotion recognition was not assessed in this work. Yet we hypothesize that a generalized face-processing impairment in Sz could impair emotion recognition to a greater extent than a specific face processing deficit alone, possibly by limiting the extent by which compensatory face processing mechanisms occur during emotional information processing. Thus the presence of a specific face processing deficit in BD may partly explain why emotion recognition impairments are subtler in magnitude than those seen in Sz.
One way to assess the influence that face processing has on facial emotion recognition in BD and Sz is to determine the extent to which facial emotion recognition is impaired when either whole or part-based facial information is presented. It stands to reason that regardless of the presence of either information type, the accuracy and speed of emotional expression recognition would be limited if a generalized impairment in both configural and featural face processing were present. In contrast, fundamental impairments in the use of either configural or featural face processing strategies would likely result in impairment in the recognition of either whole or part-based emotional face stimuli, but not both.
Few studies have compared the difference in emotion processing in Sz when the whole face is available for processing compared to restricted information such as presentation of the eye-region alone. We are aware of no studies in BD that have focused attention on this aim, despite the potential implications that findings from such studies could have for the development of emotion recognition remediation programs. In this study, we address this paucity of literature in a follow-up analysis to our previous face-processing work, using the same cohorts of Sz and BD patients that demonstrated either generalized (Sz) or specific (BD) non-emotional face processing impairments compared to controls (Joshua et al., Reference Joshua, Van Rheenen, Castle and Rossell2016). Here, we report the results of an emotion recognition paradigm using stimuli designed to encourage or to inhibit the use of configural face information in whole or part-based conditions, respectively.
In line with our previous findings, we made two main hypotheses: (i) BD patients would show specific emotion recognition impairments when responding to faces from only the whole as opposed to the part-based condition, and (ii) Sz patients would show deficits on both whole-face and part-based conditions. The extent to which the availability of whole-face information was beneficial over part-based information in each of these groups was also explored.
METHODS
This study was approved by relevant Hospital and University review boards and abided by the declaration of Helsinki. Informed consent was obtained from each participant before the study began.
Participants
Twenty-eight Sz individuals, 28 BD-I individuals, and 28 healthy controls completed the study. Patient diagnosis was ascertained using the Structured Clinical Interview for DSM-IV (First, Spitzer, Gibbon, & Williams, Reference First, Spitzer, Gibbon and Williams1996) and current symptomatology acquired using the Beck Depression Inventory (BDI) (Beck & Steer, Reference Beck and Steer1987) and the Positive and Negative Syndrome Scale (PANSS) (Kay, Fiszbein, & Opler, Reference Kay, Fiszbein and Opler1987). None of the patients experienced any co-morbid Axis 1 diagnoses at the time of testing. All BD patients were tested during a period of clinical stability (i.e., not currently meeting criteria for a mood/ psychotic episode); however, 12 BD and 14 Sz individuals were depressed (BDI score≥10). Sixteen Sz patients also had current psychotic symptoms (PANSS P1 and/or P3 scores>3). No BD patients had current psychotic symptoms. Control participants were excluded if they had any history of psychiatric disorder or a first degree relative with Sz, BD, or schizoaffective disorder. Participants from all groups also met the following criteria: (a) No history of neurological disorder/ head trauma, (b) No current drug/ alcohol abuse disorder and no illicit drug use in previous 24 hr, (c) English first language, (d) Aged 18–65 years, and (e) Estimated premorbid IQ>85 based on the National Adult Reading Test (Nelson & Willison, Reference Nelson and Willison1991), (f) no electroconvulsive therapy in past 12 months.
Materials
Participants completed a facial emotion labeling task, which involved two conditions, each with a different stimulus type; a “whole-face” condition and a “part-based” condition. Participants were presented with a single face or a single pair of eyes on screen that displayed the expression of happiness, sadness, anger, fear, or neutrality. The eyes were used for the part-based condition given that eye information has been shown to be more important in the recognition of emotional expressions more so than that provided by the mouth (Baron-Cohen, Wheelwright, Jolliffe, & Therese, Reference Baron-Cohen, Wheelwright and Jolliffe1997). Participants were required to determine which of the five emotions the face (or eyes) was showing, indicating their response via a 5-button press. The task involved 50 trials involving 10 presentations for each of the five emotions, presented in a randomized order. Each face (or pair of eyes) was presented for 2000 ms, followed by a fixation cross for 1500 ms.
The stimuli were 10 (5 male, 5 female) greyscale photographs of faces selected from the commonly used Ekman and Friesen series, Pictures of Facial Affect (Reference Ekman and Friesen1976), each displaying happiness, sadness, anger, fear, and neutrality (no emotion). The images included no facial hair, jewelry, make-up, or glasses and were cropped to exclude any head hair. The stimuli for the whole-face condition consisted of an oval shape, 220×300 pixels, spanning the top of the forehead to the bottom of the chin, excluding the ears either side. The stimuli for the eyes only condition consisted of a rectangle shape, 220×76 pixels, spanning the eyes, eyebrows and bridge of the nose only. All images were presented on a white background. Five additional stimuli were selected as practice stimuli. Participants were given instructions and five practice trials before each task commencing.
Statistical Analysis
Demographic and clinical characterization of this sample is reported in our previous work (Joshua et al., Reference Joshua, Van Rheenen, Castle and Rossell2016) and re-presented in Table 1. Two multivariate analyses of variance with post hoc Sidak tests were carried out to examine between-group differences in performance on the whole-face and eyes-only conditions in terms of both accuracy and response time. Gender and premorbid IQ were specified as covariates in the analysis, given group differences on these measures. This was followed by paired-samples t tests to assess within-group differences in performance on each of the conditions. Pearson’s correlations were used to examine associations between clinical symptomatology and task performance in the clinical groups.
Demographics and clinical characteristics of the sample
Note. These data have been presented in our previous report (Joshua et al., Reference Joshua, Van Rheenen, Castle and Rossell2016) and is re-presented here to enable clarity for the reader.
CPZe=chlorpromazine equivalents; PANSS=Positive and Negative Syndrome Scale.
RESULTS
Figure 1 shows group performance on the task. There was a significant omnibus effect (F(4,158)=3.44; p=.01) and significant group differences in accuracy on both part-based (F(2,79)=4.32; p=.02) and the whole-face conditions (F(2,79)=7.36; p=.001). Sz patients performed worse than controls on both conditions (both p’s<.01). BD patients performed worse than controls on the whole-face condition only (p=.04). Sz and BD participants did not differ from each other on either condition (both p’s>.05). There was no significant omnibus effect for response time, and no significant group difference in response time on the whole-face condition or part-based condition. Nonetheless, the BD group did have slower, albeit non-significant response times compared to controls on the whole-face condition that were in the medium effect size range (d=0.53).
Group performance on the task.
Accuracy performance was always better for the whole-face compared to the part-based condition in all groups (Sz t(27) −5.33; p<.001; d=0.73; controls t(27) −9.48; p<.001; d=1.25; BD t(27) −4.73; p<.001; d=0.59). There were no differences in response times between conditions for the Sz and control groups (both p’s>.05, both d’s<.13), but BD patients did take significantly longer in the whole-face versus the part-based condition (t(27) −2.69; p=.01; d=0.28).
There were no significant correlations between the PANSS/BDI and scores for either condition in the Sz group. In the BD group, higher negative symptoms were correlated with worse accuracy performance on both part-based (r=−.40; p=.03) and whole-face (r=−.51; p=.01) conditions.
DISCUSSION
This study aimed to understand the influence of configural and featural face processing on facial emotion recognition in Sz and BD. We used an emotion recognition task that manipulated use of holistic and part-based facial processing strategies, both of which have been previously shown as impaired in this Sz cohort using a non-emotional face processing paradigm (Joshua et al., Reference Joshua, Van Rheenen, Castle and Rossell2016). In line with previous work, the current analysis indicated that Sz patients had an emotion recognition profile consistent with a generalized impairment in face processing, evidenced by patient-control deficits on both whole and part-based conditions of the emotion processing task. In contrast, BD patients were impaired on only the whole-face condition relative to controls.
In all groups, we found that emotion recognition was more accurate for the whole-face relative to the part-based condition. Effect size differences between the conditions showed that both clinical groups failed to use the additional information available when viewing the whole face compared to viewing part-based information to the same extent as controls (d’s=0.6–0.7 vs. 1.3). Slowed response latencies (albeit non-significant) of medium effect size for the whole-face condition in the BD group only, further suggests that BD patients required a greater degree of processing when more information from the whole-face was available.
The whole-face emotion recognition impairments evident in both clinical groups speak strongly to our past findings of configural face processing abnormalities in Sz and BD. When a whole face is presented as opposed to eyes-only, configural information is present. Configural information refers to subtle spacing between facial features, such as distances between the eyes and mouth. These spatial relationships are crucial for accurate emotion perception because they enable a more efficient processing strategy than combining information from multiple features and separately adding information together in a piecemeal manner.
Thus, whole-face perception typically involves more efficient configural processing. This was reflected by the better performance of the healthy control participants compared to both clinical groups on the whole-face condition. The additional part-based emotion recognition impairment observed in Sz suggests that featural processing is also impaired; a finding consistent with our past work in this Sz cohort. Thus, it is possible that the greater and more consistent emotion recognition magnitude deficits generally observed in the Sz compared to the BD literature, may reflect in part, the manifestation of a compounding effect of generalized impairments in the use of normal configural and featural face processing strategies.
Although a mechanistic explanation for these patient group differences cannot be properly addressed in the current data, it is possible that the generalized facial emotion processing alterations observed in Sz are associated with abnormalities in early stage visual processing functions. These functions have been shown to be affected in individuals with Sz but not BD, whereas both BD and Sz patients appear to have impairments in the later stages of visual processing related to higher order perception and integration (Jahshan et al., Reference Jahshan, Wynn, McCleery, Glahn, Altshuler and Green2014). There is some neurobehavioral evidence in Sz cohorts indicating impairments in the lateral connections of the early visual cortex (Keri, Kelemen, Benedek, & Janka, Reference Keri, Kelemen, Benedek and Janka2005).
Furthermore, neuroimaging data show greater cortical thinning in functionally localized early visual cortical regions (Reavis et al., Reference Reavis, Lee, Wynn, Engel, Jimenez and Green2016) and a stronger presence of functional abnormalities in visual processing areas during facial emotion processing in Sz compared with BD (Delvecchio, Sugranyes, & Frangou, Reference Delvecchio, Sugranyes and Frangou2013). Together this suggests that abnormal visual sensory processing may be core to the facial emotion processing abnormalities seen in Sz, conferring greater stimulus ambiguity that leads to more diffuse deficits than that seen in BD. This evidence in combination with our findings suggests important implications for the development of remediation programs to enhance emotion recognition in these disorders; strategies targeting training of face-specific spatial integration abilities may be useful in BD, while more comprehensive training targeting visual information processing more generally may be important for Sz.
This study should be considered in the context of the same limitations that beset our previous report, which includes an inability to control for medication effects due to the varied medications used in the current sample and an absence of the explicit assessment of manic symptom severity (see Joshua et al., Reference Joshua, Van Rheenen, Castle and Rossell2016, for further details). Past research has also shown differences in emotion perception performance between BD patients with and without a psychosis history (Thaler et al., Reference Thaler, Strauss, Sutton, Vertinski, Ringdahl, Snyder and Allen2013). As we were unable to examine differences in psychosis history here due to unavailability of data, this represents a confound that should be addressed in future work.
Furthermore, although pre-morbid IQ scores were controlled in the between-group analysis, we did not assess the effects of general cognitive performance on facial emotion recognition in the study. It is possible that this had some impact on the results as the whole-face processing abnormalities shared between the clinical groups may reflect the outcome of a core shared cognitive deficit related to selective attention, inhibition or processing speed. Further research explicitly examining the relationship between cognition and face and emotion processing is warranted. Finally, it should be noted that the design we used here is one of many that could be used to assess the influence of configural and featural face processing on emotion recognition. Future work should aim to replicate the findings of the present analysis in an independent cohort using different, albeit related measures.
In sum, here we show evidence that the specific configural processing deficit observed in our BD cohort in an earlier analysis influences the processing of facial emotions. On the other hand, both configural and featural processing abnormalities appear to impair emotion recognition in Sz. As negative symptom scores were related to both part-based and whole-face emotion recognition in the BD sample, it appears that such symptoms have an influence on the recognition of facial expressions in this cohort, beyond the influence of face processing strategies themselves. Further work is needed to replicate these findings in a larger sample.
ACKNOWLEDGMENTS
This work was supported by an NHMRC early career fellowship (T.E.V.R.), a Menzies Foundation Scholarship (N.J.), and the Harold and Cora Brennan Trust Grant as well as the Marian & E.H. Flack Trust. Authors Van Rheenen and Joshua are joint first authors. The authors declare that they have no conflicts of interest
