INTRODUCTION
Social cognitive abilities are higher order cognitive processes that include emotion recognition, the ability to infer the beliefs, thoughts, and intentions of others (i.e., cognitive theory of mind, ToM), the ability to make inferences about the feelings of others (i.e., affective ToM), and the understanding of social norms or rules, moral judgement, and empathy (Baez et al., Reference Baez, Herrera, Villarin, Theil, Gonzalez-Gadea, Gomez and Ibanez2013). These abilities can often be impaired following an acquired brain injury (ABI) and may compromise a person’s ability to make social judgements, infer and understand other people’s feelings, and communicate effectively with others (Levin, Reference Levin1995; McDonald, Reference McDonald2013; Morton & Wehman, Reference Morton and Wehman1995). Social cognitive difficulties can therefore have severe psychosocial consequences including a negative impact on the ability to work towards rehabilitation goals, to return to or maintain work, or maintain meaningful social relationships (Ownsworth & McKenna, Reference Ownsworth and McKenna2004).
Despite these negative consequences, social cognition in ABI is rarely assessed in day-to-day clinical practice (Kelly, McDonald & Frith, Reference Kelly, McDonald and Frith2017). In a survey of 443 clinicians treating individuals with brain injury, 84% stated that more than half their patients had deficits in social communication and 78% of these reported not having tools or time to fully assess or treat social communication (Kelly et al., Reference Kelly, McDonald and Frith2017). Part of the difficulty is that few social cognition tests have been developed or validated in ABI populations, or may not be available for purchase or adoption (Sohlberg et al., Reference Sohlberg, MacDonald, Byom, Iwashita, Lemoncello, Meulenbroek, Ness and O’Neil-Pirozzi2019). In addition, many such assessments can be particularly lengthy and may focus on one sole aspect of social cognition. This can be a challenge for busy clinicians with limited time who wish to get an overall idea of their client’s social cognitive ability. Assessment tools for social cognition have often been developed for research into autism spectrum disorders (Baron-Cohen, O’Riordan, Stone, Jones & Plaisted, Reference Baron-Cohen, O’Riordan, Stone, Jones and Plaisted1999). Commonly used social cognition tests, such as the Reading the Mind in the Eyes Test (RME, Baron-Cohen, Jolliffe, Mortimore & Robertson, Reference Baron-Cohen, Jolliffe, Mortimore and Robertson1997; Baron-Cohen, Wheelwright, Hill, Raste & Plumb, Reference Baron-Cohen, Wheelwright, Hill, Raste and Plumb2001) and the Faux-Pas Test (Stone, Baron-Cohen & Knight, Reference Stone, Baron-Cohen and Knight1998), among others, have often been used experimentally to distinguish individuals with and without autism. For example, the RME aims to assess an individual’s ability to infer other people’s mental states by visually examining photos of the eye region and requesting the participant to label the emotional state. The revised version of the RME (Baron-Cohen et al., Reference Baron-Cohen, Wheelwright, Hill, Raste and Plumb2001) aimed to improve the psychometric properties of the original RME and was shown to be able to distinguish between severe TBI and controls’ performance (Henry, Phillips, Crawford, Ietswaart & Summers, Reference Henry, Phillips, Crawford, Ietswaart and Summers2006). However, this test provides little in terms of contextual information or cues and assumes participants are able to infer emotional states by looking at the eye region alone. On the other hand, the Faux Pas test (Stone et al., Reference Stone, Baron-Cohen and Knight1998) uses story vignettes as well as a set of predetermined questions to evaluate an individual’s ability to understand a faux pas (i.e., an unintentional statement that the listener might not want to hear or know, and which can have unintended negative consequences). Research has shown that individuals with a TBI or bilateral damage to the orbitofrontal cortex appear more impaired than those with damage to dorsolateral prefrontal cortex or controls in their ability to detect a faux pas (Milders, Fuchs & Crawford, Reference Milders, Fuchs and Crawford2003; Stone et al., Reference Stone, Baron-Cohen and Knight1998) or correctly rejecting a non faux-pas (Milders, Ietswaart, Crawford & Currie, Reference Milders, Ietswaart, Crawford and Currie2006).
One assessment that aims to provide a generic social cognition profile in neurological disorders by incorporating elements of other well-known social cognition tasks is the Geneva Social Cognition Scale (GeSoc; Martory et al., Reference Martory, Pegna, Sheybani, Métral, Benasconi and Annoni2015). The GeSoc is a screening tool that includes sections of ToM and emotion recognition tasks from the Faux Pas (Stone et al., Reference Stone, Baron-Cohen and Knight1998) and Reading the Mind in the Eyes (RME; Baron-Cohen, et al., Reference Baron-Cohen, Wheelwright, Hill, Raste and Plumb2001) tests, and has 62% sensitivity and 94% specificity for a cut-off of 84 in detecting social cognition deficits in patients with neurological disorders (Martory et al, Reference Martory, Pegna, Sheybani, Métral, Benasconi and Annoni2015). While the GeSoc is likely to have high convergent validity given that it includes items from the Faux Pas and RME, the tool has not been validated against other social cognition assessments.
In terms of ecological validity, the Awareness of Social Inference Test (TASIT; McDonald, Flanagan, Rollins & Kinch, Reference McDonald, Flanagan, Rollins and Kinch2003) assesses emotion recognition, ToM and the capacity to understand the meaning of spoken comments intended non-literally and the ability to distinguish between these and literally intended comments. Participants are tasked with identifying speaker beliefs, intentions and pragmatic inferences contrasting sincere exchanges with sarcasm and lies from video-vignettes with actors. The TASIT aims to provide the assessor with an overview of an individual’s social cognition by measuring a range of abilities using stimuli involving everyday interactions. The TASIT is valid and reliable in the assessment of certain aspects of social cognition in severe traumatic brain injury (TBI; McDonald et al., Reference McDonald, Bornhofen, Shum, Long, Saunders and Neulinger2006). The TASIT comprises three subtests and normative data are currently available for older children and adults (14–60 years) (McDonald, Flanagan & Rollins, Reference McDonald, Flanagan and Rollins2011). A limitation of the TASIT, however, is its administration time which is often not possible in public healthcare settings.
Other social cognition tests in ABI have focussed on specific aspects of social cognition such as ToM. Patients with severe TBI are impaired on ToM assessed using stories and static pictures (Milders et al., Reference Milders, Fuchs and Crawford2003; Shamay-Tsoory, Tomer, Berger & Aharon-Peretz, Reference Shamay-Tsoory, Tomer, Berger and Aharon-Peretz2003). However, while experimental studies tend to distinguish between affective and cognitive ToM (Shamay-Tsoory, Tibi-Elhanany & Aharon-Peretz, Reference Shamay-Tsoory, Tibi-Elhanany and Aharon-Peretz2006), few clinical measures have been developed and validated in ABI that tap both affective and cognitive ToM within the same test (Henry, Cowan, Lee & Sachdev, Reference Henry, Cowan, Lee and Sachdev2015). According to Henry et al. (Reference Henry, Cowan, Lee and Sachdev2015), the Faux Pas (Stone, Baron-Cohen & Knight, Reference Stone, Baron-Cohen and Knight1998) and the Strange Stories task (Happé, Reference Happé1994) tap both affective and cognitive ToM. Tests such as the RME (Baron-Cohen et al., Reference Baron-Cohen, Wheelwright, Hill, Raste and Plumb2001), Ekman-60 (Young, Perrett, Calder, Sprengelmeyer & Ekman, Reference Young, Perrett, Calder, Sprengelmeyer and Ekman2002), Emotion Evaluation Test from the TASIT (McDonald, Flanagan, Rollins & Kinch, Reference McDonald, Flanagan, Rollins and Kinch2003) and Florida Affect Battery (Bowers et al., Reference Bowers, Blonder and Heilman1999) are primarily affective ToM measures whilst the False-Belief Task (Gregory et al., Reference Gregory, Lough, Stone, Erzinclioglu, Martin, Baron-Cohen and Hodges2002) would mainly tap cognitive ToM.
Another aspect of social cognition that has not typically been assessed in ABI is the ability to understand social rules from interpersonal (how another person should behave) and intrapersonal (how they themselves should be behave) viewpoints. While the understanding of social norms has been explored in healthy ageing (Baksh, Abrahams, Auyeung & MacPherson, Reference Baksh, Abrahams, Auyeung and MacPherson2018; Baksh, Bugeja & MacPherson, Reference Baksh, Bugeja and MacPherson2020a; Halberstadt, Ruffman, Murray, Taumoepeau & Ryan, Reference Halberstadt, Ruffman, Murray, Taumoepeau and Ryan2011), autistic adults (Baez et al., Reference Baez, Rattazzi, Gonzalez-Gadea, Torralva, Vigliecca, Decety and Ibanez2012; Baksh et al., Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b) and patients with schizophrenia and bipolar disorder (Baez et al., Reference Baez, Herrera, Villarin, Theil, Gonzalez-Gadea, Gomez and Ibanez2013), few studies have examined social norm understanding in ABI. Beer and colleagues (Reference Beer, John, Scabini and Knight2006) found that patients with orbitofrontal damage due to trauma still had knowledge of social norms but could not apply them in social situations.
IQ and/or executive functions (Ozonoff & McEvoy, Reference Ozonoff and McEvoy1994; Sabbagh, Xu, Carlson, Moses & Lee, Reference Sabbagh, Xu, Carlson, Moses and Lee2006; see MacPherson & Della Sala, Reference MacPherson, Della Sala, Cox, Girardi and Iveson2015) can also be compromised in ABI, which can complicate social cognition assessment. In particular, ToM and executive functions have been reported to be strongly associated (Bora et al., Reference Bora, Vahip, Gonoul, Akdeniz, Alkan and Eryavuz2005; Channon & Crawford, Reference Channon and Crawford2000; Charlton, Barrick, Markus & Morris, Reference Charlton, Barrick, Markus and Morris2009). Apperly, Samson and Humphreys (Reference Apperly, Samson and Humphreys2005) argue that, due to common mechanisms, executive deficits may at least partially underlie deficits in ToM. However, there are single case studies involving ABI patients that have demonstrated a dissociation between ToM and executive functions (Bird, Castelli, Malik, Frith & Husain, Reference Bird, Castelli, Malik, Frith and Husain2004; Lough, Gregory & Hodges, Reference Lough, Gregory and Hodges2001). The debate regarding the associations between social cognition and executive abilities remains unresolved. Nonetheless, in ABI patients who may exhibit executive impairments, it is essential that the social cognition measures used show minimal associations with executive functions to accurately capture social cognitive abilities and identify difficulties, to then target rehabilitation.
It is common to use different tests to examine distinct aspects of social cognition. However, this makes direct comparisons problematic for clinical settings, since these different tests may vary in difficulty. Some variability in the results discussed above are due to the diversity of tasks used to assess ToM, as different tasks have been found to utilise different cognitive mechanisms (Ahmed & Miller, Reference Ahmed and Miller2011). The Edinburgh Social Cognition Test (ESCoT; Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018) was devised to allow clinicians and researchers to examine different aspects of social cognition within the same test. The ESCoT has been validated in healthy adults aged 18–85 years (Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018) and autistic adults (Baksh et al., Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b). ESCoT performance has also been found to dissociate from IQ measures (Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018) or executive functions such as set shifting, inhibition and updating in healthy adults (Baksh et al., Reference Baksh, Bugeja and MacPherson2020a) and autistic adults (Baksh et al., Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b). However, the ESCoT has not been validated in patients with ABI.
Current Study
Given that social cognition impairments are common following ABI, but are not typically assessed, the ESCoT would provide clinicians with a clinical tool that examines distinct aspects of social cognition within the same test. The current study aimed to examine the validity of the ESCoT in people with ABI by assessing the correlation between performance on the ESCoT and performance on other well-established social cognition assessments (i.e., Faux Pas test, Reading the Mind in the Eyes and the Social Norms Questionnaire). In addition, we assessed whether the ESCoT was better able to distinguish between the ABI and control groups than existing social cognition measures. The final aim was to evaluate the psychometric properties of the ESCoT in ABI and compare these to traditional social cognition tests by examining the influence of general cognitive abilities and executive functions on performance.
METHOD
Participants
Patients with first incidence ABI were recruited during their inpatient stay as part of a clinical rehabilitation service at the Neurorehabilitation Hospital at the Astley Ainslie Hospital in Edinburgh, UK. Exclusion criteria were: 1) a prior neurological or psychiatric history; 2) neurodegenerative condition, learning or neurodevelopmental disability; 3) registered blind or deaf; and 4) non-native English speaker. Forty-one patients were recruited (27 males) aged 20–72 years (M = 55.97, SD = 11.30) with 10–20 years of full-time education (M = 12.78, SD = 2.68). According to the Office for National Statistics (2010), our sample included 14 (34.1%) professionals, 3 (7.3%) intermediate workers, 11 (26.8 %) skilled workers, 7 (17.1%) semi-skilled workers, and 6 (14.6 %) unskilled workers prior to their ABI. Diagnoses included cerebrovascular accident (CVA, N = 20), traumatic brain injury (TBI, N = 12), hypoxic brain injury (HBI, N = 4), brain tumour (BT, N = 3) and inflammatory brain injury (IBI, N = 2). For the Hospital Anxiety and Depression Scale (HADS; Zigmond & Snaith, Reference Zigmond and Snaith1983), based on the proposed cut-off of 10 out of 21 (Crawford, Henry, Crombie & Taylor, Reference Crawford, Henry, Crombie and Taylor2001), four patients fell within the clinical range for both anxiety and depression (3 CVA and 1 TBI), whilst another patient scored highly for depression alone (CVA) and another for anxiety alone (HBI). According to Cohen (Reference Cohen1992), a sample size of 67 with power = .80 and α = .05 was required to detect a medium effect size (correlation of r = .30) and a sample size of 23 with power = .80 and α = .05 was required to detect a large effect size (correlation of r = .50) for correlations between the ESCoT and the other traditional measures of social cognition.
Forty-one healthy controls (25 males) aged 20–72 years (M = 55.37, SD = 20.37) also took part. They had 9–20 years of full-time education (M = 13.49, SD = 2.49). Controls were recruited through online advertisement and a research volunteer panel at the University of Edinburgh. No control had a self-reported history of neurological or psychiatric disorders based on the Wechsler Adult Intelligence Scale-IV (WAIS-IV; Wechsler, Reference Wechsler2008) exclusion criteria. The control group did not significantly differ from the ABI group in terms of age (p = .59), full-time education (p = .11) or gender (p = .82).
The study was conducted in accordance with the Declaration of Helsinki Ethical Principles and Good Clinical Practices and was approved by the local NHS Research Ethics Committee (18/NE/0067) and the School of Philosophy, Psychology and Language Sciences Ethics committee at University of Edinburgh (161-1314).
Measures
Premorbid ability
Test of Premorbid Functioning (TOPF; Reference WechslerWechsler, 2011). The TOPF was administered to estimate premorbid IQ. It is composed of 70 words that have atypical grapheme to phoneme translations. The TOPF is co-normed with the WAIS-IV (Wechsler, Reference Wechsler2008) and has very high reliability (.96–.99), test-retest reliability (.89–.95) and concurrent validity with the WAIS-IV Full Scale IQ (r = .70, Holdnack and Whipple Drozdick, Reference Holdnack and Whipple Drozdick2009).
General cognitive ability
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS-A, Reference RandolphRandolph, 2009 ). The RBANS-A was used as a multi-domain screening measure. An aggregate measure of overall performance was computed to provide a Total Index score out of 160. The clinical utility of the RBANS in TBI has been demonstrated with sensitivity and likelihood ratios from modest to strong, as well as high specificity (McKay, Casey, Wetheimer & Fichtenberg, Reference McKay, Casey, Wertheimer and Fichtenberg2007). Both construct validity (Pachet, Reference Pachet2007) and internal reliability (McKay et al., Reference McKay, Casey, Wertheimer and Fichtenberg2007) have been reported, supporting the use of the RBANS as a clinically valid tool for screening mild to severe TBI.
Executive abilities
Verbal Fluency Test (Delis Kaplan Executive Functioning Scale, D-KEFS; Reference Delis, Kaplan and KramerDelis, Kaplan & Kramer, 2001). The D-KEFS Verbal Fluency Test was administered to assess cognitive flexibility. The examinee is given 60 s to generate as many unique words as possible starting with a particular letter (condition 1) and within a certain category (condition 2). The total raw scores for each condition were considered separately. Frontal patients have more difficulty on the letter fluency task relative to the category fluency task, whereas patients with early Alzheimer’s disease often show the opposite pattern due to a breakdown in semantic knowledge (Delis et al., Reference Delis, Kaplan and Kramer2001). Test re-test reliability has been established for the letter and category conditions with coefficients ranging between .36 and .80. The letter fluency condition yielded the highest internal consistency coefficients, which ranged from moderate to high, with most age groups at the good to high levels. Internal consistencies are lower for category fluency (Delis et al., Reference Delis, Kaplan and Kramer2001).
Trail Making Test (D-KEFS; Delis et al., Reference Delis, Kaplan and Kramer2001 ). The Number-Letter Switching condition from the D-KEFS Trail Making Test (TMT) was also administered to assess cognitive flexibility. The examinee is presented with number and letter targets distributed across the page and is asked to switch back and forth between connecting numbers and letters in numerical and alphabetical order (i.e., 1, A, 2, B, etc.). When an error is made by the examinee, they are instructed to return to the last correct target before continuing. The D-KEFS TMT is scored in terms of completion time in seconds. Test re-test reliability coefficients for TMT ranged between .38 and .77, with internal consistency ranging between .69 and .81 (Delis et al., Reference Delis, Kaplan and Kramer2001).
Working Memory Index (WMI) from the Wechsler Adult Intelligence Scale - Fourth Edition (WAIS-IV; Reference WechslerWechsler, 2008 ). To assess working memory, two subtests from the WAIS-IV were administered: Arithmetic and Digit Span. For Arithmetic, the examinee is verbally presented with arithmetical problems that increase in difficulty. For Digit Span, the examinee is verbally presented with a string of numbers and is asked to repeat back the numbers in the same (Forward), reverse (Backward) or sequential (Sequencing) order immediately after stimuli presentation. A Working Memory Index (WMI) was calculated by combining the Arithmetic and Digit Span scores. Reliability and validity of the WAIS-IV has been established with index reliability coefficients ranging from .90 to .98 and test-retest reliability coefficients ranging from .87 to .96 (Wechsler, Reference Wechsler2008).
Theory of mind (ToM)
Faux Pas Test (Stone et al., Reference Stone, Baron-Cohen and Knight1998). The Faux Pas (FP) test is an advanced ToM task based on the ability to recognise whether a faux pas has been committed or not (i.e., a character unintentionally says something they should not have said which could hurt or upset the other character). The stories are read aloud and at the end of each story, the participant is asked questions about detecting a faux pas, understanding a faux pas, understanding the mental state of the receiver of the faux pas, understanding the mental state of the person producing the faux pas; and understanding the details of the story. One point was assigned for each correct response and a ratio score (0–1) was calculated according to Stone et al. (Reference Stone, Baron-Cohen and Knight1998) where the higher the score, the better the performance. The FP test has been administered to autistic individuals (Baron-Cohen et al., Reference Baron-Cohen, O’Riordan, Stone, Jones and Plaisted1999) and adults with behavioural-variant frontotemporal dementia (bvFTD; Gregory et al., Reference Gregory, Lough, Stone, Erzinclioglu, Martin, Baron-Cohen and Hodges2002). Baron-Cohen et al. (Reference Baron-Cohen, Jolliffe, Mortimore and Robertson1997) showed the FP test was a good measure of ToM deficits in children with Asperger’s syndrome. In addition, Stone et al. (Reference Stone, Baron-Cohen and Knight1998) investigated ToM in individuals with damage to the dorsolateral prefrontal cortex, orbitofrontal cortex (OFC) and anterior temporal cortex. They showed that individuals with damage to the OFC were able to understand the FP stories, yet were unable to state that something inappropriate had been said and concluded that the performance of OFC patients was parallel to that of Asperger’s syndrome. Gregory et al. (Reference Gregory, Lough, Stone, Erzinclioglu, Martin, Baron-Cohen and Hodges2002) showed excellent inter-rater reliability (r = .98) in ratings of patients with bvFTD and Alzheimer’s disease
Reading the Mind in the Eyes Test (RME; Baron-Cohen et al., Reference Baron-Cohen, Wheelwright, Hill, Raste and Plumb2001 ). The RME assesses an individual’s understanding of other people’s mental states. Participants are shown 37 (1 practice) photographs of the eye-region of the faces of different actors and given the choice of four adjectives to describe the emotion or internal state the actor is thinking or feeling. The total score is out of 36. Vellante et al. (Reference Vellante, Baron-Cohen, Melis, Marrone, Petretto, Masala and Preti2013) stated that the Italian version of the RME showed good internal consistency as well as good test-retest reliability. However, a review by Olderbak et al. (Reference Olderbak, Wilhelm, Olaru, Geiger, Brenneman and Roberts2015) suggested that the RME typically had poor internal consistency, though acceptable test-retest reliability. Spreng, McKinnon, Mar and Levine (Reference Spreng, McKinnon, Mar and Levine2009) found no correlation between performance on the RME and the Interpersonal Perception Task–15 (Costanzo & Archer, Reference Costanzo and Archer1994), a measure of nonverbal cue understanding in social interactions. Individuals with bvFTD are also impaired on the RME compared to controls (Gregory et al., Reference Gregory, Lough, Stone, Erzinclioglu, Martin, Baron-Cohen and Hodges2002).
The Edinburgh Social Cognition Test (ESCoT, Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018 ). Footnote 1 The ESCoT measures four social cognitive abilities within the same test: affective ToM; cognitive ToM; interpersonal understanding of social norms and intrapersonal understanding of social norms. It consists of 11 self-contained dynamic, cartoon-style everyday social interactions: 1 practice interaction, 5 interactions involving a social norm violation and 5 not involving a social norm violation. Following a video presentation, the participant is presented with four cartoon picture frames in sequential order depicting what the video has just shown. Firstly, the participant is asked to describe what occurred in the interaction to ensure they understand the animation (this was not scored). Then, they are asked four questions about what they have just observed. Each question is awarded a maximum of three points based on the quality of the answer, resulting in a score of 12 points for each social interaction. The total maximum score for the test is 120 points. The ESCoT takes about 20 min to administer and has been validated in healthy younger, middle-aged and older adults (Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018) and autistic adults (Baksh et al., Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b). Our previous work has established the reliability of the ESCoT using intraclass correlation (ICCs), demonstrating a consistency of .90, indicating high inter-rater reliability. We have also assessed internal consistency for the ESCoT by calculating Guttman’s Lambda 4 reliability which produced a coefficient of .70, which is acceptable (Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018). Baksh et al. (Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b) developed cut-off scores for the ESCoT based on the 5th percentile of their normative data: total score ≤ 83; affective ToM ≤ 19; cognitive ToM ≤ 17; interpersonal social norms ≤ 18; and intrapersonal social norms ≤ 22.
Social norm understanding
Social Norms Questionnaire (SNQ; Reference Rankin Rankin, 2008 ). The SNQ is a 22-item questionnaire that screens for potential behaviour changes and assesses how well individuals understand the social standards that govern their behaviour in UK mainstream culture. For example, “would it be socially acceptable to hug a stranger without asking them first?” A total score is obtained by summing the correct items (out of 22). A higher total score indicates greater knowledge of social norms. This measure is yet to be validated.
Procedure
Patients were tested individually over the course of two sessions during their inpatient stay. They performed the assessments in the following order: TOPF, RBANS-A, WAIS-IV, FP, RME, ESCoT, D-KEFS verbal fluency, D-KEFS TMT, and SNQ. Controls only performed the RME, ESCoT and SNQ.
Analyses
Firstly, we fitted multiple linear regression models to examine whether a diagnosis of ABI (i.e., ABI vs. not ABI) predicted performance on the ESCoT subtests, RME and SNQ, while adjusting for the impact of age, sex and years of education. We transformed scores on the social cognition tests using a square-root transformation to avoid violation of normality and results were back transformed by squaring the number. The reference group in the regression models were male for sex and the healthy controls for diagnostic group. Receiver Operator Characteristic (ROC) curve analyses were then conducted for the social cognition tests with both ABI and control data (ESCoT, RME and SNQ) to investigate the discriminant abilities of the social cognition tests to correctly assign participants to their diagnostic group. The control data were from a retrospective data set which had data from the RME and SNQ only. We reported Area under the curve (AUC) for our ROC curve analysis as a measure of diagnostic accuracy. Cronbach’s α was used to establish the internal consistency of each of the subtests of the ESCoT in an ABI population. Kline’s (Reference Kline1999) cut-off of .70 was adopted as the minimum acceptable level of internal consistency.
Pearson’s or Spearman’s correlational analyses were carried out on the ABI data depending on whether they were normally distributed or not to examine the relationship between the ESCoT, general cognitive ability and executive functioning measures and the established social cognition tests. Finally, the relationship between performance on social cognition tests and the general cognitive ability and executive measures were examined using an exploratory regression analysis. In the first stage, the background predictors (age, sex, SES, years of education, HADS-D and HADS-A) which significantly correlated with the outcome variables (ESCoT total score, RME, RMF and SNQ) at a pre-specified significance level of p < .20 were entered into the analysis (Altman, Reference Altman1991) using the enter method. We chose a significance level of p < .20 over more traditional levels since p < .05 can fail in identifying variables known to be important to the outcome variable and simulation studies have shown that a cut-off of p < .20 yields better outcomes (Bursac et al., Reference Bursac, Gauss, Williams and Hosmer2008; Lee, Reference Lee2014). TOPF IQ scores were included in the first stage of the regression analysis if scores correlated with the outcome variables at p < .20. In the second stage, the general cognitive ability and executive measures were entered using the stepwise method (entry criterion p < .05, removal criterion p > .10).
RESULTS
Background Neuropsychological Measures
Table 1 shows ABI performance on the premorbid ability, general cognitive ability, and executive measures. Figure 1 demonstrates the percentage of ABI patients who were impaired on those assessments.
Performance of the ABI patients on the premorbid ability, general cognitive ability, executive function and working memory measures
Note. D-KEFS = Delis Kaplan Executive Functioning Scale; RBANS = Repeatable Battery for Assessment of Neuropsychological Status; SD = Standard Deviation; TOPF = Test of Premorbid Functioning; WAIS-IV = Wechsler Adult Intelligence Scale 4th Edition.
Raw Scores are presented for Fluency Tests.
The percentage of ABI patients in each performance classification across the premorbid ability, general cognitive ability, and executive function measures.
ROC curves for the Edinburgh Social Cognition Test (ESCoT), the Reading the Mind in the Eyes (RME) test and the Social Norm Questionnaire (SNQ).
Group Comparisons between ABI Patients and Controls on ESCoT and Established Social Cognition Tests
Table 2 demonstrates the performance of ABI patients and controls on the ESCoT and other social cognition measures. All patients were able to describe what occurred in the interactions suggesting that they understood the animations.
ABI and control group performance on the social cognition tests
Note. ESCoT = Edinburgh Social Cognition test; FP = Faux Pas test; RME = Reading the Mind in the Eyes test; SNQ = Social Norm Questionnaire; ToM = Theory of Mind.
a Control n = 40.
The regression analyses examining whether a diagnosis of ABI predicted performance on the social cognition tests are presented in Table 3.
Summary of multiple regression analyses for the social cognition tests with diagnosis of ABI as a predictor
Note. ESCoT = Edinburgh Social Cognition test; RME = Reading the Mind in the Eyes test; SNQ = Social Norm Questionnaire; ToM = Theory of Mind.
We found that an ABI diagnosis was significantly associated with poorer performance on all four ESCoT subtests compared to controls even after adjusting for age, sex and years of education. Similarly, poorer performance on ESCoT total score and the RME were significantly associated with an ABI diagnosis. There was no statistically significant association between diagnostic group and SNQ performance.
In the Receiver Operator Characteristic (ROC) curve analyses, the AUC values and 95% confidence intervals were: ESCoT total score = 97.2 (92.5–100.0); RME = 81.1 (71.8–90.4); and SNQ = 59.6 (47.1–72.0). Therefore, the ESCoT is the most effective at distinguishing between the ABI and control groups. The ESCoT total score showed high sensitivity and good specificity (95%, 88% respectively) at detecting ABI using the established cut-off score of 83 or less (Figure 2).
Based on Baksh et al. (2020), 5th percentile cut-off scores which were derived from 236 healthy adults between the ages of 18 and 85, 58.54% of our ABI patients were impaired on the affective ToM subtest, 75.61% on the cognitive ToM subtest, 87.80% on interpersonal understanding of social norms, and 92.68% on intrapersonal understanding of social norms. In comparison, 12.20% of controls were impaired on the affective ToM subtest, 9.76% on the cognitive ToM subtest, 17.07% on interpersonal and 2.44% on intrapersonal understanding of social norms. On the total ESCoT score, 95.12% of ABI participants were impaired in comparison to 12.20% of controls (see Table 4). In Table 5, we also provide the patients’ ESCoT scores based on their diagnosis.
Impairment rate comparisons between groups based on Baksh et al. (2020) cut-off scores
Note. ABI = Acquired Brain Injury; ESCoT = Edinburgh Social Cognition Test; ToM = Theory of Mind.
ESCoT performance by ABI diagnosis
Note. BT = Brain Tumour; CAV = Cerebrovascular Accident; ESCoT = Edinburgh Social Cognition Test; HBI = Hypoxic Brain Injury; IBI = Inflammatory Brain Injury; M = Mean; SD = Standard Deviation; TBI = Traumatic Brain Injury; ToM = Theory of Mind.
Internal Consistency of ESCoT Subtest Items
Cronbach’s α for the 10 affective ToM items was .80. No subtest items were greater than the overall α level. Cronbach’s α for the 10 cognitive ToM items was .70. Similarly, no subtest items were greater than the overall α level. Cronbach’s α for the interpersonal social norm understanding items was .70 and .80 for the intrapersonal social norm understanding subtest. No subtest items were greater than the overall α level.
Comparison of ESCoT with Demographic Variables in ABI Patients
Correlational analyses between the ESCoT subtests and age, gender, SES and years of education yielded only one significant negative correlation between SES and the ESCoT affective component where the lower a patient’s SES, the poorer their performance on affective ToM (r = −.32, p = .04, see Table 6). No other correlations were significant.
Correlational analyses between performance on the measures of social cognition and demographics and background cognitive variables
Note. D-KEFS = Delis Kaplan Executive Functioning Scale; ESCoT = Edinburgh Social Cognition Test; FP = Faux Pas test; HADS-A = Hospital Anxiety and Depression Scale Anxiety Subscale; HADS-D = Hospital Anxiety and Depression Scale Depression Subscale; RBANS = Repeatable Battery for the Assessment of Neuropsychological Status; RME = Reading the Mind in the Eyes test; SES = socio-economic status; SNQ = Social Norms Questionnaire; TMT = Trail Making Test; ToM = Theory of Mind; TOPF = Test of Premorbid Functioning; WAIS-WM = Wechsler Adult Intelligence Scale 4th Ed. Working Memory Index.
** p < .01; *p < .05;
a =Spearman’s correlations.
Comparison of ESCoT with Background Neuropsychological Assessments in ABI Patients
The correlational analyses between ESCoT performance and the background measures are presented in Table 6. Pearson’s correlational analyses revealed that TOPF IQ positively correlated with the ESCoT affective subcomponent (r = .39, p = .01), where the higher the TOPF IQ, the better the affective ToM performance. However, TOPF IQ did not correlate with the other ESCoT subcomponents. The ESCoT affective scores also positively correlated with general cognitive ability (RBANS total score: r = .38; p = .02), working memory (WAIS-WMI: r = .37; p = .02) and D-KEFS Category Fluency scores (r = .34; p = .04). Again, the higher the score, the better the affective ToM performance. Spearman’s correlational analyses demonstrated a significant positive correlation between ESCoT affective scores and D-KEFS Letter Fluency (rho = .38; p = .02), where the more words generated, the better the affective ToM performance, and negatively with D-KEFS TMT (rho = −.41; p = .01), where the faster the D-KEFS TMT performance, the better the affective ToM performance. Spearman’s correlational analysis also showed a significant negative relationship between ESCoT cognitive scores and D-KEFS TMT (rho = −32; p = .05), where the faster the D-KEFS TMT performance, the better the cognitive ToM performance. The ESCoT total score also negatively correlated with D-KEFS TMT (rho = −.39; p = .02), where the faster the D-KEFS TMT performance, the better the overall ESCoT performance.
Comparison of ESCoT with Traditional Social Cognition Measures in ABI Patients
Correlational analyses were conducted between performance on the ESCoT and the FP, RME and SNQ (see Table 6). ESCoT total scores significantly correlated with the FP (r = .34; p = .03), RME (rho = .33; p = .03) and SNQ tests (rho = .36; p = .02). The better the ESCoT performance, the better the performance on the other social cognition measures. ESCoT affective ToM also significantly positively correlated with the FP test (r = .37; p = .02), RME (rho = .52; p = .001) and SNQ (rho = .52; p < .0001). Again, better affective ToM performance was associated with better performance on the traditional social cognition measures. Finally, interpersonal social norm understanding was significantly correlated with FP performance (r = .39; p = .01). No other correlations were significant.
Relationship between Social Cognition Tests and Background Measures
Table 7 provides the regression analyses involving the social cognition tests and background measures. For ESCoT total score, performance was associated with D-KEFS TMT, with higher social cognition associated with faster switching. For the FP test, there was a relationship with working memory where those with higher FP scores showed higher working memory. For RME, higher TOPF IQ and faster switching were associated with better RME scores. Finally, SNQ performance was significantly associated with D-KEFS TMT where those with higher SNQ scores had faster switching.
Summary of multiple regression analyses for the social cognition tests and background measures
Note. ESCoT = Edinburgh Social Cognition Test; FP = Faux Pas test; RME = Reading the Mind in the Eyes test; SNQ = Social Norms Questionnaire.
DISCUSSION
Changes in social behaviour are common and negative consequences of brain injury (Williams & Wood, Reference Williams and Wood2010). However, few clinicians include measures of social cognition when completing a neuropsychological assessment (Kelly et al., Reference Kelly, McDonald and Frith2017), despite evidence that patients with a brain injury are known to experience moderate to severe ToM deficits (Martín-Rodríguez & León-Carrión, Reference Martín-Rodríguez and León-Carrión2010). We demonstrated that an ABI diagnosis was significantly associated with poorer performance on all ESCoT subtests. We also demonstrated good internal consistency of ESCoT items and validity of the ESCoT against established social cognition measures. The ESCoT was most effective at distinguishing between ABI patients and healthy controls, followed by the RME and SNQ. While cut-off scores derived from normative data are not available for the other social cognition tests, the ESCoT had 95% sensitivity and 88% specificity, which is higher sensitivity than the GeSoc (62% sensitivity and 94% specificity). This highlights the ESCoT’s ability to detect social cognition difficulties that could go undetected using traditional social cognition measures.
The ESCoT total and its subcomponents, mainly affective ToM and interpersonal social norm understanding, showed significant associations with well-known and validated social cognition measures, providing evidence of the ESCoT’s convergent validity as a social cognition test. In particular, ESCoT total and the affective ToM subtest correlated with the FP, RME and SNQ. Likewise, in autistic adults, we found that ESCoT total performance significantly correlated with the RME and SNQ (Baksh et al., Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b). However, with autistic adults, it was cognitive rather than affective ToM that positively correlated with the RME. These findings add to the debate about what the RME assesses; our current findings support those studies that suggest that the RME is an affective ToM measure (Duval, Piolino, Bejanin, Eustache & Desgranges, Reference Duval, Piolino, Bejanin, Eustache and Desgranges2011), at least in ABI patients. Though traditionally thought to tap mainly cognitive ToM, Henry and colleagues (Reference Henry, Cowan, Lee and Sachdev2015) suggest the FP is both an affective and cognitive ToM test, and thus our results above would support this hypothesis. While interpersonal social norm understanding also significantly correlated with the FP test, cognitive ToM and intrapersonal social norm understanding did not correlate with any social cognition measure. It is therefore possible that some ESCoT components measure an additional dimension of social cognition (e.g., one’s ability to say how they may behave in certain situations) that traditional social cognition measures do not tap.
Neither age, gender, HADS scores, SES nor years of education predicted ESCoT total scores, although it should be noted that our lowest level of education was 9 years. We therefore cannot rule out an impact of education of having lower levels of education. Regression results showed that better overall performance on the ESCoT was predicted by better D-KEFS TMT scores. In contrast, in healthy older adults, we found that TMT performance did not predict performance on any ESCoT measure (Baksh et al., 2020). In an ABI population who have executive dysfunction and/or social cognition impairment, a relationship may possibly be evident because one function is supporting the other damaged system. However, recent studies have shown that successful performance on the TMT Part-B involves both frontal and nonfrontal regions (Chan et al., Reference Chan, MacPherson, Robinson, Turner, Lecce, Shallice and Cipolotti2015; Jacobson, Blanchard, Connolly, Cannon & Garavan, Reference Jacobson, Blanchard, Connolly, Cannon and Garavan2011) and task performance most likely depends upon several cognitive processes rather than simply executive processes. Therefore, the relationship between ESCoT performance and D-KEFS TMT in our ABI patients might reflect general cognitive impairment rather than an executive impairment. Moreover, as three of the four social cognition tests (i.e., the ESCoT, the RME and the SNQ) correlated with the TMT, the common variance among these tests may be due to general cognitive or executive ability rather than social cognition. It was also a little surprising that the correlations between general cognitive and executive abilities and social cognition were stronger for affective ToM than cognitive ToM. Further work should examine the underlying cause of the relationship between these tests in people with ABI and establish whether our findings also extend to other alternate switching tasks.
Our results support the notion that ToM is a multidimensional construct where two separate systems are involved in processing judgements about others’ beliefs and intentions and judgements about other people’s emotions and feelings (Shamay-Tsoory & Aharon-Peretz, Reference Shamay-Tsoory and Aharon-Peretz2007; Shamay-Tsoory et al., Reference Shamay-Tsoory, Tibi-Elhanany and Aharon-Peretz2006). However, previous work would suggest that damage to the dorsolateral prefrontal cortex overlaps with impairment in cognitive rather than affective ToM and executive abilities. Several of our ABI patients are likely to have diffuse rather than focal brain damage, affecting a number of cortical areas, as well as their white matter connections so our current findings may depend on the brain areas involved. However, our ABI sample were recruited as part of a clinical rehabilitation service so clinical scan data were not available to investigate the focal damage of our ABI group.
Higher working memory scores predicted better FP performance. This is not surprising given that complex ToM tasks possibly involve other cognitive functions, such as executive ability, attention, speed of information processing, and memory (Bibby & McDonald, Reference Bibby and McDonald2005; Henry et al., Reference Henry, Phillips, Crawford, Ietswaart and Summers2006). Li et al. (Reference Li, Wang, Wang, Tao, Xie and Cheng2012) found that inhibition, updating, speed processing and memory mediated age differences on the FP task. Our current findings suggest that there is a relationship between working memory and ToM in ABI, at least in terms of FP performance. Within our social cognition battery, the FP was the only measure where the stimuli were read aloud to patients. Therefore, the FP task may place additional demands on working memory as patients are required to remember the events of a verbal story. These findings highlight the importance of the modality of social cognition measures (Henry et al., Reference Henry, Phillips, Ruffman and Bailey2013), especially in clinical populations.
We also found that premorbid estimation of ability using a single-word reading task, the TOPF IQ, predicted RME performance. This suggests that verbal ability predicts performance on certain social cognition tests. Indeed, previous findings, including our own, have found that verbal ability predicts performance on traditional social cognition tests (Baker et al., Reference Baker, Peterson, Pulos and Kirkland2014; Baksh et al., Reference Baksh, Abrahams, Auyeung and MacPherson2018, Reference Baksh, Abrahams, Bertlich, Cameron, Jany, Dorrian and Auyeung2020b; McDonald et al., Reference McDonald, Flanagan, Rollins and Kinch2003). A similar result, however, has not been found for the ESCoT.
Importantly, a significant proportion of our sample scored within the average range on the cognitive indices such as the RBANS and the executive tasks and yet failed the ESCoT. This highlights the need for assessing social cognition in ABI and including social cognitive assessments such as the ESCoT in clinical settings where decisions regarding risk, capacity, community living, among many others, may be required. Overall, the results showed that patients who showed a better understanding of others’ thoughts and social rules also performed better on cognitive flexibility tasks.
This is the first study to examine affective and cognitive ToM as well as the interpersonal and intrapersonal understanding of social norms in an ABI population. However, our study has some limitations. The sample size and heterogeneity of the ABI sample mean that the results should be interpreted with caution and different aetiologies could not be investigated systematically. The original sample size calculation was based on a conservative correlation of .30 (Cohen, Reference Cohen1992); however, correlations between social cognition and ESCoT measures were substantially higher than this, suggesting that our study had sufficient power to detect correlations of the level obtained. Future work should include a larger ABI sample to allow for a systematic investigation of different aetiologies and their performance on the ESCoT. Another possible limitation of our study is that it does not assess emotion recognition, and as such, other social cognition tests would need to be included in the assessment in order to assess the entirety of social cognition abilities.
In conclusion, the ESCoT appears to be a clinically useful tool to provide clinicians with relevant information about ABI individuals’ appraisal of social situations and interaction with others. The ESCoT was the most effective social cognitive test at distinguishing between ABI and healthy controls. Inclusion of a social cognition measure in day-to-day clinical practice and assessment will improve clinicians’ ability to support individuals in the community and target their rehabilitation plans.
FINANCIAL SUPPORT
No funding was sought for the completion of this study.
CONFLICT OF INTEREST
None.
