Introduction
Executive functioning (EF) involves top-down regulation of behavior, emotion, and cognition, and is called for in non-routine situations where habitual responses and prior experiences are insufficient. Executive dysfunction (ED) is common in a vast array of neurological and neuropsychiatric conditions (Bonelli & Cummings, Reference Bonelli and Cummings2007). Traumatic brain injury (TBI) and other acquired brain injuries are particularly prone to cause ED (Bonelli & Cummings, Reference Bonelli and Cummings2007; Ponsford et al., Reference Ponsford, Downing, Olver, Ponsford, Acher, Carty and Spitz2014; Sigurdardottir et al., Reference Sigurdardottir, Andelic, Wehling, Roe, Anke, Skandsen and Schanke2015), but EF is also commonly affected in other neurological conditions such as Parkinson’s disease (PD) (Narayanan, Rodnitzky, & Uc, Reference Narayanan, Rodnitzky and Uc2013). Additionally, many neuropsychiatric disorders have ED as part of their symptomatology. For example, impaired impulse regulation, controlled attention, and working memory are core aspects of attention deficit hyperactivity disorder (ADHD) (Hervey, Epstein, & Curry, Reference Hervey, Epstein and Curry2004; Marchetta, Hurks, Krabbendam, & Jolles, Reference Marchetta, Hurks, Krabbendam and Jolles2008; Nigg et al., Reference Nigg, Stavro, Ettenhofer, Hambrick, Miller and Henderson2005; Woods, Lovejoy, & Ball, Reference Woods, Lovejoy and Ball2002). Over the past decade it has, furthermore, been increasingly recognized that major psychiatric conditions such as bipolar disorder (BD) and schizophrenia involve disturbances of cognition, with ED as a key aspect (Carpenter et al., Reference Carpenter, Bustillo, Thaker, van Os, Krueger and Green2009; Goldberg, Andrews, & Hobbs, Reference Goldberg, Andrews and Hobbs2009). Borderline personality disorder (BPD) is yet another condition where emotional dysregulation and impulsivity are regarded as diagnostic hallmarks (Bøen et al., Reference Bøen, Hummelen, Elvsåshagen, Boye, Andersson, Karterud and Malt2015; Lieb, Zanarini, Schmahl, Linehan, & Bohus, Reference Lieb, Zanarini, Schmahl, Linehan and Bohus2004).
Assessment of Executive Functioning: In Need of Ecologically Valid Measures
Identifying and characterizing ED poses one of the most common and challenging issues in clinical neuropsychology (Royall et al., Reference Royall, Lauterbach, Cummings, Reeve, Rummans, Kaufer and Coffey2002), since executive problems are both multifaceted and most prominent in unstructured environments. It is well-established that patients with ED may have severe problems in occupational, leisure, social, and emotional domains, despite normal or near-normal neuropsychological test profiles (Knight & Stuss, Reference Knight and Stuss2002; Levine, Katz, Dade, & Black, Reference Levine, Katz, Dade and Black2002; Zald, Reference Zald2002; Zald & Andreotti, Reference Zald and Andreotti2010). Indeed, performance-based tests have been suggested to account for less than half of the variance in everyday executive functioning (Chaytor & Schmitter-Edgecombe, Reference Chaytor and Schmitter-Edgecombe2003), highlighting the need for additional standardized measures with predictive value in relation to everyday functioning (Burgess et al., Reference Burgess, Alderman, Forbes, Costello, Coates, Dawson and Channon2006; Zald & Andreotti, Reference Zald and Andreotti2010).
Collection of self- and informant-reported data can provide valuable additional information in this regard (Gioia & Isquith, Reference Gioia and Isquith2004; Gioia, Kenworthy, & Isquith, Reference Gioia, Kenworthy and Isquith2010). Several questionnaires of EF are in clinical use, although the availability of normative data and descriptions of psychometric properties vary (see Malloy & Grace, Reference Malloy and Grace2005; Zald & Andreotti, Reference Zald and Andreotti2010; for review). However, the correspondence between test measures and questionnaires of EF has repeatedly been shown to be low. It has consequently been suggested that questionnaires measure ED more at the behavioral than at the cognitive level (Anderson, Parmenter, & Mok, Reference Anderson, Parmenter and Mok2002), and that performance-based and rating measures of EF assess different aspects of cognitive and behavioral functioning that independently contribute to clinical problems (Toplak, West, & Stanovich, Reference Toplak, West and Stanovich2013). This interpretation suggests that questionnaires do measure EF, just other dimensions or levels of the construct, which are more sensitive to difficulties of daily living. However, verification of this assumption requires knowledge of the relationship between rating scales of EF and other self-report measures specifically targeting other functional domains, such as emotional distress.
The BRIEF-A
One widely used self-report rating scale is the Behavior Rating Inventory of Executive Function, Adult version (BRIEF-A) (Roth, Isquith, & Gioia, Reference Roth, Isquith and Gioia2005). The children versions of the questionnaire (BRIEF and BRIEF-P) (Gioia, Espy, & Isquith, 2002; Gioia, Isquith, Kenworthy, & Barton, Reference Gioia, Isquith, Kenworthy and Barton2002; Gioia, Isquith, Guy, & Kenworthy, Reference Gioia, Isquith, Guy and Kenworthy2000) have been extensively investigated (Gioia et al., Reference Gioia, Kenworthy and Isquith2010; Isquith, Roth, Kenworthy, & Gioia, Reference Isquith, Roth, Kenworthy and Gioia2014). Despite a growing body of empirical studies involving the BRIEF-A, there is still a paucity of publications. Some reports have demonstrated elevated BRIEF-A scores in adults with ADHD (Grane, Endestad, Pinto, & Solbakk, Reference Grane, Endestad, Pinto and Solbakk2014; Miranda, Mercader, Fernandez, & Colomer, Reference Miranda, Mercader, Fernandez and Colomer2013; Roth, Lance, Isquith, Fischer, & Giancola, Reference Roth, Lance, Isquith, Fischer and Giancola2013), pathological gamblers (Reid, McKittrick, Davtian, & Fong, Reference Reid, McKittrick, Davtian and Fong2012), ecstasy/polydrug users (Hadjiefthyvoulou, Fisk, Montgomery, & Bridges, Reference Hadjiefthyvoulou, Fisk, Montgomery and Bridges2012), and adult survivors of pediatric acute lymphoblastic leukemia (Tamnes et al., Reference Tamnes, Zeller, Amlien, Kanellopoulos, Andersson, Due-Tonnessen and Fjell2015).
Studies of patients with brain injury have also supported clinical usefulness (Matheson, Reference Matheson2010; Rabin et al., Reference Rabin, Roth, Isquith, Wishart, Nutter-Upham, Pare and Saykin2006; Waid-Ebbs, Wen, Heaton, Donovan, & Velozo, Reference Waid-Ebbs, Wen, Heaton, Donovan and Velozo2012). To our knowledge, only two studies have explored frontal anatomical specificity. In patients with schizophrenia, self-reported working memory problems on the BRIEF-A were significantly related to smaller bifrontal volume (Garlinghouse, Roth, Isquith, Flashman, & Saykin, Reference Garlinghouse, Roth, Isquith, Flashman and Saykin2010). In one of the samples included in this study, it was shown that orbitofrontal lesions were associated with higher BRIEF-A scores than dorsolateral prefrontal injuries, and that BRIEF-A scores were strongly correlated with emotional distress, but not with cognitive test measures (Løvstad, Funderud, Endestad, et al., Reference Løvstad, Funderud, Endestad, Due-Tonnessen, Meling, Lindgren and Solbakk2012). A recent TBI-study demonstrated that depressive symptoms, but not neuropsychological test results, predicted BRIEF-A scores 2–5 years following injury (Finnanger et al., Reference Finnanger, Olsen, Skandsen, Lydersen, Vik, Evensen and Indredavik2015). The lack of correlation with cognitive test measures is in accordance with pediatric BRIEF-studies (Anderson, Anderson, Northam, Jacobs, & Mikiewicz, Reference Anderson, Anderson, Northam, Jacobs and Mikiewicz2002; Mangeot, Armstrong, Colvin, Yeates, & Taylor, Reference Mangeot, Armstrong, Colvin, Yeates and Taylor2002; Vriezen & Pigott, Reference Vriezen and Pigott2002).
In summary, there are a modest number of empirical studies involving the BRIEF-A, and no studies have compared symptom levels across clinical populations known to show ED. More knowledge is needed about the clinical properties of the BRIEF-A, that is, establishing its association to other functional domains than EF, such as emotional distress. There is, furthermore, a need to explore whether the BRIEF-A normative data are appropriate across cultural settings, as studies involving both the BRIEF (Hovik et al., Reference Hovik, Egeland, Isquith, Gioia, Skogli, Andersen and Oie2014), and the BRIEF-A in a Northern-European country (Grane et al., Reference Grane, Endestad, Pinto and Solbakk2014; Løvstad, Funderud, Endestad, et al., Reference Løvstad, Funderud, Endestad, Due-Tonnessen, Meling, Lindgren and Solbakk2012; Sølsnes, Skranes, Brubakk, & Lohaugen, Reference Sølsnes, Skranes, Brubakk and Lohaugen2014), have indicated mean scores below the U.S. normative mean of T=50 in healthy respondents.
Study Aims
The main aim was to study EF in daily living using the BRIEF-A in neurological and neuropsychiatric clinical samples as well as in a large healthy comparison group (HCG), to explore the association of the BRIEF-A to self-reported psychological problems, and to explore the applicability of the U.S. normative data in a Northern-European country.
A joint database derived from clinical studies that included the BRIEF-A in their assessment protocols, was established. The following specific research questions were addressed: (1) What is the level of self- and informant reported dysexecutive symptoms in various neurological and neuropsychiatric patient groups? (2) What is the relationship between the BRIEF-A and self-reported emotional distress? Based on earlier studies (Finnanger et al., Reference Finnanger, Olsen, Skandsen, Lydersen, Vik, Evensen and Indredavik2015; Løvstad, Funderud, Endestad, et al., Reference Løvstad, Funderud, Endestad, Due-Tonnessen, Meling, Lindgren and Solbakk2012), we expected that both the HCG and patients would display strong covariations between the BRIEF-A and self-reported emotional functioning. We, furthermore, expected to confirm no or low associations with performance-based cognitive tests of EF. (3) What are the reported BRIEF-A score levels in a large Northern-European healthy comparison sample? Existing studies (Hovik et al., Reference Hovik, Egeland, Isquith, Gioia, Skogli, Andersen and Oie2014; Sølsnes et al., Reference Sølsnes, Skranes, Brubakk and Lohaugen2014) gave reason to expect lower scores than the U.S. normative mean of T=50.
Methods
Participants
This work represents a collaboration between researchers in Norway who have applied the BRIEF-A, neuropsychological tests of intelligence (IQ) and EF, and measures of psychological functioning to adult patients with severe TBI (sTBI), focal prefrontal (PFC) or cerebellar (CC) lesions, PD, ADHD, type II BD-II, or BPD. All studies were approved by a Norwegian Regional Committee for Medical and Health Research Ethics and adhered to the Declaration of Helsinki.
HCGs from the studies involving PFC and cerebellar lesions, ADHD, and BD-II and BPD, were collapsed, with the following numbers in each substudy: PFC (n=21), CC (n=20), BD-II and BPD (n=43), and ADHD (n=31), providing an aggregated HCG of n=115.
The patients with sTBI participated in a prospective national population-based multicentre study carried out from 2009 to 2012. The data presented are from the 1-year follow-up. For details on study design and sample characteristics, see Røe et al. (Reference Røe, Skandsen, Anke, Ader, Vik, Lund and Andelic2013) and Sigurdardottir et al. (Reference Sigurdardottir, Andelic, Wehling, Roe, Anke, Skandsen and Schanke2015).
The PFC group was derived from a study on neuropsychological and electrophysiological indices of cognitive control. Mode of lesion verification and mapping, and detailed sample descriptions are provided in Løvstad, Funderud, Endestad, et al. (Reference Løvstad, Funderud, Endestad, Due-Tonnessen, Meling, Lindgren and Solbakk2012) and Løvstad, Funderud, Lindgren, et al. (Reference Løvstad, Funderud, Lindgren, Endestad, Due-Tonnessen, Meling and Solbakk2012).
The CC group consisted of patients participating in a study of the effect of focal cerebellar lesions, see Moberget et al. (Reference Moberget, Andersson, Lundar, Due-Tonnessen, Heldal, Endestad and Westlye2015) for details.
The PD group was recruited from a study on the effect of bilateral deep brain stimulation (DBS) of the subthalamic nucleus, with pre-surgery data included here. For further details, see Pham et al. (Reference Pham, Solbakk, Skogseid, Toft, Pripp, Konglund and Malt2015).
The patients with ADHD participated in a study exploring neurocognitive and electrophysiological functioning in newly diagnosed and unmedicated adult ADHD, recruited from 2008–2011. For details, see Grane et al. (Reference Grane, Endestad, Pinto and Solbakk2014).
The BD-II and BPD groups were part of a study examining brain function with neurocognitive tests and neuroimaging methods. Patients meeting the DSM-IV criteria for BPD and BP-II were recruited, see Bøen et al. (Reference Bøen, Hummelen, Elvsåshagen, Boye, Andersson, Karterud and Malt2015).
Measures
BRIEF-A
The BRIEF-A is a 75-item questionnaire capturing adults’ self-reported everyday EF. Participants answer the following question: “During the past 6 months, how often has each of the following behaviors been a problem?,” with responses scored as never=1; sometimes=2; or often=3. Results yield a composite index score, Global Executive Composite (GEC), and two sub-index scores; Behavioral Regulation Index (BRI) and Metacognition Index (MI), based on nine subscales. Raw scores are transformed into age-corrected T-scores. There is a self-report and an informant version (Roth et al., Reference Roth, Isquith and Gioia2005). Self- and informant report versions were applied in all samples, except self-report only in the BD-II/BPD study. Scores on the validity scales of the BRIEF-A did not exceed the recommended cutoff values (Negativity >6, Infrequency >3, and Inconsistency >8) (Roth et al., Reference Roth, Isquith and Gioia2005) in any group. Participants with more than one validity scale score above recommended cutoff values were excluded, and no protocols with elevated scores on the Inconsistency scale were retained.
Cognitive Test Measures
Using the Wechsler Abbreviated Scale of Intelligence subtests, full-scale IQ was either established using all four (WASI; PFC group), or the Vocabulary and Matrix Reasoning subtests (CC, BD-II, BPD, PD, sTBI), both procedures as recommended in the WASI manual (Wechsler, Reference Wechsler1999). In the ADHD group, the Wechsler Adult Intelligence Scale Third Edition (WAIS-III) was applied (Wechsler, Reference Wechsler1997). The neuropsychological tests reported are those overlapping in all substudies: Letter-Number Sequencing from the WAIS-III, and Color Word Interference Test (CWIT) conditions 1–4 from the Delis-Kaplan Executive Function System (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001). An overall EF index score (EF Index) was established based on Z-scores on Letter-Number Sequencing (working memory), and CWIT3 minus 1 (inhibition), and CWIT4 minus 1 (switching). Letter-Number Sequencing scores were reversed before the measures were summed and divided by three, rendering high EF Index scores indicative of deficit.
Emotional Functioning
The Symptom Checklist-90-Revised (SCL-90-R; Derogatis, Reference Derogatis1994) was applied by all studies except the sTBI study, where SCL-5, a short form of the SCL-90-R, was used, providing a total score, and the Hospital Anxiety and Depression Scale (HAD; Zigmond & Snaith, Reference Zigmond and Snaith1983), which provides a total, anxiety, and depression score. Analyses focused on total scores indicating general psychological distress, and scores on anxiety and depression scales.
Statistical Analysis
Group differences in BRIEF-A scores, neuropsychological test measures, and psychological symptom load scores were explored with analysis of variance, with Group as between-subjects factor. Due to expected covariations between BRI and MI, and between the Depression and Anxiety scales of the SCL-90-R, data involving these pairs of data were entered in multivariate analyses of variance, while group effects on the GEC of the BRIEF-A and the GSI of the SCL-90-R were explored with univariate analysis. Wilks’ lambda statistics are reported for multivariate tests. Bonferroni corrected post hoc analyses are provided. Linear regression analysis showed that gender did not significantly predict any measure, while age and education predicted results on some variables.
Therefore, age and education, but not gender, were entered as covariates in all analyses. Differences between BRIEF-A Self- and Informant reports were explored with paired samples t tests. Relationships between measures were examined with partial Pearson’s correlation coefficients (two-tailed). Dichotomous variables were explored with chi-square tests. Partial eta squared (η p 2), R 2, and odds ratios are reported as effect size measure in the analysis of variance, correlation, and chi square tests, respectively. η p 2 of .01, .06, and .14, and R 2 of .01, .09, and .25 were considered small, medium and large (Cohen, Reference Cohen1992). Due to the high number of comparisons performed, a conservative significance level of p≤.01 was chosen.
Results
Demographics
There was a group effect of age (F(7,402)=34.14; p<.001), gender (F(7,401)=10.55; p<.001), and education (F(7,398)=7.61; p<.001) (see Table 1). The HCG was younger than the sTBI, PFC, and PD groups. In accord with typical age of onset, the PD group was significantly older than all other groups. The sTBI group was older than the BD-II and CC groups, while the CC, ADHD, and BD-II groups were younger than patients with PFC lesions. As expected from TBI demographics, the sTBI group had a higher male:female ratio than the PD, BD-II, and BPD groups. The ADHD group had lower levels of education than all groups except BD-II and CC.
Demographic characteristics of the study samples.
Group Differences on BRIEF-A Index Scores
The self-report form
There was an overall group effect on all three BRIEF-A self-report index scores: (GEC (F(7,402)=32.16; p<;001; η p 2=.36); MI and BRI (F(14,784)=15.95; p<.001; Wilks’Λ=0.61; η p 2=.22). See Table 2 for univariate statistics. The BRI and MI differed significantly from each other in all groups, thus GEC, BRI, and MI were analyzed separately.
The Behavior Rating Inventory of Executive Function (BRIEF-A): Self- and Informant Report Forms.
Note. Statistical comparisons are based on raw-scores. Scaled T-scores (SD), and raw scores (SD).
* p<.01.
** p<.001.
n.a.=not available.
The PFC group did not deviate significantly from the HCG on any BRIEF-A index. The PD group did not differ from the HCG on the GEC and BRI. The CC group did not differ on the BRI. All other post hoc tests showed that patients reported more dysexecutive symptoms than the HCG (p-values<.001–.02). Of note, the HCG group tended to have index T-scores in the mid-40 range, with the 95% confidence interval falling below the expected normative mean of T=50 for all three indexes for both self- and informant rated BRIEF-A protocols. While patients with neurological conditions (sTBI, PD, CC, and PFC lesions) had index T-scores in the 45–50 range, the neuropsychiatric subgroups (ADHD, BD-II, and BPD) rated themselves around or above the recommended clinical cutoff score of T=65. See Figure 1.
Mean self-reported BRIEF-A Index T-scores (± 2 standard errors); healthy comparison group and all patient groups.
In accord with this, post hoc analyses showed that the neurological groups (sTBI, PFC, CC, and PD) did not differ from each other, but from the ADHD, BD-II, and BPD groups on all indexes (p<.001), with the exception that the PD and CC groups did not deviate significantly from patients with BPD on the MI, nor the CC from the BPD group on the GEC. With these exceptions, the ADHD, BD-II, and BPD groups did not differ from each other, but from the other groups on all three indexes (p<.001–.01).
Table 3A shows the proportion of participants and informants in each group having BRIEF-A index scores in the clinical range according to the U.S. norms (T-score >65), as well as the proportion with scores more than 1.5 SD from the mean of the HCG of the current study. The table illustrates that a larger proportion of patients with neuropsychiatric conditions are classified in the clinical range compared to patients with neurological disease, and that basing clinical classification on the Norwegian HCG results in a major increase in the number of clinical cases. The results in Table 3B also indicate that, while the PFC and PD groups did not differ significantly from the HCG based on the U.S. norms, they did so based on the mean of the current HCG, suggesting improved detection of potentially clinically significant impairment.
The BRIEF-A Global Executive Composite (GEC), self- and informant report form.
a Odds ratio (OR) in Table 3A is an effect size estimation of the difference between the patient group in question and the HCG; i.e., the odds that a person with the condition will be classified as above suggested cut-off values compared to a healthy comparison person.
b OR in Table 3B provides an estimate of the odds that a person with this condition will be classified as different from an HCG person.
c OR cannot be computed due to all HCG subjects having below cut-off values according to U.S. norms.
d Chi square test cannot be computed due to all subjects in both the HCG and the PD groups having below cut-off values according to U.S. norms.
BRIEF-A and Cognition
For raw scores and statistics with regard to group differences in neuropsychological test measures, see Supplementary Table 1. As hypothesized, no significant correlations where found between IQ or the EF Index, and BRIEF-A. Regarding individual tests, analysis involving the aggregated sample showed no significant correlations. At a group level, the only significant finding was a positive correlation between CWIT1 and BRI in the PD group (r=.39; p<.01; R 2=.15).
Group Differences in Psychological Functioning
Group comparison of psychological distress was performed without the sTBI group due to the use of other measures than the SCL-90-R. There were clear group effects on the SCL-90-R General Severity Index (GSI) (F(8,261)=30.28; p<.001; η p 2=.42), (see Fig. 2), and on the Anxiety and Depression scales (F(12,504)=16.12; p<.001; Wilk’s Λ=0.52; η p 2=.28). Patients displayed higher symptom load than the HCG (p<.01), with the exception that the CC group did not deviate on any measure, the PFC group did not differ on the Anxiety scale, and the PD group did not differ from the HCG regarding depressive symptoms. Of note, the relationship between the GEC and the EF Index remained nonsignificant also when level of emotional distress was controlled for. See Figure 2 for EF Index, BRIEF-A GEC, and SCL-90-R GSI scores across groups. For raw scores and statistics, see Supplementary Table 2.
Mean self-reported standardized BRIEF-A Global Executive Composite (GEC), EF Index, and the SCL-90-R Global Severity Index (SCL-90-R GSI) (±2 standard errors); healthy comparison group and all patient groups.
BRIEF-A and Psychological Distress
In the total sample, except the sTBI group, scores on the BRIEF-A indexes were strongly correlated with the SCL-90-R GSI, Anxiety, and Depression scores (r range .65–.76; p<.001; R 2=.43–.58). The GSI, Anxiety and Depression measures correlated significantly with all BRIEF-A indexes in the HCG (r range .54–.73; p<.001; R 2=.29–.53), patients with PFC lesions (r range .63–.78; p<.001; R 2=.4–.61), and the PD group (r range .43–.69; p<.01–.001; R 2=.18–.47). The same pattern was present in the CC group (r range .6–.70; p<.01–.001; R 2=.36–.49), except that the Anxiety scale, correlated at the p<.02–.03 level only (r=.49–.54; R 2=.24–.3). Likewise, the GEC, BRI, and MI correlated with the SCL-5, HAD total, HAD anxiety, and HAD depression scores in the sTBI group (r range .45–.6; p<.001; R 2=.2–.36).
For patients with ADHD, the SCL-90-R GSI only correlated significantly with BRIEF-A GEC (r=.51; p<.01; R 2=.26) and BRI (r=.56; p<.001; R 2=.31), and anxiety correlated with the BRI (r=.48; p<.01; R 2=.23). The only significant correlations seen in the BP-II and BPD groups, was between SCL-90-R GSI and the BRI (r=.59; p<.01; R 2=.35).
Distinction between Neurological and Neuropsychiatric Samples
As the BRIEF-A scores tended to cluster in a “neurological” group that included the sTBI, PFC, CC, and PD samples, and a “neuropsychiatric” group that encompassed the ADHD, BD-II, and BPD groups, the analyses were repeated with the patient data collapsed into these two overarching groups.
This showed clear overall group differences for the BRIEF-A (GEC (F(2,402)=107.46; p<.001; η p 2=.35); MI and BRI (F(14,784)=15.95; p<.001; Wilks’Λ=0.61; η p 2=.22), and all three groups differed from each other (p<.001). See Figure 3.
Mean self-reported standardized BRIEF-A Global Executive Composite (GEC), EF Index, and the SCL-90-R Global Severity Index (SCL-90-R GSI) (± 2 standard errors); healthy comparison group, neurological and neuropsychiatric groups.
Correlations between BRIEF-A and neuropsychological test measures were nonsignificant in the neurological and neuropsychiatric groups. Comparison of emotional distress levels was performed without the patients with sTBI, and showed overall group effects for both the SCL-90-R GSI (F(2,261)=85.45; p<.001; η p 2=.40), and the Anxiety and Depression scales (F(4,794)=51.19; p<.001; Wilks’Λ=0.63; η p 2=.21). The three groups differed from each other (p<.001), with lower scores for the HCG, followed by the neurological and neuropsychiatric groups, respectively. In the neurological group, both the GSI, Anxiety and Depression scales correlated in the .51–.69 range (p<.001; R 2=.26–.48) with BRIEF-A indexes. Significant correlations were lower in the neuropsychiatric group (r range .35–.55; p<.01–.001; R 2=.12–.30), and the MI Index did not correlate significantly with the Anxiety and Depression scales.
Informant versus Self-Reports
Note that the sample sizes of informant data deviated from the self-report, and that no informant data were available for the BD-II and BPD groups (see Table 2). There were no significant differences between self- and informant scores for the HCG, PFC, CC, or PD groups, while informants of patients with ADHD reported lower scores (GEC: t(33)=4.08; p<.001; BRI: t(31)=4.17; p<.001; MI: t(31)=3.51; p<.001). In the sTBI group, informants reported higher scores than patients on the GEC (t(93)=−2.55; p<.01) and MI (t(94)=−2.58; p<.01). Correlations between Self- and Informant data (Table 2) demonstrated higher levels of covariations in the HCG, PFC, CC, and ADHD groups compared to sTBI and PD. At a group level, informants did not rate any patient group in the established clinical range of T>65.
No significant associations were found between informant BRIEF-A and IQ or the EF index. In the aggregated sample, the CWIT1 correlated with GEC (r=.19; p<.01; R 2=.04) and MI (r=.23; p<.001; R 2=.05), and the CWIT4 with MI (r=.17; p<.01). In the sub-groups, the only significant effect was that CWIT1 correlated with the MI index (r=.17; p<.01; R 2=.03) in the CC group.
The aggregated informant data confirmed an association between BRIEF-A indexes and SCL-90-R GSI, Anxiety and Depression scales (r range .36–52; p<.001; R 2=.13–.27). In the HCG, GEC and MI, but not BRI, correlated significantly with SCL-90-R GSI, Anxiety and Depression (r range .41–.52; p<.01–.001; R 2=.17–.27). In the PFC group, only the GEC and GSI were associated (r=.56; p<.01; R 2=.31), and in the CC group, BRI and Depression, were significantly associated (r=.61; p<.01; R 2=.37). In the sTBI group, informant-reported GEC and BRI correlated significantly with the HAD total score (GEC: r=.29; p<.01; R 2=.08; BRI: r=.37; p<.001; R 2=.14). No significant correlations were found in the PD and ADHD groups.
Discussion
This study showed that patients with neuropsychiatric disorders reported more ED in everyday life, and higher levels of emotional distress than patients with neurological conditions and healthy participants. As expected from previous studies, self- and informant reported EF showed weak or no correlations with cognitive test performance. Of interest, self-reported EF and emotional symptom levels were strongly inter-correlated in neurological groups and healthy participants, an association that was weaker or non-existent in the neuropsychiatric groups. The same pattern was seen in informant data, although with weaker associations. Healthy Norwegian respondents had BRIEF-A scores 1/2–3/4 SD below the U.S. normative mean.
BRIEF-A Scores across Clinical and Healthy Samples
A noteworthy finding was the relatively low GEC-scores in the sTBI and PFC groups, as ED is common and, therefore, could be expected in these samples. The use of self-rating tools is prone to result in inaccurate reports in patients with reduced symptom awareness (Koenigs & Tranel, Reference Koenigs and Tranel2006). On the other hand, informant reports did not differ significantly from the patient reports, suggesting that lack of insight might not be the sole explanation for low scores. Also, it is of interest that the ADHD group resembled the BPD and BD-II groups, with low EF-Index scores, high emotional symptom load and high BRIEF-A scores.
Relationship to Emotional Distress
As hypothesized, there were no significant associations between the BRIEF-A and neurocognitive test measures, except an association with processing speed in the PD group. The overall pattern was comparable in BRIEF-A informant data, except a correlation with processing speed and switching in the total sample, suggesting that informants might recognize functional limitations that patients do not. The lack of or weak associations between performance-based neuropsychological measures and self- and informant-reported symptoms on the BRIEF-A is in line with studies involving both adults (Niendam, Horwitz, Bearden, & Cannon, Reference Niendam, Horwitz, Bearden and Cannon2007; Rabin et al., Reference Rabin, Roth, Isquith, Wishart, Nutter-Upham, Pare and Saykin2006) and children (Anderson, Anderson et al., Reference Anderson, Anderson, Northam, Jacobs and Mikiewicz2002; MacAllister et al., Reference MacAllister, Bender, Whitman, Welsh, Keller, Granader and Sherman2012; McAuley, Chen, Goos, Schachar, & Crosbie, Reference McAuley, Chen, Goos, Schachar and Crosbie2010; Teunisse et al., Reference Teunisse, Roelofs, Verhoeven, Cuppen, Mol and Berger2012), although there are some exceptions to this pattern in studies applying the BRIEF-A (Garcia-Molina, Tormos, Bernabeu, Junque, & Roig-Rovira, Reference Garcia-Molina, Tormos, Bernabeu, Junque and Roig-Rovira2012; Sølsnes et al., Reference Sølsnes, Skranes, Brubakk and Lohaugen2014). Also, in the present ADHD study group, moderate negative correlations have been found between a subset of MI subscales and reaction time variability, omission-, and commission errors (Grane et al., Reference Grane, Endestad, Pinto and Solbakk2014).
Any measure that increases the ability of neuropsychological assessments to capture symptoms of ED is welcomed by clinicians and probably contributes to the eagerness to implement measures such as the BRIEF-A. However, in assessing self-reported everyday EF functioning rather than cognitive functioning at an impairment level, there is a risk that other factors than ED, such as emotional symptom load, affect the scores. Highly interesting findings in this regard are described in a sample with mild TBI (Donders, Oh, & Gable, Reference Donders, Oh and Gable2015), where not neuropsychological functioning, but having a history of out-patient psychiatric treatment was associated with higher symptom reports on the BRIEF-A, particularly related to emotional dysregulation (Donders & Strong, Reference Donders and Strong2016).
Likewise, Hanssen, Beiske, Landrø, and Hessen (Reference Hanssen, Beiske, Landro and Hessen2014) found that, while level of neurological impairment predicted neuropsychological test measures of EF in patients with multiple sclerosis, depression was the strongest predictor of subjective complaints of ED on the BRIEF-A. A similar concern that the child version of the BRIEF might tap into general behavioral disruption and impairment, and thus not be a specific measure of executive function, has been raised (McAuley et al., Reference McAuley, Chen, Goos, Schachar and Crosbie2010). In the neurological samples of the current study, there were significant intercorrelations between self-reported emotional and dysexecutive symptoms, even when explored in relation to informant reports.
Surprisingly, the correlation was weaker or non-existent in the neuropsychiatric groups, particularly the BPD and BD-II groups, despite the higher symptom loads on both the BRIEF-A and SCL-90-R. One possible explanation is that the causes of emotional distress vary with disorder etiology. While emotional distress in the neurological groups may largely represent a secondary reaction to the injury, illness or deficit they are experiencing, emotional distress is a primary aspect of both BPD and BD-II. Corroborating this interpretation is the fact that the neuropsychiatric samples performed better on neuropsychological measures of EF than the neurological groups. Thus, one might expect higher correlations in situations where the emotional distress is a result of or closely connected to the underlying disorder causing ED.
Cultural Issues in Self-Reported Executive Functioning
The BRIEF-A is theoretically based and was established after the child version. There is still a limited evidence-base regarding its clinical utility in various patient groups. A recent survey among Norwegian neuropsychologists indicates that the BRIEF-A has become very popular, with 43% using the BRIEF-A on a regular basis (unpublished data in preparation). Local normative data do not exist, rendering the U.S. validation data the only available normative data. As the BRIEF-A asks whether a behavior in question is considered a problem by the patient and/or informant, there is a strong normative aspect imbedded, with perceived cultural expectations forming the baseline for self-evaluation.
To our knowledge, no other studies with non-U.S. samples have presented data on a healthy comparison group comparable in size to the current study. The tendency toward lower scores has potential implications for clinical use of the questionnaire. BRIEF-A scores should be interpreted with caution, and a T-score of 65 should not be considered a necessary threshold to consider symptoms to be of clinical interest, as patient scores of, for example, 56–64 might be in an actual clinical range. Using a clinical cutoff at 1.5 SD above the mean of the healthy comparison sample vastly increased the number of individuals defined as above cutoff.
Few published studies of the BRIEF-A include comparison groups. One exception is a Spanish study of adults with a history of ADHD, where the normally developing comparison group displayed BRIEF-A scores at or above the expected T=50 (Miranda et al., Reference Miranda, Mercader, Fernandez and Colomer2013). A recent Norwegian study found somewhat higher mean values in their healthy comparison group (Finnanger et al., Reference Finnanger, Olsen, Skandsen, Lydersen, Vik, Evensen and Indredavik2015). The current results should also be interpreted with caution, as the HCG was not recruited to a normative study, but to studies with other aims. We cannot rule out the possibility of these comparison groups being biased toward reporting low symptom load. The fact that the comparison group had an average IQ approximately 2/3 SD above the normative mean might point in this direction. On the other hand, other Norwegian studies have described the same tendency toward lower than T=50 mean BRIEF-A scores (Hovik et al., Reference Hovik, Egeland, Isquith, Gioia, Skogli, Andersen and Oie2014; Sølsnes et al., Reference Sølsnes, Skranes, Brubakk and Lohaugen2014).
Study Limitations
One strength of the current study is that it provides data on the BRIEF-A in a broad array of clinical populations, and to our knowledge with the largest non-U.S. healthy comparison group thus far. However, since data were collapsed across studies with different initial research questions, the number of common measures was limited. The limited sample sizes in several of the groups warrant caution of interpretation and generalization. We note, however, that a clear pattern emerged across these individual studies, with neurological and neuropsychiatric samples clustering together into two distinct overarching groups. It should, however, be noted that due to the subsamples being collected at different study sites and with differing scientific aims, this might have introduced sampling biases affecting representativeness of the patient groups. For example, the BPD group is more skewed in gender distribution than expected from known epidemiology, although it is common that more females than males seek treatment (Lieb et al., Reference Lieb, Zanarini, Schmahl, Linehan and Bohus2004). Finally, due to the cross-sectional nature of the study design, causal inferences cannot be made.
Summary
This study confirms the complexity of assessment of EF, and the challenges associated with establishing measures with high ecological validity. While the use of questionnaires such as the BRIEF-A clearly represents valuable additions to performance-based cognitive tests of EF, these findings call for interpretive caution in the face of elevated BRIEF-A scores, as psychological distress seems to covary with the subjective report of EF, particularly in neurological samples. The findings suggest that the BRIEF-A might not be a pure measure of everyday executive functioning, but also reflects general psychological adjustment. It has been highlighted that the children’s BRIEF is not intended as a diagnostic tool, and should represent one component of broad assessment strategies (Gioia et al., Reference Gioia, Isquith, Kenworthy and Barton2002). The current study shows that the same holds true for the adult version.
Additionally, uncertainty about the feasibility of existing normative data in non-U.S. countries adds to the need for interpretive caution when using the BRIEF-A. This study points to the need to build more knowledge about cross-cultural variability in symptom endorsement, and ideally, establishment of non-U.S. normative data.
Acknowledgments
We thank all participants and collaborators. The authors report no conflicts of interest. In addition to the author affiliations, the substudies have been funded by: sTBI: The Research Council of Norway (grant 185267). PFC: The South-Eastern Norway Regional Health Authority (grants SUN-001-SS and 2008047) and The Research Council of Norway (grant 186504/V50). ADHD: The Northern Norway Regional Health Authority (grant 5154/PEP1012-11). CC: The South-Eastern Norway Regional Health Authority (grant 3b-122). PD: The South-Eastern Norway Regional Health Authority (grant 2010053).BP-II/BPD: The South-Eastern Norway Regional Health Authority (Norwegian Research Network On Mood Disorders).
Supplementary Material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S135561771600031X
