Introduction
Problem domains of psychopathology are highly correlated, both concurrently and over time (Eaton et al. Reference Eaton, Rodriguez-Seijas, Carragher and Krueger2015). Consequently, studies have put forward that the structure of psychopathology is best captured by a bi-factor model of general psychopathology on the one hand and specific problem domains on the other (Lahey et al. Reference Lahey, Applegate, Hakes, Zald, Hariri and Rathouz2012; Caspi et al. Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013; Laceulle et al. Reference Laceulle, Vollebergh and Ormel2015; Noordhof et al. Reference Noordhof, Krueger, Ormel, Oldehinkel and Hartman2015). These specific problem domains include internalizing (INT), and externalizing (EXT) (Lahey et al. Reference Lahey, Applegate, Hakes, Zald, Hariri and Rathouz2012; Caspi et al. Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013; Laceulle et al. Reference Laceulle, Vollebergh and Ormel2015), as well as attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) problem domains (Noordhof et al. Reference Noordhof, Krueger, Ormel, Oldehinkel and Hartman2015). A bi-factor model parts the variance in two, separating what is common across different psychopathology domains from what is unique for each domain. Capturing psychopathology by such a bi-factor model may shed light on hitherto unclear associations with external variables, such as impairments in executive function (EF).
EF is crucial for our daily functioning in guiding effortful and goal-directed behavior (Diamond, Reference Diamond2013). EF includes a broad range of cognitive processes, such as suppressing automatic responses, switching between task sets, maintaining or updating information for a short period of time, maintaining attention over a longer period of time, and responding to feedback (Diamond, Reference Diamond2013). Similar discussions of overlap and uniqueness as in psychopathology play a role in EF. Different theoretical models of EF all share the idea that the structure of EF includes both general and specific parts (Duncan et al. Reference Duncan, Emslie, Williams, Johnson and Freer1996; Miyake et al. Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000; Baddeley, Reference Baddeley2012). That is, EFs have something in common, but are also separable from one another (Miyake et al. Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000). A recent study showed that the structure of EF may also be captured by a bi-factor model of general EF, together with separable specific EF components (Friedman et al. Reference Friedman, Miyake, Young, Defries, Corley and Hewitt2008). Neuroimaging studies have likewise shown that different EFs share common neural substrates in the frontal and parietal regions of the brain, but are also associated with separable regions elsewhere (Collette et al. Reference Collette, Hogge, Salmon and Van der Linden2006; Niendam et al. Reference Niendam, Laird, Ray, Dean, Glahn and Carter2012).
Impairments in EFs are widely accepted as a common characteristic of a range of psychiatric disorders (Millan et al. Reference Millan, Agid, Brüne, Bullmore, Carter, Clayton, Connor, Davis, Deakin, DeRubeis, Dubois, Geyer, Goodwin, Gorwood, Jay, Joëls, Mansuy, Meyer-Lindenberg, Murphy, Rolls, Saletu, Spedding, Sweeney, Whittington and Young2012; Snyder et al. Reference Snyder, Miyake and Hankin2015). The focus has long been on finding distinct cognitive profiles for these disorders, which has accumulated in many inconsistent findings. The literature has thus not converged on which EFs are impaired in each disorder. What does emerge from the literature is that EF dysfunctions are generally widespread, severe, and pronounced in disorders that are (often) severe and chronic, such as schizophrenia (Stefanopoulou et al. Reference Stefanopoulou, Manoharan, Landau, Geddes, Goodwin and Frangou2009). In contrast, null findings have especially been reported in younger, and therefore, still less chronically affected, populations with (generally) mild to moderately severe disorders such as depression or anxiety (Baune et al. Reference Baune, Fuhr, Air and Hering2014; Vilgis et al. Reference Vilgis, Silk and Vance2015). Studied in the context of a longitudinal bi-factor model of psychopathology (modeled from INT, EXT, and thought problems assessed multiple times over 20 years), Caspi et al. (Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013) showed that poorer performance on several EF tasks was associated with the general psychopathology factor (or p factor; capturing the chronic transdiagnostic severity of problems) but not with the specific INT and EXT factors. Castellanos-Ryan et al. (Reference Castellanos-Ryan, Brière, O'Leary-Barrett, Banaschewski, Bokde, Bromberg, Büchel, Flor, Frouin, Gallinat, Garavan, Martinot, Nees, Paus, Pausova, Rietschel, Smolka, Robbins, Whelan, Schumann and Conrod2016) and Martel et al. (Reference Martel, Pan, Hoffmann, Gadelha, do Rosário, Mari, Manfro, Miguel, Paus, Bressan, Rohde and Salum2017) also showed that poorer EF was associated with a general psychopathology factor and several other studies showed that fluid intelligence was associated with a general psychopathology factor (Lahey et al. Reference Lahey, Rathouz, Keenan, Stepp, Loeber and Hipwell2015; Neumann et al. Reference Neumann, Pappa, Lahey, Verhulst, Medina-Gomez, Jaddoe, Bakermans-Kranenburg, Moffitt, van IJzendoorn and Tiemeier2016). Moreover, EFs seem more severely impaired when multiple conditions are concurrently present (Brunnekreef et al. Reference Brunnekreef, de Sonneville, Althaus, Minderaa, Oldehinkel, Verhulst and Ormel2007) or develop over time (Roy et al. Reference Roy, Oldehinkel and Hartman2017). Together, these findings may suggest that EF impairments are more strongly associated with severity and chronicity of psychiatric problems than with distinct diagnoses. This could explain discrepancies found in the literature, where the focus has been on distinct disorders.
At the same time, one could ask whether impairments in general EF or in specific EFs are related to psychopathology. Such direct comparisons have not been previously made, as studies have either examined EF performance on lumped or, more frequently, on separate neuropsychological tasks. Some studies using separate tasks showed uniform impairments across tasks in relation to psychopathology (Caspi et al. Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013). Such uniform impairments may indicate that EF is not process specific when associated with psychopathology. This is also suggested by other studies that have lumped scores on EF tasks (e.g. through sum scores or factor scores) (Stordal et al. Reference Stordal, Mykletun, Asbjørnsen, Egeland, Landrø, Roness, Rund, Sundet, Lundervold and Lund2005; McGrath et al. Reference McGrath, Braaten, Doty, Willoughby, Wilson, O'Donnell, Colvin, Ditmars, Blais, Hill, Metzger, Perlis, Willcutt, Smoller, Waldman, Faraone, Seidman and Doyle2015). For instance, low sum scores in EF were strongly associated with high scores on psychopathology (Stordal et al. Reference Stordal, Mykletun, Asbjørnsen, Egeland, Landrø, Roness, Rund, Sundet, Lundervold and Lund2005). Another study that modeled a latent EF factor based on multiple tasks also showed a substantial association with psychopathology (McGrath et al. Reference McGrath, Braaten, Doty, Willoughby, Wilson, O'Donnell, Colvin, Ditmars, Blais, Hill, Metzger, Perlis, Willcutt, Smoller, Waldman, Faraone, Seidman and Doyle2015). Although these studies are suggestive of an association at a general EF level, aggregation through sum or factor scores still captures the scores on individual tasks. Given this overlap, no firm conclusion on associations of general EF or specific EFs with psychopathology can be made. In contrast, as mentioned before, a bi-factor model splits the variance into general and separable specific parts. Thus, bi-factor models allow for a clear-cut interpretation of the extent to which the associations between psychopathology and EF can be summarized as generic (i.e. between general EF and psychopathology) or more separable (i.e. between specific EFs and psychopathology). This may bring us further in understanding how EF is connected to psychopathology.
To summarize, it is generally agreed upon that multiple impairments in EF are involved in a wide range of psychiatric disorders. However, extensive research has not converged on distinct EF profiles for distinct disorders. In the current study, we investigate whether EF problems relate to severity and chronicity of psychopathology rather than type. Concurrently, we address the question whether general or specific EF impairments are associated with psychopathology. This will be accomplished by, on the one hand, modeling psychopathology as measured over multiple occasions during the course of adolescence in a bi-factor model. We extend Caspi et al.’s (Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013) study by not only including INT, EXT, and thought problems but also ADHD and ASD problems, as these are commonly characterized by EF problems. On the other hand, we model EF as measured at two occasions during the course of adolescence in a bi-factor model of general and specific EF, unlike previous studies that used inconclusive sum scores or factor scores of EF to study the relation with psychopathology. Through this double bi-factor approach, the present study aims to understand the relationship between psychopathology and EF. We hypothesize that the association between EF and psychopathology is generic. That is, EF impairments are only associated with general psychopathology, and that this concerns impairments in general EF rather than in specific EFs.
Method
Sample
Participants were part of the Tracking Adolescents’ Individual Lives Survey (TRAILS; Oldehinkel et al. Reference Oldehinkel, Rosmalen, Buitelaar, Hoek, Ormel, Raven, Reijneveld, Veenstra, Verhulst, Vollebergh and Hartman2015). TRAILS is a large prospective cohort study following 2230 adolescents [response rate 76.0%, mean age = 11.1 (s.d. = 0.6 years)] from urban and rural areas in the Northern Netherlands every 2–3 years. The study was approved by the Dutch National Ethical Committee and written informed consent was obtained from both the parents and adolescents. A detailed description of the study is given elsewhere (de Winter et al. Reference de Winter, Oldehinkel, Veenstra, Brunnekreef, Verhulst and Ormel2005; Oldehinkel et al. Reference Oldehinkel, Rosmalen, Buitelaar, Hoek, Ormel, Raven, Reijneveld, Veenstra, Verhulst, Vollebergh and Hartman2015).
For our study, we used data from the first, second, third, and fourth waves. Follow-up response rates were 96.4% [mean age = 13.6 (s.d. = 0.5)] at the second wave, 81.4% [mean age = 16.3 (s.d. = 0.7)] at the third wave and 84.4% [mean age = 19.1 (s.d. = 0.6)] at the fourth wave. Participants were more likely to drop-out at any of the follow-up measurements if they were male, had a non-western ethnicity, divorced parents, low socio-economic status, low IQ and academic achievement, poor physical health, or with behavior and substance use problems (Nederhof et al. Reference Nederhof, Jörg, Raven, Veenstra, Verhulst, Ormel and Oldehinkel2012).
Measures
Child Behavior Checklist
Various mental problems were assessed with the Child Behavior Checklist (CBCL; Dutch version; Verhulst et al. Reference Verhulst, Ende and Koot1996) at the first, second, and third waves. The CBCL is a parent-rated questionnaire with items scored on a three-point scale [0 (not), 1 (sometimes), or 2 (very often)]. The current study used the subscales anxious-depressed (i.e. ANX), somatic complaints (i.e. SC), aggressive behavior (i.e. AGG), rule-breaking behavior (i.e. DEL), attention problems (i.e. AP), and thought problems (i.e. TP). Anxious-depressed and somatic complaints can be subsumed under the INT problem domain, while aggressive and rule-breaking behavior can be subsumed under the EXT problem domain. Attention problems and thought problems remain separate from these problem domains (Achenbach, Reference Achenbach1966).
Child Social Behavior Questionnaire
ASD problems were assessed with the parent-rated Child Social Behavior Questionnaire (CSBQ; Dutch version) at the first, second, third, and fourth waves (Hartman et al. Reference Hartman, Luteijn, Serra and Minderaa2006, Reference Hartman, Moorlag, de Bildt and Minderaa2007). Items are also scored on a three-point scale. The CSBQ captures six symptom dimensions that are typically seen in children with ASD (Hartman et al. Reference Hartman, Luteijn, Serra and Minderaa2006). These are: reduced contact and social interests (i.e. CON), difficulties in understanding social information (i.e. SNA), stereotyped behavior (i.e. STE), fear and resistance to change (i.e. ANG), behavior/emotions not optimally tuned to the situation (i.e. AFS), orientation problems in time, place, or activity (i.e. ORI). The first four symptom dimensions are core autistic problems; the latter two dimensions, although characteristic of ASD, are outside the formal DSM criteria of ASD and are also seen in children with aggression and ADHD, respectively (Hartman et al. Reference Hartman, Luteijn, Serra and Minderaa2006; Noordhof et al. Reference Noordhof, Krueger, Ormel, Oldehinkel and Hartman2015).
Executive functioning
EFs were assessed at the first and fourth waves using computerized tasks from the Amsterdam Neuropsychological Tasks program (ANT; de Sonneville, Reference de Sonneville2003, Reference de Sonneville2005). The following tasks were administered, with mean reaction time or the within-subject standard deviation of the mean reaction time (for sustained attention only) as the output measure: (1) Baseline Speed task, measuring psychomotor speed; (2) Feature Identification task, measuring controlled visuospatial pattern recognition (i.e. pattern search); (3) Sustained Attentional Dots task, measuring sustained attention and responsiveness to feedback on errors; (4) Memory Search Letters task, measuring working memory maintenance (i.e. working memory); (5) Shifting Attentional Set task, measuring cognitive flexibility and response inhibition. An extensive description is given in Appendix A. High reaction times or within-subject standard deviations indicate poor EF function.
Statistical analyses
Structural equation modeling was performed with Mplus version 7.3 using maximum likelihood estimation with robust standard errors. Specifically, two measurement models were developed with confirmatory factor analysis. The first was a psychopathology bi-factor model with the symptom dimensions at the first level, and at the second level the INT (somatic complaints, anxious-depressed), EXT (delinquent behavior, aggressive behavior, behavior/emotions not tuned), ADHD (difficulties understanding, orientation problems, attention problems), and ASD domains [behavior/emotions not tuned, reduced contact, orientation problems, difficulties understanding, stereotyped behavior, fear and resistance to change (Noordhof et al. Reference Noordhof, Krueger, Ormel, Oldehinkel and Hartman2015)], alongside the p factor [which also captures the thought problems, similar to Laceulle et al.’s model (Reference Laceulle, Vollebergh and Ormel2015); Fig. 1]. The second was an EF bi-factor model with specific EF factors (psychomotor speed, pattern search, sustained attention, feedback responsiveness, working memory maintenance, response inhibition and cognitive flexibility), and a general EF factor (Fig. 2). In all models, we included an additional wave factor to partial out time of assessment-specific variance.
Bi-factor psychopathology measurement model.
Bi-factor EF measurement model.
To answer our research questions, estimates of both measurement models were fixed and the associations between EF and psychopathology problem domains were estimated in two steps. A stepwise approach is necessary because estimation of an unrestricted model allowing associations of both general EF and all specific EFs with the p factor and all specific problem domains does not converge to a solution. In step 1, we assessed the associations between a general EF factor and the psychopathology problem domains with the following structural models:
1a. General EF factor with p factor, INT, EXT, ADHD, and ASD.
1b. General EF factor with INT, EXT, ADHD, and ASD.
1c. General EF factor with p factor.
In step 2, we assessed the associations between specific EFs and the psychopathology problem domains, similar to step 1:
2a. Specific EFs with p factor, INT, EXT, ADHD, and ASD.
2b. Specific EFs with INT, EXT, ADHD, and ASD.
2c. Specific EFs with p factor.
Within each step, we chose the model with the best fit to the data from the respective model a, b, or c (models b and c are each nested in model a). Subsequently, each best-fitting model was made more parsimonious by fixing the non-significant correlations to zero, while checking if the more parsimonious model significantly deteriorated the model fit. Finally, we compared the optimal models from steps 1 and 2 and chose the model with the best fit as our final model.
Model fit was evaluated using the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). A CFI >0.95 indicates a good fit, as well as RMSEA and SRMR scores <0.05. The threshold values for acceptable model fit are 0.90 for the CFI, 0.08 for the RMSEA, and 0.06 for the SRMR (Bollen & Curran, Reference Bollen and Curran2006). Models were compared using the Akaike information criterion (AIC) in the case of non-nested models, while nested models were compared with the Satorra–Bentler χ2 difference test. Estimates of the associations between EF and psychopathology were corrected for multiple testing by using a two-stage adaptive procedure to control the false discovery rate (FDR), resulting in FDR adjusted p values with a maximum acceptable FDR of 5% (α FDR-corrected = 0.05; Benjamini et al. Reference Benjamini, Krieger and Yekutieli2006).
Results
Measurement models
Model fit indices for the bi-factor psychopathology model show that the fit was adequate [χ2(754) = 2678.59, CFI = 0.95, RMSEA = 0.03, SRMR = 0.046]. Model fit indices for the bi-factor EF model show an adequate fit as well [χ2(63) = 411.98, CFI = 0.92, RMSEA = 0.05, SRMR = 0.041]. Factor loadings of both models can be found in Tables B1 and B2 in Appendix B.
Relationship between executive functioning and psychopathology
Model fit indices for the models that describe the relationship between a general EF factor and psychopathology problem domains (i.e. step 1) and for the models that describe the relationship between specific EFs and psychopathology problem domains (i.e. step 2) are found in Tables C1 and C2 in Appendix C. In step 1, the best fit was found for model 1a. The model fit barely deteriorated after making it more parsimonious by constraining the non-significant estimates of the associations between EF and psychopathology to zero (Table C3 in Appendix C). The remaining associations were between the general EF factor and ADHD, ASD, and the p factor, while general EF showed no relations with INT and EXT (i.e. model 1). In step 2, the best fit was found for model 2a (model 2b did not converge to a solution, also indicating that it misrepresents the associations present in the data). After making model 2a more parsimonious, by constraining the non-significant associations while keeping similar goodness-of-fit (Table C4 in Appendix C), the remaining associations were between specific EFs and the p factor, INT, EXT, ADHD, and ASD (i.e. model 2). Finally, comparing models 1 and 2 revealed that the best-fitting model was model 2 (Table 1). Parameter estimates for model 2 are shown in Table 2 and graphically displayed in Fig. 3; those for model 1 are shown in Table C5 in Appendix C.
Model 2: specific EFs with psychopathology.
Model fit indices: optimal models 1 and 2
Correlations model 2: specific EFs with psychopathology
*p uncorrected < 0.05, **p FDR-corrected < 0.05; estimated Pearson correlations.
Inspection of Table 2 (discussed in more detail below) showed that all specific EFs were associated with the p factor. This uniform pattern of results may indicate that further model simplification without loss in model fit is possible (i.e. model 3; Tables C6 and C7 in Appendix C). That is, we fitted a more parsimonious model which specified an association between general EF (thus capturing the consistent association found for all specific EFs) and the p factor (part of model 1), combined with the associations of specific EFs with INT, EXT, ASD, and ADHD (part of model 2). Model fit did not change (as shown by the RMSEA, SRMR, and CFI) and the fit was worse compared to model 2 (as shown by the AIC). Therefore, the best-fitting and final model was indeed model 2, but with minimal differences from model 3.
As mentioned before and shown in Table 2, model 2 revealed that the p factor was associated with reduced performance on all specific EFs (r between 0.14 and 0.39). The association with cognitive flexibility was no longer significant following FDR correction (p FDR-corrected = 0.052).
The psychopathology problem domains showed patterns of associations in addition to the associations already captured by the p factor. The ADHD problem domain also showed a rather consistent widespread pattern of EF problems (r between 0.14 and 0.40). The ASD problem domain, on the other hand, showed a slightly more specific pattern. ASD was primarily associated with decreases in psychomotor speed, sustained attention, feedback responsiveness, and cognitive flexibility (r between 0.09 and 0.19). The association between psychomotor speed and ASD was non-significant after FDR correction (p FDR-corrected = 0.051). Furthermore, the INT and EXT problem domains each showed one specific association. INT was only associated with decreases in cognitive flexibility (r = 0.30), while EXT was only associated with increases in working memory maintenance (r = −0.10).
Sensitivity analyses
INT with self-report
From middle adolescence on, self-report is considered a better indicator of INT problems (Smith, Reference Smith2007). Our data capture both pre- and early adolescence and middle and late adolescence. Therefore, in a sensitivity analysis, we changed the informant to test the robustness of the association between INT and cognitive flexibility. INT problems were indicated by self-report measures using the Youth Self-report [YSR; and the Adult Self-report (ASR) at wave 4], where items are similar to the CBCL. The specific association with cognitive flexibility remained, showing that it is not dependent on the parent as the informant (r = 0.24, p FDR-corrected < 0.001).
EXT with self-report
It is unclear if self-report or parent report is a better indicator of EXT problems (Smith, Reference Smith2007). Therefore, we also decided to test the robustness of the association between EXT and working memory maintenance, with EXT problems indicated by self-report measures using the YSR (and ASR at wave 4). EXT was not significantly associated with working memory maintenance (r = −0.05, p FDR-corrected = 0.19), which indicates that the association was not robust.
Discussion
We modeled both psychopathology and EF as bi-factor models to examine whether EF impairments are transdiagnostic or relate to individual syndromes and concurrently, whether such associations are with general EF or in specific EF impairments. With this double bi-factor approach, the best model showed that impairments in multiple specific EFs are associated with general psychopathology, and are above and beyond this generic association also present in specific problem domains. This is especially true for ADHD and ASD problems. Furthermore, INT problems have a distinct association with cognitive flexibility. Of note is that this conclusion is based on a model that fitted only slightly better than model 3, in which the general EF factor accounted for the associations with general psychopathology, alongside the same pattern of associations between specific EFs and specific problem domains of psychopathology as found for our best model 2. It follows that we can draw stronger conclusions with regard to the findings from the ‘psychopathology side’ than the ‘EF side’ of the double bi-factor model. Thus, we conclude, firstly that the inconsistent findings in the literature may be explained by substantial transdiagnostic EF impairments. Secondly, once these transdiagnostic impairments are captured, ADHD, ASD, and INT problems still have their specific EF profile. Thirdly, whether general EF or specific EFs are related to the p-factor needs to be further studied, as differences in fit between these two models were small.
We extended the research by Caspi et al. (Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013) by including ADHD and ASD problems. Consistent with their findings, we show substantial associations with the p factor. Note that this important finding relates to a discussion on how the p factor should be interpreted (Caspi et al. Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013; Laceulle et al. Reference Laceulle, Vollebergh and Ormel2015; Noordhof et al. Reference Noordhof, Krueger, Ormel, Oldehinkel and Hartman2015). This factor captures all the shared variance between different problem domains. Therefore, it is often considered to reflect severity and chronicity of psychopathology, particularly when based on multiple measures across time as in our study. However, other explanations have been suggested as well (Lahey et al. Reference Lahey, Applegate, Hakes, Zald, Hariri and Rathouz2012; Noordhof et al. Reference Noordhof, Krueger, Ormel, Oldehinkel and Hartman2015). In particular, the p factor may capture response tendencies of informants (i.e. shared method variance). However, the widespread and substantial associations between specific EFs and the p factor would not be found if this factor merely represented response tendencies. As such, our findings are consistent with the interpretation of the p factor reflecting cross-domain severity and chronicity.
INT and EXT problems alone do not show much impairment in EF beyond severity and chronicity captured by the p factor. Although these findings are mostly comparable to findings by Caspi et al. (Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013) who found no associations, we did find a robust association between cognitive flexibility and the INT problem domain. Differences between the two studies could be due to cognitive flexibility being measured with different tasks. Caspi et al. (Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2013) used the Trail Making Test part B, which requires continuously alternating between two response sets in a particular order, making it fairly predictable (e.g. with numbers and letters: 1, A, 2, B, etc.). We used the shifting attention set (visual) task from the ANT, which is unpredictable, because it requires alternation between two response sets that cannot be anticipated. Presumably, this task is more demanding and therefore more sensitive to detecting cognitive flexibility problems. In addition, our design allowed us to model a latent trait of cognitive flexibility based on two EF assessments over time. This stable estimate of cognitive flexibility, separated from random error and state-specific variance, may yield stronger associations. The presence of an association between cognitive flexibility and the INT problem domain above and beyond severity and chronicity of psychopathology suggests that cognitive inflexibility may be a core cognitive deficit of INT problems. Surely, it is a highly plausible association when considering a connection between cognitive flexibility and rumination, a key component of INT disorders (Yang et al. Reference Yang, Cao, Shields, Teng and Liu2017). That is, individuals with impaired cognitive flexibility may be less able to ‘reset’ their minds following daily hassles or disappointments. Subsequently, they fixate and dwell on their thoughts (i.e. ruminate) and this inability to shift set further evokes feelings of worry and sadness.
The ASD and ADHD problem domains also show associations with specific EFs above and beyond the associations with severity and chronicity of psychopathology. Both show impairments in visuospatial working memory, sustained attention, feedback responsiveness, and cognitive flexibility, while impairments in psychomotor speed and working memory maintenance were specific to ADHD problems. Consistent with the literature, the EF deficits are more widespread relative to INT and EXT problems (Hill, Reference Hill2004a, Reference Hillb; Willcutt et al. Reference Willcutt, Doyle, Nigg, Faraone and Pennington2005; Doyle, Reference Doyle2006; Baune et al. Reference Baune, Fuhr, Air and Hering2014; Vilgis et al. Reference Vilgis, Silk and Vance2015), and heterogeneous (Sergeant et al. Reference Sergeant, Geurts and Oosterlaan2002; Nigg et al. Reference Nigg, Willcutt, Doyle and Sonuga-Barke2005; Martel et al. Reference Martel, Roberts, Gremillion, von Eye and Nigg2011). A relatively novel finding is that although our adolescent general population sample actually represents the absent-to-very mild end of severity in ASD problems (given the prevalence of 1%; Elsabbagh et al. Reference Elsabbagh, Divan, Koh, Kim, Kauchali, Marcín, Montiel-Nava, Patel, Paula, Wang, Yasamy and Fombonne2012), we demonstrate that EF impairments are present. Of further note is that, although ASD is often considered a more severe neurodevelopmental disorder than ADHD, we found that the EF impairments are somewhat more widespread and stronger in ADHD problems. The relative severity of ASD may not necessarily be demonstrated by more severe EF deficits but rather by the wider range of cognitive problems that go beyond EF, most notably in the socio-cognitive domain.
One of the biggest strengths of this study is the use of neuropsychological data in an epidemiological context. We included a large sample with psychopathology assessed at four time points and EF measured at two time points. Importantly, the latter has not often been accomplished and shows that we have fairly unique data that served our research questions well. Moreover, this study included ADHD and ASD problems, lifespan conditions that should not be ignored as specific problem domains when trying to understand the structure of psychopathology, particularly when studied in relation to EF (Hartman et al. Reference Hartman, Geurts, Franke, Buitelaar and Rommelse2016). Finally, we modeled EF with a bi-factor model. By partialling out shared variance between the EF tasks, we minimized measurement error and state-specific variance; and by partitioning the variance in general EF and specific EFs, we were better adept to assess the cognitive processes measured by their respective tasks. Our specific EFs may in part still measure non-EF-related variance, as we were unable to use multiple tasks for each specific cognitive process. Nonetheless, we tackled the task impurity problem, which is an often-mentioned point of criticism in the literature, at least to some extent. In all, our approach seemed to have paid off with associations that not only fit well with the literature, but also exceed it, as we show effect sizes that are rarely reported in the literature, especially when regarding general population samples.
The study has several limitations. First, we must consider the dropout between the four waves. Although we were able to contain approximately 80% of our sample in 10 years, which is an outstanding response rate, our sample may not be fully representative of the general population (de Winter et al. Reference de Winter, Oldehinkel, Veenstra, Brunnekreef, Verhulst and Ormel2005; Nederhof et al. Reference Nederhof, Jörg, Raven, Veenstra, Verhulst, Ormel and Oldehinkel2012; Oldehinkel et al. Reference Oldehinkel, Rosmalen, Buitelaar, Hoek, Ormel, Raven, Reijneveld, Veenstra, Verhulst, Vollebergh and Hartman2015). Second, we have a limited set of EF measures that assess fairly specific EF dimensions, rather than the more complex processes or higher order EFs, which are more closely related to behavior in everyday life (e.g. planning). A related limitation is that the use of EF measures differs greatly between studies; our study is only one way of measuring different EFs. This emphasizes the need for future studies to determine if our conclusions hold up in samples using different EF tasks. Moreover, although there is no literal overlap in items and constructs, the fact that items tend to not be fully specific for the construct they measure does limit the specificity of the psychopathology domains. This is a limitation that is unavoidable in these types of studies. Finally, the use of parent-reported questionnaires may have different properties for INT and EXT problems, although our sensitivity analyses did show similar associations between EF and the INT and EXT problem domains based on self-report. Nonetheless, the results may still depend to some extent on the precise measures, as well as the choice of rater.
By studying very diverse psychopathology domains simultaneously, we have shown that EF problems cross diagnostic boundaries. In addition, EF problems play a domain-specific role in ADHD, ASD, and INT symptomatology. Whether both general EF and specific EFs are related to psychopathology needs to be further studied. We conclude that the association between psychopathology and EF cannot simply be considered either generic or specific and examining both general and specific components of EF and psychopathology enables clearer conclusions on distinct EF profiles for distinct disorders.
Supplementary Material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291717003269.
Acknowledgements
This research is part of the TRacking Adolescents’ Individual Lives Survey (TRAILS). Participating centers of TRAILS include various departments of the University Medical Center and University of Groningen, the Erasmus University Medical Center Rotterdam, the University of Utrecht, the Radboud Medical Center Nijmegen, and the Parnassia Bavo group, all in the Netherlands. The authors are grateful to everyone who participated in this research or worked on this project to make it possible.
Financial support
TRAILS has been financially supported by various grants to J.O. and A.J.O from the Netherlands Organization for Scientific Research (NWO) (Medical Research Council program grant no. GB-MW 940-38-011; ZonMW Brainpower grant no. 100-001-004; ZonMw Risk Behavior and Dependence grant no. 60-60600-97-118; ZonMw Culture and Health grant no. 261-98-710; Social Sciences Council medium-sized investment grants no. GB-MaGW 480-01-006 and GB-MaGW 480-07-001; Social Sciences Council project grants no. GB-MaGW 452-04-314 and GB-MaGW 452-06-004; NWO large-sized investment grant no. 175.010.2003.005; NWO Longitudinal Survey and Panel Funding 481-08-013), the Dutch Ministry of Justice (WODC), the European Science Foundation (EuroSTRESS project FP-006), Biobanking and Biomolecular Resources Research Infrastructure BBMRI-NL (CP 32), and the participating universities.
Conflicts of interest
On behalf of all authors, the corresponding author states there are no conflicts of interest.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
