INTRODUCTION
Historically, conceptual models of neuropsychiatric disorders have emphasized relationships between particular conditions and cognitive decrements that are potentially pathognomonic (e.g., inhibitory control in attention-deficit/hyperactivity disorder [ADHD]; Barkley, Reference Barkley1997). Yet, as Pennington (Reference Pennington2006) and colleagues (Willcutt, Doyle, Nigg, Faraone, & Pennington, Reference Willcutt, Doyle, Nigg, Faraone and Pennington2005) point out, models that include multiple cognitive deficits that overlap are more consistent with growing evidence that neuropsychiatric disorders are complex, multifactorial conditions that share some of their underlying genetic risk (PGC, 2013). The literature to date supports these more complex models. Although not all studies are consistent, meta-analyses that summarize these data implicate weaknesses in general cognitive ability, executive functions (EFs), and reaction time variability (RTV) in a range of conditions (see Table 1 for examples).
Meta-analyses, conceptual reviews, and studies of relatives support the relevance of impairments in general ability, RTV, and EFs to neuropsychiatric conditions known to share genetic underpinnings
Note. Rather than an exhaustive review, Table 1 includes representative studies from the literature and emphasizes meta-analyses in youth. Where such analyses are not available, we include meta-analyses of adults or, alternatively, reviews and individual studies. We also note cells where evidence for cognitive impairment is weak or limited. ADHD=attention-deficit/hyperactivity disorder; ASD=autism spectrum disorder; BPD=bipolar disorder; MDD=major depressive disorder; SCZ=schizophrenia.
a Indicates meta-analysis.
b Indicates twin study.
c Indicates study of first-degree relatives.
d Indicates review.
Despite this evidence, studies that directly compare cognition across multiple psychopathological diagnoses are limited. Although there are some exceptions (e.g., Goldberg et al., Reference Goldberg, Mostofsky, Cutting, Mahone, Astor, Denckla and Landa2005), the bulk of the data supporting cross-disorder cognitive impairments reflects extrapolation from studies examining single conditions versus controls (Willcutt, Sonuga-Barke, Nigg, & Sergeant, Reference Willcutt, Sonuga-Barke, Nigg and Sergeant2008). The few cross-disorder meta-analyses have extended the evidence for domains of common weakness (Lipszyc & Schachar, Reference Lipszyc and Schachar2010; Stefanopoulou et al., Reference Stefanopoulou, Manoharan, Landau, Geddes, Goodwin and Frangou2009; Willcutt et al., Reference Willcutt, Sonuga-Barke, Nigg and Sergeant2008). Yet, as Willcutt et al. (Reference Willcutt, Sonuga-Barke, Nigg and Sergeant2008) have noted, direct within-sample analyses are needed to estimate the relative magnitude of decrements on a common metric and also to clarify whether comorbidity accounts for or exacerbates decrements across conditions.
Gaining a better understanding of the domains of cognitive weaknesses across conventional diagnostic boundaries is important for both clinical and research purposes. In the child clinical assessment field, while neuropsychiatric diagnoses are not made based on scores from psychometric tests, cognitive decrements are often taken into consideration; yet, the degree to which clinicians should expect cognitive weaknesses to contribute to differential diagnosis is not clear. Clarifying the extent of overlapping deficits across conditions will improve the evidence base regarding the implications of particular cognitive weaknesses. In the research literature, there is growing evidence from molecular genetic studies that different forms of neuropsychiatric illness share aspects of their underlying risk, and family and twin studies suggest that cognitive decrements may index liability in at least some conditions (Table 1). Confirming domains of cognition that are compromised across different forms of psychopathology will facilitate the use of cognitive constructs in studies aiming to examine cross-disorder risk mechanisms (Craddock et al., Reference Craddock, Kendler, Neale, Nurnberger, Purcell, Rietschel and Thapar2009). In conjunction with emerging genomic findings, such studies may help to incorporate cognition into a more biologically informed psychiatric nosology, as advocated by the NIMH’s Research Domain Criteria framework (Cuthbert, Reference Cuthbert2015).
The current study aimed to address gaps in the literature by examining cognitive weaknesses and the impact of comorbidity in youth with different neuropsychiatric conditions known to share genetic underpinnings (Lee et al., Reference Lee, Ripke, Neale, Faraone, Purcell, Perlis and Wray2013; Malhotra & Sebat, Reference Malhotra and Sebat2012; PGC, 2013). Specifically, we focused on youth ascertained from a single cohort with ADHD, mood disorders, autism spectrum disorder (ASD), and psychotic symptoms. We predicted decrements in general cognitive ability, EFs, and RTV across multiple conditions because weaknesses in these constructs are implicated in meta-analyses of affected individuals and because family and twin studies suggest their role in underlying disease liability.
METHOD
Subjects
Participants were from the Longitudinal Study of Genetic Influences on Cognition (LOGIC). LOGIC recruits youth referred for evaluation at a pediatric assessment clinic within the Psychiatry Department at Massachusetts General Hospital (MGH). Patients with neuropsychiatric symptomatology are referred to this clinic for cognitive and psychiatric evaluation to assist with differential diagnosis and/or treatment or educational planning. To enroll, youth must contribute their clinical data. They are also asked to provide a DNA sample and to supplement assessments to create a uniform cognitive and psychiatric battery across subjects. Study procedures were in compliance with the Partners Institutional Review Board and the Helsinki Declaration. Parents and youth 18 and older provided written informed consent after a description of risks and benefits; youth 7–17 provided written assent.
Subjects in the current analysis were consecutively enrolled patients meeting the following criteria: (1) Full Scale IQ (FSIQ)≥70; (2) ages 8 to 21 years old (i.e., eligible to be assessed on measures reflecting cognitive domains of interest); and (3) a DSM-IV-TR diagnosis from one of the following categories: ADHD, mood disorders (major depressive disorder, bipolar disorder or mood disorder not otherwise specified [NOS]), ASD (pervasive developmental disorder [PDD] NOS, Asperger’s syndrome, or autistic disorder), or positive symptoms of psychosis (i.e., hallucinations and/or delusions). These four groups were selected because recent large-scale genomic studies indicate that these forms of psychopathology share common genetic variation that contributes to their risk (Lee et al., Reference Lee, Ripke, Neale, Faraone, Purcell, Perlis and Wray2013; Malhotra & Sebat, Reference Malhotra and Sebat2012; PGC, 2013). LOGIC is an ongoing project. At the time of these analyses, there were 486 unrelated youth who met these criteria. Their mean age was 11.8 ± 3.1 years and 34.8% are female.
Cognitive Assessments
Tests were administered using published instructions by licensed psychologists or by advanced trainees or psychometricians under their supervision. Based on the literature (Table 1), we examined the following constructs, using measures with robust psychometric properties that are commonly used in child clinical practice and research:
IQ/general cognitive ability
We assessed cognitive ability using the General Ability Index (GAI) from the Wechsler Intelligence Scale for Children – Fourth Edition (Wechsler, Reference Wechsler2004) for youth 8 to 16 and the Wechsler Adult Intelligence Scale – Fourth Edition (Wechsler, Reference Wechsler2008) for youth 17 to 21. We used GAI because this score estimates ability without the use of processing speed and working memory (WM) tests, which may show relative weaknesses in clinical populations (Prifitera, Weiss, & Saklofske, Reference Prifitera, Weiss and Saklofske1998; Tulsky, Saklofske, Wilkins, & Weiss, Reference Tulsky, Saklofske, Wilkins and Weiss2001). Additionally, WM was examined separately in our analyses.
RTV
RTV represents intra-individual consistency in reaction time. Increased variability is often considered to reflect failures of sustained attention; however, it may additionally reflect the regulation of arousal or executive allocation of attentional resources (Tamm et al., Reference Tamm, Narad, Antonini, O’Brien, Hawk and Epstein2012). Although RTV has been studied extensively in ADHD, it may be relevant to other forms of psychiatric illness (Kaiser et al., Reference Kaiser, Roth, Rentrop, Friederich, Bender and Weisbrod2008; Lipszyc & Schachar, Reference Lipszyc and Schachar2010). Our measure of this construct was obtained from the Conners’ Continuous Performance Test – Second Edition (Conners, Reference Conners2000) based on the Hit Reaction Time Standard Error.
EFs
By definition, EFs support goal-directed behavior and environmental adaptation (Loring, Reference Loring1999). We targeted components of EFs (i.e., WM, inhibition, and shifting/mental flexibility) that overlap with major domains of the overarching EF construct (Miyake & Friedman, Reference Miyake and Friedman2012; RDoC Cognitive Group, 2011). We operationalized WM using the Working Memory Index from the Wechsler Intelligence scales (Wechsler, Reference Wechsler2004, Reference Wechsler2008). We operationalized inhibition using the Commission Errors score on the Conners’ Continuous Performance Test – Second Edition (CPT II; Conners, Reference Conners2000). We examined shifting/mental flexibility using the Switching condition on the Delis-Kaplan Executive Function System (D-KEFS) Trail Making Test (Delis, Kramer, Kaplan, & Holdnack, Reference Delis, Kramer, Kaplan and Holdnack2004).
Diagnoses
DSM-IV-TR Axis I diagnoses were made by or under the supervision of MGH/Harvard Medical School faculty who were licensed clinical psychologists. Our source clinic is a training site for neuropsychiatric assessment for pre- and post-doctoral Clinical Psychology Fellows. Thus, accurate and thorough diagnostic assessment is one of the “deliverables” of the clinic. Diagnostic procedures include: (1) clinical interviews with a parent/ legal guardian and patient, (2) review of available medical records, and (3) review of omnibus and targeted behavioral rating scales (including the Child Behavior Checklist/6–18 or Adult Behavior Scale, and the Child Symptom Inventory-IV, which includes specific DSM-IV-TR criteria).
Diagnoses were made if full DSM-IV-TR criteria were met based on information from these sources. The one exception was that we allowed for a diagnosis of ADHD in the presence of an ASD, as in DSM-5. This strategy was implemented in our clinic prior to the publication of DSM-5 because of the academic/therapeutic implications of high levels of inattention and hyperactivity/impulsivity and our group’s interest in studying comorbidity in child neuropsychiatric conditions. We examined the reliability of the diagnostic process by having four independent licensed clinical psychologists blindly review and rate a subsample of 30 youth per diagnosis. These cases were randomly selected regardless of comorbidity.
Following the guidelines of Landis and Koch (Reference Landis and Koch1977) we interpreted kappa coefficients between .61 and .80 as representing a substantial agreement and kappa coefficients between .81 and 1.00 as indicative of almost perfect agreement. The inter-rater reliability using Cohen’s Kappa was .93 for ADHD, ASD, and mood disorders (95% confidence interval [CI] [.80, 1.06]) indicative of almost perfect agreement, and .80 for the presence or absence of psychosis (95% CI [.59, 1.01]), indicative of substantial agreement. Further corroboration of clinician diagnoses of ADHD from our source clinic occurred in 12 youth who were not part of the current study. These youth received the KSADS-E as part of a separate research project that recruited ADHD cases from our clinic. This structured diagnostic interview confirmed clinician diagnoses of ADHD in 100% of these cases.
Diagnostic characteristics of the sample
Four hundred and eighty-six youth met criteria for one or more of our target diagnoses. As noted, we ascertained n=383 individuals with ADHD. There were n=106 youth with an ASD. As per DSM-IV-TR, 8.5% of these youth had autistic disorder, 51.9% had Asperger’s syndrome, and 39.6% had PDD NOS. There were 157 youth with mood disorders (13.4% with bipolar disorder, 41.4% with major depressive disorder, 4.5% with dysthymia, and 40.8% with mood disorder NOS). Finally, n=29 of the individuals exhibited positive symptoms of psychosis (i.e., hallucinations and/or delusions). Of these, 24% had a diagnosis of schizophrenia and 76% were categorized as psychotic disorder -NOS given the emerging, fluid nature of the symptoms. The breakdown of comorbidity within these four categories is shown in Table 2.
Distribution of 486 individual patients across comorbid disorders conditions
Note. Youth with a single (non-comorbid) diagnosis are represented in the shaded cells on the diagonal. Frequencies above the diagonal in gray are included, despite their redundancy, to allow easier calculation along each vertical column of total numbers of subjects with and without comorbid conditions in the overall cohort.
* Because n=30 youth have>1 comorbid disorder, youth may appear more than one time across this row (e.g., the ADHD, mood disorders, and ASD columns include the same n=16 youth who met criteria for all three conditions). Thus, while column totals at the bottom reflect the total n for each diagnosis, they cannot be added together to reach 486 because they include overlapping subjects.
Other characteristics of the sample
Detailed data (dose, type, onset, offset) regarding use of psychotropic medication was obtained as part of the clinical evaluation. A total of n=154 (31.7%) children were taking stimulants, n=71 (14.6%) were on non-stimulant medication to treat ADHD (e.g., atomoxetine), n=60 (12.3%) were taking an atypical antipsychotic, n=88 (18.1%) were taking a selective serotonin reuptake inhibitor (SSRI), n=32 (6.6%) were taking a non-SSRI antidepressant, n=17 (3.5%) were taking a benzodiazepine, and n=39 (8.0%) were taking another type of psychotropic medication. Totals exceed 100% because some youth were taking more than one type of medication. Based on this information, we created a binary variable to indicate current use of one or more types of psychotropic medications versus non-use. This variable yielded a total of n=273 (56.2%) youth using psychotropic medication.
Analytic Approach
Our goals were to determine the presence and magnitude of cognitive weaknesses across youth in the four target diagnostic groupings and to clarify the impact of comorbidity.
Phase 1: Youth with non-comorbid diagnoses
First, we aimed to focus on patients from each psychopathology group who were free of comorbidity; however, because only three youth with psychosis were free of comorbidity, we included all 29 youth with psychosis in our fourth category to study youth with this relatively rare but severe presentation. Within this group, 45% met criteria for ADHD, 31% met criteria for diagnosis within the ASD category, and 52% meet criteria for a mood disorder (21% major depressive disorder, 14% bipolar disorder, and 17% mood disorder NOS). Rates exceed 100% due to youth with multiple conditions. For all four groups in this phase of inquiry, we operationalized cognitive weaknesses in two ways: first relative to population norms (phase 1a) and then as a discrepancy relative to their own general ability (phase 1b).
Covariates
Table 3 shows demographics for subjects analyzed in phase 1. Full Scale IQ is provided for descriptive purposes because components of this measure (i.e., GAI and WM) are outcome measures. Significant differences in the distribution of sex were found across groups (χ2 (3)=16.41; p = .001), with more boys in the ASD group (85.3%) and more girls in the mood disorders group (58.5%). For age, we found a significant group effect (F(3,353)=23.02; p<.001), with participants with psychosis and mood disorders slightly but significantly older than youth with ASD and ADHD, respectively (ASD vs. psychosis p=.005; ASD vs. mood disorders p<.001; psychosis and mood vs. ADHD both <.001). Finally, significant differences in rates of medication use were found (χ2 (3)=19.67; p<.001), with more youth taking medication in the psychosis (75.9%) and mood disorder (68.3%) groups and fewer in the ASD group (35.3%). We thus controlled for age, sex, and current medication use in subsequent analyses.
Demographic characteristics of participants included in phase 1 analyses
Note. These include youth with non-comorbid ADHD, mood disorders, and ASD, as well as youth with psychotic symptoms regardless of comorbidity.
a Percentage of the total sample of 486 participants
1a. Comparisons between groups and comparison of groups versus normative data
To compare cognition between groups and in relation to published norms, we used analysis of covariance (ANCOVA), controlling for age, sex, and medication use. Comparisons between groups on each of the five cognitive domains (GAI, RTV, and the three EF constructs) allowed us to estimate marginal means (adjusted for potential confounders) that could be used post hoc to compare the performance of youth in each diagnostic group with published norms. We note that the normative samples from our measures are considered generally representative of the U.S. population (Conners, Reference Conners2000; Delis et al., Reference Delis, Kramer, Kaplan and Holdnack2004; Wechsler, Reference Wechsler2004, Reference Wechsler2008) and are large for the ages (8 to 21) overlapping our participants (i.e., WISC-IV n=1600, WAIS-IV n=600, D-KEFS n=1050, and CPT-II n=1632).
1b. Comparisons versus GAI
Second, we used a mixed modeling approach to determine the presence of weaknesses on RTV and EFs compared to participants’ own GAI for each diagnostic grouping. Mixed modeling is an extension of regular regression appropriate when data are hierarchically structured (as in our five cognitive domains nested within a subject). This statistical technique does not require the data to be balanced (i.e., not every subject will have a score on each domain) presuming missing data are random (Snijders & Bosker, Reference Snijders and Bosker2012), and our analyses support this assumption.
We note that the Conners’ CPT was included after the start of our study and thus there are greater numbers of missing data in relation to the RTV and inhibition measures across diagnoses. In the total sample, there were 132 youth (27.2%) that had one or more missing cognitive measures. The group with data on all cognitive measures did not differ from the group with missing data on age (t(484)=1.20; p=.23), FSIQ (t(484)=1.32; p=.19; sex (χ2(1)=1.60; p=.21), a diagnosis of ADHD (χ2(1)=.07; p=.80), or any of the diagnostic groups we examined (χ2(5)=6.79; p=.24). Here, cognitive scores were converted into Z-scores to compare GAI to RTV and EF. Converted scores were entered as a within-subjects variable, referred to as “cognition,” consisting of five cognitive domains. When a significant main effect was found, we ran post hoc comparisons of RTV and the three EF domains to GAI.
Phase 2: Impact of comorbidity
Next, we examined the influence of comorbidity on cognitive profiles. To reduce the possibility of Type II error, we only examined comorbid groups that exceeded n=30 subjects in this set of analyses. As shown in Table 2, only groups with ASD+ADHD (n=35) and mood disorders+ADHD (n=66) met our threshold. We, therefore, ran two mixed modeling analyses, comparing ASD with ASD+ADHD groups and then mood disorders versus mood disorders+ ADHD groups, controlling for confounders.
In each model, we first estimated a “full” model including a diagnostic group by cognition interaction term. A significant interaction would indicate that the shape of the cognitive profile differed between the comorbid and non-comorbid groups. Given a non-significant interaction effect, we dropped the interaction term and tested the resultant model (via Wald χ2) to determine a main effect for diagnostic group and/or a main effect for cognition. A main effect for diagnostic group represents a difference in the magnitude of the decrement between the comorbid and non-comorbid groups. A main effect of cognition reflects within-subject differences between particular cognitive domains and GAI. Like phase 1, post hoc comparisons determined which cognitive domains differed from GAI, but here include subjects regardless of comorbidity.
Analyses were conducted with STATA14 (StataCorp, 2015). We used a significance level of .05 except where we applied a Bonferroni correction (with a significance threshold of .0125) in relation to (1) the post hoc comparisons with the norm scores (.05/4 target groups) and (2) the post hoc mixed models in phase 1b and phase 2 (.05/4 cognitive domains vs. GAI).
RESULTS
Phase 1: Cognitive Decrements in Non-comorbid Diagnostic Groups and Psychosis
Phase 1a: Differences among groups and relative to age-based norms
We examined the extent to which youth from the psychopathology groups differed from one another on the five cognitive domains via an F-test (Table 4). No significant differences were found between groups after controlling for age, sex, and medication use. We then compared means generated from these comparisons (adjusted for potential confounders) to population norms (also Table 4). Figure 1 shows effect sizes (Cohen’s d) from these post hoc comparisons of the estimated marginal means. Consistent with the lack of group differences, effect sizes for the diagnostic groups were close in range, with some of the differences from the normative mean reaching statistical significance. For GAI, slightly but significantly higher than normative performance was noted in youth with ADHD and with mood disorders (p<.001 and p=.01, respectively). For EFs, after correction for multiple testing, the ADHD group showed statistically worse performance in all three domains and the psychosis group showed lower WM (all p-values<.001). For RTV, significantly greater variability (worse performance) occurred in youth with ADHD (p<.001).
Cognitive performance relative to normative data in referred youth with different neuropsychiatric conditions.
Results of phase 1 comparisons, including (1) ANCOVA between-group comparisons (correcting for age, sex, and current medication use) and (2) post-hoc comparisons between estimated marginal means (corrected for covariates) and age-based normative data
Note. Values in table represent adjusted marginal means (SD), (95% CI), and sample size from ANCOVA.
* p ≤ 0.0125 (significant after Bonferroni correction); Low scores indicate worse performance, with the exception of measures designated with **, in which high scores reflect greater difficulty.
Phase 1b: Comparison to GAI
We then performed mixed modeling analyses for each diagnostic group to examine weaknesses in RTV and domains of EFs within subjects, relative to their own ability and controlling for potential confounders. Per convention, numbers of subjects for these mixed models (phase 1b) as well as those in phase 2 (discussed below) are shown in Table 5. Results are shown in Figure 2. Each of these four analyses yielded a significant main effect for the within-subjects factor cognition, indicating differences between cognitive domains (for ADHD [Wald χ2 (4)=122.52; p<.001], mood disorders [Wald χ2 (4)=31.53; p<.001], ASD [Wald χ2 (4)=14.84; p=.005], and psychosis [Wald χ2 (4)=13.62; p=.01]).
Cognitive impairments relative to participants’ own general ability. *p ≤ 0.0125 (significant after Bonferroni correction).
Distribution of the number of subjects per diagnosis on each cognitive domain for mixed modeling analyses
a Non-comorbid.
b Psychosis with any comorbidity.
In post hoc comparisons, differences with GAI were significant across RTV and all EF measures for ADHD and mood groups (all p-values<.001). For youth with ASD and psychosis, significant effects were found for WM, inhibition, and shifting (for ASD: WM p=.005, inhibition p=.004, and shifting p=.001; for psychosis: WM p=.002, inhibition p=.002, and shifting p=.008). Decrements on RTV versus GAI did not achieve statistical significance after Bonferroni correction (for ASD, p=.04; for psychosis, p=.08 ). Thus, post-test comparisons indicated that weaknesses in EF versus GAI were significant across the four groups but that RTV was only significantly impaired relative to GAI in the ADHD and Mood groups.
Phase 2: Impact of Comorbidity on Cognition in ASD and Mood Disorders
Table 6 shows the results of mixed modeling analyses to determine the effect of comorbidity with ADHD on cognition in youth with ASD and mood disorders. Here, diagnostic group was a between-subjects factor and cognition was a within-subjects factor.
Mixed model analyses of decrements in cognitive functioning in youth with ASD and mood disorders with and without comorbid ADHD (controlling for age, sex, and medication use)
* Asterisk=not significant; critical value after Bonferroni correction is 0.0125.
a Interaction between diagnosis and cognition is not shown because of a lack of statistical significance.
In youth with ASD, there was no significant group × cognition interaction (Wald χ2 (4)=3.89; p=.42), indicating that the shape of the cognitive profile did not differ between the comorbid and non-comorbid groups. When the interaction term was dropped, the final model yielded a significant main effect for cognition (Wald χ2 (4)=28.58; p<.001), but not for group (Wald χ2 (1)=.20; p=.65). Thus, there were no significant differences in the magnitude of impairment in children with ASD with and without ADHD. Post hoc comparisons suggest that for youth with ASD regardless of comorbidity, performance in the three EF domains, but not RTV (p=.04), was significantly worse than GAI (all 3 EF p-values ≤.001).
Regarding mood disorders, the shape of the cognitive profiles for those with and without ADHD did not differ, as indicated by the non-significant group × cognition interaction (Wald χ2 (4)=6.07; p=.19). As with ASD, the model without the interaction yielded a significant main effect for cognition (Wald χ2 (4)=48.59; p<.001), with post hoc comparisons indicating significantly lower performance on all four cognitive domains RTV, WM, inhibition, and shifting compared to GAI (all p-values <.001). Additionally, in contrast to the comorbidity analysis for ASD, the group effect in the analyses of mood disorders comorbidity was significant (Wald χ2 (1)=5.41; p=.02). Thus, while the shape and relative weaknesses within the cognitive profile were similar for the two groups, cognitive difficulties were of greater magnitude in the group with mood disorders + ADHD compared to the group with mood disorders alone.
DISCUSSION
We extended evidence for cognitive weaknesses that are relevant across disorders by examining youth with different forms of psychopathology from a single cohort and addressing the impact of comorbidity. We focused on ADHD, ASD, mood disorders, and psychosis because these conditions share genetic liability and because twin and family studies suggest that cognitive impairments potentially index their underlying risk. Results indicated that aspects of EFs show decrements in a range of conditions and are not simply a result of their comorbidity. Such findings have implications for clinical practice and for studies seeking to understand mechanisms of shared liability.
We first examined cognition in youth with non-comorbid diagnoses where possible to associate cognitive weaknesses with specific conditions. Given few non-comorbid cases, youth with psychosis were included regardless of comorbidity to allow this severe form of psychopathology to be included in our analyses. Significant differences in cognition between the four diagnostic groupings were not observed. We then aimed to identify decrements present in multiple disorders. Here, we operationalized impairment in relation to normative data as well as one’s own general ability.
When impairment was defined in relation to participants’ ability, strong evidence for cross-disorder EF weaknesses emerged. Using this approach, significant decrements were observed for all diagnostic groups on measures of WM, inhibition, and shifting after correcting for potential confounders and multiple testing. Cross-disorder weaknesses in EF were less robust when normative data were the referent. In this case, after Bonferroni correction, cognitive performance was only significantly worse in youth with ADHD for the three EF domains and in youth with psychosis for WM. Yet, effect sizes based on adjusted marginal means for youth with ASD, mood disorders, and psychosis were small to moderate and generally comparable to effect sizes in youth with ADHD (Cohen’s d=−.2 to −.5). Thus, findings were generally consistent with the cross-disorder decrements found in relation to GAI in the mixed models, which benefitted from greater statistical power. For RTV, significant decrements were identified in youth with ADHD and mood disorders compared to their own GAI, and in the ADHD group when normative data were the referent. However, we cannot conclude that weaknesses in this domain were specific to ADHD and mood disorders, given that effect sizes (albeit non-significant) in youth with ASD and with psychosis also fell in the small to moderate range.
Finally, we addressed the impact of comorbidity with ADHD on cognition for mood disorders and ASD, using GAI as the referent. In both cases, there was no significant group×cognition interaction and youth exhibited significant weaknesses on all three EF domains, regardless of comorbidity. Thus, in these cases, EF impairment per se was not specific to the groups showing comorbidity with ADHD. RTV was also significantly worse than GAI in both the mood disorders and mood disorders plus ADHD groups, but not in either group with ASD. In youth with mood disorders, however, those with comorbid ADHD showed a greater magnitude of decrement than the group with mood disorders alone, whereas the magnitude of impairment did not differ in youth with ASD with and without ADHD.
These results have implications for research and clinical practice. In the literature, evidence for a shared genetic liability across different neuropsychiatric conditions has sparked interest in phenotypes that may index shared risk mechanisms. Our data support further investigation of the role of EF decrements (and the disturbances in frontally mediated neural networks that they reflect) in the liability shared across different types of psychopathology (Craddock et al., Reference Craddock, Kendler, Neale, Nurnberger, Purcell, Rietschel and Thapar2009). Given that EFs impact one’s ability to problem solve and adapt to challenges, it is plausible that such functions impact mental health generally. Whether or not EF decrements lie directly in the pathway between genes and a general risk for psychopathology is, therefore, an important question to examine, since support for this possibility would yield new implications for cognition as a therapeutic target.
These findings also have relevance to evidence-based assessment in child clinical settings. To date, no prior studies have documented cognitive profiles in these four types of psychopathology within a single sample. Our results suggest that youth whose diagnoses fall within these groupings are unlikely to be distinguished from one another based on the pattern or magnitude of weaknesses in the cognitive domains we investigated. For example, although ADHD is strongly associated with inhibition decrements in the theoretical literature, our data highlight the lack of specificity of weaknesses in inhibition to ADHD, particularly when considered in relation to patients’ own GAI.
Although RTV performance was more uneven across groups, results did not support diagnostic specificity for impaired RTV, despite its strong association with ADHD in the literature (Kuntsi et al., Reference Kuntsi, Rogers, Swinard, Borger, van der Meere, Rijsdijk and Asherson2006). Rather, our findings echo Pennington’s conceptualization of multiple overlapping deficits across different conditions (2006). This pattern is consistent with multifactorial inheritance and suggests that psychometric test scores in isolation do not speak to differential diagnosis. Although neuropsychologists are generally aware of this conclusion, those referring youth for evaluations may benefit from education regarding the limited diagnostic specificity of test scores per se. Importantly, these data do not negate the value of testing. A large literature associates impaired test scores to real world academic, occupational, and emotional functioning (e.g., Biederman et al., Reference Biederman, Petty, Fried, Fontanella, Doyle, Seidman and Faraone2006; Dajani, Llabre, Nebel, Mostofsky, & Uddin, Reference Dajani, Llabre, Nebel, Mostofsky and Uddin2016; Green, Reference Green2006), thus highlighting the value of identifying strengths and weaknesses for academic, rehabilitative, or treatment planning (Dajani et al., Reference Dajani, Llabre, Nebel, Mostofsky and Uddin2016; Seidman, Bruder, & Giuliano, Reference Seidman, Bruder and Giuliano2008).
This study supplemented clinical assessments already being undertaken. Despite the advantages of this approach for amassing a large, well-characterized, multi-diagnostic sample in a cost-efficient manner, it is not without limitations. First, we used clinician diagnoses to create diagnostic groupings. Although confirmation by structured diagnostic interviews would have enhanced our approach, these were not available for diagnoses beyond ADHD. Nonetheless, given the role of our source clinic in training at an academic teaching hospital, considerable attention is given to whether patients fulfill specific diagnostic criteria. Moreover, our blinded ratings showed high levels of inter-clinician agreement.
Second, our conclusions about cognitive weaknesses in youth with psychosis are limited by the comorbidity and sample size of this group. We cannot rule out the possibility that the impairments in this particular group are due to co-occurring conditions. However, given that (1) comorbidity within youth-onset psychosis appears to be the rule rather than the exception in our cohort as well as in the literature (Buckley et al., Reference Buckley, Miller, Lehrer and Castle2009) and also that (2) there is a significant gap in literature involving cross-disorder comparisons, the exclusion of this group would have omitted potentially informative data that domains of cognitive weaknesses overlap not only between more common conditions of youth but also in relation to this less common but severe manifestation of psychopathology. Additionally, impairments versus normative data for domains other than WM did not reach statistical significance despite generally comparable effect sizes to other groups, potentially due to the smaller sample size of this group. Nonetheless, effect sizes are informative, and comparisons versus GAI, which capitalize on the power of within-individual comparisons, were more robust.
Third, we acknowledge the heterogeneity within our mood disorders group. Although the majority of subjects received a mood disorder NOS diagnosis, this group included youth with major depressive disorder and bipolar disorder. While combining across youth with these diagnoses is reasonable given that mood symptoms in these children may evolve across diagnostic boundaries over time, the field would benefit from comparison of youth with specific forms of mood disorders to other psychopathology.
Fourth, the conclusion that these findings may have implications for clinical practice presumes reasonable generalizability of our findings. Impairments in our clinical cohort are generally comparable with the literature (Table 1) and thus may help to bridge the gap between research and clinical samples. Although performance on GAI was slightly but significantly better than normative samples in the ADHD and mood disorder groups, high average to above average ability estimates are often observed in child clinical research populations (Seidman et al., Reference Seidman, Biederman, Valera, Monuteaux, Doyle and Faraone2006), potentially in part due to exclusion of subjects with FSIQ ≤ 70, as in our study. We note that the mean Full Scale IQ (FSIQ) of four groups ranged from 94.0 to 102.9 and that GAI is expected to be higher than FSIQ in neuropsychiatric samples (Iverson, Lange, Viljoen, & Brink, Reference Iverson, Lange, Viljoen and Brink2006). Thus it is reasonable to expect generalizability of our findings. Moreover, discrepancy with overall ability is considered functionally important (Sattler, Reference Sattler2008), and asymmetric cognitive performance within individuals may help to identify difficulties obscured by group means (Jacobson, Delis, Bondi, & Salmon, Reference Jacobson, Delis, Bondi and Salmon2002). Thus, our use of mixed models to capitalize on patients’ own ability as a referent for identifying areas of weakness (Denckla, Reference Denckla1994) not only augments statistical power but helps to extend the generalizability of our data.
Fifth, a medication-naϊve sample would have been preferable; however, cognitive impairment is notoriously unresponsive to psychotropic medications (Frazier et al., Reference Frazier, Giuliano, Johnson, Yakutis, Youngstrom, Breiger and Hooper2012), and we conservatively controlled for medication use in all analyses. Sixth, as in any youth sample, subjects have not passed through the age of risk for mood disorders and psychosis, which could reduce differences between youth with ADHD and ASD and those already manifesting these conditions if cognitive impairments are trait- rather than state-related phenomena. Studies of adults across these different diagnostic groupings and/or longitudinal follow-up of youth in this sample would, therefore, complement the current findings.
Despite these issues, this study advances the literature by examining multiple diagnostic groups and cognitive domains in a single youth sample with attention to comorbidity. By investigating conditions known to share genetic liability, we aimed to highlight cognitive functions suitable for consideration in studies of cross-disorder risk mechanisms. Our findings suggest that domains of EFs including WM, inhibition, and shifting/mental flexibility show decrements in ADHD, mood disorders, ASD, and psychosis that are not simply a function of comorbidity. Furthermore, although comorbidity with ADHD increases the magnitude of deficits in mood disorders, the overall profile of findings for GAI, RTV, and EFs does not differ in youth with ASD and mood disorders in the presence of comorbidity with ADHD. These data support a complex relationship between cognition and psychopathology, as described in Pennington’s (Reference Pennington2006) multiple deficit models and set the stage for further investigation of the role of EF weaknesses, and the disrupted neural networks they reflect, in the heritable risk mechanisms common to different forms of neuropsychiatric illness. They further highlight that test scores themselves do not speak to differential psychiatric diagnosis, despite having other utility in the context of neuropsychiatric evaluations.
ACKNOWLEDGMENTS
This research was supported by funding from the David Judah Foundation, the Stanley Center for Psychiatric Research and NIH R03MH106862 to AD. The authors report no potential conflicts of interest. Drs. Doyle, Vuijk, Doty, McGrath, Willoughby, O’Donnell, Wilson, Colvin, and Seidman, and Mss. Toner, Hudson, Blais, and Ditmars have no affiliations to disclose. Dr. Braaten serves on the boards of Magination Press and Beyond Booksmart. She receives royalties from Guilford Press. Dr. Faraone serves on the advisory boards to Alcobra Ltd., Arbor Pharmaceuticals, LLC, CogCubed, Ironshore Pharmaceuticals & Development, Inc., and NeuroLifeSciences. He is a consultant to Akili Interactive Labs, Inc., and Alcobra Ltd., Ironshore Pharmaceuticals & Development, Inc., and VAYA Pharma. He receives grant or research support from Framework Programmes for Research and Technological Development and Shire Pharmaceuticals. He receives royalties from Guilford Press and Oxford University Press. He holds stock/equity in CogCubed and holds US Patent US20130217707 A1 for Framework Programmes for Research and Technological Development.
