Although symptoms of anxiety and depression tend to coexist throughout the life span, empirical evidence indicates that rates of comorbidity are highest among youths (Rohde, Lewinsohn, & Seeley, Reference Rohde, Lewinsohn and Seeley1991). High rates of comorbidity have led some researchers to conclude that anxiety and depression actually represent a unitary construct in youths (e.g., Achenbach, Connors, Quay, Verhulst, & Howell, Reference Achenbach, Connors, Quay, Verhulst and Howell1989; Brady & Kendall, Reference Brady and Kendall1992; Jacques & Mash, Reference Jacques and Mash2004). Research also indicates that anxiety and depression symptom differentiation in youths may differ as a function of psychopathological disturbance and/or age (cf. Brady & Kendall, Reference Brady and Kendall1992). Such research has led some to question the validity of the current conceptualizations of anxiety and depressive disorders in the DSM (American Psychological Association, 2000). The difficulty discriminating between anxiety and depression coupled with the high rates of comorbidity in both adult and child populations suggest that depression and anxiety are strongly related to one another and therefore may not be best conceived as separate and discrete disorders (e.g., Lahey et al., Reference Lahey, Rathouz, Van Hulle, Urbano, Krueger and Applegate2008; Watson, Reference Watson2005; Higa-McMillan, Smith, Chorpita, & Hayashi, Reference Higa-McMillan, Smith, Chorpita and Hayashi2008). Treatments have also more recently begun to be developed and tested to treat this broad class of emotional problems and disturbances related to anxiety and depression, such as the Unified Protocol for Treatment of Emotional Disorders in Youth (Ehrenreich, Goldstein, Wright, & Barlow, Reference Ehrenreich, Goldstein, Wright and Barlow2009). Proposed changes for the upcoming fifth addition of the DSM reflect this growing concern. For example, although a mixed anxiety/depression disorder currently exists in the appendix of the text revision of the fourth edition of the DSM, the Mood Disorders workgroup has proposed that it be included in the fifth edition as a separate diagnostic category (Fawcett, Reference Fawcett2009). Although research in this area is growing in the adult literature, it is critical that research in developmental psychopathology examine the implications of development and severity of symptoms on the structural similarities and differences between anxiety and depression.
Comorbidity and Symptom Severity
The high rate of comorbid anxious and depressive symptomology in youths has been well established in the research literature (e.g., Kovacs, Gatsonis, Paulauskas, & Richards, Reference Kovacs, Gatsonis, Paulauskas, Richards, Chess and Hertzig1990; Masi, Mucci, Favilla, Romano, & Poli, Reference Masi, Mucci, Favilla, Romano and Poli1999; van Lang, Ferdinand, Ormel, & Verhulst, Reference van Lang, Ferdinand, Ormel and Verhulst2006). In a review of research studies that examined anxiety and depression comorbidity in youths, Brady and Kendall (Reference Brady and Kendall1992) found that comorbidity rates of the studies assessed ranged from 15.9% to 61.9% depending on the sample studied. For example, lower comorbidity rates were found in nonclinical samples (Anderson, Williams, McGee, & Silva, Reference Anderson, Williams, McGee and Silva1987; Costello et al., Reference Costello, Costello, Edelbrock, Burns, Dulcan and Brent1988) and higher rates were found among youth inpatients (Carey, Finch, & Imm, Reference Carey, Finch and Imm1989) and adolescent outpatients (Kovacs et al., Reference Kovacs, Gatsonis, Paulauskas, Richards, Chess and Hertzig1990). In contrast, research that has attempted to address the unitary versus two-factor structure of depression and anxiety in youths has concluded that higher comorbidity rates are found among less disturbed samples and that symptom differentiation may be a function of psychopathology severity (Gurley, Cohen, Pine, & Brook, Reference Gurley, Cohen, Pine and Brook1996). Among a large community sample of youths (N = 976, ages 9–20) and their caregivers who completed structured clinical interviews, Gurley et al. (Reference Gurley, Cohen, Pine and Brook1996) found that anxiety and depression could not be discriminated among those without a diagnosis but could be discerned in their diagnosed subsample. Results also indicated that the overlap between anxiety and depressive symptoms decreased (i.e., symptoms became more differentiated) as emotional psychopathology increased. Comorbidity was slightly more common among younger youths and a unitary depression/anxiety factor best fit those who were below the most stringent diagnostic threshold for all disorders, regardless of age. The authors concluded that the ability to discriminate between depression and anxiety increases as the severity of emotional psychopathology increases. Although studies using adult participants have replicated these findings (Clark & Watson, Reference Clark and Watson1991; Hiller, Zaudig, & von Bose, Reference Hiller, Zaudig and von Bose1989; Katon & Roy-Byrne, Reference Katon and Roy-Byrne1991; Zinbarg et al., Reference Zinbarg, Barlow, Liebowitz, Street, Broadhead and Katon1994), few, if any, studies have done so with youth samples. Thus, the present study seeks to address this research need.
Tripartite Model and Development
Research on the tripartite model has also shed light on the overlap and differentiation of anxiety and depression symptoms in youths. The tripartite model purports that anxiety and depression can be understood along three dimensions (Clark & Watson, Reference Clark and Watson1991). The first dimension, negative affect (NA), accounts for general emotional distress and includes states such as fear, anger, guilt, and sadness (Watson & Clark, Reference Watson and Clark1984; Watson & Tellegen, Reference Watson and Tellegen1985). Individuals with anxiety and/or depression are hypothesized to have moderate to high levels of NA. The second dimension, known as physiological hyperarousal (PH), is theorized to be specific to anxiety (i.e., high levels of PH should be found among individuals with anxiety, and PH and depression should have low or nonsignificant correlations). Finally, positive affect (PA), characterized by energy, engagement, and high levels of interest, should be significantly negatively correlated with depression and relatively unrelated to anxiety symptoms.
Some tripartite model research has found that the tripartite model is applicable to both younger children and adolescents. For example, results from Turner and Barrett (Reference Turner and Barrett2003) indicate that the tripartite model differentiates between anxiety and depression in nonreferred third-, sixth-, and ninth-grade children. Results from this study provided little evidence for increasing differentiation between depression and anxiety. Among a sample of child and adolescent psychiatric inpatients, Joiner, Catanzaro, and Laurent (Reference Joiner, Catanzaro and Laurent1996) found that the tripartite model was well supported. In addition, Chorpita, Albano, and Barlow (Reference Chorpita, Albano and Barlow1998) assessed a clinical sample of children with anxiety disorder and/or mood disorder diagnoses and found that a model in which fear, anxiety, and depression were conceptualized as distinct but related factors fit the data best. Chorpita, Plummer, and Moffitt (Reference Chorpita, Plummer and Moffitt2000) found that among a child clinical sample, nearly all aspects of the tripartite model were supported (e.g., NA was significantly related to anxiety and PA was significantly related to depression), except the PH dimension, which did not correlate significantly with generalized anxiety disorder symptomology. Similarly, in a large school-based sample, Chorpita (Reference Chorpita2000) found that PH was only related to panic disorder but not to other anxiety disorders. Hence, it appears that the constructs of NA and PA are well supported in the child psychopathology literature. However, there is evidence that suggests that the PH factor may not be applicable to youths.
Despite the growing empirical support for the applicability of the tripartite model to younger youth samples, there is stronger support for its applicability in samples of older children and adolescents. For example, Jacques and Mash (Reference Jacques and Mash2004) found that among a large community sample of elementary school children and high school youths (N = 472), the tripartite model was better supported in the older sample. Using confirmatory factor analytic methods to assess items from a battery of youth anxiety and depression measures, Cole, Truglio, and Peeke (Reference Cole, Truglio and Peeke1997) found that a unified construct model (combining depression and anxiety) fit data from a nonclinical sample of thirds graders (N = 280) better than the tripartite model as well as a dual model of anxiety and depression (i.e., anxiety and depression specified as separate factors). On the contrary, these researchers found that the tripartite model fit the sixth-grade sample (N = 211) data better compared to the unified one-factor model. Furthermore, anxiety and depression symptoms were less correlated in the older sample compared to the younger sample. Similarly, Lonigan, Hooe, David, and Kistner (Reference Lonigan, Hooe, David and Kistner1999) found that the tripartite model fit data from older children from a nonclinical sample (N = 213, ages 12–17) better than their younger counterparts (N = 152, ages 9–11). Results from these studies provide support for the notion that there is an increasing differentiation between anxiety and depression across development (De Bolle & De Fruyt, Reference De Bolle and De Fruyt2010).
Alternative theories used to explain the development of anxiety and depression in youths are also worth noting. Cannon and Weems (Reference Cannon and Weems2006) and Weems, Saltzman, Reiss, and Carrion (Reference Weems, Saltzman, Reiss and Carrion2003) have found support for the notion that primary anxiety symptoms may precede the development of depression. For instance, in an investigation of the applicability of the tripartite model in a large community sample of youths, Cannon and Weems (Reference Cannon and Weems2006) found that in the younger subsample (ages 6–11) there was a distinct anxiety-only group but a less distinct depression-only group. In the older subsample (ages 12–17), there were more instances of depression-only and fewer youths who solely had anxiety problems. The authors suggest that the comorbidity of anxiety and depression in youths may not be a result of heightened susceptibility to a common component (such as NA). Rather, comorbidity may be better explained by the distress and impairment caused by anxiety, which leads to increased depressive symptomology. Similarly, results from a study of children with a history of traumatic experience(s) suggest that the emotional numbing of depression may develop as a result of the chronic hyperarousal associated with anxiety in youths (Weems et al., Reference Weems, Saltzman, Reiss and Carrion2003).
In sum, previous studies indicate that the applicability of the tripartite model, specifically the differentiation between depression and anxiety, may differ as a function of age in both clinical and nonreferred samples. However, other research indicates that the tripartite model is appropriate for both younger children and adolescents. In addition, some epidemiological research suggests that anxiety and depression symptoms may be less differentiated among nonclinical samples (Gurley et al., Reference Gurley, Cohen, Pine and Brook1996). This phenomenon may be partially explained by the higher comorbidity rates of anxiety and depression in younger versus older youths. Given the inconsistent research evidence from previous studies, we sought to assess the applicability of three related factor structures in order to shed light on the two-factor versus unitary conceptualizations of depression and anxiety symptomology in youths.
Present Study
In order to examine anxiety and depression symptom differentiation in youths, we assessed competing factor structures based on the items from the DSM-oriented scales of the Child Behavior Checklist (CBCL; Achenbach et al., Reference Achenbach, Dumenci and Rescorla2003) in a large sample of referred and nonreferred youths as well as subsamples of older (ages 11–18) and younger (ages 7–10) youths. This age cutoff was chosen because the formal operations stage begins around age 11 (Piaget, Reference Piaget and Mussen1983). At this time, youths begin to have abstract thoughts and are capable of metacognitions and thus worry and self-consciousness. Moreover, the definition of “adolescence” has recently been modified so as to include children as young as 11 years of age; this is because children at this age have begun to experience more adolescent-typical problems (e.g., substance abuse, eating disorders, and depression; Saluja et al., Reference Saluja, Iachan, Scheidt, Overpeck, Sun and Giedd2004). Given that past research has assessed the tripartite model and symptom differentiation of anxiety and depression through child self-report, use of parent report also represents an important step in understanding these constructs as reported by parents. The CBCL is one of the most commonly used parent-report measures of youth behavioral and emotional problems and competencies. The rationally derived CBCL DSM-oriented scales were created by a panel of child mental health professionals with the aim of producing a CBCL model (and a set of DSM-oriented scales) congruent with DSM nosology (Achenbach et al., Reference Achenbach, Dumenci and Rescorla2003). Each scale represents one or more DSM diagnoses. The scales and their associated diagnoses include affective problems, anxiety problems, attention-deficit/hyperactive problems, conduct problems, oppositional defiant problems, and somatic problems. The construction of the DSM-oriented scales thus provided us a set of items upon which to examine the differentiation of anxiety and depressive problems across sample types.
Specifically, we examined the six-factor CBCL DSM-oriented structure as well as the applicability of two competing, yet related, DSM-oriented models. We tested a five-factor DSM-oriented model by collapsing the affective problems and anxiety problems scales into a broader NA factor. We also tested a two-factor model of the DSM-oriented scales, which consisted of the internalizing problems and the externalizing problems scales (Achenbach, Reference Achenbach1991). The internalizing problems scale includes the affective, anxiety, and somatic problems scales and the externalizing problems scale includes the oppositional defiant, conduct, and attention-deficit/hyperactive problems scales.
Based on previous psychometric research on the CBCL DSM-oriented scales, we hypothesized that the six-factor model would yield good fit for the entire sample (N = 1,470) as well as all subsamples (e.g., younger, older, clinical, and nonclinical youths). However, given the inconsistent results regarding the relationship among age, psychopathology severity, and depression and anxiety symptom differentiation in youths, we predicted that the six- and five-factor structures would provide variable fit for the subsamples. In other words, we expected that the model encompassing a one-factor construct combining depression and anxiety (i.e., the five-factor structure) to differentially fit younger and older participants as well as clinic-referred and nonreferred youths, compared to the six-factor structure. Finally, we were interested in potential differences in reporting styles and item interpretations of the CBCL items across sex. Thus, we examined the data for differential item functioning across boys and girls. We expected that measurement invariance tests would identify differences (with respect to differential item functioning) across genders because previous studies have found that symptom presentation (e.g., anxiety and depression) often differs between boys and girls (Essau, Conradt, & Petermann, Reference Essau, Conradt and Petermann2000; Lewinsohn, Rohde, & Seeley, Reference Lewinsohn, Rohde and Seeley1998).
Method
Participants
The present sample (N = 1,470) consisted of clinic-referred (n = 757) and school-based (n = 713) youths whose caregivers completed the CBCL for Ages 6–18.
Clinic-referred sample
The clinical sample comprised youths referred to a university clinic located in Honolulu, Hawaii, and a clinic in a children's hospital in Boston, Massachusetts, for mental health evaluations. The mean age of the clinic-referred sample was 13.07 years (SD = 3.04, range = 7–18). This sample consisted of 528 males (69.7%) and 224 females (29.6%); youth gender data were missing for 5 (0.7%) participants. Major ethnic groups of the youths included multiethnic (n = 397; 52.4%), Asian American (n = 85; 11.2%), White (n = 84; 11.1%), Hawaiian/Pacific Islander (n = 42; 5.5%), other (n = 18; 2.4%), African American (n = 12; 1.6%), and Latino/Hispanic (n = 9; 1.2%). Ethnicity data for this sample were missing for 110 (14.5%) participants. Of the 757 youths in the clinical sample, 372 had at least one diagnosis (58.2% had one diagnosis and 18.1% had two or more diagnoses). Of youths with a diagnosis, 168 had an anxiety disorder (separation anxiety disorder n = 24, 3.1%; obsessive compulsive disorder n = 20, 2.5%; social anxiety disorder n = 52, 6.9%; specific phobia n = 41, 5.4%; generalized anxiety disorder n = 31, 4.1%), 42 (5.5%) were diagnosed with major depressive disorder, 119 (15.7%) were diagnosed with oppositional defiant disorder, and 130 (17.2%) youths received a conduct disorder diagnosis. Eighteen (2.4%) participants were diagnosed with pervasive developmental disorder, and 28 (3.7%) received an “other” diagnosis.
School-based sample
The nonclinical (school-based) sample comprised children and adolescents recruited from private and public schools in Grades 3–12 on Oahu, Hawaii. The mean age of this sample was 12.86 years (SD = 2.8, range = 7–18) and consisted of 295 males (41.4%) and 416 females (58.3%); youth gender data were missing for 2 (0.3%) participants. Major ethnic groups of the youths included Asian American (n = 380; 53.3%), multiethnic (n = 288; 40.4%), Hawaiian/Pacific Islander (n = 20; 2.8%), White (n = 19; 2.7%), and Latino/Hispanic (n = 2; 0.3%). Ethnicity data for this sample was missing for 24 (3.4%) participants.
Six separate t tests were conducted in order to investigate CBCL DSM-oriented scale score differences between the clinic-referred and the school-based samples. All analyses were performed using a 99.7% confidence interval (α = 0.003). Results indicated that the clinic-referred group had significantly higher total scores on all DSM-oriented scales compared to the school-based, nonreferred group (p < .003).
Procedure
For both samples, participants and their caregivers underwent standardized institutional review board-approved notice of privacy and consent procedures. Upon receiving signed consent, the primary caregivers of the school-based youths were mailed a blank CBCL (with a university-addressed, stamped envelope) to fill out and mail back to the university. For the clinic-referred sample, the caregivers were asked to fill out a CBCL regarding their children (following consent provided at the initial meeting with the youths and their primary caretakers). Clinic-referred youths underwent diagnostic intake evaluations (including completing questionnaires and structured diagnostic interviews), and their caregivers also completed a packet of intake questionnaires and participated in structured diagnostic interviews. Diagnostic determinations were made by PhD level clinical child psychologists and doctoral students in clinical psychology.
Measures
The CBCL for Ages 6–18 (Achenbach & Rescorla, Reference Achenbach and Rescorla2001) is a parent-report instrument that includes 113 items measured on a 3-point Likert scale (0 = not true, 1 = somewhat or sometimes true, and 2 = very true or often true). The DSM-oriented scales comprise 55 of the 113 items. The six DSM-oriented scales include affective problems, anxiety problems, attention-deficit/hyperactive problems, conduct problems, oppositional defiant problems, and somatic problems. We transformed the narrow-band six-factor structure into a more broadband five-factor structure by collapsing the affective problems and anxiety problems into a single NA factor. A third broadband factor structure, consisting of two factors, was also tested. The two-factor structure is composed of an internalizing problems scale (affective problems, anxiety problems, and somatic problems scales are collapsed) and an externalizing problems scale (attention-deficit/hyperactive problems, conduct problems, and oppositional defiant problems scales are collapsed; see Figure 1).
The Child Behavior Checklist DSM-oriented six-factor model and competing five- and two-factor models.
The DSM-oriented scales have evidenced strong test–retest reliability, cross-informant agreement, and internal consistency (Achenbach et al., Reference Achenbach, Dumenci and Rescorla2003). Supportive scale reliability and convergent and discriminative validity have been demonstrated in a large sample of clinic-referred children (Nakamura, Ebesutani, Bernstein, & Chorpita, Reference Nakamura, Ebesutani, Bernstein and Chorpita2009), and support for the factor structure has been found in a community sample of youths (Achenbach et al., Reference Achenbach, Dumenci and Rescorla2003). The clinical utility of the CBCL DSM-oriented scales has also been explored by Ferdinand (Reference Ferdinand2008) and Ebesutani et al. (Reference Ebesutani, Bernstein, Nakamura, Chorpita, Higa-McMillan and Weisz2010). Ferdinand (Reference Ferdinand2008) found that when using parent/child impairment ratings, the anxiety problems and affective problems scales could predict relevant diagnoses (e.g., scores on the anxiety problems scale could predict separation anxiety disorder, generalized anxiety disorder, and specific phobia). Among a large sample of clinic-referred children, five of the six CBCL DSM-oriented scales (excluding the somatization problems scale, which was not tested owing to absence of somatization disorder diagnoses in the sample) corresponded significantly with the individual diagnoses targeted by the scale (Ebesutani et al., Reference Ebesutani, Bernstein, Nakamura, Chorpita, Higa-McMillan and Weisz2010). Furthermore, each of these scales was able to discriminate youths within the diagnostic group targeted by the scale from youths without such diagnoses.
Data analytic strategy
Confirmatory factor analysis (CFA)
To examine competing factor structures of the CBCL DSM-oriented scales, we conducted CFA using Mplus version 4.21 (Muthén & Muthén, Reference Muthén and Muthén2006). Due to the categorical (i.e., ordinal) nature of our data, we employed the robust weighted least squares with mean and variance adjustment (WLSMV) estimator because it has been found to handle ordinal data well (Brown, Reference Brown2006; Flora & Curran, Reference Flora and Curran2004; Muthén, du Toit, & Spisic, Reference Muthén, du Toit and Spisic1997). The fit of the six-, five-, and two-factor DSM-oriented structures were examined for the entire sample (clinical and nonclinical samples combined) and the clinical and nonclinical samples separately. We also examined the fit of these factor structures among different subsamples, including (a) younger (ages 7–10) and (b) older (ages 11–18) youth subsamples (both clinical and nonclinical participants combined in each subgroup) as well as more specific subgroups crossing age by sample type: (c) younger clinical, (d) older clinical, (e) younger nonclinical, and (f) older nonclinical. The fit of the DSM-oriented scale structures were evaluated via various fit indices including the root mean square error of approximation (RMSEA; Steiger, Reference Steiger1990), the comparative fit index (CFI; Bentler, Reference Bentler1990), and the Tucker–Lewis index (TLI; Tucker & Lewis, Reference Tucker and Lewis1973). Values of 0.90 and 0.95 and above represent good and excellent model fit, respectively, for the CFI and the TLI indices (Bentler, Reference Bentler1990; Hu & Bentler, Reference Hu and Bentler1999). For the RMSEA index, values of 0.08 or lower indicate good fit, and RMSEA values of 0.05 or lower indicate excellent fit (Browne & Cudeck, Reference Browne, Cudeck, Bollen and Long1993). To examine the relative fit of the two-, five-, and six-factor structures, we conducted chi-square (χ2) difference tests for each subsample.Footnote 1
Multigroup CFA (MG-CFA)
We then conducted MG-CFA using Mplus version 4.21 to determine whether the best fitting CBCL DSM-oriented structure was associated with measurement invariance across sex (i.e., male and female youths) and sample type (i.e., clinical and nonclinical participants). MG-CFA is commonly considered to be the most robust technique for the examination of measurement invariance and offers researchers the ability to examine all aspects of population heterogeneity and measurement invariance (Brown, Reference Brown2006). As recommended by Brown (Reference Brown2006), we employed a “step-up” approach for evaluating model restrictions within MG-CFA. The model-building approach allowed us to identify and examine any aspects of poor fit associated with each additional model restriction (Brown, Reference Brown2006). Hence, after testing the CFA model in each group separately (i.e., single-sample CFA tests), we conducted simultaneous tests (for all samples of interest) of equal form (i.e., equal factor structure, also referred to as configural invariance). Due to the categorical nature of our data, we did not model item intercepts (as would be done with continuous data), but rather, we modeled item thresholds (Lubke & Muthén, Reference Lubke and Muthén2004). The number of thresholds is equal to the number of categories for each item minus one (Brown, Reference Brown2006). Thus, we modeled two thresholds for each item owing to the 3-point Likert scale underlying the CBCL. We also set all scaling factors to 1.0, as is required when conducting MG-CFA with categorical data (cf. Mplus FAQ; UCLA, Academic Technology Services, Statistical Consulting Group, 2011).
We tested the various forms of measurement invariance in the following ways. We assessed the fit of equal form across groups using the same fit index cutoffs mentioned above. Support for equal form across groups indicates configural invariance and justifies proceeding to the test of equal factor loadings (i.e., metric invariance, also referred to as weak factorial invariance) and then to the test of equal item thresholds. We freed and constrained both factor loadings and item thresholds in tandem so that meaningful comparisons of factor distributions across groups could be made owing to the categorical nature of our data (Muthén & Asparouhov, Reference Muthén and Asparouhov2002). We determined whether these constraints significantly degraded fit using the χ2 difference test as well as by examining the change in CFI values between the “equal form and equal thresholds” test and the nested “equal factor loading” test. If the ΔCFI is greater than 0.01 or the χ2 difference test results are significant, the model fit of the equal thresholds and equal factor loading model is considered significantly degraded compared to the equal form model (Cheung & Rensvold, Reference Cheung and Rensvold1998), suggesting a lack of full invariance of the tested parameters (i.e., that the parameters generally vary) across groups. Multiple tests (i.e., ΔCFI and χ2 difference testing) were used to provide a more conservative and robust evaluation of measurement invariance. Partial measurement invariance was examined if these results indicated that full measurement invariance was not supported. Partial measurement invariance may be present and supported when the lack of overall measurement invariance is due to only some of the indicators being noninvariant across groups (Byrne, Shavelson, & Muthén, Reference Byrne, Shavelson and Muthén1989). To test for partial measurement invariance, the most noninvariant parameters are freed one at a time and tests are performed to examine for invariance under such conditions.
Results
CFA
The 55 items comprising the six-, five-, and two-factor CBCL DSM-oriented scales were subject to CFA for the full sample (nonclinical and clinical samples; N = 1,470). The resulting fit statistics and chi-square difference test results appear in Table 1. The two-factor model (composed of the externalizing problems factor and the internalizing problems factor) demonstrated poor to moderate fit in nearly all subsamples, except for the nonclinical (all ages) sample and the nonclinical older youth sample, which evidenced good fit (i.e., RMSEA < 0.08, CFI > 0.90, and TLI > 0.95).
Fit indices of the CBCL DSM-oriented scales
Note: CBCL, Child Behavior Checklist; CFI, comparative fit index; TLI, Tucker–Lewis Index; RMSEA, root mean square error of approximation.
The five-factor model, which is composed of the NA (combined affective problems and anxiety problems), attention-deficit problems, somatic problems, conduct problems, and oppositional defiant problems scales, provided good fit for five of the nine samples examined, including all nonclinical samples (all ages nonclinical, young nonclinical, and old nonclinical) as well as the younger and older subsamples (both clinical and nonclinical participants). The five-factor model provided moderate fit for the entire sample and all clinical samples (all ages clinical, young clinical, and old clinical). Results from the chi-square difference tests also indicated that the six- and five-factor models provided significantly better fit in all samples compared to the two-factor model as evidenced by p values of less than .0001 for each test. These results suggest that the five-factor represents the structure of the data better than the two-factor model.
The original DSM-oriented scales are composed of six factors (similar to the five-factor model tested in the present paper but splitting NA into affective problems and anxiety problems). The resulting fit statistics represent good fit for this six-factor model in the entire sample as well as adequate to good fit for all subsamples. Although the CFI values for both the clinical and the older clinical youth samples do not represent good fit by conventional standards (i.e., CFI < 0.90), both TLI and RMSEA values indicate good fit for the six-factor structure of the DSM-oriented scales in these two samples. Chi-square difference tests were also conducted to compare the fit of the six- and five-factor models for each subsample. Results from these tests indicate that the six-factor model provides significantly better fit for all samples (ps < .01) except for the nonclinical young subsample, χ2diff (4) = 9.377, p = .052. The nonsignificant results indicate that the six-factor model does not provide a significantly better fit relative to the five-factor model for the nonclinical young sample, thereby supporting the more parsimonious, good-fitting five-factor model among the nonclinical young subsample. In other words, results suggest that the five-factor structure, which represents a unitary NA factor (i.e., depression and anxiety are undifferentiated), provides better fit for the younger youth sample characterized by less severe psychopathology (i.e., nonreferred). Conversely, the six-factor structure, which includes separate factors for affective and anxiety problems, provides optimal fit for the younger clinic-referred sample (i.e., the younger subsample with more severe psychopathology).
MG-CFA
Measurement invariance across gender
As tested and reported above, single-sample solutions indicated that the six-factor DSM-oriented model provided good fit for all subsamples assessed and also provided significantly better fit relative to the five-factor model for all subsamples, except the nonclinical young sample (whose data are best represented by a five-factor structure). Therefore, we proceeded to test measurement invariance of the six-factor structure across gender for the entire sample excluding the nonclinical young participants (n = 1,238). To determine whether model fit (of the six-factor model) differed across gender, we first conducted a test of equal form by placing equality constraints on the general factorial structure across boys and girls. Strong model fit indices (i.e., CFI = 0.960, TLI = 0.968, RMSEA = 0.064) resulting from the test of equal form indicated that configural invariance of the six-factor model was supported across gender among this subsample. We then assessed the next level of measurement invariance by simultaneously constraining factor loadings and item thresholds to be equal across groups. Results from this test indicated that these constraints significantly degraded model fit, as evidenced by significant chi-square difference test results, χ2diff (45) = 265.289 p < .0001, and a ΔCFI value greater than 0.01 (ΔCFI = 0.039) between the restricted and the unrestricted solutions. In other words, results indicated that the CBCL items had different relationships to their respective DSM-oriented factor across males and females.
We then proceeded to test partial metric invariance by freeing constraints on noninvariant items. We determined which items to free by calculating the difference in factor loadings for each item across groups. For example, the somatic problems scale item 56G, “vomiting, throwing up” was associated with the largest difference between boys and girls in factor loadings. For the boys-only group, the item factor loading was 0.70 and for girls it was 0.95; thus, we eliminated the constraints previously imposed on this item's factor loading and item threshold to allow this item to freely load on the somatic problems factor across group. We then tested for partial invariance by constraining all other factor loading and threshold parameters to be equivalent across samples. Results indicated that partial metric invariance was not supported, as evidenced by a significant χ2diff test result and a change in ΔCFI greater than 0.01, χ2diff (44) = 257.558 p < .0001, ΔCFI = 0.039). This procedure was repeated four more times (i.e., four items associated with large factor loading differences were added to the model, one at a time). Chi-square difference test results and ΔCFI values indicated that the model continued to be significantly degraded compared to the less constrained model, with each new addition. After freeing the five items associated with the largest factor loading differences between groups, we concluded that partial invariance was not supported; that is, after five parameters were freed, χ2diff (43) = 222.360 p < .0001, ΔCFI = 0.036. We decided to discontinue partial invariance testing at this juncture because Vandenberg and Lance (Reference Vandenberg and Lance2000) suggest that if a large number of indicators are found to be noninvariant, partial invariance tests should not be continued and full invariance should be assumed. These results thus suggest that the item–factor relationships are not the same across boys and girls, and that the DSM-oriented scales are associated with differential item functioning with respect to gender among this subsample.
Measurement invariance across sample type
The single-sample CFA results indicate that the five-factor model fit the young nonclinical data best and the six-factor model fit the young clinical sample data best. Therefore, we did not assess measurement invariance of any given factor structure across sample type in the young subsample but did so for the older subsample (because the six-factor structure fit the best among the older clinical and older nonclinical youths). Results from the test of equal form across these groups indicate that configural invariance was present across clinical and nonclinical older youths (CFI = 0.916, TLI = 0.954, RMSEA = 0.065). We then tested the next level of measurement invariance by constraining factor loadings and item thresholds to be equal across groups. Results from this analysis revealed that measurement invariance was not supported at this level; that is, χ2diff (47) = 917.011 p < .0001 and ΔCFI = 0.095. In other words, results indicated that the CBCL items had different relationships to their respective DSM-oriented factor across older clinical and older nonclinical samples. We tested partial invariance by following the steps outlined above and concluded that partial invariance was not supported for the six-factor model across clinical and nonclinical participants in the older youth sample; that is, after freeing the five most noninvariant parameters, χ2diff (44) = 834.298 p < .0001 and ΔCFI = 0.083. These results suggest that older youth participants answered many CBCL items associated with the DSM-oriented scales in systematically different ways based on sample type (i.e., clinic referred or school based).
Discussion
This study examined symptom differentiation of anxiety and depression across development and sample types by examining various competing factor structures using the DSM-oriented scales of the CBCL. All three models tested provided at least adequate fit, suggesting that the CBCL DSM-oriented scales (and hierarchical combinations of these scales) are structurally consistent with DSM nosology. Given that past research of these scales has been limited to samples of youths who had moderate to high CBCL total problems scale scores (Achenbach et al., Reference Achenbach, Dumenci and Rescorla2003) and given the mixed results regarding the factorial validity of the scales (Hartman et al., Reference Hartman, Hox, Mellenbergh, Boyle, Offord and Racine2001), this study provides more conclusive supportive evidence for the structural validity of the CBCL DSM-oriented scales.
As hypothesized, we found that the six-factor model (i.e., affective problems, anxiety problems, somatic problems, attentional problems, conduct problems, and oppositional defiant problems) provided good fit for all subsamples. The five-factor model (with anxiety problems and affective problems collapsed into a single NA factor) was supported above the six-factor model among the young, nonclinical sample. These findings are consistent with previous research on symptom differentiation in clinical versus nonclinical samples (Gurley et al., Reference Gurley, Cohen, Pine and Brook1996) and younger versus older youths (Cole et al., Reference Cole, Truglio and Peeke1997; Jacques & Mash, Reference Jacques and Mash2004; Lonigan et al., Reference Lonigan, Hooe, David and Kistner1999), suggesting that anxiety and depression are less differentiated in nonclinical youths and younger children. In addition to providing further support for the unique lack of differentiation of anxiety and depression among nonreferred children, the present study also extended these findings to parent reports of youth emotional and behavioral problems.
Regarding this lack of differentiation among younger youths, it is possible that depression is a result of the distress and impairment caused by anxiety as Weems and colleagues suggest (Cannon & Weems, Reference Cannon and Weems2006; Weems et al., Reference Weems, Saltzman, Reiss and Carrion2003); thus, the difference between anxiety and depression is less distinct in younger children. However, given that anxiety and depression were differentiated from each other in the younger clinical sample, an alternative explanation is that, at a subclinical level, anxiety and depression are more alike than different in younger children. This conclusion appears to support the notion that there is an underlying trait of NA that accounts for this class of symptoms. Developmental theories and research have also shed light on possible mechanisms by which anxiety and depression differentiate in youths. For example, some research suggests that advances in cognitive development (i.e., self-concept development and self-esteem; Burnett, Reference Burnett1995; Ohannessian, Lerner, Lerner, & von Eye, Reference Ohannessian, Lerner, Lerner and von Eye1999) and/or contextual factors in adolescence (e.g., peer relationships; Boivin, Hymel, & Bukowski, Reference Boivin, Hymel and Bukowski1995; Boivin, Poulin, & Vitaro, Reference Boivin, Poulin and Vitaro1994; Kupersmidt & Patterson, Reference Kupersmidt and Patterson1991) may play a role in the onset of depressive symptoms in older youths. In addition, some research indicates that as emotions become more specific and outcome dependent with development, children's increasingly sophisticated attributions result in previously undifferentiated feelings states (e.g., anxiety or depression; Weiner & Graham, Reference Weiner, Graham, Izard, Kagan and Zajonc1988). Based on this theory, De Bolle, Decuyper, De Clercq, and De Fruyt (Reference De Bolle, Decupyer, De Clercq and De Fruyt2010) suggest that younger children who experience problems of depression or anxiety are likely to report a general malaise. Later in development, specific symptoms may be attributed to specific environmental entities or may be interpreted as signs of imminent danger, which may result in specific fears or phobias and generalized anxiety, respectively. In contrast, these signals may be interpreted as a failure to effectively cope with the situation, which will eventually affect one's self-esteem and thus increase the risk for depression. In sum, these theories and supporting research suggest that cognitive development plays a primary role in NA symptom differentiation.
Beyond symptom differentiation among clinical and nonclinical samples and younger versus older youths, the partial invariance test results indicate that the parents of boys answered many CBCL items in systematically different ways than did the parents of girls. These findings are not surprising, given that research on gender differences across developmental psychopathology consistently find that the nature of symptom expression differs between girls and boys (Essau et al., Reference Essau, Conradt and Petermann2000; Lewinsohn et al., Reference Lewinsohn, Rohde and Seeley1998). Specifically, past research has often failed to find gender differences in depressive and anxiety symptomology among prepubescent male and female youths (e.g., Cohen et al., Reference Cohen, Cohen, Kasen, Velez, Hartmark and Johnson1993; Nolen-Hoeksema, Girgus, & Seligman, Reference Nolen-Hoeksema, Girgus and Seligman1991). However, compared to male adolescents, higher rates of anxiety and depressive disorders are consistently found among female adolescents (Brooks-Gunn & Petersen, Reference Brooks-Gunn and Petersen1991; Essau et al., Reference Essau, Conradt and Petermann2000). Similar results were found for older clinical and older nonclinical youths (i.e., responses differed based on sample type). Although these samples may be responding differently, Brown (Reference Brown2006) notes that some parameters may differ by chance, especially if the sample size is large. Therefore, future research in this area should explore this matter more closely because it is of relevance to the generalizability and interpretation of the DSM-oriented scales.
Limitations to this study should be noted. First, this study only included parent report of child symptoms. Although more research is needed on the parent version of the Achenbach System of Empirically Based Assessment scales (i.e., the CBCL), the literature on cross-informant agreement suggests that, on average, parents and children agree on symptom presence to a moderate degree (Achenbach, McConaughy, & Howell, Reference Achenbach, McConaughy and Howell1987; De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005). Nevertheless, understanding parents' perspectives on the nature of their child's symptoms and the resulting structure of psychopathology because necessary as multiple-informant assessment is routinely recommended (Barbosa, Tannock, & Manassis, Reference Barbosa, Tannock and Manassis2002; Jensen et al., Reference Jensen, Rubio-Stipec, Canino, Bird, Dulcan and Schwab-Stone1999). Furthermore, the possibility that parents are unable to detect slight differences in anxiety and affect among younger children with less severe psychopathology cannot be ruled out. Thus, longitudinal studies are needed to fully understand the complex interactions between the severity of symptoms, sequence of symptoms, developmental stage, and contextual factors in symptom differentiation between anxiety and depression across development. Second, whereas the clinical sample in this study was composed of clinic-referred youths from two different cities, the nonreferred sample was only composed of youths from Hawaii. It is possible that youths in Hawaii differ in meaningful ways from youths from Massachusetts. Thus, assumptions regarding the differences between the clinical and the nonclinical samples should be made with caution. Third, the present sample size was not large enough to conduct analyses that would elucidate the developmental trajectory of NA differentiation. It is recommended that a larger sample size be utilized in future studies in order to identify the particular age at which the differentiation between depression and anxiety begins to emerge.
Nonetheless, the present study provided additional needed empirical support for the relatively recently constructed CBCL DSM-oriented scales. Despite being rationally derived based on the consensus of raters, the present confirmatory factor analytic findings empirically supported the six-factor structure among younger and older clinic-referred samples. As noted above, the present study also further supports the lack of symptom differentiation of anxiety and depression among younger youths with less severe psychopathology, aiding to resolve the discrepant findings in the literature and also highlighting the need for further research to better understand why this particular subsample of young children tend to be associated with their own unique (unitary) representation of anxiety and depression.
