Introduction
All children display anger and frustration, especially in early childhood. However, despite the ubiquity of such behaviors, previous work shows that irritable children (i.e., those who are easily annoyed, show persistently irritable mood, and have frequent temper outbursts) are at risk for a range of psychopathologies later in development (Beauchaine & Tackett, Reference Beauchaine and Tackett2019; Brotman, Kircanski, & Leibenluft, Reference Brotman, Kircanski and Leibenluft2017; Leibenluft & Stoddard, Reference Leibenluft and Stoddard2013; Stringaris, Reference Stringaris2011; Stringaris, Cohen, Pine, & Leibenluft, Reference Stringaris, Cohen, Pine and Leibenluft2009). Irritability is also a core feature of disruptive mood dysregulation disorder (DMDD), as well as a diagnostic criterion common to several DSM-5 (Diagnostic and Statistical Manual of Mental Disorders, fifth edition) disorders (e.g., major depressive disorder, oppositional defiant disorder [ODD]). For these reasons, interest in early irritability as a transdiagnostic construct has grown dramatically.
However, this interest has outpaced methodological work on how irritability should be conceptualized and assessed. Much of the work on childhood irritability to date has been conducted using clinical populations; a review of this literature shows extensive variability in terms of how this construct is conceptualized. While some researchers frame irritability as excessive negative reactivity characterized by anger and possibly aggression (Brotman et al., Reference Brotman, Kircanski and Leibenluft2017; Copeland, Brotman, & Costello, Reference Copeland, Brotman and Costello2015; Leibenluft, Reference Leibenluft2011; Leibenluft & Stoddard, Reference Leibenluft and Stoddard2013), others emphasize chronic grumpiness and ill-tempered mood (Craig, Hietanen, Markova, & Berrios, Reference Craig, Hietanen, Markova and Berrios2008). In addition, some investigators distinguish between “phasic” and “tonic” irritability, where phasic irritability refers to acute temper outbursts that peak and decline rapidly and tonic irritability refers to chronic, relatively low-grade angry mood (Beauchaine & Tackett, Reference Beauchaine and Tackett2019; Brotman et al., Reference Brotman, Kircanski and Leibenluft2017; Leibenluft, Reference Leibenluft2017; Moore et al., Reference Moore, Lapato, Brotman, Leibenluft, Aggen, Hettema and Roberson-Nay2019). In contrast, behavioral neuroscience approaches conceptualize irritability in context; that is, as a response to blocked goal attainment or frustrative nonreward (Beauchaine & Tackett, Reference Beauchaine and Tackett2019; Brotman et al., Reference Brotman, Kircanski and Leibenluft2017; Deveney et al., Reference Deveney, Connolly, Haring, Bones, Reynolds, Kim and Leibenluft2013; Tseng et al., Reference Tseng, Moroney, Machlin, Roberson-Nay, Hettema, Carney and Brotman2017), with “irritable” children having a relatively low threshold for exhibiting anger when goal attainment is thwarted or otherwise challenging.
The varying conceptualizations of irritability have, unsurprisingly, given rise to various approaches to its assessment. In studies of child psychopathology, investigators often use items taken from parent-reported child symptom questionnaires or clinical interviews to assess irritability. For example, the widely used Child Behavior Checklist (CBCL; Achenbach, Reference Achenbach1991) is frequently used to assess irritability based on three items from the Aggressive Behavior scale that reflect mood lability (e.g., “sudden changes in mood or feelings”; Evans et al., Reference Evans, Bonadio, Bearman, Ugueto, Chorpita and Weisz2019; Roberson-Nay et al., Reference Roberson-Nay, Leibenluft, Brotman, Myers, Larsson, Lichtenstein and Kendler2015; Savage et al., Reference Savage, Verhulst, Copeland, Althoff, Lichtenstein and Roberson-Nay2015; Stringaris, Zavos, Leibenluft, Maughan, & Eley, Reference Stringaris, Zavos, Leibenluft, Maughan and Eley2012; Wiggins, Mitchell, Stringaris, & Leibenluft, Reference Wiggins, Mitchell, Stringaris and Leibenluft2014). Although these items are included in scales assessing ODD, Stringaris, Goodman, et al. (Reference Stringaris, Goodman, Ferdinando, Razdan, Muhrer, Leibenluft and Brotman2012) argued that they tap aspects of irritability relevant to depressive symptoms in particular and that irritability assessed in this way accounts for the heterotypic continuity between oppositionality and depression, highlighting the transdiagnostic relevance of the construct as measured by these items. Similarly, using the CBCL irritability scale, Wiggins et al. (Reference Wiggins, Mitchell, Stringaris and Leibenluft2014) found associations between longitudinal trajectories of irritability and parental psychopathology, such that children whose parent-reported irritability increased in early childhood were more likely to have a parental history of depression and substance use problems.
Less frequently, ad hoc irritability scales have been derived from semi-structured parent interviews of child psychopathology, such as the Preschool Age Psychiatric Assessment (PAPA; Egger, Ascher, & Angold, Reference Egger, Ascher and Angold1999) and the Child and Adolescent Psychiatric Assessment (CAPA; Angold et al., Reference Angold, Prendergast, Cox, Harrington, Simonoff and Rutter1995). Here, items assessing irritability are drawn from the depression and ODD sections, and reflect both a proneness to experience angry mood and excessive temper episodes (Dougherty et al., Reference Dougherty, Smith, Bufferd, Stringaris, Leibenluft, Carlson and Klein2013). Similar to the literature using parent-report measures of irritability, these studies find both cross-sectional and longitudinal associations child irritability and parent psychopathology. In addition, interview-based irritability indices predict children's later symptoms also assessed via these interviews (Dougherty et al., Reference Dougherty, Smith, Bufferd, Stringaris, Leibenluft, Carlson and Klein2013, Reference Dougherty, Smith, Bufferd, Kessel, Carlson and Klein2015; Vogel, Jackson, Barch, Tillman, & Luby, Reference Vogel, Jackson, Barch, Tillman and Luby2019).
Toward developing a concise measure designed specifically to assess irritability, Stringaris, Goodman, et al. (Reference Stringaris, Goodman, Ferdinando, Razdan, Muhrer, Leibenluft and Brotman2012) developed the Affective Reactivity Index (ARI), a questionnaire consisting of six symptom items and one impairment item about irritability for use with youth between ages 6 and 18. While this brief measure is easy to use for screening purposes in clinical settings, its single-factor structure has not been well-supported (Mulraney, Melvin, & Tonge, Reference Mulraney, Melvin and Tonge2014; Stringaris, Goodman, et al., Reference Stringaris, Goodman, Ferdinando, Razdan, Muhrer, Leibenluft and Brotman2012). That is, certain items show high covariance and may be redundant (i.e., “lose temper easily” and “often lose temper”), although they are intended to assess different aspects of irritability; that is, threshold and frequency of temper loss, respectively. Furthermore, the ARI was specifically designed to be completed by youth and parents, rendering it less useful when the goal is to assess childhood irritability prior to the age when youth can accurately self-report.
The lack of irritability questionnaires focused more specifically on children's emotions and related behavior, rather than psychopathology symptoms, is problematic (Wakschlag et al., Reference Wakschlag, Estabrook, Petitclerc, Henry, Burns, Perlman and Briggs-Gowan2015). The Multidimensional Assessment Profile of Disruptive Behavior (MAP-DB; Wakschlag et al., Reference Wakschlag, Choi, Carter, Hullsiek, Burns, McCarthy and Briggs-Gowan2012), a rare example of a behaviorally focused questionnaire, offers a developmentally sensitive parent-report questionnaire for the measurement of temper dysregulation, but whether it captures irritability as a transdiagnostic construct is unclear. In particular, the Temper Loss scale is composed of 22 items that assess mood dysregulation and tantrums, capturing information about frequency (i.e., number of days in the previous month), quality (e.g., “has a tantrum until exhausted”), interactional context (e.g., loss of temper with parent vs. other adults), and triggers (e.g., “lose temper or have a tantrum to get something he or she wanted”). However, existing work links the temper loss scale to aggression and impulsivity (Wakschlag et al., Reference Wakschlag, Choi, Carter, Hullsiek, Burns, McCarthy and Briggs-Gowan2012), suggesting that this scale may not capture aspects of mood dysregulation relevant to internalizing symptoms.
This literature review, while not exhaustive, captures the diverse perspectives in the field regarding the conceptualization and measurement of irritability. Given the current status of the field, it would be premature to emphasize one conceptualization over another, as studies of the relations between these different conceptualizations and measurement approaches are needed to optimize the valid assessment of irritability. The scant available literature on the convergence of existing measures indicates that different ways of assessing children's irritability have distinct correlates and yield potentially conflicting findings, as is typical of studies of measurement of child psychopathology. For example, Deveney et al. (Reference Deveney, Grasso, Hsu, Pine, Estabrook, Zobel and Briggs-Gowan2019) found that child irritability was differentially related to neural responses to facial stimuli during a dot-probe task depending on whether it was assessed via semi-structured clinical interview or observer ratings. Unfortunately, most studies rely on a single, usually parent-reported index of irritability, although basic psychometric theory favors a multimethod approach, particularly for the assessment of child behavior (Majdandžić & Van Den Boom, Reference Majdandžić and Van Den Boom2007; Pavlova & Uher, Reference Pavlova and Uher2020; Stifter, Dollar, & Cipriano, Reference Stifter, Dollar and Cipriano2011). Further, while affordable and capable of covering child behavior in a broad array of contexts, parent report has several key limitations, including potential biases related to parent disorder (Goodman et al., Reference Goodman, Rouse, Connell, Broth, Hall and Heyward2011), as well as parent personality traits and stress exposure (Clark, Durbin, Donnellan, & Neppl, Reference Clark, Durbin, Donnellan and Neppl2017; De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005). Finally, as noted in our earlier review of studies of child irritability and psychopathology risk, it is standard practice to use extant child symptom measures to derive parent-reported irritability scales, which are in turn used to predict children's symptoms (Dougherty et al., Reference Dougherty, Smith, Bufferd, Stringaris, Leibenluft, Carlson and Klein2013; Evans et al., Reference Evans, Bonadio, Bearman, Ugueto, Chorpita and Weisz2019; Vogel et al., Reference Vogel, Jackson, Barch, Tillman and Luby2019; Wiggins et al., Reference Wiggins, Mitchell, Stringaris and Leibenluft2014). Although overlapping items in measures of irritability and symptoms can be eliminated or otherwise addressed, this approach nevertheless introduces undesirable circularity to the extent that measures of irritability are often drawn from the very same tools used to measure child psychopathology. This approach also introduces common method variance, potentially artificially inflating associations between children's risk factors (e.g., irritability) and outcomes (e.g., psychopathology symptoms).
As previously mentioned, research on child irritability has not typically capitalized on the developmental literature on child temperament and its assessment. Measures of child temperament may tap behaviors potentially more clearly differentiated from clinical symptoms, at least compared to measures of irritability drawn from symptom checklists; thus, these measures may help address the significant conceptual and methodological question of how to best distinguish between traits and clinical symptoms (Goldsmith & Lemery, Reference Goldsmith and Lemery2000). Although developmental approaches to child temperament may provide useful insights and assessment approaches for the study of early irritability, these have rarely been used in the field, despite irritability's strong heritability and stability across development, features that partially define temperament constructs (Clifford, Lemery-Chalfant, & Goldsmith, Reference Clifford, Lemery-Chalfant and Goldsmith2015; Copeland et al., Reference Copeland, Brotman and Costello2015; Moore et al., Reference Moore, Lapato, Brotman, Leibenluft, Aggen, Hettema and Roberson-Nay2019; Riglin et al., Reference Riglin, Eyre, Cooper, Collishaw, Martin, Langley and Thapar2017; Roberson-Nay et al., Reference Roberson-Nay, Leibenluft, Brotman, Myers, Larsson, Lichtenstein and Kendler2015). Temperament research has also yielded observational paradigms designed to provide an ecologically valid, contextual method for capturing normative and atypical variation in children's behavior. Further, standardized laboratory paradigms and coding systems have been developed to provide systematic and objective assessment of children's emotions, independent of informant biases which overcome some of the inherent limitations of parent report (Gagne, Van Hulle, Aksan, Essex, & Goldsmith, Reference Gagne, Van Hulle, Aksan, Essex and Goldsmith2011b). These paradigms also expose children to the same stimuli, thereby reducing situational influences on behavior and emphasizing individual differences in emotion.
Although behavioral paradigms exist for use with children, these, to our knowledge, have not been used to study irritability as a temperamental traitFootnote 1. For example, the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS; Wakschlag, Hill, et al., Reference Wakschlag, Hill, Carter, Danis, Egger, Keenan and Briggs-Gowan2008) has been developed specifically for the assessment of anger regulation and other dimensions of disruptive behavior. The DB-DOS was therefore validated on youth oversampled for significant externalizing problems, raising the concern that it is less appropriate for tapping the full range of behaviors related to irritability. Also, the DB-DOS examines anger only in interpersonal contexts (e.g., Parent×Child interactions) involving anger-provoking stimuli (Wakschlag, Briggs-Gowan, et al., Reference Wakschlag, Briggs-Gowan, Hill, Danis, Leventhal, Keenan and Carter2008; Wakschlag, Hill, et al., Reference Wakschlag, Hill, Carter, Danis, Egger, Keenan and Briggs-Gowan2008); thus ratings of children derived from interactions with the parent may conflate child characteristics with problematic caregiver–child relationships. Also, while considering child behavior during anger-eliciting contexts may index anger that is extreme in its intensity, this approach is limited in its ability to tap other aspects of maladaptive anger expression, such as context-inappropriate anger. This is potentially problematic given that context-inappropiate anger (e.g., anger expressed during tasks designed to be enjoyable) appears to capture difficulties in emotional processing and regulation (Locke, Davidson, Kalin, & Goldsmith, Reference Locke, Davidson, Kalin and Goldsmith2009), a mechanism identified in the pathophysiology of irritability (Brotman et al., Reference Brotman, Kircanski and Leibenluft2017). Consistent with this, context-inapproriate affect, primarly anger, has been associated with children's behavioral and biological maladaptation (Buss, Davis, Ram, & Coccia, Reference Buss, Davis, Ram and Coccia2018; Locke et al., Reference Locke, Davidson, Kalin and Goldsmith2009; Locke, Miller, Seifer, & Heinze, Reference Locke, Miller, Seifer and Heinze2015). Thus, children's observed anger in contexts not designed to elicit this emotion per se may prove to be a valuable index of maladaptive irritability (i.e., irritability expressed in situations in which it is neither adaptive nor expected).
The Laboratory Temperament Assessment Battery (Lab-TAB; Goldsmith, Reilly, Lemery, Longley, & Prescott, Reference Goldsmith, Reilly, Lemery, Longley and Prescott1995) is a well-established observational paradigm designed to assess child temperament across a range of contexts. Lab-TAB episodes are designed to elicit a broad array of children's emotions, in addition to anger. Typically, child temperament indices from the Lab-TAB are derived from relevant behavior during episodes intended to elicit the emotion of interest only (Gagne, Van Hulle, Aksan, Essex, & Goldsmith, Reference Gagne, Van Hulle, Aksan, Essex and Goldsmith2011a; Goldsmith et al., Reference Goldsmith, Reilly, Lemery, Longley and Prescott1995), although our group and others have derived child temperament indices based on aggregated ratings of the emotion of interest across all Lab-TAB episodes, regardless of the emotion targeted by individual tasks (Durbin, Hayden, Klein, & Olino, Reference Durbin, Hayden, Klein and Olino2007; Hayden, Klein, & Durbin, Reference Hayden, Klein and Durbin2005; Mackrell et al., Reference Mackrell, Sheikh, Kotelnikova, Kryski, Jordan, Singh and Hayden2014). Both approaches have advantages; while the former approach benefits from the use of tasks in which the relevant behavior occurs frequently, the latter strategy is better equipped to capture child behavior across a variety of contexts. However, neither approach captures child anger expressed during non-provocative or neutral contexts, although such contexts may be especially useful toward identifying children with a low threshold for experiencing anger, an important aspect of irritability. For example, in late childhood, observer-rated context-inappropriate anger has been related to variations in children's cortisol stress reactivity and externalizing problems above and beyond what is predicted by context-appropriate anger (Locke et al., Reference Locke, Davidson, Kalin and Goldsmith2009, Reference Locke, Miller, Seifer and Heinze2015), suggesting the potential value of this approach to defining childhood irritability. Thus, in the current study, we operationalized irritability as child anger expressed during Lab-TAB tasks designed to elicit emotions and traits other than anger (i.e., tasks in which anger displays were somewhat unusual and contextually inappropriate).
The current study had several goals. With the goal of providing descriptive information for future studies aimed at optimizing the assessment of irritability, we examined associations between children's symptoms andirritability conceptualized as individual differences in the tendency to express anger with little or no provocation, assessed using observational ratings from the Lab-TAB. This index of irritability was contrasted with children's anger expressed during tasks specifically designed to evoke anger. We referenced these approaches to the CBCL irritability scale, one of the most widely used measures of childhood irritability (Evans et al., Reference Evans, Bonadio, Bearman, Ugueto, Chorpita and Weisz2019; Stringaris, Zavos, et al., Reference Stringaris, Zavos, Leibenluft, Maughan and Eley2012), and also included the anger scale from the Child Behavior Questionnaire (CBQ; Rothbart, Reference Rothbart1989). The CBQ uses parent report to assess reactive and self-regulatory aspects of childhood temperament, with the anger scale particularly tapping reactivity to blocked goal attainment (Rothbart, Reference Rothbart1989; Rothbart, Ahadi, Hershey, & Fisher, Reference Rothbart, Ahadi, Hershey and Fisher2001). While widely used to assess child temperament, the CBQ, specifically the anger scale, has not been previously used to measure childhood irritability to the best of our knowledge. We used these indices to provide broad, descriptive information concerning associations between different assessment approaches to childhood irritability at age 3 and children's internalizing and externalizing symptoms at ages 5 and 8 years.
Method
Participants
A sample of 409 families was recruited from the community using advertisements placed in local daycares, preschools, recreational facilities, and the University of Western Ontario's developmental participants pool to participate in a longitudinal study of children's development. At baseline (T1), children were 3 years old (M = 3.43; SD = .30; 208 girls), had at least one biological parent who could participate in the study, and were free of significant medical and psychological problems. Children were also screened using the Peabody Picture Vocabulary Test (PPVT; Dunn & Dunn, Reference Dunn and Dunn1997), and were of average cognitive ability (M = 112.00; SD = 14.05). Families were predominantly White (93.2%), and the majority were in the middle class: 15.4% reported an annual family income less than CAD $20,000, 53.3% reported an annual income between CAD $40,000 and CAD $100,000, and 31.4% reported an annual income greater than CAD $100,000. Most of the children came from two-parent homes (87.6%); 64.8% of mothers and 95% of fathers had a job that required them to work outside the home. At baseline, mothers’ mean age was 33.3 years (SD = 4.62 years) and 78.6% had attained college- or university-level education. Similarly, fathers’ mean age was 35.01 years (SD = 4.89 years); 71.1% had attained college or university level education. These demographic characteristics are consistent with that of the population of Southwestern Ontario from which participants were recruited (Statistics Canada, Reference Statistics Canada2011). Study procedures were approved by the University of Western Ontario's Research Ethics Board. The primary caregiver provided consent for their own and their child's participation.
Procedure
Data used here were collected over a 6-year period. At T1, children completed observational tasks in the laboratory, described below. In addition, the primary caregiver (mothers for 93% of children) completed questionnaires assessing child symptoms and temperament. Primary caregiver report on child symptoms was again collected 2.5 (T2; N = 394; mean child age = 5.93 years), and 5 years (T3; N = 365; mean child age = 8.59) after baseline.
Measures
Parent-reported irritability
We indexed irritability in four different ways, two derived from parent report and two derived from observational ratings. As has been done in previous studies of childhood irritability (Evans et al., Reference Evans, Bonadio, Bearman, Ugueto, Chorpita and Weisz2019; Stringaris, Goodman, et al., Reference Stringaris, Goodman, Ferdinando, Razdan, Muhrer, Leibenluft and Brotman2012), three items (“temper tantrums or hot temper,” “stubborn, sullen or irritable,” and “sudden changes in mood or feelings”) from the CBCL (Achenbach, Reference Achenbach1991) were used to create a brief irritability scale (α = .76). We also used the anger/frustration subscale of the Children's Behavior Questionnaire (CBQ; Rothbart, Reference Rothbart1989), in which parents rate their child's characteristic reactions to circumscribed situations on a 7-point scale. The anger/frustration subscale (N items = 13; α = .79) is part of the higher-order factor negative affectivity (Rothbart et al., Reference Rothbart, Ahadi, Hershey and Fisher2001) and includes items such as “gets angry when told to go to bed,” “has temper tantrums when s/he doesn't get what s/he wants,” and “gets angry when s/he can't find something s/he wants to play with.”
Observational measures
Children participated in the Lab-TAB (Goldsmith et al., Reference Goldsmith, Reilly, Lemery, Longley and Prescott1995). The Lab-TAB consists of 12 emotionally evocative tasks, each two to six minutes long, designed to elicit behaviors relevant to child temperament (see Durbin et al., Reference Durbin, Hayden, Klein and Olino2007; Johnson et al., Reference Johnson, Olino, Klein, Dyson, Bufferd, Durbin and Hayden2016; Mackrell et al., Reference Mackrell, Sheikh, Kotelnikova, Kryski, Jordan, Singh and Hayden2014). Specifically, each Lab-TAB task, described subsequently, is designed to elicit an emotion such as positive affect, sadness/anger, or fear, although children typically exhibit multiple emotions during each task regardless of its nature. Children's behavior during these tasks was video recorded and subsequently coded for facial, vocal, and bodily expressions of affect. Full coding procedures are available from the first author upon request. Briefly, for each episode, instances of facial, vocal, and bodily positive affect, sadness, fear, and anger were recorded by trained coders. As examples, eyebrows being drawn together, irritable or cranky tone of voice, and forceful movements such as stomping feet would be coded as expressions of anger. For the different expressions of affect in each episode, each instance of relevant facial, vocal, and bodily behavior was coded as low (multiplied by 1), moderate (multiplied by 2), or high (multiplied by 3). Ratings for each affect were then aggregated across selected episodes to derive scales of interest as described below. Undergraduate, postbaccalaureate, and graduate student raters were trained by a “master” coder and had to reach 80% agreement before coding independently. Intermittent reliability checks were conducted to maintain an intraclass correlation (ICC) of .80 and prevent coder drift. Coders were blind to information concerning children's psychopathology symptoms and parent-reported temperament.
Although the Lab-TAB is designed to tap multiple aspects of child emotionality, we know of no study using it to assess irritability; thus, there is no extant irritability scale derived from the Lab-TAB. Some operationalizations of irritability emphasize a low threshold for experiencing anger (Brotman et al., Reference Brotman, Kircanski and Leibenluft2017; Toohey & DiGiuseppe, Reference Toohey and DiGiuseppe2017); we therefore chose to index irritability based on child behavior during nine non-frustrative Lab-TAB tasks in which anger was not the emotion the task was intended to elicit, thereby tapping anger that is context-inappropriate or expressed in response to little-to-no provocation (e.g., annoyance in response to waiting for a toy). This operationalization is also consistent with the methodology used in work that demonstrates associations between context-inappropriate anger during the Lab-TAB and other indices of emotion dysregulation (Locke et al., Reference Locke, Davidson, Kalin and Goldsmith2009, Reference Locke, Miller, Seifer and Heinze2015). While these studies focused specifically on anger during episodes designed to elicit positive affect, we aggregated affect ratings across a wider range of nonprovoking tasks; thus, our first observational index of irritability was an aggregate of ratings of facial, vocal, and bodily anger across tasks designed to elicit nonanger emotions (N items = 81; α = .78). We refer to this scale as Lab-TAB irritability.
Lab-TAB irritability
The nine tasks used for this scale are listed below, along with the intended target emotion and typical length of the episode:
Risk room (fear; 7 min). The child is left alone in a room with a set of novel and ambiguous stimuli (e.g., a short staircase, a balance beam, mattress). The child is allowed to play freely with these items for 5 min, after which the experimenter returns to the room and asks the child to approach each object.
Tower of patience (effortful control; 6 min). The child and the experimenter take turns building a tower using carboard blocks. The experimenter follows a schedule of delays of increasing length before placing her block such that the child has to wait to take his or her turn.
Puzzle with parent (positive affect; 4 min). In this parent–child play task, the dyad are presented with block puzzle pieces that can be solved in multiple ways, and pictures of six completed puzzles. The dyad is instructed to work together as a team to arrange the blocks to make each of the puzzles.
Stranger approach (fear; 2 min). The child is left alone in the room with a toy. After a few moments, a friendly male research assistant unknown to the child enters the room and speaks to the child while slowly walking closer.
Make that car go (positive affect; 5 min). The experimenter and the child race with two remote-controlled race cars.
Pop-up snakes (positive affect; 4 min). The experimenter introduces the child to what appears to be a can of potato chips, actually containing coiled spring snakes. The child is encouraged to surprise his or her caregiver with the snakes.
Jumping spider (fear; 2 min). The child is introduced to a terrarium containing a fuzzy, fake, black spider. The experimenter asks the child to touch the spider. As the child approaches it, the experimenter manipulates the spider using an attached wire making it appear to jump. At the end of several trials, the experimenter shows the child that it was a fake spider.
Snack delay (effortful control; 5 min). The child is seated in front of desirable snacks and told to wait for a signal before eating them.
Popping bubbles (positive affect; 6 min). The child and experimenter play with a bubble-shooting toy.
Lab-TAB anger
As a basis for comparing the utility of the above-described irritability scale based on child behavior during nonanger-eliciting tasks, and following common practice (Gagne et al., Reference Gagne, Van Hulle, Aksan, Essex and Goldsmith2011b; Goldsmith et al., Reference Goldsmith, Reilly, Lemery, Longley and Prescott1995), we used a second observational index of child irritability based on behavior during tasks specifically designed to elicit child anger (N items = 27; α = .74). The tasks used to form this scale were:
Transparent box (anger; 4 min). The child is brought to a locked transparent box that contains an appealing toy. The experimenter hands the child a set of inoperable keys to use to open the box. The experimenter returns after a brief interval, explaining that she had given the child the wrong keys and allows the child to unlock the box and play with the toy.
Impossibly perfect green circles (anger; 2 min). The child is asked to draw a green circle on a piece of paper and is mildly criticized by the experimenter and told repeat the drawing several times.
Box empty (anger; 7 min). The child is given an elaborately wrapped box that is in fact empty, under the pretense that it conains an appealing toy and is left alone to open it. After a brief interval, the experimenter returns with several small toys for the child to keep, explaining that she forgot to put the toy in the box.
Child symptoms
We used scales drawn from the CBCL, excluding items that constituted the irritability scale to prevent redundancy. The primary caregiver completed the preschool version of the CBCL at T1 and T2 (i.e, child ages 3 and 5; Achenbach, Reference Achenbach1992) and the school-aged version at T3 (i.e., child age 8; Achenbach, Reference Achenbach1991). The CBCL asks the parent to rate the frequency and intensity of their child's emotional and behavioral problems over the past 6 months. We used scales indexing broadband internalizing (Nitems = 29), and externalizing problems (Nitems = 33). Alphas for these scales across the three waves of data collection waves ranged from .78 to .88. We also used the relatively short, empirically derived subscales of CBCL items relevant to specific disorders (Lengua, Sadowski, Friedrich, & Fisher, Reference Lengua, Sadowski, Friedrich and Fisher2001). The specific subscales were: depression (N items = 12; α = .52–.71), ODD (N items = 3; α = .50–.70; AIC = .41–.50Footnote 2), attention problems/hyperactivity ( attention-deficit/hyperactivity disorder (ADHD); N items = 3; α = .68–.75; AIC = .25–.45), and conduct problems (conduct disorder [CD]; N items = 17; α = .76–.80).
Statistical analyses
We first examined bivariate associations between all major study variables. To examine the utility of indices of irritability in predicting children's risk, we used linear hierarchical regressions predicting children's broadband internalizing and externalizing symptoms, as well as children's depressive, ODD, attention problems/hyperactivity, and CD symptoms at child ages 5 and 8 years. The equivalent baseline symptom measure was entered in the first step as a covariate (e.g., age-3 depressive symptoms when predicting age-5 or age-8 depressive symptoms), followed by parent-reported irritability measures in the second step.Footnote 3 To determine the whether our Lab-TAB irritability index had incremental validity over Lab-TAB anger, these observational indices were entered in separate steps. That is, Lab-TAB anger was entered in the third step, and finally, Lab-TAB irritability in the fourth step. In total, we conducted 12 regression analyses. We did not apply a p value adjustment given the broad descriptive goals of our analyses, which are intended to provide comprehensive information concerning irritability–symptom associations with the goal of informing future assessment practices in this field. All analyses were conducted on SPSS v.24.
Results
Descriptive statistics and bivariate correlations
Descriptives of all major study variables and their bivariate correlations with one another are presented in Tables 1 and 2. Parent-reported irritability on the CBCL and the CBQ anger scale were highly correlated, as were Lab-TAB irritability and Lab-TAB anger (Table 1). CBQ anger was weakly correlated with Lab-TAB irritability and Lab-TAB anger, and associations between the CBCL irritability scale and observational indices were absent.
Bivariate correlations among measures of irritability
Note. ** p < .01; * p < .05; Child sex: boys = 1, girls = 2; CBCL = Child Behavior Checklist; CBQ Ang/Frust = anger/frustration subscale of the Child Behavior Questionnaire; PPVT = Peabody Picture Vocabulary Test.
Bivariate correlations among major study variables
Note. ** p < .01; * p < .05; CBCL = Child Behavior Checklist; CBQ Ang/Frust. = anger/frustration subscale of the Child Behavior Questionnaire; Int = Internalizing; Ext = Externalizing; ODD = oppositional defiant disorder; ADHD = attention-deficit hyperactivity disorder; CD = conduct disorder.
Broadband internalizing and externalizing problems at child ages 3, 5, and 8 years were correlated with questionnaire measures of irritability. However, only externalizing problems were associated with observational indices of irritability; that is, Lab-TAB irritability and Lab-TAB anger. With respect to specific symptom scales, correlations with parent-reported irritability (i.e., CBCL irritability and CBQ anger/frustration) ranged from small to large (Cohen, Reference Cohen1988), and were significant for all syndrome scales assessed (i.e., depression, ODD, attention problems/hyperactivity, and CD). Symptom scales were weakly correlated with Lab-TAB irritability and Lab-TAB anger.
Hierarchical regressions
Broadband internalizing/externalizing symptoms
The CBQ, but not the CBCL irritability scale, predicted broadband internalizing symptoms at child age 5, even after controlling for age-3 symptoms (Table 3). Observational indices of irritability were not associated with age-5 internalizing symptoms. Parent-reported and observational indices of irritability were unrelated to broadband internalizing symptoms at age 8 (Table 3).
Hierarchical linear regression analyses of irritability measures predicting children's internalizing problems at ages 5 and 8 years
Note. ** p < .01; * p < .05; CBCL = Child Behavior Checklist; CBQ Ang/Frust. = anger/f.
Similarly, the CBQ anger/frustration subscale predicted children's broadband externalizing problems at age 5, but not age 8 (Table 4). The CBCL irritability scale was associated with age-5 externalizing problems, although this relationship was negative and became nonsignificant when Lab-TAB irritability was added to the model. Although the CBCL irritability scale was significantly correlated with age-8 broadband externalizing symptoms (Table 2), it did not predict symptoms when entered with the CBQ anger/frustration in the same step. While neither observational measure (i.e., Lab-TAB anger and irritability) predicted age-5 broadband externalizing problems, the LAB-TAB irritability scale contributed to the prediction of age-8 externalizing problems, above and beyond parent-report (Table 4).
Hierarchical linear regression analyses of irritability measures predicting children's externalizing problems at ages 5 and 8 years
Note. ** p < .01; * p < .05; CBCL = Child Behavior Checklist; CBQ Ang/Frust. = anger/frustration subscale of the Child Behavior Questionnaire; Ext = Externalizing symptoms.
Specific syndrome scales
CBQ anger/frustration subscale predicted age-5 depressive symptoms, even when observational indices of irritability were added to the model (Table 5). Lab-TAB irritability predicted age-5 depressive symptoms above and beyond parent-report, whereas Lab-TAB anger did not. At age 8, the CBCL irritability scale, but not the CBQ, predicted depressive symptoms (Table 6), despite significant zero-order correlations (Table 2). Lab-TAB anger did not contribute to the prediction of age-8 depressive symptoms (i.e., Step 3); however, when Lab-TAB irritability was added (Step 4), both Lab-TAB anger and irritability predicted age-8 depressive symptoms above and beyond parent report, although in opposite directions. That is, Lab-TAB anger was negatively associated with age-8 depressive symptoms, likely reflecting a suppression effect,Footnote 4 while symptoms were positively correlated with Lab-TAB irritability.
Hierarchical linear regressions of irritability measures predicting children's later symptoms of specific disorders at age 5 years
Note. ** p < .01; * p < .05; CBCL = Child Behavior Checklist; CBQ Ang/Frust. = anger/frustration subscale of the Child Behavior Questionnaire; ODD = oppositional defiant disorder; ADHD = attention-deficit hyperactivity disorder; CD = conduct disorder.
Hierarchical linear regressions of irritability measures predicting children's symptoms of specific disorders at age 8 years
Note. ** p < .01; * p < .05; CBCL = Child Behavior Checklist; CBQ Ang/Frust. = anger/frustration subscale of the Child Behavior Questionnaire; ODD = oppositional defiant disorder; ADHD = attention-deficit hyperactivity disorder; CD = conduct disorder.
The CBQ anger/frustration, but not the CBCL, predicted age-5 ODD, ADHD, and CD symptoms (Table 5). Although Lab-TAB anger showed significant zero-order correlations with age-5 ADHD symptoms, it did not contribute to models predicting any of the specific externalizing symptom scales at age 5 above and beyond parent-report. Lab-TAB irritability uniquely predicted all specific externalizing syndromes, above and beyond parent-report and Lab-TAB anger. For age-8 symptoms, parent-report measures of irritability were unrelated to ODD, ADHD, and CD symptoms (Table 6). Lab-TAB anger did not contribute to the prediction of these symptom scales. Lab-TAB irritability predicted age-8 ODD and ADHD symptoms at age 8, above and beyond parent report and Lab-TAB anger (Table 6).
Discussion
Our findings provide initial support for observed irritability, conceptualized as a low threshold for anger/context-inappropriate anger, as a predictor of later child depressive symptoms and externalizing problems above and beyond other measures of irritability, including parent report. Moreover, we found that associations between irritability and children's symptoms were often dependent on the measure used to assess irritability. This pattern is unsurprising based on similar multimethod studies of child behavior (Hayden et al., Reference Hayden, Klein and Durbin2005; Majdandžić & Van Den Boom, Reference Majdandžić and Van Den Boom2007); however, given that current research on irritability has relied heavily on parent report, our findings highlight the need for dedicated study of the construct and convergent validity of current measures of irritability. Our findings also provide preliminary support for the notion that child irritability expressed in non-provocative or neutral contexts may be especially relevant to youth adjustment compared to more normative expressions of anger that are context-appropriate. Irritability assessed using this approach was associated with depressive, ODD, ADHD, and CD symptoms in regression models, even after accounting for symptoms’ associations with several other indices of irritability.
Observational measures of children's context-inappropriate anger demonstrated incremental validity above and beyond parent report in predicting children's symptoms. In particular, preschoolers’ observed irritability, expressed in response to little-to-no provocation, explained additional variance in depressive (2%) and hyperactivity (30%) symptoms, particularly at age 8 years, even after removing the variance in these symptoms predicted by other measures of irritability and the same symptoms assessed at baseline. These findings are consistent with previous work demonstrating that laboratory temperament paradigms capture aspects of behavior that are not readily detected by questionnaire measures or by parents as informants, and that observational indices have unique associations with children's psychopathology symptoms and risk (Hayden et al., Reference Hayden, Klein and Durbin2005; Karp, Serbin, Stack, & Schwartzman, Reference Karp, Serbin, Stack and Schwartzman2004).
Across measures of irritability, we found associations with both internalizing and externalizing problems, supporting the transdiagnostic relevance of this construct (Beauchaine & Tackett, Reference Beauchaine and Tackett2019; Brotman et al., Reference Brotman, Kircanski and Leibenluft2017; Dougherty et al., Reference Dougherty, Smith, Bufferd, Kessel, Carlson and Klein2015). However, we found that our observational measure of irritability consistently predicted depressive and, to a greater extent, hyperactivity symptoms, respectively explaining 2% and 30% of the variance in these symptoms. Although irritability characterizes several disorders, researchers have questioned whether its phenomenology varies between disorders. For example, Leibenluft et al. (Reference Leibenluft, Cohen, Gorrindo, Brook and Pine2006) demonstrated that tonic irritability predicted ADHD and depressive disorders whereas phasic irritability was linked to phobias and mania. In contrast, Copeland et al. (Reference Copeland, Brotman and Costello2015) found little evidence for the distinction between tonic and phasic irritability components in normative samples. Stringaris, Goodman, et al. (Reference Stringaris, Goodman, Ferdinando, Razdan, Muhrer, Leibenluft and Brotman2012) demonstrated that the irritability component of childhood ODD accounted for its heterotypic continuity with later depressive problems, in that it may manifest differently in these disorders. Similarly, in clinical samples of children with ADHD, varying operationalizations of irritability predicted different patterns of comorbidity (Mick, Spencer, Wozniak, & Biederman, Reference Mick, Spencer, Wozniak and Biederman2005).
This mixed literature illustrates the need for research dedicated to comparing the validity of different conceptualizations as well as different assessment approaches to irritability. That is, the transdiagnostic nature of irritability has been argued such that irritability leads to increases in diverse symptoms of psychopathology (Beauchaine & Tackett, Reference Beauchaine and Tackett2019; Brotman et al., Reference Brotman, Kircanski and Leibenluft2017); however, it is unclear how the strength of irritability–symptom associations vary based on how the construct is defined and assessed. Systematic study of this issue is needed to permit strong conclusions concerning irritability's transdiagnostic nature.
Distinguishing irritability from near-neighbor constructs such as anger, aggression, and general negative affectivity is challenging (Avenevoli, Blader, & Leibenluft, Reference Avenevoli, Blader and Leibenluft2015; Beauchaine & Tackett, Reference Beauchaine and Tackett2019; Malhi, Bell, & Outhred, Reference Malhi, Bell and Outhred2019; Toohey & DiGiuseppe, Reference Toohey and DiGiuseppe2017). In this study, we distinguished between irritability and context-appropriate anger; that is, we compared anger responses to little-or-no provocation with observed anger in frustrating tasks that are expected to elicit this emotion. Our findings suggest that considering the context in which irritability occurs may be useful in conceptualizing this construct; in the current study, behavioral expressions of irritability disproportionate to the context were more consistently linked to children's later symptoms compared to anger expressed during tasks designed to elicit this emotion. This finding complements and builds upon previous work (Locke et al., Reference Locke, Davidson, Kalin and Goldsmith2009) linking context-inappropriate anger to children's cortisol expression, assessed concurrently. Although our battery of tasks was not designed specifically to differentiate between irritability and closely related constructs, an important goal for future research, our findings suggest that child anger expressed in situations that are not inherently anger-provoking may be especially relevant to maladaptive expressions of irritability. Future studies using factor analytic approaches may be useful in determining the extend to which irritability expressed in nonprovocative contexts, irritability that is context-appropriate, anger, and other facets of negative affectivity are distinguishable.
While observed irritability predicted children's later depressive and hyperactivity symptoms, symptom associations with parent report were mixed. The CBCL irritability scale predicted age-8 depressive symptoms only, while the CBQ anger scale predicted age 5, but not age 8, depressive, ODD, hyperactivity, and conduct symptoms. The CBCL irritability scale has been used widely in the literature to predict later symptoms, often assessed using the CBCL as well (Evans et al., Reference Evans, Bonadio, Bearman, Ugueto, Chorpita and Weisz2019; Savage et al., Reference Savage, Verhulst, Copeland, Althoff, Lichtenstein and Roberson-Nay2015; Stringaris, Goodman, et al., 2012). Surprisingly, we found very few associations between CBCL irritability and CBCL symptoms in our study; rather, the CBQ anger scale tended to account for more variance in later CBCL symptoms than the irritability scale derived from the CBCL itself. Although the CBQ and CBCL scales had bivariate correlations with later symptoms of comparable magnitude, it may be that the CBQ scale, as a measure of temperament, is better equipped than the CBCL to capture broad variation in irritability in typically developing children. It is also possible that associations between the CBCL irritability scale and symptoms reported in the literature (Evans et al., Reference Evans, Bonadio, Bearman, Ugueto, Chorpita and Weisz2019; Savage et al., Reference Savage, Verhulst, Copeland, Althoff, Lichtenstein and Roberson-Nay2015; Stringaris, Goodman, et al., Reference Stringaris, Goodman, Ferdinando, Razdan, Muhrer, Leibenluft and Brotman2012) reflect variance attributable to the measurement tool, rather than veridical associations between irritability and outcomes, even with accounting for overlapping items. If so, including a superior irritability measure in multivariate models would have reduced the predictive power of the CBCL irritability scale.
The low convergence we found between observed and parent-reported irritability is unsurprising, and consistent with previous work (Durbin & Wilson, Reference Durbin and Wilson2012; Gagne et al., Reference Gagne, Van Hulle, Aksan, Essex and Goldsmith2011b; Gartstein & Marmion, Reference Gartstein and Marmion2008; Majdandžić & Van Den Boom, Reference Majdandžić and Van Den Boom2007; Pavlova & Uher, Reference Pavlova and Uher2020; Stifter, Willoughby, & Towe-Goodman, Reference Stifter, Willoughby and Towe-Goodman2008). Extant work supports the utility of multimethod, multi-informant approaches in the assessment of psychopathology, citing unique predictive contributions from these various sources (Pavlova & Uher, Reference Pavlova and Uher2020). Nevertheless, despite meager intercorrelations between our four measures of irritability, there were consistent associations between these and child symptoms, particularly in the prediction of depressive and hyperactivity symptoms. The observation of consistent patterns despite lack of convergence between measures supports the importance of irritability for child adjustment, and renders focused psychometric study of the construct especially compelling.
The current study has several strengths, including the use of multiple methods of assessing irritability, ecologically valid laboratory tasks, and a longitudinal design. In addition, our sample was fairly large with minimal attrition over the 6-year data collection period. However, our study also had a number of limitations. Perhaps the most serious of these is our use of measures not specifically designed to assess irritability, although one (the three-item CBCL scale) is possibly the most widely used approach in the field to date. While our use of extant measures has benefits (e.g., the Lab-TAB is a widely used child temperament measure with well-developed coding procedures), this likely constrained our ability to answer important questions about how irritability is best operationalized and assessed. That is, our methodology precludes conclusions about which conceptualization of irritability is most valid; studies directly comparing various conceptualizations are needed to accomplish this important goal. Similarly, the development of new measures of both parent-reported and observed child irritability “from the ground up” is almost certainly needed. However, we note that our current goal was to generate preliminary hypotheses regarding the nature of the construct and its assessment in service of the development of new measures. Our finding that atypical anger expressions are more consistently associated with children's later psychopathology may be useful in ongoing work on this important issue.
We studied early irritability as a predictor of symptoms of internalizing and externalizing disorders, rather than its association with actual diagnoses in later childhood. This approach is justified considering the low-risk nature of our sample and literature demonstrating the continuity between normative and maladaptive traits in relation to psychopathology (Clark, Reference Clark2005; Krueger & Piasecki, Reference Krueger and Piasecki2002; Wakschlag et al., Reference Wakschlag, Estabrook, Petitclerc, Henry, Burns, Perlman and Briggs-Gowan2015). We also note that families in this study were mostly in the middle to upper-middle class, parents were highly educated, and children were of average to above average cognitive ability; these demographic characteristics may limit the generalizability of our findings and replication in high-risk samples where clinical diagnostic measures may be more appropriately used is an important direction for future work. Finally, we did not correct for multiple tests, given that the goal of our descriptive study was to provide broad information characterizing associations between different irritability assessment tools and youth symptoms.
Our findings corroborate previous work underscoring the importance of multimethod approaches in the study of child behavior. In particular, we demonstrate the promise of observational laboratory measures of irritability in predicting children's adaptation. Importantly, despite limitations of the current assessment approaches, our work lends additional support for the role of irritability in shaping youth outcomes. The field will benefit greatly from extensive psychometric work dedicated to the study of different conceptualizations and assessment approaches to irritability.
Acknowledgments
The authors would like to that all the families who participated in this study, especially for their continued involvement in the Child Development Project.
Funding Statement
This research was supported by grants from Canadian Institutes of Health Research (CIHR) (MOP 86458), the Children's Health Research Institute (CHRI), and Ontario Ministry of Colleges and Education.
Conflicts of Interest
None
