Three Approaches to Assessing The Validity of Hot and Cool EF Systems

As described by Peterson and Welsh (Reference Peterson and Welsh2014), the current research does not make a compelling case for the separability of the hot and cool forms of EF in childhood, particularly when examining behavioral measures of EF (vs. self-reports and rating scales completed by parents). In younger children (3 to 6 years of age), both hot and cool EF tasks are typically correlated with age and do not show different rates of development or levels of difficulty (Carlson, Reference Carlson2005; Carlson & Wang, Reference Carlson and Wang2007; Hongwanishkul, Happaney, Lee, & Zelazo, Reference Hongwanishkul, Happaney, Lee and Zelazo2005). In contrast, some studies with older children and adolescents support a more protracted development for hot EF (Prencipe et al., Reference Prencipe, Kesek, Cohen, Lamm, Lewis and Zelazo2011). However, across-age comparisons of hot and cool task performance require equivalent difficulty which is difficult to assess.

The construct validity of hot and cool EF also has been examined by means of bivariate correlational patterns and more sophisticated multivariate procedures, such as confirmatory factor analyses and structural equation models. Divergent and convergent correlational patterns would indicate relatively independent constructs, such that cool tasks and hot tasks should significantly intercorrelate within but not across domain. Again, the data are mixed. Recent studies have found correlational patterns that do not align with the expected divergent and convergent validity predictions (Hongwanishkul et al., Reference Hongwanishkul, Happaney, Lee and Zelazo2005; Thorell, Reference Thorell2007). In a meta-analysis of the convergent validity of “self-control” tasks (Delay of Gratification, and thus “hot” measures) and EF tasks (“cool” measures) across development, uniformly low correlations were obtained both within and across task domain (Duckworth & Kern, Reference Duckworth and Kern2011). Regarding confirmatory factor analyses and structural equation models of hot and cool EF on early childhood samples, evidence supports a single factor (Allan & Lonigan, Reference Allan and Lonigan2011; Masten et al., Reference Masten, Herbers, Desjardins, Cutuli, McCormick, Sapienza and Zelazo2012; Sulik et al., Reference Sulik, Huerta, Zerr, Eisenberg, Spinrad, Valiente and Taylor2010), two hot and cool factors (e.g., Brock, Rimm-Kaufman, Nathanson, & Grimm; Reference Brock, Rimm-Kaufman, Nathanson and Grimm2009; Kim, Nordling, Yoon, Boldt, & Kochanska, Reference Kim, Nordling, Yoon, Boldt and Kochanska2013; Willoughby, Kupersmidt, Voegler-Lee, & Bryant, Reference Willoughby, Kupersmidt, Voegler-Lee and Bryant2011), or one- and two-factor models demonstrating equally good fit (Denham, Warren-Khot, Bassett, Wyatt, & Perna, Reference Denham, Warren-Khot, Bassett, Wyatt and Perna2012; Kim et al., Reference Kim, Nordling, Yoon, Boldt and Kochanska2013). It is possible that EF is more unidimensional in early childhood and specializes into separate functions with development (Zelazo & Carlson, Reference Zelazo and Carlson2012). However, research with older children and adolescents has failed to find significant correlations between two hot EF tasks, Iowa Gambling and Delay Discounting (Lamm, Zelazo, & Lewis, Reference Lamm, Zelazo and Lewis2006; Prencipe et al., Reference Prencipe, Kesek, Cohen, Lamm, Lewis and Zelazo2011). Moreover, an exploratory factor analysis (Prencipe et al., Reference Prencipe, Kesek, Cohen, Lamm, Lewis and Zelazo2011) did not identify dissociable cool and hot factors in their adolescent sample.

A third approach to distinguishing hot and cool EF constructs involves exploring the degree to which each gives rise to different developmental outcomes. In young children, Hongwanishkul et al. (Reference Hongwanishkul, Happaney, Lee and Zelazo2005) found that performance on cool EF tasks covaried with intelligence, whereas, hot EF task performance did not, and only the cool task performance correlated with the temperament dimension of effortful control. Consistent with Kim et al. (Reference Kim, Nordling, Yoon, Boldt and Kochanska2013), both Brock et al. (Reference Brock, Rimm-Kaufman, Nathanson and Grimm2009) and Willoughby et al. (Reference Willoughby, Kupersmidt, Voegler-Lee and Bryant2011) found that cool task performance only was related to academic outcomes in young children, while hot effortful control task performance predicted parent-reported behavior problems at age 5. Finally, Thorell (Reference Thorell2007) found that, for 6-year-old children diagnosed with ADHD, performance on hot and cool EF tasks predicted different sets of clinical symptoms. Therefore, the evidence for differential predictive patterns provides somewhat stronger evidence for separable hot and cool EF constructs; however, this research has primarily examined outcomes in early childhood.

Building Conceptual Clarity

The difficulty in establishing an empirical foundation that clearly delineates hot versus cool EF may lie squarely in the lack of conceptual clarity. Indeed, it is only in the last few years that a theoretical discussion has emerged regarding the conceptual overlaps among EF, effortful control, self-control, and self-regulation (Allan & Lonigan, Reference Allan and Lonigan2011; Blair & Ursache, Reference Blair and Ursache2011; Denham et al., Reference Denham, Warren-Khot, Bassett, Wyatt and Perna2012; Duckworth & Kern, Reference Duckworth and Kern2011; Hofmann, Schmeichel, & Baddeley, Reference Hofmann, Schmeichel and Baddeley2012; McClelland & Cameron, Reference McClelland and Cameron2012; Zhou, Chen & Main, Reference Zhou, Chen and Main2012). EF and effortful control share many cognitive components (e.g., inhibition), as well as experimental tasks, and the major difference may be the research traditions from which each evolved: cognitive neuropsychology, and personality/temperament research, respectively (Zhou et al., Reference Zhou, Chen and Main2012). Similarly, the overlap between the domains of self-regulation and EF has been discussed (Hofmann et al., Reference Hofmann, Schmeichel and Baddeley2012; McClelland & Cameron, Reference McClelland and Cameron2012), as well as the facilitation of effective self-regulatory behaviors, such as emotion regulation, by traditional EF processes (e.g., working memory; Hofmann et al., Reference Hofmann, Schmeichel and Baddeley2012). If cool EFs are recruited in “hot” contexts such as emotion regulation (Rueda & Paz-Alonzo, Reference Rueda and Paz-Alonzo2013) and delay of gratification tasks (Anderson & Reidy, Reference Anderson and Reidy2012), teasing apart the two putative constructs will be challenging.

A striking example of this conceptual confusion is evidenced by the differing descriptions of “hot processes” provided by the cognitive-affective model (CAPS, Mischel & Ayduk, Reference Mischel and Ayduk2011; see also Metcalfe & Mischel, Reference Metcalfe and Mischel1999) as compared to hot EF, introduced by Zelazo and Müller (Reference Zelazo and Müller2002). The similarity of language (“hot”) and the shared task, the seminal Delay of Gratification paradigm, led to an inaccurate melding of these two models that are actually quite distinct. The hot processes of CAPS are bottom-up affective, automatic, and stimulus-driven responses to the environment, evident early in development, primarily mediated by the amygdala, and steadily dominated and controlled by the more adaptive cool processes of the prefrontal systems across development. In contrast, hot EF involves top-down, goal-oriented, and deliberative responses to “hot” tasks and situations, shows a protracted period of development, involves the orbitofrontal cortex, and coordinates with cool EF processes, depending on task conditions.

Consistent with the theoretical and conceptual overlap among EF, effortful control, and self-regulation, the domains have shared specific tasks. Indeed, the tasks in Kochanska's (e.g., Reference Kochanska2002) investigations of early compliance and moral behavior of children—delay of gratification and prohibition tasks—are precisely the measures that have been used to assess hot EFs in recent research (Brock et al., 2009). Moreover, many early childhood tasks have been considered “cool” in EF or effortful control research because they do not include an “extrinsic or proximal reward for performance” (Allan & Lonigan, Reference Allan and Lonigan2011, p. 906). However, one could argue that these tasks do have heightened motivational or emotional significance, thus qualifying as hot EF measures (Zelazo & Müller, Reference Zelazo and Müller2002). For example, game-like tasks with strong conflict and prepotent responses, such Heads-Toes-Knees-Shoulders (McClelland & Cameron, Reference McClelland and Cameron2012), Simon Says (Carlson, Reference Carlson2005), Walk-a-Line-Slowly (Allan & Lonigan, Reference Allan and Lonigan2011), and Statue (Klenberg, Korkman, & Lahti-Nuuttila, Reference Klenberg, Korkman and Lahti-Nuuttila2001) are presumed to be cool; however, one need only test young children once on such tasks to appreciate the emotional experience for young children (e.g., giggling, grimacing, hooting, and hollering, particularly when they err). Moreover, different tasks may be relatively warmer or cooler across individuals. As an illustration, Garon, Longard, Bryson, and Moore (Reference Garon, Longard, Bryson and Moore2012) offered a thoughtful, post hoc discussion of their unexpected finding that preschool age boys delayed gratification longer than the girls in their sample. During testing, they observed that the boys did not enjoy interacting with the female experimenter or playing with the stickers as much as the girls. Thus, as the authors noted, the experimental setting may have been less hot for the boys.

The challenge of unequivocally labeling a task as hot or cool has implications for the research findings from which we draw our conclusions. For example, Allan and Lonigan (Reference Allan and Lonigan2011) found that their hot effortful control tasks (delay and conflict tasks with rewards) and cool effortful control tasks (Heads-Toe and Walk-a-Line-Slowly) loaded on one common factor, perhaps reflecting that their cool tasks were “hotter” than they suspected. Alternatively, separable factors for hot and cool EF may reflect other unmeasured non-EF processes (e.g., motor demands; Denham et al., Reference Denham, Warren-Khot, Bassett, Wyatt and Perna2012).

Unresolved Issues in Hot Versus Cool EF

Whereas the past few years has seen an explosion in the theoretical discussion of the intersections among research traditions and domains, thorny issues remain that have substantial implications for developing valid measures of hot EF across childhood and adolescence. The fundamental question remains: What constitutes a hot EF task? It is our position that the answer is: It depends. The “heat” of a task, which presumably varies along a continuum, may depend on age, task demands and context, and individual differences such as temperament and personality. Moreover, we suggest that there are complex interactions among these variables in determining the relative heat of a given task (Somerville & Casey, Reference Somerville and Casey2010). As an example of the current lack of consensus, there is disagreement whether a hot EF task must include rewards or some type of appetitive stimulus (Allan & Lonigan, Reference Allan and Lonigan2011) or that it need only elicit emotional arousal or heightened motivation (Zelazo & Müller, Reference Zelazo and Müller2002).

As reviewed in Peterson and Welsh (Reference Peterson and Welsh2014), an interesting approach to overcoming some of the problems associated with contrasting performance in hot and cool tasks is to manipulate the temperature of a single task (e.g., increasing or decreasing reward salience). This approach enables researchers to contrast performance in the presumably warmer versus cooler conditions while holding cognitive demands and overall difficulty level constant. Following the seminal methods of Mischel (Mischel & Ayduk, Reference Mischel and Ayduk2011), recent research on hot EF modifies the salience of various reward conditions with the “hotter” versions of tasks resulting in impaired performance in young children (Carlson, Davis, & Leach, Reference Carlson, Davis and Leach2005; Lewis, Lamm, Segalowitz, Stieben, & Zelazo, 2006) and greater risk taking in adolescents (Crone, Bullens, van der Plas, Kijkuit, & Zelazo, Reference Crone, Bullens, van der Plas, Kijkuit and Zelazo2008; Figner, Mackinley, Wilkening, & Weber, Reference Figner, Mackinlay, Wilkening and Weber2009). While temperature manipulation studies have certainly advanced our knowledge, the methodology does highlight an assumption that may not always be true. Should decreased task performance be interpreted as reflecting a hotter temperature? While heating a task is expected to impair performance in young children, Somerville and Casey (Reference Somerville and Casey2010) review evidence demonstrating that rewards (“heat”) may impair or facilitate task performance in adolescence depending upon task demands to either focus on, or suppress attention to, these rewards, respectively. Furthermore, it may be that the effect of a given manipulation is influenced by a range of factors such as gender (Garon et al., Reference Garon, Longard, Bryson and Moore2012), age (Lewis et al., Reference Lewis, Lamm, Segalowitz, Stieben and Zelazo2006), and individual differences in personality.

Temperament research has shown that children vary in the degree to which they experience arousal within a given context. In some studies (Kagan, Snidman, Zentner, & Peterson, Reference Kagan, Snidman, Zentner and Peterson1999), children are presented with presumably-cool tasks such as Matching Familiar Figures, requiring reflection rather than impulsive responses. By measuring behavior and physiology, researchers have demonstrated that children vary in their response to an experimental context. In EF research, consider how many presumably cool EF tasks involve rich face-to-face interaction (Simon Says, Statue). It may be fruitful to consider two lessons from the temperament literature: The heat of a context varies across individuals; by combining behavioral and psychophysiological measures, as well as task and context manipulations, we can explore both task and person factors.

Summary

Across the past decade or so, EF researchers have embraced the challenges presented by emotion-laden, real-world contexts. Today, previously disparate fields acknowledge the commonalities across our questions, constructs, and methods. For researchers interested in mechanisms supporting goal-directed behavior both in the laboratory and across the full range of potential contexts, this is an exciting time. Currently, there is consensus that some level of heightened motivation must exist for hot EF processes to be engaged, but here is where the agreement ends. By its very nature, motivation is an individual-difference construct; contextual manipulations that will heighten motivation differ across person and situation, and these factors have not been considered comprehensively in the research to date. Whereas the most well-established hot EF tasks include a “reward,” either immediate or promised, the degree to which a hot task must involve a reward is in dispute, and how one operationalizes “reward” will be task-, person-, and situation-specific. Consistent with the equivocal findings regarding a single- or dual-factor model of hot and cool EF, evidence supports some shared cognitive processes across hot and cool tasks; however, the degree to which cognitive processes contribute uniquely to the two types of tasks remains unclear. Finally, the genesis of the hot versus cool EF discussion began with the neuropsychological evidence of a functional dissociation between two prefrontal cortical regions in adult brain damage cases. Systematic, longitudinal, cross-level studies of children and adolescents will provide a richer understanding of the brain basis of hot and cool EF across development.