Introduction
TBI and Deficits in Awareness
Traumatic brain injury (TBI) is sustained by approximately 1.7 million Americans yearly, and leaves an estimated 5.3 million people living with disabilities (Faul, Xu, Wald, & Coronado, Reference Faul, Xu, Wald and Coronado2010). Individuals with TBI often experience persisting cognitive impairments in the domains of attention, learning and memory, perception, and executive functioning that are largely resistant to rehabilitation efforts (Prigatano & Fordyce, Reference Prigatano and Fordyce1986; Whyte & Rosenthal, Reference Whyte and Rosenthal1993).
One additional consequence following TBI can be reduced awareness, or the capacity to reflect upon one's own condition. The level of awareness maintained by individuals who have sustained TBI has important consequences for patient outcome and may influence the development of psychopathology, perceived quality of life, feelings of subjective well-being, vocational potential, adherence to rehabilitation efforts (e.g., medication compliance), and return to independent living (Evans, Sherer, Nick, Nakase-Richardson, & Yablon, Reference Evans, Sherer, Nick, Nakase-Richardson and Yablon2005; Flashman & McAllister, Reference Flashman and McAllister2002; Godfrey, Partridge, Knight, & Bishara, Reference Godfrey, Partridge, Knight and Bishara1993; Kervick & Kaemingk, Reference Kervick and Kaemingk2005; Ownsworth & Fleming, Reference Ownsworth and Fleming2005; Prigatano, Reference Prigatano1997; Sherer et al., Reference Sherer, Hart, Nick, Whyte, Thompson and Yablon2003; Trahan, Pépin, & Hopps, Reference Trahan, Pépin and Hopps2006). For these reasons, understanding self-appraisal following TBI has important clinical implications.
Anosognosia is a term coined by Babinski to reflect diminished self-awareness and, in particular, failure to recognize personal disability (Babinski, Reference Babinski1914); it has since received widespread application in clinical populations to describe unawareness of illness (Barrett, Eslinger, Ballentine, & Heilman, Reference Barrett, Eslinger, Ballentine and Heilman2005; Goldenberg, Müllbacher, & Nowak, Reference Goldenberg, Müllbacher and Nowak1995; Heilman, Barrett, & Adair, Reference Heilman, Barrett and Adair1998; Lebrun, Reference Lebrun1987; Leritz, Loftis, Crucian, Friedman, & Bowers, Reference Leritz, Loftis, Crucian, Friedman and Bowers2004; McGlynn & Schacter, Reference McGlynn and Schacter1997; Pia & Tamietto, Reference Pia and Tamietto2006; Rickelman, Reference Rickelman2004; Wagner, Spangenberg, Bachman, & O'Connell, Reference Wagner, Spangenberg, Bachman and O'Connell1997). Implicit to this literature is that awareness is multi-faceted; Amador et al. (Reference Amador, Strauss, Yale, Flaum, Endicott and Gorman1993) found patients to have variability in the level of insight held about different aspects of illness (awareness of mental disorder, effects of medication, and social consequences of mental disorder). Specific to TBI, Fleming and Strong (Reference Fleming and Strong1995) proposed three levels or areas in which impairments of self-awareness can manifest: awareness of injury related deficits (which can include cognitive impairments), awareness of functional implications of deficits, and awareness to set realistic goals or predict prognosis. The current study focuses on awareness of cognitive performance after TBI, specifically, decrements in metacognition.
Metacognition
As defined early on by Flavell, the term “metacognition” describes one's ability to track their cognitive processing and includes both knowledge and experience (Flavell, Reference Flavell1979). Metacognitive knowledge refers to cumulative beliefs that can guide cognitive pursuits; while metacognitive experiences pertain to monitoring and control processes that occur in the moment of cognitive engagement (Flavell, Reference Flavell1979). For the purpose of this study, we focus on the latter, that is, the ability to monitor one's performance and the recognition of on-line performance during information processing.
One well established theoretical model of metacognition maintains that this higher order cognition includes a “meta” level that monitors and controls processes that are occurring at an “object” level (Nelson & Narens, Reference Nelson and Narens1990). Metacognitive experience can be measured objectively by making comparisons between actual performance and the individual's judgments of performance collected before (prospective monitoring) or after performing (retrospective monitoring) a task (for a review of these methods, see Dunlosky & Metcalfe, Reference Dunlosky and Metcalfe2009). Typically, these judgments include ease of learning (EOL), judgment of learning (JOL), feeling of knowing (FOK), and retrospective confidence judgments (RCJ) (Nelson, Reference Nelson1992; Nelson & Narens, Reference Nelson and Narens1990). These approaches have been used to examine metacognitive ability in several cognitive domains, including control of action (Augustyn & Rosenbaum, Reference Augustyn and Rosenbaum2005), comprehension of text (Dunloskly, Baker, Rawson, & Hertzog, Reference Dunloskly, Baker, Rawson and Hertzog2006; Griffin, Jee, & Wiley, Reference Griffin, Jee and Wiley2009; Griffin, Wiley, & Thiede, Reference Griffin, Wiley and Thiede2008; Lefèvre & Lories, Reference Lefèvre and Lories2004; Maki & Berry, Reference Maki and Berry1984; Rawson, Dunlosky, & Thiede, Reference Rawson, Dunlosky and Thiede2000; Thiede, Wiley, & Griffin, Reference Thiede, Wiley and Griffin2010), and a great number of studies have investigated memory (i.e., metamemory) (Hager & Hasselhorn, Reference Hager and Hasselhorn1992; Hertzog, Dixon, & Hultsch, Reference Hertzog, Dixon and Hultsch1990; Kaszniak & Zak, Reference Kaszniak and Zak1996; Leonesio & Nelson, Reference Leonesio and Nelson1990; McDonald-Miszczak, Hertzog, & Hultsch, Reference McDonald-Miszczak, Hertzog and Hultsch1995; Pannu & Kaszniak, Reference Pannu and Kaszniak2005; Tiede, Derksen, & Leboe, Reference Tiede, Derksen and Leboe2009).
Little is known about metacognitive experiences after TBI as tested by using reports of judgment on an item by item basis (such as EOLs, JOLs, and RCJs). Instead, the majority of studies that examine awareness of cognitive ability after TBI rely on inventories based upon patient and/or other report [e.g., Self Awareness of Deficits Interview (SADI), Scale of Unawareness of Mental Disorders (SUM-D), Patient Competency Rating Scale (PCRS) (Ciurli, et al., Reference Ciurli, Bivona, Barba, Onder, Silvestro, Azicnuda and Formisano2010; Fleming, Strong, & Ashton, Reference Fleming, Strong and Ashton1996; Garmoe, Newman, O'Connell, Reference Garmoe, Newman and O'Connell2005; Hart, Whyte, Kim, & Vaccaro, Reference Hart, Whyte, Kim and Vaccaro2005; McAvinue, O'Keeffe, McMackin, & Robertson, Reference McAvinue, O'Keeffe, McMackin and Robertson2005; Ownsworth & Fleming, Reference Ownsworth and Fleming2005; Ownsworth, McFarland, & Young, Reference Ownsworth, McFarland and Young2002; Prigatano & Klonoff, Reference Prigatano and Klonoff1998; Roche, Fleming, & Shum, Reference Roche, Fleming and Shum2002; Satz et al., Reference Satz, Forney, Zaucha, Asarnow, Light, McCleary and Becker1998; Sawchyn, Mateer, & Suffield, Reference Sawchyn, Mateer and Suffield2005; Sherer et al., Reference Sherer, Hart, Nick, Whyte, Thompson and Yablon2003; Trahan et al., Reference Trahan, Pépin and Hopps2006)]. While such measures offer clinically relevant information, they are subjective and biased by perceptual differences in raters; there is often limited reliability between raters and an inherent difficulty in establishing a “gold standard” for distinct sets of ratings. Moreover, the items in these inventories can be broad in nature (i.e, items query how one might feel about his/her memory in general) and address only the individual's overall ability in a domain; while this method provides information regarding more general metacognitive knowledge, it has limited ability to directly assess accuracy of awareness in the moment. In contrast, item-by-item reports of judgments were developed to allow for confidence to be directly tied to cognitive decision making, thus providing a more sensitive and accurate measure of metacognitive experience. Although these types of judgments were initially developed for study of healthy individuals, more recently, investigators have successfully used these measurements to study clinical populations as well (Cosentino & Stern, Reference Cosentino and Stern2005).
In the limited literature that does exist of objectively measured metacognitive experiences after TBI, impairments have been documented in the domain of metamemory; individuals with TBI make less accurate judgments of learning than healthy adults (Kennedy, Carney, & Peters, Reference Kennedy, Carney and Peters2003; Kennedy & Yorkston, Reference Kennedy and Yorkston2000). However, these findings were challenged by Anderson and Schmitter-Edgecombe (Reference Anderson and Schmitter-Edgecombe2009), who found that despite performing poorly on memory tasks, adults with TBI were still able to accurately predict their performance. It has also been documented that the accuracy of RCJs made by individuals with TBI were comparable to those of healthy adults; however, qualitative differences emerged such that when errors were made, they tended to be more overconfident, while healthy adults were underconfident (Kennedy, Reference Kennedy2001). The limited number of studies investigating metacognition after TBI and the focus on metamemory reflects a need for research in this population examining distinct cognitive domains.
Metacognition and Executive Functioning
“Executive functioning” is a term that captures several higher order cognitive processes that are responsible for complex behaviors such as planning, organization, and problem-solving. In the model posed by Norman and Shallice (Reference Norman and Shallice1986), executive functioning acts in a supervisory manner by monitoring and manipulating more basic schemas, which ultimately affects the outcome. Of interest, this model holds important similarities to Nelson and Narens's (Reference Nelson and Narens1990) model of metacognition; both models include a higher order level that monitors and controls “lower,” more basic information processing. The similarity between metacognition and executive functioning to act as regulatory systems has convinced some that the two processes may be related (Fernandez-Duque, Baird, & Posner, Reference Fernandez-Duque, Baird and Posner2000; Shimamura, Reference Shimamura2000). One hypothesis is that the complex attentional control afforded by executive functions is necessary for self-reflective processes, such as metacognition (Hart et al., Reference Hart, Whyte, Kim and Vaccaro2005). Another stance asserts that metacognitive judgments are influential in the exertion of executive control over behaviors; studies in healthy individuals have found that adjustments in behavior are linked to their judgments of performance (Karpicke, Reference Karpicke2009; Metcalfe, Reference Metcalfe2009; Redford, Reference Redford2010). Finally, results from neuroimaging studies have identified an association between frontal networks and executive functioning (Chen, Wei, & Zhou, Reference Chen, Wei and Zhou2006; Collette, Hogge, Salmon, & Van Der Linden, Reference Collette, Hogge, Salmon and Van Der Linden2006; Markela-Larenc et al., Reference Markela-Lerenc, Ille, Kaiser, Fiedler, Mundt and Weisbrod2004); similar networks have also been found to be associated with metacognition (Chua, Schacter, Rand-Giovannetti, & Sperling, Reference Chua, Schacter, Rand-Giovannetti and Sperling2006; Chua, Schacter, & Sperling, Reference Chua, Schacter and Sperling2009; Kikyo, Ohki, & Miyashita, Reference Kikyo, Ohki and Miyashita2002). The allocation of similar neural resources further supports the notion that executive functioning and metacognition are related.
Indeed, age related differences in metamemory appear to track with differences in executive abilities (Perrotin, Belleville, & Isingrini, Reference Perrotin, Belleville and Isingrini2007; Perrotin, Isingrini, Souchay, Clarys, & Taconnat, Reference Perrotin, Isingrini, Souchay, Clarys and Taconnat2006; Perrotin, Tournelle, & Isingrini, Reference Perrotin, Tournelle and Isingrini2008; Souchay, Isingrini, & Espagnet, Reference Souchay, Isingrini and Espagnet2000). However, it also appears that metamemory performance is captured by a unique component of processing self-relevant information that is independent of executive functioning in older adults (Cosentino, Metcalfe, Holmes, Steffener, & Stern, in press), and in individuals with schizophrenia (Koren et al., Reference Koren, Seidman, Poyurovsky, Goldsmith, Viksman, Zichel and Klein2004). Furthermore, the relationship between executive functioning and metacognition is questionable in light of evidence from studies of animals that demonstrate higher order functioning but do not show self-awareness (Gallup & Suarez, Reference Gallup and Suarez1991). Similar observations of retained executive functioning abilities despite impairment in self-awareness have also been documented in humans (Stuss & Alexander, Reference Stuss and Alexander2000; Stuss & Levine, Reference Stuss and Levine2002). Thus, while metacognitive processes and executive functioning seem to share a common neural substrate and serve similar regulatory functions, the degree to which they overlap is uncertain; in particular, it remains unknown whether deficits in the abilities reflected in these constructs are dissociable following neurological disruption.
The evidence thus far for a relationship between executive functioning and metacognition after TBI remains inconclusive; using self-report inventories, some studies have found that deficits in executive functioning are related to poor metacognitive knowledge (Allen & Ruff, Reference Allen and Ruff1990; Bogod, Mateer, & MacDonald, Reference Bogod, Mateer and MacDonald2003; Ciurli et al., Reference Ciurli, Bivona, Barba, Onder, Silvestro, Azicnuda and Formisano2010; Hart et al., Reference Hart, Whyte, Kim and Vaccaro2005), while others have found no relationship (Bach & David, Reference Bach and David2006). The literature examining the relationship between in-the-moment metacognitive experiences and executive functioning following TBI is much more limited, and findings here are also mixed. One study of metacognition in a heterogeneous sample (including stroke and TBI) found mixed results when comparing feeling of knowing judgments with measurements of frontal lobe dysfunction; statistical analyses using a gamma coefficient suggested a significant correlation, but the use of Hamann's coefficient did not confirm those findings (Schneyer et al., Reference Schneyer, Verfaellie, Alexander, LaFleche, Nicholls and Kaszniak2004). Another study of metamemory failed to find a relationship between prospective monitoring judgments and tasks of executive functioning, suggesting that these may be two distinct processes (Anderson & Schmitter-Edgecombe, Reference Anderson and Schmitter-Edgecombe2009). Thus, the relationship between executive dysfunction (and its subcomponents) and metacognitive experience remains unclear and the current study aims to directly examine these two supervisory systems.
Study Goals
A review of the literature has identified gaps in the understanding of metacognitive experiences objectively after TBI (especially in domains other than metamemory). Thus, the primary goal of the current study was to investigate metacognition (defined here as beliefs, perceptions, and thoughts about one's own cognitive performance) by using objective measures. In particular, in-the-moment metacognitive accuracy in participants with TBI and healthy adults was examined using item-by-item confidence judgments of performance. Through the use of these confidence judgments, it was an important goal of this study to document the relationship between metacogntive accuracy and performance on executive functioning tasks. The literature in metacognition following TBI to this point has focused almost exclusively on metamemory; and it was a goal of this study to also examine metacognition in TBI in a task requiring abstract reasoning.
Methods
Subjects
Participants consisted of 21 individuals between the ages of 18 and 65 with moderate to severe TBI and 21 healthy adults (HCs) matched for age and education. Injury severity was determined by Glasgow Coma Scale (GCS) score reported in medical charts at the time of admission to hospital; moderate and severe injuries were defined by GCS scores between 3 and 12, or were substantiated as positive findings on neuroimaging scans as noted in patient medical charts. One participant had a GCS score of 14; however, the results from her imaging scans revealed significant injury, including verifiable subdural hematoma. A detailed description of imaging findings, GCS scores, and time post injury are presented in Table 1; demographic and clinical descriptive variables for the individuals with TBI and HCs are summarized in Table 2. There was no significant difference in age or years of education between the two groups. Exclusion criteria for both the TBI sample and HCs included history of: psychiatric illness, colorblindness, and substance abuse that required hospitalization or rehabilitation. All participants received and agreed to the informed consent approved by the Institutional Review Board at the institution.
Clinical descriptors for participants with TBI
Note. HEM = hemorrhage; SAH = subarachnoid hematoma; DAI = diffuse axonal injury; SDH = subdural hematoma; IPH = intraperitoneal hemorrhage; IVH = intraventricular hemorrhage.
**GCS score not available; injury severity was based upon imaging findings and reported loss of consciousness of 1 day.
Participant demographic information: mean(standard deviation)
Procedure and Measures
An identical battery of paper and pencil neuropsychological tests was administered to all participants. The neuropsychological tests used to examine executive functioning were administered as published; for tests of metacognition, modifications were made to existing tests to capture retrospective confidence judgments (RCJs). To collect RCJs, participants used a 6-point Likert scale to rank how certain they answered the item correctly immediately after responding to each item in a task. All metacognitive tasks (described in further detail below) in the battery followed this format for reporting RCJs. Before metacognitive testing, a practice task was given to familiarize participants with making RCJs. During the practice task, participants completed 20 multiple choice trivia questions (e.g., “Buenos Aires is the capital of which country?”), each followed immediately with an inquiry of confidence.
Tests of Executive Functioning
There is existing evidence that executive functioning is not a unitary construct; rather, it includes components of set shifting, inhibition, monitoring, productive fluency, and cognitive flexibility (Busch, McBride, Curtiss, & Vanderploeg, Reference Busch, McBride, Curtiss and Vanderploeg2005; Miyake et al., Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wagner2000). Tests of executive functioning were chosen based on their ability to assess these components; the tests included: the Trailmaking Tests (Army Individual Test Battery, 1944; Reitan & Wolfson, Reference Reitan and Wolfson1985), Verbal fluency subtest from the Delis-Kaplan Executive Functioning System (DKEFS) (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001), and the Stroop Color Word Test (Trenerry, Crosson, Deboe, & Leber, Reference Trenerry, Crosson, Deboe and Leber1989). Additionally, modified administrations (modifications described below) of the Matrix Reasoning subtest from the Wechsler Adult Intelligence Scale-III (WAIS-III) (Wechsler, Reference Wechsler1997) and the Abstraction subtest from the Shipley's Institute of Living Scale (Shipley, Reference Shipley1946) were included as measures of executive functioning (refer to Figure 1). Consideration was warranted to the complexity of executive functioning as a construct that may involve other basic cognitive processes; thus, to control for these more basic processes and provide a “purer” measure of executive functioning, the Digit Span subtest from the WAIS-III (Wechsler, Reference Wechsler1997) was also included in the battery as a measure of simple attention/working memory. With the exception of the modified Matrix Reasoning and Shipley's Abstraction subtests, standard administration was followed on these tests, thus Z-scores were calculated using normative data provided by the test publishers.
Organization of neuropsychological tests used to measure executive functioning and metacognition.
In-the-Moment Metacognitive Confidence Judgments
As noted, to measure metacognitive ability, standard neuropsychological tests were modified so that immediately after answering each item of a task, participants were required to make a RCJ. After providing a response for each item, participants were asked to complete the sentence: “I am _______that my answer is correct.” Participants reported their answer by selecting one of six options: completely certain, certain, somewhat certain, somewhat uncertain, uncertain and completely uncertain. These item-by-item confidence judgments were completed on the following tests: the Matrix Reasoning subtest from the WAIS-III (Wechsler, Reference Wechsler1997), the Abstraction subtest from the Shipley's Institute of Living Scale (Shipley, Reference Shipley1946), and the recognition portion of the Hopkins Verbal Learning Test-R (HVLT-R) (Benedict, Schretlen, Groninger, & Brandt, Reference Benedict, Schretlen, Groninger and Brandt1998). These measures were chosen for modification as these tasks are non-timed, and consist of individual items that are amenable to confidence judgments. RCJs from the modified Matrix Reasoning and Shipley's Abstraction subtests were used to reflect the domain of meta-abstract reasoning, while the judgments from the modified HVLT-R served as measurements of metamemory. Of note, during the administration of the HVLT-R, participants were exposed only once to the word list instead of the standard three learning trials to reduce ceiling effects during the recognition trial. Due to the deviation from test standardization for these measures, the means and standard deviations of the HC sample from this study were used to gauge performance in the sample with TBI.
Performance and confidence judgment of the metacognitive items were coded so that a Goodman and Kruskal gamma coefficient, a measure of resolution, could be calculated (Goodman & Kruskal, Reference Goodman and Kruskal1954). These rank-order coefficients, ranging between −1 and 1, describe the concordance between pairs of performance and confidence data in a given task; that is, the extent to which confidence judgments are high when performance is high, and judgments are low when performance is low. Gamma coefficients with values closer to 1 suggest a more accurate match between confidence and performance (more accurate awareness). A gamma coefficient was calculated for each test of metacognition, for each participant (for details of the computation and use of the gamma coefficient, please refer to: Gonzalez & Nelson, Reference Gonzalez and Nelson1996; Goodman & Kruskal, Reference Goodman and Kruskal1954; Nelson, Reference Nelson1984). The gamma coefficients were then converted into Z scores using a Fisher's r to z transform to improve the properties of the distribution (Howell, Reference Howell1987) and to enable comparison against scores from the executive functioning tasks.
Executive Functioning Indices and Metacognition Indices
To more easily examine the different components of executive functioning, the scores from the different tests were combined to create indices reflecting the components noted earlier of set shifting/switching and inhibition, abstract reasoning and mental flexibility, and production fluency. Results from a factor analysis demonstrated a common organizing factor underlying the trailmaking tests and the Stroop (de Frias, Dixon, & Strauss, Reference de Frias, Dixon and Strauss2006); thus, in this study the scores were grouped together in a Switching/Inhibition index. The averaged scores from the Matrix Reasoning and Abstraction subtests were grouped into an Abstract Reasoning index based upon evidence of a high correlation and relationship between the two types of tasks (Pringle & Haanstad, Reference Pringle and Haanstad1971). Scores from the DKEFS Verbal Fluency test made up the Verbal Fluency index. The dimensionality of metacognition was examined by dividing the scores from the metacognition tests into two groups based upon domain membership. The gamma coefficients from the Matrix reasoning and Abstraction subtests were averaged and grouped into a Meta-Abstract Reasoning (meta-AR) score, while the coefficient from the HVLT-R task formed the Metamemory (metaMEM) score.
Data Analyses and Results
All data analyses (including calculation of the gamma coefficients) were performed using Statistical Package for the Social Sciences (SPSS) Version 17.0.
Group Differences in Executive Functioning Performance and Metacognitive Accuracy
Differences in executive functioning performance between the participants with TBI and HCs were determined by using an independent samples t-test to compare the Z-scores obtained on the three executive functioning indices described earlier. Participants with TBI showed significantly worse performance compared to HCs on all three executive functioning indices (data including performance on the indices as well as on each task of executive functioning are summarized in Table 3).
Performance on executive functioning (indices and separate tasks): Mean Z scores (standard deviation)
*Normative data do not apply here; HC sample represents normative data.
Differences in metacognitive accuracy between the HCs and participants with TBI were determined by comparing gamma coefficients using an independent samples t-test. Participants with TBI had significantly lower gamma coefficients in the metaMEM domain compared to HCs; meaning they were less accurate in judgments of their memory performance. In contrast, there was no significant difference between gamma coefficients in the meta-AR domain across participants with TBI and HCs (data are summarized in Table 4). Of note, for individuals with TBI, the number of years post injury did not predict metacognitive accuracy.
Differences in gamma coefficients: Mean (standard deviation)
*Denotes statistical significance at p < .05.
Relationship Between Executive Functioning and Metacognition
The relationship between executive functioning and metacognitive accuracy was investigated using correlation analyses. Analyses were conducted using the Z-scores from each of the executive functioning indices and the Z-scores converted from gamma coefficients. In an effort to work with relatively “pure” measures of executive functioning, initial analyses were conducted to determine the contribution of basic attention and working memory processes to each executive index. In this group of participants with TBI, attention/working memory, as measured by the Digit Span subtest of the WAIS-III (Wechsler, Reference Wechsler1997), was significantly correlated to all three indices of executive functioning (Verbal Fluency r = 0.55; p = .01; Abstract Reasoning r = 0.65; p = .00; and Switching/Inhibition r = 0.54; p = .02), but not related to any of the metacognitive indices. To account for this covariance, a partial correlation analysis was performed between executive function performance and metacognitive accuracy while controlling for attention/working memory performance. Using this “purer” measure of executive functioning, no significant relationships between executive functioning and metacognition were found in the sample of HCs. For the participants with TBI, the only significant correlation found was between Abstract Reasoning and meta-AR; that is, only performance on abstract reasoning tasks was related to accuracy of metacognitive judgments. A summary of these correlational results are found in Table 5.
Correlation between executive functioning indices and metacognition domains (Pearson's r values)
*Indicates significance at p = .05.
†Controlling for working memory.
Differences in Metacognitive Domains
The overlap in the two metacognitive domains studied here was examined by conducting both a first order correlation analysis and a repeated-measures analysis of variance (ANOVA); these analyses were chosen to test both the magnitude to which metacognition in the domains were related to one another (correlation) and the significance of any differences (repeated-measures ANOVA). A repeated-measures ANOVA was used because the same participants were engaged in two different conditions that were to be compared; that is, each participant provided a measure of metaMEM as well as a measure of meta-AR. For each group, no significant relationships were found between the metacognitive domains (results in Table 6). The repeated-measures ANOVA revealed a statistically significant difference in gamma coefficients between the two metacognitive domains (F(1,40) = 14.98; p = .000) and demonstrated that all participants showed better metacognitive accuracy (higher gamma coefficients) on the tasks of meta-AR (mean = 0.65; SD = 0.22) compared to metaMEM (mean = 0.36; SD = 0.50). A significant interaction (depicted in Figure 2) was also found between metacognitive domain and group (TBI or HC) membership (F(1,40) = 4.66; p = .04). The findings from these analyses suggest a difference in metacognitive appraisal between the domains of memory and abstract reasoning.
Correlations of gamma coefficients between and within metacognitive domains
Graph of differences in metacognitive accuracy across domains of MetaMEM and Meta-AR.
The meta-AR domain was constructed from the combination of two tasks (Shipley's Abstraction and Matrix Reasoning); thus, it was necessary to also examine if there were differences based on which measurement of meta-AR was used. The separate gamma coefficients for each of the meta-AR tasks were computed and the values are provided in Table 4. A correlation analysis conducted between the metaMEM gamma coefficients and each separate subtest of the meta-AR index found no significant relationships (results presented in Table 6). These confirm the earlier finding that metacognitive abilities differ between domains.
Of interest, when comparing the gamma coefficients from different tasks within the same meta-AR domain (gamma coefficients from Matrix Reasoning and coefficients from Shipley's Abstraction) no significant relationship was found between the individual meta-AR gamma coefficients either (results shown in Table 6), suggesting differing metacognitive abilities within a domain as well.
Discussion
This study is the first to use item-by-item judgments to objectively examine the relationship between the processes of executive functioning and metacognition in a sample of participants with moderate and severe TBI.
Metacognition and Executive Functioning
Importantly, the data here demonstrate that impairment in various aspects of executive functioning does not uniformly predict impairment in metacognitive ability. The failure to find a relationship between metacognitive accuracy and independent tasks of executive functioning reveals dissociable components to these processes and is at least partially attributable to the known multi-dimensionality of “executive functions.” The findings in TBI are similar to what has been observed in other samples; examinations of metamemory in the aging and schizophrenia literature corroborate the observation that executive functioning and aspects of metacognitive accuracy are dissociable (Cosentino et al., in press; Koren et al., Reference Koren, Seidman, Poyurovsky, Goldsmith, Viksman, Zichel and Klein2004; Souchay, Isingrini, Clarys, Taconnat, & Eustache, Reference Souchay, Isingrini, Clarys, Taconnat and Eustache2004). The contribution of this study's findings to the understanding of the relationship between executive functioning and metacognition has significant implications for clinical application. As evidence that metacognitive processes are distinct from executive functioning mounts, clinical tools need to be developed that ensure independent and accurate assessment of these constructs.
In considering why the Abstract Reasoning index in particular was found to be related to metacognitive abilities, it is possible that the nature of the abstract reasoning task used in this study places demand on self-monitoring processes that are also inherent to metacognition. Successful completion of the Matrix Reasoning subtest, for example, involves search of a pattern or “rule” that can be applied to complete each sequence. Abstract reasoning tasks such as this require evaluation of one's own thought processes and some degree of “perspective taking” to determine how the problem was intended to be solved. The ability to engage in abstract reasoning may be linked to metacognitive accuracy due to similar evaluative processes required for each. If this were the primary basis of the association between abstract reasoning and metacognitive processes, an association between abstract reasoning and the metaMEM measure may be expected as well. Such an association was not found in this study, thus, we cannot rule out the possibility that the observed association reflects the tendency for performance on a measure to be correlated with metacognitive accuracy for that measure (Koriat & Helstrup, Reference Koriat and Helstrup2007), as RCJs for the Abstract Reasoning index comprise the meta-AR score. However, evidence elsewhere in the metamemory literature indicates that such a correlation is not a consistent and necessary phenomenon (Hager & Hasselhorn, Reference Hager and Hasselhorn1992; Leonesio & Nelson, Reference Leonesio and Nelson1990), thereby suggesting that the association found in this study between the abstract reasoning component of executive functioning and metacogntive accuracy may indeed be due to shared evaluative processes.
The absence of a significant relationship between the executive functioning and metacognition in the sample of HCs may be due to a threshold effect whereby some degree of impairment is required, or due to increased observable variance measured in the TBI sample. That is, the intact Abstract Reasoning skills in the HC group could have left little variance to be examined during correlational analysis; or there was an all-or-none phenomenon, where above some threshold, intact abstract reasoning skills have little relationship to intact metacognitive accuracy.
Support for Dimensions to Metacognition
The co-existence of high accuracy in one metacognitive task and low accuracy in another for individuals with TBI indicates that metaMEM and meta-AR may be distinct. Similarly, the failure to find a relationship between the metacognitive accuracy on metaMEM and meta-AR tasks in either groups supports the notion that these metacognitive capacities may represent distinct domains. Importantly, this dissociation indicates that preserved ability in one domain may not predict similar skills in another. This has important implications for how deficits in metacognition are conceptualized after neurological injury; such deficits may not be universally observed across cognitive areas and require independent assessment.
Differences in metacognitive accuracy within domains were found as well, as metacognitive accuracy on individual subtests from the meta-AR composite were not correlated. This difference in accuracy may be attributed to a dissociation in task demands; for example, Shipley's Abstraction requires the participant to produce the answer while Matrix Reasoning requires decision making between options provided. These distinct task demands are also associated with differential feedback received—less ambiguity may be experienced when an item requires a self-generated response, ultimately affecting the reported metacognitive judgment. This hypothesis is supported by the observation that all participants in this study had better metacognitive accuracy on the Shipley's Abstraction subtest compared to the Matrix Reasoning subtest and participants with TBI were seen to benefit the most (i.e., have greatest awareness) when completing tasks that require self-generated responses.
In the metaMEM domain, the impaired accuracy demonstrated by the individuals with TBI found in this study was inconsistent with findings in an earlier study of RCJs by Kennedy (Reference Kennedy2001), where adults with TBI did not demonstrate such deficits. One possible reason for this discrepancy has to do with how metamemory was measured; a free recall task was used in the Kennedy (Reference Kennedy2001) study, while this study used a recognition format. In another study, Kennedy (Reference Kennedy2004) documented differences in metacognitive accuracy during recall when using two different types of tasks (noun pair recall vs. recall of narrative information). Together, these observations raise the question of whether judgments of metacognition can be generalized across tasks within the same domain; there is certainly evidence that performance on judgments of learning (recall-based) and feeling of knowing (recognition-based) tasks are not correlated (Souchay et al., Reference Souchay, Isingrini, Clarys, Taconnat and Eustache2004), likely reflecting important differences in the task demands for metacognitive judgments about self-generated content (i.e., recall) versus those made regarding information generated by the experimenter (i.e., recognition).
The influence of how a construct is assessed is also relevant for the study of “dimensionality” in metacognition; research to date has not controlled for how metacognitive ability is assessed. It is possible that the difference in accuracy observed between metaMEM and meta-AR is attributable to subtle differences in assessment; that is, judgments may be influenced depending on the procedure used to assess metacognition in each domain. Similarly, a “task load” effect may also be possible whereby two tasks measuring the same construct differentially tap into metacognitive accuracy due to differences in overall task difficulty. Future work requires specific manipulations (e.g., use of identical task demands) within each cognitive domain to tease apart the potential influences of assessment and task load on the dimensionality of metacognition. Identifying the contribution of task characteristics effects will be important to determine if metacognition can be divided along cognitive dimensions. Even so, the current data minimally demonstrate that the approach to assessing metacognitive ability is crucial and certain formats may be more sensitive to deficit than others (e.g., recall vs. recognition formats).
The use of gamma coefficients in this study has resulted in important insights into metacognition after TBI; and although to date gamma coefficients have frequently been used in studies of metacognition, there remain limitations in using this measure. In particular, the gamma coefficient requires variance in the data to be calculated; coefficients cannot be generated if one or more variables is constant (e.g., items all answered correctly or report of identical confidence ratings). Test items must thus cover a breadth of difficulty; this may have been difficult to achieve for participants who performed at the floor or ceiling.
While the gamma coefficient provides a measure of relative accuracy, important supplemental information might be gained from examining judgment directionality. Kennedy (Reference Kennedy2001) created an “absolute value” (AV) measure which determined judgment directionality (under or overconfidence). AVs were computed for the data from this study (for detailed description of the procedure, please refer to Kennedy, Reference Kennedy2001); and in post hoc analyses, a significant main effect of domain was found demonstrating consistent underconfidence in both groups on the task of metaMEM, but overconfidence on tasks of meta-AR (F(1,40) = 19.9; p = .000). These results again support the finding that evaluative processes differ across domains of metacognition. Additionally, the magnitude of confidence of participants with TBI did not differ significantly from HCs in either domain (metaMEM: t(40) = 0.67; p = .50; Meta-AR(Shipley's): t(25.5) = 0.15; p = .88; Meta-AR(Matrix): t(40) = 0.55; p = .58), suggesting that the magnitude and direction of judgments made by participants with TBI were similar to the HCs. Taken together with the data from the gamma coefficients, the correspondence of low coefficients and underconfident responses reported by the participants with TBI help to confirm that the deficits in metamemory reflect a genuine effect. The AV measure appears to capture important information about the quality of awareness that may be helpful in separating the effect of potential influencing factors (e.g., response bias) and is deserving of further examination in future studies.
Future studies can expand in several other areas to increase the understanding of metacognitive experience after TBI. While this study focused on a retrospective monitoring process, there is evidence that prospective monitoring processes in metamemory are compromised after TBI (Kennedy et al., Reference Kennedy, Carney and Peters2003; Kennedy & Yorkston, Reference Kennedy and Yorkston2000); however, these have yet to be tested in the domain of meta-AR, and their relationship to executive functioning remains uncertain. A longitudinal examination of metacognition through the different temporal stages of recovery would provide useful information regarding the course of change in metacognitive abilities after neurological insult. As interest of metacognitive processes in neurological populations grows, it will be important that future studies further evaluate and confirm the validity of these measures such that there can be appropriate application in clinical samples.
Conclusions
The capacity for self-awareness is critical in day-to-day functioning and often disrupted following TBI. While subjective questionnaires dominate the literature examining awareness of deficit and offer flexibility in assessment, such methods pose significant challenges for making determinations regarding awareness of on-task performance, and for investigating the nature of metacognitive deficits in clinical populations. In this study, the use of RCJs paired with cognitive testing permits direct examination of metacognitive accuracy and its relationship to other cognitive domains. The results of this study demonstrate that metacognition in TBI is task dependent and may not be a unitary construct; the deficit observed in metaMEM did not extend to meta-AR, which was preserved in this sample. Moreover, the results of this study failed to demonstrate a consistent relationship between executive functioning and metacognition, providing evidence that while related, these constructs seem to have unique characteristics and divergent demands.
Acknowledgments
Much appreciation to Philip Schatz, Ph.D., for his consultation and to Neal Fitzpatrick, Tia Bochnakova, Julia Slocomb, and Britney Wardecker for their assistance in recruitment and data collection. The authors declare that there are no conflicts of interest associated with this study and manuscript. This research was funded in part by a grant from the New Jersey Commission on Brain Injury Research (grant number 0120090178).
