INTRODUCTION
Metacognitive processes may play a crucial role in daily life functioning. Deficits in metacognitive capacity have real-world implications, as they may impact the ability to form complex ideas about how to respond to psychological and social challenges (Hamm et al., Reference Hamm, Renard, Fogley, Leonhardt, Dimaggio, Buck and Lysaker2012; Lysaker et al., Reference Lysaker, Shea, Buck, Dimaggio, Nicolo, Procacci and Rand2010; Tas, Brown, Esen-Danaci, Lysaker, & Brune, Reference Tas, Brown, Esen-Danaci, Lysaker and Brune2012). According to Flavell (Reference Flavell1976): “Metacognition refers to one’s knowledge concerning one’s own cognitive processes and products or anything related to them […] Metacognition refers, among other things, to the active monitoring and consequent regulation and orchestration of these processes […] usually in the service of some concrete goal or objective.” This definition distinguishes two metacognitive dimensions. First, “metacognitive knowledge,” that is, knowledge about one’s own cognitive processes, is a trait dimension independent from actually performing a cognitive task. Second, “metacognitive awareness,” that is, regulation of the former processes, is a state or “on-line” dimension that is active while performing a cognitive task. In daily life, “on-line” metacognition is used to continuously monitor level of functioning through self-assessment of level of performance (Koriat, Reference Koriat2007). According to Nelson and Narens (Reference Nelson and Narens1994), metacognitive awareness can also be subdivided into two components. These operations include “monitoring” (supervision) and “control” (top-down regulation) of basic cognitive function involved in information processing. Whereas metacognition is of great interest for neuropsychological practice, little is known about the way that metacognitive questions should be included in neuropsychological assessment. Assessing metacognition in clinical neuropsychological practice fits well with recent attempts to improve the ecological validity of neuropsychological assessment procedures; that is, to bridge the gap between laboratory measures of cognitive deficits and real-world information processing difficulties.
Metacognitive knowledge and metacognitive awareness may be measured with various tools (Quiles, Prouteau, & Verdoux, Reference Quiles, Prouteau and Verdoux2013). Several rating scales or self-report questionnaires assessing metacognitive knowledge have been developed for healthy subjects or for persons suffering from psychiatric disorders (Beck, Baruch, Balter, Steer, & Warman, Reference Beck, Baruch, Balter, Steer and Warman2004; Schoo, van Zandvoort, Biessels, Kappelle, & Postma, Reference Schoo, van Zandvoort, Biessels, Kappelle and Postma2013; Semerari et al., Reference Semerari, Carcione, Dimaggio, Falcone, Nicolò, Procacci and Alleva2003). For example, metacognitive knowledge for specific cognitive domains (memory, attention, executive functioning, and visuoperception) has been investigated with vertical visual analogue scales in healthy people, showing that participants have domain-specific metacognitive knowledge of their cognitive functioning (Schoo et al., Reference Schoo, van Zandvoort, Biessels, Kappelle and Postma2013). Studies using the Subjective Scale to Investigate Cognition in Schizophrenia, a 21-item self-report questionnaire assessing metacognitive knowledge by measuring cognitive complaints about several cognitive domains (working memory, explicit long-term memory, attention, language, and praxia) (Beck et al., Reference Beck, Baruch, Balter, Steer and Warman2004), showed that patients with schizophrenia express subjective complaints that do not strictly correspond with objective performances (Prouteau et al., Reference Prouteau, Verdoux, Briand, Lesage, Lalonde, Nicole and Stip2004). Finally, Metacognitive Assessment Scale (MAS) is a rating scale for assessing metacognitive knowledge, as manifested in individuals’ verbalizations (Semerari et al., Reference Semerari, Carcione, Dimaggio, Falcone, Nicolò, Procacci and Alleva2003). Studies using the MAS showed that metacognitive knowledge varies from one person to another according to the degree of complexity a person may reach regarding integrated ideas about the self, which synthesizes different aspects of experience (Dimaggio, Vanheule, Lysaker, Carcione, & Nicolo, Reference Dimaggio, Vanheule, Lysaker, Carcione and Nicolo2009; Semerari et al., Reference Semerari, Carcione, Dimaggio, Falcone, Nicolò, Procacci and Alleva2003). A key implication is that some persons may consider their own mental activities more readily than others and hence may form ideas about the self, which synthesizes different aspects of experience.
Unlike the relatively large number of tools measuring metacognitive knowledge, few tools are available to measure metacognitive awareness. They have been mainly used in the domain of metamemory; prospective judgments (i.e., a prediction of future memory performance upstream of the memory task itself) include, for example, judgment of learning and feeling of knowing tasks. Retrospective measures corresponding to an estimate of performance after the task include measurement of confidence level, in which the participant has to give his/her level of confidence in his/her answer on a Likert scale just after the recognition task. However, such measures have rarely been applied to other domains of cognitive functioning. An original experimental protocol was developed by Koriat and Goldsmith (Reference Koriat and Goldsmith1996). In this paradigm, 71 undergraduates were first asked to answer a question exploring general knowledge (for example “what was the name of the first emperor of Rome?”). Then, to measure metacognitive “monitoring,” they had to rate their confidence in their answer between 0 and 100%. Lastly, to measure metacognitive “control,” the participants had to choose whether or not to validate their answer for the total score. The decision to validate the answer (or metacognitive control) was significantly correlated with participants’ confidence (or monitoring) in their answers. Moreover, metacognitive monitoring (capacity to discriminate correct from incorrect answers) was positively correlated with answer accuracy (Koriat & Goldsmith, Reference Koriat and Goldsmith1996). Koren et al. (Reference Koren, Seidman, Poyurovsky, Goldsmith, Viksman, Zichel and Klein2004, Reference Koren, Poyurovsky, Seidman, Goldsmith, Wenger and Klein2005; Koren, Seidman, Goldsmith, & Harvey, Reference Koren, Seidman, Goldsmith and Harvey2006) applied Koriat and Goldsmith’s paradigm to develop a metacognitive version of the Wisconsin Card Sorting Test (WCST) aimed at assessing “on-line” metacognitive monitoring and control during the execution of the test. For each card of the test, two questions were added to evaluate “on-line” metacognitive monitoring (“What is your degree of confidence in this answer?”) and control (“Do you want to take into account this response in your total score?”). A study using this metacognitive version of the WCST showed that metacognitive control was significantly less correlated with metacognitive monitoring in persons suffering from schizophrenia than in normal volunteers (Danion, Gokalsing, Robert, Massin-Krauss, & Bacon, Reference Danion, Gokalsing, Robert, Massin-Krauss and Bacon2001). Bacon and collaborators, using a comparable metacognitive version of memory tests, found that persons with schizophrenia had higher metamemory monitoring scores for incorrect answers and lower metamemory monitoring scores for correct answers compared to normal volunteers (Bacon, Danion, Kauffmann-Muller, & Bruant, Reference Bacon, Danion, Kauffmann-Muller and Bruant2001; Bacon & Izaute, Reference Bacon and Izaute2009).
To include metacognitive awareness assessment in clinical practice, information is needed about the potential impact of on-line metacognitive questions on neuropsychological test performances. To our knowledge, only one study has specifically addressed this issue, showing that metacognitive questions induced poorer learning (Begg, Martin, & Needham, Reference Begg, Martin and Needham1992). On-line metacognition could thus affect learning by interacting with the encoding process. In line with the cognitive load theory, we hypothesize that adding metacognitive questions to each item of a neuropsychological test may constitute an intercurrent task (i.e., an additional cognitive load) soliciting attentional resources (Lavie, Reference Lavie2005; Lavie, Hirst, de Fockert, & Viding, Reference Lavie, Hirst, de Fockert and Viding2004; Rees, Frith, & Lavie, Reference Rees, Frith and Lavie1997). Indeed, if the processing load of the target task exhausts available capacity, as in problem-solving tests, distracting stimuli such as metacognitive questions will not be perceived. However, if the target-processing load is low, attention will spill over to the processing of distractors. Thus, the aim of the present study was to assess the impact of “on-line” metacognitive questions on neuropsychological performances in a non-clinical sample.
METHODS
Participants
Participants were recruited among persons attending the blood donation center (Etablissement Français du Sang Aquitaine) in Bordeaux, France, between November 2011 and April 2012. In France, blood donors are unpaid volunteers. Persons consecutively attending the center for platelet donation were invited to participate in the study. We chose this population because the duration of platelet donation (90–100 min) was comparable to that of neuropsychological assessment. As machines used in this center draw blood from only one arm, donors had one hand free for writing.
To be allowed to donate their platelets, donors had to (i) be aged between 18 and 70 years; (ii) weigh more than 50 kg; (iii) not present with a current infection, sexually transmitted infection, anemia, any chronic disease, pregnancy; (iv) not be under guardianship.
Inclusion criteria for the present study were (i) informed consent to participate in the study; (ii) aged from 18 to 60 years; (iii) living in the urban community of Bordeaux; (iv) speaking French; (v) no history of neurological illness or trauma; (vi) no history of severe mental disorder (i.e., psychotic disorder or bipolar disorder) as evaluated with the Mini International Neuropsychiatric Interview (MINI) (Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller and Dunbar1998); (vii) no alcohol or drug dependence (except nicotine) as evaluated with the MINI; (viii) no regular use of psychotropic drugs (less than once a week over the last month). These inclusion criteria allowing inclusion of persons with minor psychiatric disorders such as anxiety disorders were developed to avoid selecting a sample of “hypernormal” persons. The study conformed to French bioethics legislation.
Neuropsychological Assessment
Participants were randomly assigned to one of the two groups, each participant of the standard group (n=19) being matched with two participants of the metacognitive group (n=38) for age and gender. The first group performed the “standard” version of the neuropsychological tests, while the second performed the “metacognitive” version of the neuropsychological tests. The sample size difference across groups was due to logistic constraints as we had a time limited access to the setting of recruitment. As the present study was part of a project aimed at developing a new method to assess metacognitive awareness, the recruitment of participants performing the metacognitive version of the tests was favored.
“Standard” Version
The Digit Symbol Test of the Wechsler Adult Intelligence scale – 3rd edition (WAIS-III) assessed processing speed (Wechsler, Reference Wechsler1997). The test requires copying symbols that match the numbers 1–9 to a key. The outcome measured was the number of correct symbols drawn within 120 s.
The modified version of the Wisconsin Card Sorting Test (MCST) (Godefroy & Groupe de Réflexion et d’Evaluation des Fonctions Executives, Reference Godefroy2008; Nelson, Reference Nelson1976) assessed executive functioning. Participants were required to sort 48 stimulus cards according to defined rules. The sorting rules could be color, shape or number. For each sorting, the subject received feedback (i.e., “right” or “wrong”). When the participants had given six consecutive correct responses, the rule changed. The scores included the number of categories completed and the number of errors.
The digit span of the WAIS-III assessed short-term memory and working memory (Wechsler, Reference Wechsler1997). Verbal short-term memory and verbal working memory were measured by forward digit recall and backward digit recall, respectively. Participants heard a sequence of digits and were asked to recall each sequence in the correct order for the former and in reverse order for the latter. The score corresponded to the number of correct sequences in correct and reverse order.
The Rappel Libre/Rappel Indicé 16 Test (RL/RI 16) assessed episodic memory (Van der Linden & Groupe de Réflexion sur l’Evaluation de la Mémoire, Reference Van der Linden2004). This episodic memory measurement tool is an adaptation of the procedure proposed by Grober and Buschke (Buschke, Reference Buschke1984; Grober, Buschke, Crystal, Bang, & Dresner, Reference Grober, Buschke, Crystal, Bang and Dresner1988). French calibration is available (Van der Linden & Groupe de Réflexion sur l’Evaluation de la Mémoire, Reference Van der Linden2004). Participants learned a list of 16 words belonging to 16 different semantic categories. To learn these words, a sheet on which 4 words were written was shown. Subjects had to read the word corresponding to the semantic category requested by the experimenter. When the four words were read, the sheet was hidden and the participant had to repeat the four words without seeing them. The same procedure was replicated four times to allow learning of the 16 words. Then, the participant had to perform a free recall of the 16 words. A cued recall (semantic category) was proposed for words not given in free recall. The experimenter then repeated two rounds of free recall / cued recall, and another round after 20 min. The score corresponded to the number of correct words for each free recall.
The Faces Test assessed social cognition through recognition of facial emotions (Baron-Cohen, Wheelwright, & Jolliffe, Reference Baron-Cohen, Wheelwright and Jolliffe1997; Merceron & Prouteau, Reference Merceron and Prouteau2013). During this test, 20 pictures of faces were presented to the participants who had to choose the most appropriate emotion in facial image among four emotions proposed. The score corresponded to the number of correct emotions among the four emotions proposed.
“Metacognitive” Version
We used a variant of the metacognitive version of the WCST developed by Koren et al. (Reference Koren, Seidman, Poyurovsky, Goldsmith, Viksman, Zichel and Klein2004) on the basis of Koriat and Goldsmith (Reference Koriat and Goldsmith1996). Two “on-line” metacognitive questions added to the test were explained immediately after receiving the general instructions. The first “on-line” metacognitive question explored level of confidence (monitoring). After each answer, the participants were asked “What is your degree of confidence in this answer?” to rate their level of confidence in the correctness of their response. In the original version of this protocol, participants had to rate their level of confidence on a visual analogue scale ranging from 0 (just guessing) to 100% (completely confident). In the present study, we used a 5-point Likert scale instead of a visual analogue scale to rate level of confidence: “not at all confident,” “slightly confident,” “moderately confident,” “very confident,” “completely confident.” The second “on-line” metacognitive question explored the impact of confidence on behavior (control). Participants were asked to validate each response, that is, to decide whether they wanted (“yes” or “no”) their response to be included in their total score. In the original version of Koren’s protocol, the participants were assigned a dollar value for each correct response. In the present study, no financial incentive was used to keep closer to ecological conditions, as in daily life there is no financial reward for efficient use of cognitive functions. For the present study, we developed a “metacognitive” version of the other neuropsychological tests by applying to these the same metacognitive assessment protocol. Participants rated their confidence in their response and decided whether to validate their answer after each digit sequence of the digit span test, and after selecting one emotion for each of the 20 faces of the Faces test. Concerning the RLRI16 test, participants had to rate their confidence in the correctness of their response and to decide to validate their response after each free and cued recall of the entire list. As the questions were related to the entire list of 16 words, they were asked in a slightly different way. The questions were as follows: “what is your degree of confidence in your success at recalling the list of 16 words?” and “do you want do validate your response in order to include your result in your total score?”.
Statistical Analyses
Statistical analyses were carried out using STATA software 11.0 (Statacorp, Reference STATA2009). Distributions of variables were examined and transformations were made to remove skewness where appropriate. Univariate analyses (χ2 test and Student’s t test) were used to compare the demographic characteristics of the two groups (“standard” vs. “metacognitive” procedure) and their neuropsychological performances. Occupational status was categorized into (i) employed and students (ii) other (housewives, retired). Psychiatric status evaluated by the MINI was categorized into (i) no psychiatric disorder (ii) at least one minor psychiatric disorder.
RESULTS
Over the study period, seventy participants were included. Twenty participants were assessed using the “standard” version of neuropsychological tests. Fifty were assessed using the “metacognitive” version of the neuropsychological tests with two metacognitive questions added for each item of each test. No significant difference was found between the two groups with respect to demographic characteristics (Table 1). With respect to psychiatric history, 8 (14%) participants presented with at least one current MINI diagnosis: major depressive disorder of mild severity (n=1), agoraphobia (n=3), social phobia (n=3), generalized anxiety disorder (n=3). No significant difference was found in the frequency of psychiatric disorder between the two groups (Table 1).
Comparison of demographic and clinical characteristics of groups assessed using “metacognitive” vs. “standard” versions of neuropsychological tests: univariate analyses
a Addition of two metacognitive questions after each item of each test.
b At least one current disorder as evaluated by Mini International Neuropsychiatric Interview.
The comparison of neuropsychological performances in the two groups are given in Table 2. There was no difference between the two groups concerning processing speed. The two groups significantly differed with respect to working memory, with more sequences recalled in the backward condition by the “metacognitive” group. Concerning episodic memory, there was no significant difference between the two groups for the first free recall of words. In the second, third, and delayed rounds of free recall, participants in the “standard” group recalled more words than those in the “metacognitive” group. No significant difference was found between the two groups with respect to short-term memory or social cognition. Concerning executive functions assessed by MCST, 100% of participants in the “standard” group and 96% in the “metacognitive” group completed at least 5 of 6 categories. No difference was found between the two groups on MCST scores.
Comparison of neuropsychological performances of groups assessed using “metacognitive” vs. “standard” version of tests: univariate analyses
a Addition of two metacognitive questions after each item of each test.
b MCST=Modified Card Sorting Test.
c RL/RI 16=Rappel Libre Rappel Indicé 16.
DISCUSSION
The present study shows that “on-line” metacognitive questions have an impact on neuropsychological performances that is of opposing direction according to the type of task. Working memory performances were better in the “metacognitive” compared to the “standard” version of the task. Conversely, episodic memory performances were poorer in the “metacognitive” compared to the “standard” version of the task. Executive functioning and social cognition performances were not significantly different between the two versions.
Methodological Limitations
First, our sample included platelet donors, which may limit the generalization of our results. Indeed, this selection may have favored the recruitment of participants not representative of the general population with respect to personality characteristics. However, we have little reason to believe that personality traits such as altruism—blood donation is unpaid in France—are associated with specific metacognitive characteristics, and hence that the differences between the two versions of the protocol can be explained by such a selection bias. Moreover, it could be hypothesized that having blood drawn may impact on neuropsychological testing performance, owing to the possible source of distraction and the physiological effects that might affect cognition. High altitude studies showed that decreased perfusion has a detrimental effect on cognitive functions (Crowley et al., Reference Crowley, Wesensten, Kamimori, Devine, Iwanyk and Balkin1992). However, Tuboly et al. (Reference Tuboly, Rudas, Csillik, Nagy, Benedek, Benedek and Braunitzer2012) found that cognitive performance (as reflected by P300 changes) was a poor marker of volume loss. In our study, blood donation did not explain the differences between the two groups as they were assessed in the same conditions, but it may limit the generalization of the results. Second, we used a single test for each cognitive function, which may limit the generalization of our findings to the whole cognitive function. Nevertheless, these tests were chosen because they are commonly used in the literature, thereby allowing comparisons with other studies. Third, this study did not assess metacognitive knowledge and did not control whether metacognitive knowledge was similar in the two groups. Metacognitive knowledge is composed of relatively autonomous and independent functions that may be selectively impaired, and could hence differently interfere with metacognitive awareness from one person to another (Dimaggio, Vanheule, Lysaker, Carcione, & Nicolo, Reference Dimaggio, Vanheule, Lysaker, Carcione and Nicolo2009). Further studies are needed to investigate the relationships between metacognitive knowledge and awareness. Finally, it is worth emphasizing that the testing used does not necessarily correspond to what happens in emotionally charged and unexpected challenges that occur in life. Further studies are needed to investigate the relationships between real-life functioning and the measure of metacognitive awareness obtained with neuropsychological tests. Finally, the sample size of the “standard” group was relatively small. However, a type II error is unlikely as statistical differences were obtained between the two groups.
Interpretation of Findings
The differential impact of “on-line” metacognitive assessment on cognitive performance depending on the cognitive task may be explained by the balance between selective attention and cognitive load. Indeed, each cognitive task requires, on the one hand, intrusion from irrelevant stimuli to be minimized (selective attention), and on the other hand, the availability of cognitive resources to perform the task (cognitive load) (Lavie, Reference Lavie2005). Whereas better selective attention may enhance performances, higher cognitive load may alter them (Lavie et al., Reference Lavie, Hirst, de Fockert and Viding2004). In our study, the lower episodic memory performances in the “metacognitive” compared to the “standard” group may be explained by interference with the learning procedure. Metacognitive questions may generate a higher cognitive load, leading to increased distractor interference (Lavie et al., Reference Lavie, Hirst, de Fockert and Viding2004) and, therefore, altering learning performances. Our findings are concordant with those obtained in a study carried out by Begg et al. (Reference Begg, Martin and Needham1992). Participants had to memorize a list of items and to repeat them later. Before repeating each item, they were asked whether they thought they would be able to repeat it. This metacognitive question theoretically aimed at promoting accurate monitoring in fact induced poorer learning performance. It may be hypothesized that accurate monitoring may have a positive impact on learning performance only when individuals are free to use or not use this information to regulate learning (Thiede, Anderson, & Therriault, Reference Thiede, Anderson and Therriault2003). When regulation is controlled by the experimenter (as in our study and in the study by Begg et al., Reference Begg, Martin and Needham1992), better monitoring has a deleterious impact on learning.
In our study, adding “on-line” metacognitive questions increased working memory performances. In working memory tasks, metacognitive questioning may not act as an interfering task but rather by enhancing selective attention to executive control components of the tasks. For example, it has already been shown that selective attention is positively linked to working memory (Awh, Vogel, & Oh, Reference Awh, Vogel and Oh2006; Gazzaley & Nobre, Reference Gazzaley and Nobre2012), which Baddeley conceptualized as a “temporary storage system under attentional control” (Baddeley, Reference Baddeley2007). Adding metacognitive questions after each response in a working memory test may require the participant to focus more attentively on the task, hence, optimizing selective attention. This leads him/her to better monitor and control his/her cognitive operations, consequently improving response accuracy. This result also supports Nelson and Narens’s model (Reference Nelson and Narens1994). Indeed, forced cognitive monitoring induced by the metacognitive questions results in better metacognitive control and adjustment to the task, thus improving cognitive performance.
A paradoxical finding was the lack of impact of adding metacognitive questions on MCST performances. Executive functions involve working memory, which was improved by the metacognitive protocol. Hence, better MCST performances might have been expected when adding metacognitive questions. However, the lack of improvement in performances on the MCST may be explained by a ceiling effect, as 97% of participants completed at least five categories. This result might have been different with a more complex version of the test, such as the original WCST. Indeed, the potentializing effect of metacognitive questions on neuropsychological performances is likely to occur in tasks requiring sophisticated cognitive processes such as working memory or executive processes. In our study, performances were enhanced in the working memory condition of the Digit span (backward), whereas no improvement was noticeable in the short-term memory condition (forward). Regarding Baddeley’s working memory model (Baddeley, Reference Baddeley2000), the short-term memory condition suggests a phonological loop, which is a slave system. Conversely, the working memory condition requires the central executive component, which is a high-level system, close to sophisticated executive processes.
These results are also consistent with previous findings suggesting that problem-solving performance and efficiency (such as mathematics) may be improved by metacognitive prompting (Hoffman & Spatariu, Reference Hoffman and Spatariu2008). Unlike feedback, which provides knowledge about results (Butler & Winne, Reference Butler and Winne1995), metacognitive prompting is an externally generated stimulus activating reflective cognition. It promotes the use of strategies such as self-monitoring, leading to enhanced learning or a better problem-solving outcome (Kauffman, Reference Kauffman2004). Indeed, problem-solving accuracy is partially influenced by the ability to use strategies to monitor and adjust the problem-solving process (Zimmerman, Reference Zimmerman1989). Prompting stimulates awareness of the task’s characteristics, performance strategies, and the evaluation of outcomes (Butler & Winne, Reference Butler and Winne1995). Under conditions of increasing complexity, metacognitive prompting may induce greater cognitive awareness and use of unmindful problem-solving strategies (Hoffman & Spatariu, Reference Hoffman and Spatariu2008). The benefits of increased metacognitive awareness are more limited for less complex tasks not requiring advanced problem-solving strategies (Veenman, Prins, & Elshout, Reference Veenman, Prins and Elshout2002). In the present study, we hypothesize that metacognitive questions were acting as metacognitive prompting. This is consistent with their significant impact on the backward digit span test, which is a complex and effortful task, and their lack of impact on the MCST and Faces Test, which are relatively easy.
CONCLUSIONS
The present findings emphasize the key role of “on-line” metacognition (or metacognitive awareness) on cognitive performances. They could be of interest for improving learning strategies in healthy persons by developing metacognitive training to improve problem-solving. They also have implications for neuropsychological assessment, as the addition of on-line metacognitive questions during testing has an impact on neuropsychological performances. Further research is needed to explore the impact of “on-line” metacognitive questions in cognitively impaired persons (Chiou, Carlson, Arnett, Cosentino & Hillary, 2011), to further develop metacognitive remediation programs.
ACKNOWLEDGMENT
The authors thank Professor J.M. Boiron and the Etablissement Français du Sang Aquitaine’s staff for their precious help in recruiting participants for this study. The authors also thank Ray Cooke for supervising the English of this manuscript. Conflict of interest: none. Funding sources: internally funded by INSERM U657 and EA 4139, University Bordeaux Segalen.
