INTRODUCTION
Executive functions refer to the ability to respond in an adaptive manner to novel situations and are the basis of many cognitive, emotional, and social skills. Executive functions include volition, planning, purposive action, and effective performance, each of which involves a set of activity-related behaviors (Lezak, Howieson, & Loring, Reference Lezak, Howieson and Loring2004) that can be referred to as “cognitive control.” Deficits in executive functions frequently occur after acquired brain injury, particularly after damage to the dorsolateral prefrontal circuit that connects the lateral convexity or the frontal lobe to subcortical structures, such as the dorsolateral portions of the caudate nucleus, and distinct areas of the globus pallidus, substantia nigra, and the thalamus (Cummings, Reference Cummings1995). Decreased executive functioning is also linked to age-related cognitive decline (Brennan, Welsh, & Fisher, Reference Brennan, Welsh and Fisher1997), possibly in relation to impairment of (parts of) the supervisory attentional system (SAS) and largely independent of a more general decline in processing speed (Andres & Van der Linden, Reference Andres and Van der Linden2000). Furthermore, impaired executive functioning has been reported in several psychiatric disorders, for example, schizophrenia (Kerns, Nuechterlein, Braver, & Barch, Reference Kerns, Nuechterlein, Braver and Barch2008) and major depressive disorder (Channon & Green, Reference Channon and Green1999).
Several measures have been developed to tap crucial aspects of executive functioning, such as rule detection, concept shifting, and sorting, in clinical populations. Well-known tests that are used in clinical practice are the Wisconsin Card Sorting Test (WCST; Heaton, Reference Heaton1981), the Tower of London (ToL; Shallice, Reference Shallice1982), and the Trail Making Test, Part B (TMT-B; Reitan, Reference Reitan1956). Although frequently used, these tests typically only tap single aspects of executive functioning (e.g., either planning or rule detection) and a patient’s performance on one executive function test may thus have little predictive value for performance on another test, let alone in complex real-world situations (“ecological validity”; see also Burgess, Alderman, Evans, Emslie, & Wilson, Reference Burgess, Alderman, Evans, Emslie and Wilson1998; Chan, Shum, Toulopoulou, & Chen, Reference Chan, Shum, Toulopoulou and Chen2008; Odhuba, van den Broek, & Johns, Reference Odhuba, van den Broek and Johns2005). Tests that resemble “real-world” tasks, such as the Behavioural Assessment of the Dysexecutive Syndrome (BADS; Wilson, Alderman, Burgess, Emslie, & Evans, Reference Wilson, Alderman, Burgess, Emslie and Evans1996) may help to overcome this issue, but are usually time-consuming as part of a full neuropsychological examination. Accurate and valid assessment of executive functions in clinical populations is also hampered by the paradoxical need to structure a situation in which patients can show whether and how well they can make structure for themselves (Lezak et al., Reference Lezak, Howieson and Loring2004).
A relatively new measure designed to assess aspects of executive functioning is the Brixton Spatial Anticipation Test, developed by Burgess and Shallice (Reference Burgess and Shallice1997) as part of the Hayling and Brixton tests. The task primarily measures a person’s ability to detect a rule, to follow it, and to switch to a new rule. It does not require a verbal or complex motor response by the patient, it has no time restriction, and takes up to 10 minutes to administer. To date, normative data on this task are limited, particularly in the elderly. Burgess and Shallice (Reference Burgess and Shallice1997) report on a relatively small sample (n = 121) of healthy individuals aged 18–80 years, which included only 18 persons >66 years. Another study showed significantly worse performance of older persons (n = 48; aged between 60 and 70 years) compared to younger adults (n = 47; aged between 20 to 30 years) (Andres & Van der Linden, Reference Andres and Van der Linden2000). More recently, Bielak, Mansueti, Strauss, & Dixon (Reference Bielak, Mansueti, Strauss and Dixon2006) published age-stratified normative data in a large sample of elderly persons (n = 457; ages between 53 and 90) Although several authors of the aforementioned studies point out modest correlations between the Brixton test and measures of (fluid) intelligence, educational level has not been included and corrected for in the normative data in any of these studies. This is important because executive functioning is closely related to intelligence in healthy (de Frias, Dixon, & Strauss, Reference de Frias, Dixon and Strauss2006), as well as in brain-injured persons (Wood & Liossi, Reference Wood and Liossi2007). Furthermore, although elderly persons in the general population more frequently have a lower level of education, the education levels in the reported samples was substantially above average (Burgess & Shallice: estimated mean IQ ~110, Andres et al.: all participants ≥12 years of schooling, Bielak et al.: >90% of the participants had at least 12 years of education).
Additionally, studies on the applicability and validity of the Brixton test in different patient groups are also limited. Examples include a study by Burgess and Shallice (Reference Burgess and Shallice1997) who report on 77 neurological patients with heterogeneous lesions of mixed etiology, showing worse performance than healthy controls. A limited number of recent studies report on Brixton test performance in patients with Parkinson’s disease (Rochester et al., Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2004, Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2005), frontal lesions (Reverberi, Lavaroni, Gigli, Skrap, & Shallice, Reference Reverberi, Lavaroni, Gigli, Skrap and Shallice2005), eating disorders (Tchanturia et al., Reference Tchanturia, Anderluh, Morris, Rabe-Hesketh, Collier, Sanchez and Treasure2004), unipolar depression (Gohier et al., Reference Gohier, Ferracci, Surguladze, Lawrence, El and Kefi2008), and in children with Attention-Deficit Hyperactivity Disorder (Shallice et al., Reference Shallice, Marzocchi, Coser, Del, Meuter and Rumiati2002), but a careful examination of the Brixton test performance in clinical populations is lacking. In addition, test-retest reliability was reported by Burgess and Shallice only (r = .71, p < .001).
The aim of the present study is twofold. First, we present age- and education-adjusted regression-based normative data for the Brixton test based on a large sample of healthy older persons. Second, we examine the applicability and validity of these norms in different patient groups and explore test-retest reliability using the classification according to Strauss, Sherman, & Spreen (Reference Strauss, Sherman and Spreen2006).
METHODS
Participants
The present study included a group of healthy participants (n = 283; ages between 55 and 92 years) and five different patient groups, namely, patients with stroke (n = 106; ages between 17 and 83 years), diabetes mellitus (n = 376; ages between 52 and 86 years), MCI/early dementia (n = 70; ages between 32 and 86 years), psychiatric disorders (n = 63; ages between 18 and 57 years), and patients with Korsakoff’s syndrome (n = 41; ages between 34 and 68 years).
The healthy participants were recruited through an advertisement in the local newspaper, through the patients participating in the Hoorn study (van den Berg et al., Reference van den Berg, Dekker, Nijpels, Kessels, Kappelle and De Haan2008), the Anglo-Danish-Dutch Study of Intensive Treatment in People with Screen Detected Diabetes in Primary Care (ADDITION) study (Sandbaek et al., Reference Sandbaek, Griffin, Rutten, Davies, Stolk and Khunti2008), the Utrecht Diabetic Encephalopathy Study (UDES) (Brands et al., Reference Brands, van den Berg, Manschot, Biessels, Kappelle, De Haan and Kessels2007), and through a database of Utrecht University, the Netherlands. Only healthy participants >50 years of age were included.
The patients with stroke were recruited through the departments of Neurology of the University Medical Center Utrecht and the St. Elisabeth and TweeSteden Hospital in Tilburg and were examined between 6 and 10 months after the stroke (mean 7.5 ± 1.3 months) (Cognition in Acute Stroke Patients and After Recovery study (CASPAR; Nys et al., Reference Nys, Van Zandvoort, De Kort, Jansen, Van der Worp, Kappelle and De Haan2005). Ninety-six patients had an ischemic cerebral infarct, 10 patients had an intracerebral hemorrhage. Forty-two patients had a stroke in the left hemisphere, 37 in the right hemisphere, 3 patients had a bilateral stroke, 15 patients had a stroke in the brain stem or cerebellum, and 9 patients had a stroke with an unclear localization.
The patients with diabetes (diabetes duration >1 year) were recruited through general practitioners in the regions of Breda (ADDITION), Hoorn (Hoorn Study), Utrecht, the Hofpoort Hospital in Woerden, and the Groene Hart Hospital in Gouda (UDES). Forty patients had type 1 diabetes and 336 had type 2 diabetes.
The patients with Mild Cognitive Impairment (MCI) or early dementia were recruited through the Alzheimer Center at the Radboud University Nijmegen Medical Center in Nijmegen, the Netherlands. Most patients had relatively mild cognitive decrements: mean Mini-Mental State Examination (MMSE) score was 26.3 ± 2.7, 16% had an MMSE score ≤23.
The patients with psychiatric disorders were recruited thought the outpatient clinic of the Regional Psychiatric Center in Woerden, the Netherlands, who were referred with cognitive complaints. The majority of patients was diagnosed with schizophrenia/psychosis (n = 30) or mood/anxiety disorders (n = 15). Other diagnoses included addiction/substance abuse (n = 2), personality disorder (n = 3), obsessive-compulsive disorder (n = 3), or other diagnoses (n = 10). None of the patients were psychotic at the time of testing.
The patients with Korsakoff’s syndrome were inpatients of the Korsakoff clinic of the Vincent van Gogh Institute for Psychiatry in Venray, the Netherlands. All patients were in the chronic, amnesic stage of the syndrome; none of the patients was in the confusional phase at the moment of testing. The amnesic syndrome was confirmed by extensive neuropsychological testing. All patients were alcohol-abstinent since their admittance to the clinic. They fulfilled the criteria for Korsakoff’s syndrome as described by Kopelman (Reference Kopelman2002). None of the patients fulfilled the clinical criteria for alcohol dementia (Oslin, Atkinson, Smith, & Hendrie, Reference Oslin, Atkinson, Smith and Hendrie1998).
All participants were of Caucasian descent and were able to communicate in the Dutch language. None of the patients or healthy participants had visual impairments or neglect that hampered the administration of the tests. Exclusion criteria for healthy participants were a fasting blood glucose ≥7.0 mmol/l (i.e., diabetes mellitus), history of alcohol abuse, psychiatric illness, or neurological disease (self report). Healthy participants recruited via the Hoorn Study were included only if they had no more than one risk factor for the metabolic syndrome (i.e., increased waist circumference, increased triglycerides, low high-density lipoprotein cholesterol, high blood pressure). The studies were approved by the local medical ethics committees and were completed in accordance with the guidelines of the Helsinki Declaration. Education level was recorded using seven categories (1 being the lowest, less than primary school, and 7 the highest, academic degree; Verhage, Reference Verhage1964) and transformed to number of years of education (Hochstenbach, Mulder, van Limbeek, Donders, & Schoonderwaldt, Reference Hochstenbach, Mulder, van Limbeek, Donders and Schoonderwaldt1998).
Brixton Spatial Anticipation Test
Although the Brixton test was developed as part of the Hayling and Brixton tests (Burgess & Shallice, Reference Burgess and Shallice1997), the present study focuses on performance on the Brixton test only. The Brixton test was administrated and scored according to the instructions in the manual (Burgess & Shallice, Reference Burgess and Shallice1997). Participants were presented with a 56-page stimulus booklet. Each page contained an array of 10 circles (2 rows of 5 circles), which were numbered from 1 to 10 (Figure 1). On each page, one circle was blue. The position of the blue circle changed form one page to the next and the changes were governed by a series of simple rules that changed without warning. Participants were presented with one page at a time and were asked to point to where they thought the blue circle would be on the next page based on the rule inferred form the previous pages. Responses were considered correct if they followed the present rule, and on trials where the rule changed, a response was correct if it followed the previous rule. The total number of errors across 55 trials was used as an outcome measure, higher scores thus reflect worse performance (maximum number of errors = 55).
Illustration of an actual sequence of the Brixton Spatial Anticipation Test.
Other Neuropsychological Measures
All participants performed a full neuropsychological examination covering the major cognitive domains. However, for the present analysis of the Brixton test, we included only the TMT, as another measure of executive functioning, and the Rey Auditory Verbal Learning Test (RAVLT; van der Elst, van Boxtel, van Breukelen, & Jolles, Reference van der Elst, van Boxtel, van Breukelen and Jolles2005), as a measure of verbal learning, to explore construct validity. It should be noted that performance on these tests will not be independent, for example, because memory at least in part relies on intact executive functioning (Duff, Schoenberg, Scott, & Adams, Reference Duff, Schoenberg, Scott and Adams2005), but theoretically it can be argued that performance on the Brixton test is more closely related to TMT (part B) performance than to performance on the RAVLT. For the TMT the score for part A (seconds), part B (seconds), and the (B − A)/A ratio were included, to control for performance on the TMT part A. For the RAVLT, the total score (the number of words recalled over the five learning trials, possible range 0–75), the delayed recall score (the total number of words after a 30 minute delay, possible range 0–15), and the delayed recognition (total correct hits and rejections after a 30 minute delay, possible range 0–30) were computed.
Participants also performed the Dutch version of the National Adult Reading Test (NART; Schmand, Lindenboom, & Van Harskamp, Reference Schmand, Lindenboom and Van Harskamp1992) as an estimate of (premorbid) verbal general intelligence. The NART and the TMT were not administered in the Korsakoff patients, and only half (n = 18) of the patients in this group completed the RAVLT. The TMT was not performed by the persons participating in the CASPAR study. The NART, TMT, and RAVLT scores were missing in 24, 15, and 17 healthy control participants, respectively, because of study design and/or logistic reasons.
Statistical Analysis
For presentation purposes only, the age distribution and mean Brixton error score of the healthy control group was calculated in 10-year sections (Table 1). Between-group differences in Brixton error scores were examined with analysis of covariance (ANCOVA) with age, sex, and years of education entered as covariates. The relation between the Brixton number of errors, age, sex, years of education, and NART-IQ was determined with Pearson correlations. Regression-based norms for use in clinical practice were determined by means of linear regression analysis, which included the Brixton error score as the dependent variable and the relevant demographical variables (based on the correlation analysis) as predictors (see Results section for details). Subsequently, these normative data were used to examine performance of the five patient groups, and the number of patients scoring “below average” (<16%) and “impaired” (<5%) were calculated. The test’s construct validity was explored by means of Principal Axis Factoring (PAF) with oblique (direct oblimin) rotation, including only those factors with an eigenvalue >1, which allows for correlation between different factors. Receiver Operating Characteristics (ROC) analyses were performed to examine classification accuracy for the control group compared to the patients with stroke and the patients with Korsakoff’s syndrome, respectively. The Area Under the Curve (AUC) was determined for each ROC curve, and cut-off points, sensitivity, and specificity were determined for the Brixton error score. Test-retest correlation was examined in 83 healthy controls who performed the Brixton test two times.
Age distribution and mean Brixton error score of the control group (n = 283)
RESULTS
The age distribution of the healthy control group (Table 1) shows that the majority of this group was aged between 60 and 79 (mean 67.4 ± 8.5 years). Table 2 shows characteristics of the healthy control group and the five patient samples. Between-group differences in Brixton error score, examined with age-, sex- and years of education-adjusted ANCOVA, revealed a main effect of group on the Brixton errors score, F(5, 930) = 18.0, p < .001. Subsequent age-, sex- and years of education-adjusted analysis showed that the patients with stroke, F(1, 384) = 21.5, p < .001, Korsakoff’s syndrome, F(1, 319) = 48.9, p < .001, and psychiatric disorders, F(1, 341) = 5.1, p < .05, performed significantly worse than the healthy control group. The Brixton error score of the other patient groups did not significantly differ from the healthy control group. Within the MCI/early dementia group, persons with a low MMSE score (≤23 points) performed significantly worse than persons with a higher MMSE score, F(1, 64) = 7.0, p < .05. Within the group of patients with stroke there was no difference in Brixton error score between the right- and left-hemisphere patients (p = .26). Also, no difference in Brixton score was found between the type 1 and type 2 diabetes patients, p = .77).
Characteristics of the healthy controls and the four patient groups
Note
Data are presented as mean ± SD unless otherwise specified.
Correlation analysis within the healthy control group showed that the Brixton number of errors correlated with age (r = .30, p < .001), years of education (r = −.18, p < .01), and NART-IQ (r = −.22, p < .001), but not with sex (r = .04, p = .50). To determine regression-based normative data, linear regression analysis was performed for the Brixton number of errors, with age and years of education entered as predictor variables to determine the function for the expected score (ES). Since a NART-IQ is not always available in clinical practice, this variable was not used as a predictor variable for the normative data. Linear regression analyses resulted in the following function for the Brixton number of errors:
Residue scores (RS) were computed by subtracting each individual’s expected score from the observed score (RS = OS − ES). The frequency distribution for the residue scores for the Brixton errors were converted into a percentile distribution (Table 3). Clinically, a score below the 5th percentile can be used as a cut-off point for an “impaired” performance (Lezak et al., Reference Lezak, Howieson and Loring2004).
Percentile distribution of the residue scores of the Brixton error score, based on the healthy control group (n = 283)
Application of these age- and education-adjusted norms on the five patient samples (Table 4) shows significant between-group differences of the residue scores, F(5, 933) = 21.1, p < .001. The patients with Korsakoff’s syndrome, stroke, and psychiatric disorders performed significantly worse than the control group. Performance of the patients with diabetes was largely similar to the control group. The number of patients performing at the “below average” and “impaired” level also differed significantly between the groups, χ2 (5) = 52.6, p < .001 and χ2 (5) = 72.9, p < .001, respectively, (Table 4). Interestingly, whereas only the patients with Korsakoff’s syndrome showed “impaired” performance significantly more often, three of the five patients groups (Korsakoff, stroke, psychiatric disorders) performed in the “below average” range more frequently, suggesting that the Brixton test is also sensitive to more subtle impairments. Although the patients with MCI/early dementia showed clear impairments on the RAVLT compared to the controls and most patient groups (Table 2), they did not show worse performance on the Brixton test (Table 4).
Brixton performance of different patient groups compared to the age- and education-adjusted normative data from healthy controls (n = 283)
Note
All comparisons were performed against the reference group.
* p < .05.
PAF with direct oblimin rotation was used to explore the rest’s construct validity. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.60, and Bartlett’s test of sphericity was significant, χ2(15) = 335.5, p < .001, indicating high sampling adequacy. The PAF yielded a multicomponent solution consisting of 2 factors with eigenvalues >1.0 that accounted for 60% of the variance. The RAVLT total score and delayed recall score loaded high on the first factor, and the TMT A showed a negative factor loading on this factor, which could be regarded as a verbal memory factor (eigenvalue 2.33, 39% of variance explained, factor loadings RAVLT total score 0.90, RAVLT delayed recall 0.82, TMT A –0.40). The second factor encompassed high loadings for the TMT (B − A)/A ratio score and the Brixton score. The RAVLT total score, delayed recall score, as well as the NART score showed negative factor loadings on this factor, which could be regarded as a mental flexibility factor (eigenvalue 1.28, 21% of variance explained, factor loadings TMT (B − A/A) ratio 0.76, Brixton 0.37, RAVLT total score −0.44, RAVLT delayed recall −0.38, NART-IQ –0.44).
The ROC-analysis of the Brixton test for the control group compared to the patients with Korsakoff’s syndrome resulted in a reasonable overall classification accuracy (AUC = 0.74; 95% confidence interval [CI] 0.66 – 0.82, p < .001). The optimal cut-off score of 19/20 errors resulted in a sensitivity of 81% and a specificity of 62%, which can be interpreted as adequate (Blake, McKinney, Treece, Lee, & Lincoln, Reference Blake, McKinney, Treece, Lee and Lincoln2002). For the control group, compared to the patients with stroke, however, the AUC was 0.56 (95% CI 0.50 – 0.63, p = .06), which is only marginally above chance level. Similarly, the optimal cut-off score of 18/19 resulted in a sensitivity of 56% and a specificity of 58%.
Eighty-three healthy controls performed the Brixton test again after 6 to 48 months. At baseline, the mean Brixton error score for these controls was 19.6 ± 8.0, at follow-up the mean score was 19.6 ± 5.3. Test-retest correlation was 0.61 (p < .001).
DISCUSSION
The current study presents age- and education-adjusted regression-based normative data for the Brixton Spatial Anticipation Test based on a sample of 283 older healthy adults that can be used in clinical practice. Application of these normative data in five different patient samples further showed that patients with Korsakoff’s syndrome, the stroke patients, and the patients with psychiatric disorders, as a group, showed significantly worse performance than healthy controls. Dysexecutive symptoms have been previously described in Korsakoff patients in relation to frontal lobe damage (Kessels, Kortrijk, Wester, & Nys, Reference Kessels, Kortrijk, Wester and Nys2008; Reed et al., Reference Reed, Lasserson, Marsden, Stanhope, Stevens and Bello2003). In stroke patients, the presence of impairments in executive functioning is largely dependent on lesion location, the frontal lobe in particular (Vataja et al., Reference Vataja, Pohjasvaara, Mantyla, Ylikoski, Leppavuori and Leskela2003), but also related to damage in other parts of the brain (Nys et al., Reference Nys, Van Zandvoort, De Kort, Jansen, Van der Worp, Kappelle and De Haan2005). Although an in-depth examination of the relation of stroke location and performance on the Brixton test is beyond the scope of the present study, lateralization differences in Brixton performance could not be demonstrated. In addition to a significantly worse performance on the Brixton test, the group of patients with psychiatric disorders also showed a significantly greater proportion of patients with a “below average” score, indicating that the Brixton test is also sensitive to more subtle impairments. When considering that the main psychiatric diagnosis of our sample of patients was psychotic disorders, this finding is in line with other studies showing dysexecutive problems in these patients (Hutton et al., Reference Hutton, Puri, Duncan, Robbins, Barnes and Joyce1998; Kerns et al., Reference Kerns, Nuechterlein, Braver and Barch2008). The results of this study further support the notion that persons with MCI or early dementia, probably due to Alzheimer’s disease, generally do not show marked decline in executive functioning in the early stages of their disease (Levinoff et al., Reference Levinoff, Phillips, Verret, Babins, Kelner, Akerib and Chertkow2006; Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999). As expected, these patients predominantly showed a worse performance on the RAVLT (Table 2). However, it should be noted that because the Brixton data for the MCI and early dementia group were collected as part of clinical assessment at a memory clinic, selection bias could have occurred. That is, dementia patients with more severe cognitive deficits may have been incapable of adequately performing complex executive tasks, such as the Brixton or the TMT B according to the test instructions (Ashendorf et al., Reference Ashendorf, Jefferson, O’Connor, Chaisson, Green and Stern2008), resulting in exclusion from the present study. The patients with diabetes in the present study largely performed at a similar level as the healthy controls, although there was a trend towards a larger proportion of patients performing at the “below average” level. This finding is in line with previous findings of cognitive decrements in relation to diabetes (Awad, Gagnon, & Messier, Reference Awad, Gagnon and Messier2004; Brands, Biessels, De Haan, Kappelle, & Kessels, Reference Brands, Biessels, De Haan, Kappelle and Kessels2005).
To our knowledge this is the first study that carefully examined the applicability and validity of the Brixton test in circumscribed clinical population samples. Several small case-control studies have used the Brixton test as part of a broader neuropsychological examination. These studies show that older patients with Parkinson’s disease (n = 20) show worse performance on the Brixton test compared to age-matched healthy controls (n = 10) (Rochester et al., Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2004, Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2005). Patients with unipolar depression (n = 20) also showed a marginally worse performance compared to age-matched healthy controls (n = 20) (Gohier et al., Reference Gohier, Ferracci, Surguladze, Lawrence, El and Kefi2008). In addition, females with anorexia nervosa (n = 34), but not females with bulimia nervosa (n = 19) performed worse than age-matched controls (n = 35) (Tchanturia et al., Reference Tchanturia, Anderluh, Morris, Rabe-Hesketh, Collier, Sanchez and Treasure2004). Two studies used a modified version of the original Brixton test and showed worse performance in a group of patients with prefrontal lesions (n = 40) (Reverberi et al., Reference Reverberi, Lavaroni, Gigli, Skrap and Shallice2005) and in children with Attention Deficit Hyperactivity Disorder (n = 31) (Shallice et al., Reference Shallice, Marzocchi, Coser, Del, Meuter and Rumiati2002), compared to age-matched controls.
The normative data presented in this study are based on linear regression analysis and included age and years of education as predictor variables. Although the reliability of regression-based norms is still debated (e.g., Fastenau, Reference Fastenau1998; Heaton, Avitable, Grant, & Matthews, Reference Heaton, Avitable, Grant and Matthews1999), we feel that this method allows for (1) careful examination of relevant predictor variables of the Brixton error score and (2) the use of the data of the entire sample instead of stratified norms based on different age and education groups, resulting in small comparison groups (see also, van Breukelen & Vlaeyen, Reference van Breukelen and Vlaeyen2005). Although normative data for the Brixton test are limited, the norms presented in the present study can be compared with the previously published norms in older persons by Bielak et al. (Reference Bielak, Mansueti, Strauss and Dixon2006). Examination of the residue scores (RS = OS – ES) for each age group presented in their study shows that their participants generally perform at the 30th–40th percentile of our distribution. This shows that our results are consistent with previously published norms, and further suggests that, compared to our norms, the norms by Bielak et al. may overestimate Brixton performance by ~10 percentiles, which may have been caused by the relatively high level of education of their study sample.
To date, examination of construct validity of the Brixton test is lacking altogether. The present factor analysis resulted in a two-factor solution in which a mental flexibility factor, which included the Brixton error score and TMT (B − A)/A ratio score, could be dissociated from a verbal memory factor. These results indicate that the Brixton error score is indeed more related with another measure of executive functioning (TMT (B − A)/A ratio) than with measures of memory and speed, which would be expected because both tests rely on mental flexibility and set shifting. This suggests satisfactory convergent validity for the Brixton test. Aspects such as rule detection are, however, not measured by the TMT, and future studies on the relation between the Brixton test and other measures of executive functioning should complement these findings. Moreover, the Brixton error score was not unrelated to memory function (RAVLT total score and delayed recall score). Although inferences about causality cannot be made based on the present cross-sectional study, this association suggests that adequate performance on learning and memory tasks presumes, at least in part, intact executive functioning (for a discussion see, Duff et al., Reference Duff, Schoenberg, Scott and Adams2005). The present study also provides a first indication of the diagnostic accuracy of the Brixton test. ROC analyses resulted in adequate sensitivity and specificity for the Brixton test when comparing the patients with Korsakoff’s syndrome with healthy controls (AUC = 0.74 (0.66 – 0.82)). However, for the stroke patients, compared to the healthy controls, diagnostic accuracy was less satisfying (AUC = 0.56 (0.50 – 0.63)), possibly reflecting larger heterogeneity in cognitive impairments in the stroke group compared to the Korsakoff group. Test-retest correlation was 0.61 in the present study, which is somewhat lower than the 0.71 that was reported by Burgess and Shallice (Reference Burgess and Shallice1997). According to the classification by Strauss et al. (Reference Strauss, Sherman and Spreen2006), this would imply marginal reliability, but it is comparable to test-retest reliability coefficients that are reported for several traditional measures of executive functioning, such as the WCST (test-retest reliability coefficients 0.39 to 0.72 for different WCST measures; Heaton, Chelune, Talley, & Kay, Reference Heaton, Chelune, Talley and Kay1993) and the BADS (0.22 to 0.85 for different BADS subtests; Jelicic, Henquet, Derix, & Jolles, Reference Jelicic, Henquet, Derix and Jolles2001). There was no apparent learning effect between the two measurements.
Strengths of the present study include the large number of healthy older participants, the use of regression-based norms that are adjusted for the confounding effects of age and educational level, and the systematic examination of the applicability and validity in different patient samples. Although the regression-based norms can be applied across a wide age-range, one should keep in mind that our population of healthy persons was aged >50 and <80, and application of these norms in very young or very old persons should be performed with some caution. Furthermore, the present study included both persons who participated in a research project (and therefore invariably performed the Brixton test) and clinical samples of patients, in which selection bias may have played a role.
In summary, the Brixton Spatial Anticipation Test is a relatively new measure of executive functioning. Although it measures only a single aspect of executive functioning, namely to detect and follow a rule, the Brixton test can be viewed as a useful addition to existing measures of executive functions because it does not require a verbal or complex motor response by the patient, it has a relatively short duration, and no apparent learning effect as demonstrated in test-retest analysis. The present study provides age- and education-adjusted regression-based norms for elderly persons, and shows that the Brixton test is a valid and useful measure of executive functioning in different groups of clinical patients.
ACKNOWLEDGMENTS
The authors report no conflict of interest. The information in this manuscript and the manuscript itself have never been published either electronically or in print.
Esther van den Berg and Ineke Brands were supported by Grant No. 2001.00.023 and 2003.01.004 of the Dutch Diabetes Research Foundation. Roy Kessels was supported by a 2006 Utrecht University High Potential research grant. Gudrun Nys and Martine van Zandvoort were supported by the Netherlands Heart Foundation (NHS 2000.023) and the Brain Foundation (The Netherlands). The ADDITION study in the Netherlands was funded by grants from Novo Nordisk Netherlands, GlaxoSmithKline Netherlands, and Merck Netherlands.
The authors would like to thank P. de Kort and B. Jansen (CASPAR), M. de Goede, H. Kortrijk and A. Wester (Korsakoff patients), L. Joosten-Weyn Banningh and N. van Schuylenborg-van Es (MCI/early dementia patients) for enrolment of patients. We also thank the Hoorn study group of the EMGO institute (VU University Medical Center), the Utrecht Diabetic Encephalopathy Study Group (University Medical Center Utrecht), the “Utrecht Diabetes Programma (UDP)” and the “IJsselstein Diabetes Project” (mentor: Ph.L. Salomé) for their assistence.
