Introduction
Evaluation of executive function is important in clinical settings because it is associated with functional outcomes, activities of daily living (ADL), decision-making capacity, and disruptive behaviors (Royall et al., Reference Royall, Mahurin and Gray1992; Stout et al., Reference Stout, Wyman, Johnson, Peavy and Salmon2003; Schillerstrom et al., Reference Schillerstrom, Rickenbacker, Joshi and Royall2007; Pereira et al., Reference Pereira, Yassuda, Oliveira and Forlenza2008). Executive function can be assessed by tests of frontal lobe functions, but many of these tests are impractical for routine use in older patients due to the time needed. For example, the Wisconsin Card Sorting Test (WCST) is commonly used as the gold standard for the assessment of executive function. However, the WCST takes time, and the instructions are difficult to understand for older people. Therefore, the Executive Interview (EXIT25) was developed to screen for executive function at the bedside (Royall et al., Reference Royall, Mahurin and Gray1992). EXIT25 provides a brief, clinically based, bedside measure of executive function that can be completed in 10–15 minutes, and EXIT25 scores have been used to predict functional status (Pereira et al., Reference Pereira, Yassuda, Oliveira and Forlenza2008), decision-making capacity (Schillerstrom et al., Reference Schillerstrom, Rickenbacker, Joshi and Royall2007), disruptive behaviors (Royall et al., Reference Royall, Mahurin and Gray1992; Stokholm et al., Reference Stokholm, Vogel, Gade and Waldemar2005), problems in self-care (Thabit et al., Reference Thabit2009), Instrumental Activities of Daily Living (IADL) (Royall et al., Reference Royall, Palmer, Chiodo and Polk2005), care level (Royall et al., Reference Royall, Mahurin and Gray1992), and learning ability (Allen et al., Reference Allen, Jain, Ragab and Malik2003). EXIT25 is available in many languages, including Dutch (Stokholm et al., Reference Stokholm, Vogel, Gade and Waldemar2005), Chinese (Chan et al., Reference Chan, Chiu and Lam2006), Portuguese (Pereira et al., Reference Pereira, Yassuda, Oliveira and Forlenza2008), and Spanish (Osorio et al., Reference Osorio, Lozar, Ramos and Aguera2009). The Clock Drawing Test (CDT) is a simple task that is often used to screen dementia. A certain level of cognitive function, including visuospatial ability, executive function, comprehension, and semantic memory, is needed to complete the CDT (Royall et al., Reference Royall, Cordes and Polk1998; Matsuoka, et al., Reference Matsuoka2013). The Executive Clock Drawing Task (CLOX) is a form of CDT designed to discriminate between executive and non-executive elements (Royall et al., Reference Royall, Cordes and Polk1998). The CLOX has been translated into Spanish (Royall et al., Reference Royall2003) and Chinese (Wong et al., Reference Wong2004). Both EXIT25 and CLOX scores have been used to evaluate executive function (Royall et al., Reference Royall, Mahurin and Gray1992; Reference Royall, Cordes and Polk1998), but they may assess different aspects of executive function. In this study, we developed Japanese versions of EXIT25 (J-EXIT25) and CLOX (J-CLOX), and examined their validity and reliability in older people. Next, we evaluated the aspects of executive function that these two tests will be examining.
Methods
Translation of J-EXIT25 and J-CLOX
The EXIT25 has 25 items, including an aural Trail Making task, verbal and design fluency, repetition of anomalous sentences, interference task, primitive reflex, go–no go task, echopraxia, and Luria hand sequences. Each item is rated on a 3-point scale: 0 (intact performance), 1 (a specific partial error or equivocal response), and 2 (specific incorrect response or failure to perform). The total score ranges from 0 to 50, with a higher score indicating greater impairment. A cut-off of 15/16 is recommended; essentially no disruptive behavior, as measured by the Nursing Home Behavior Problem Scale, was found in older participants with scores below this cut-off score (Royall et al., Reference Royall, Mahurin and Gray1992). A score of 15/50 best discriminates normal older participants from those with all-cause dementia (sensitivity = 93%, specificity = 83%, receiver operating characteristic (ROC) = 0.93, n = 200) (Royall et al., Reference Royall, Rauch, Roman, Cordes and Polk2001). In other studies, no normal participants have been found to score higher than 15 (Stokholm et al., Reference Stokholm, Vogel, Gade and Waldemar2005; Chan et al, Reference Chan, Chiu and Lam2006).
Quick EXIT (Larson and Heinemann, Reference Larson and Heinemann2010) and EXIT15 (Atkinson et al., Reference Atkinson2007) have been developed as short versions of EXIT25, and both use a number–letter task (Item 1), word fluency (Item 2), design fluency (Item 3), thematic perception (Item 4), anomalous sentence repetition (Item 5), a memory/distraction task (Item 6), an interference task (Item 7), motor impersistence (Item 12), a go–no go task (Item 15), echopraxia I (Item 16), a Luria hand sequence I (Item 17), a Luria hand sequence II (Item 18), and a serial order reversal task (Item 22). Quick EXIT also includes a grip task (Item 19), and EXIT15 uses a complex command task (Item 21) and imitation behavior (Item 24).
The CLOX is divided into two parts. CLOX1 is an unprompted clock-drawing task and CLOX2 is a copying task. In CLOX1, the participant is shown the blank side of a piece of paper on a light colored surface with a circle visible from the underside. On the blank side of the paper, the participant is instructed to “Draw a clock that says 1:45. Set the hands and numbers on the face so that a child could read them.” Once the individual begins to draw, no further assistance is provided. In CLOX2, the examiner turns over the paper and draws a clock face in the pre-drawn circle in the lower left corner. The examiner writes 12, 6, 3, and 9, fills in the rest of the numbers, and sets the hands to show 1:45. The participant is asked to observe this procedure and copy the clock in the lower right corner. The CLOX has 15 items and assesses the accuracy of the clock's overall form, size, and elements (hands and numbers); the forms of these elements (e.g., hands as arrows, relative lengths, Arabic numbers); and the ability to employ a constructional sequence (placing the numbers 12, 6, 3 and 9 first). The CLOX is scored on a 15-point scale, with a higher score reflecting greater CDT performance. A CLOX1 score of 10/15 or a CLOX2 score of 12/15 represents the fifth percentile (2 standard deviations (SD) below the mean) for the young adult reference group (Royall et al., Reference Royall, Cordes and Polk1998).
With the original authors’ permission, the English versions of EXIT25 (Royall et al., Reference Royall, Mahurin and Gray1992) and CLOX (Royall et al., Reference Royall, Cordes and Polk1998) were translated into Japanese by one of the authors (Teruyuki Matsuoka). In the translation, some items of EXIT25 had to be modified to fit with Japanese culture. In the number–letter task (Item 1), the alphabet was changed to Japanese hiragana. For word fluency (Item 2), letter “a” was changed to letter “ka,” which is commonly used in Japanese word fluency tasks. In anomalous sentence repetition (Item 5), the original anomalous sentences were changed to famous Japanese proverbs and songs. In the interference task (Item 7), the word “brown” printed in black was changed to the word “red” (in Japanese) printed in blue because the word “red” (in Japanese) printed in blue was used in the Japanese version of the Stroop Test. In the serial order reversal task (Item 22), the participants recite the months of a year backwards in the original version. Since the months of a year are simple numerical values in the Japanese language, citing the months of a year backwards is equivalent to a simple digit span backwards. Therefore, in J-EXIT25, this task was changed and the participants were asked to recite the days of a week backwards. We followed a standard back-translation procedure to ascertain the semantic equivalence of the Japanese version with the original English version. The initial versions of J-EXIT25 and J-CLOX were administered preliminarily to 14 participants. Minor modifications to these versions were made to produce the final versions of the scales.
Participants
The participants were 201 consecutive older persons (62 men and 139 women; mean age ± SD, 78.4 ± 6.4 years; mean education ± SD, 11.4 ± 3.0 years) who visited the Center for Diagnosis of Dementia at Kyoto Prefectural University of Medicine (n = 172), Ayabe City Hospital (n = 8) or Fukuchiyama City Hospital (n = 2), or who lived in a nursing home (n = 19). The inclusion criteria were (i) age: ≥65 years; (ii) a Clinical Dementia Rating (CDR) (Hughes et al., Reference Hughes, Begr, Danziger, Coben and Martin1982) score: <3; and (iii) a Mini-Mental State Examination (MMSE) (Folstein et al., Reference Folstein, Folstein and Mchugh1975; Sugishita, Reference Sugishita2012) score: ≥10. All participants were assessed comprehensively by a geriatric psychiatrist and all had information available for medical and social histories, physical and neurological examinations, neuropsychological tests, laboratory data, and brain imaging. Cognitive impairments were assessed by clinical examination and neuropsychological tests, and functional status was confirmed by family members or other caregivers using interview, the IADL scale, Physical Self-Maintenance Scale (PSMS) (Lawton and Brody, Reference Lawton and Brody1969), and the Neuropsychiatric Inventory (NPI-12) (Cummings et al., Reference Cummings, Mega, Gray, Rosenberg-Thompson, Carusi and Gornbein1994). The diagnosis was made by a geriatric psychiatrist according to the International Classification of Disease (ICD-10). Dementia with Lewy bodies (DLB) and mild cognitive impairment (MCI) were diagnosed using the DLB consensus criteria (McKeith et al., Reference Mckeith2005) and the Petersen (Reference Petersen2004) criteria respectively. The Ethics Committee of Kyoto Prefectural University of Medicine approved the study, and informed consent was obtained from all the participants.
Assessment
All the participants were evaluated using the J-EXIT25, J-CLOX, CDR, MMSE, Frontal Assessment Battery (FAB) (Dubois et al., Reference Dubois, Slachevsky, Litvan and Pillon2000), IADL scale and PSMS, and NPI-12. Except for J-EXIT25 and J-CLOX, the Japanese versions of these tests have been validated and are commonly used in Japan.
The CDR is an observation scale for assessment of six domains – memory, orientation, judgment, community affairs, home and hobbies, and personal care – with the following scorings: CDR 0, healthy; CDR 0.5, questionable dementia; CDR 1, mild dementia; CDR 2, moderate dementia; and CDR 3, severe dementia. The CDR rater was aware of the MMSE score.
The FAB consists of six items that explore different aspects of frontal lobe function. Six items include similarities (conceptualization), lexical fluency (mental flexibility), motor series (programming), conflicting instructions (sensitivity to interference), go–no go (inhibitory control), and prehension behavior (environmental autonomy). The score for each item ranges from 0 to 3 and the total FAB score is 18, with a lower score indicating more frontal lobe dysfunction. The IADL scale assesses five abilities of the participants to use telephone, go shopping, use public or private transport, manage medication, and manage money for both genders, and additionally to prepare meals, do housework, and do laundry for women. The total IADL score is 5 in men and 8 in women. The PSMS consists of six items related to physical activities: toileting, feeding, dressing, grooming, ambulating, and bathing. On the IADL and PSMS, a lower total score indicates greater impairment of ADL.
The NPI is a caregiver-based clinical instrument that evaluates 12 domains of neuropsychiatric symptoms in dementia: delusions, hallucinations, agitation, depression, anxiety, euphoria, apathy, disinhibition, irritability, aberrant motor behavior, sleep disturbance, and eating problems. The frequency score ranges from 0 to 4 points and the severity score ranges from 0 to 3 points. The score for each NPI subscale is created by multiplying the frequency and severity scores, with a maximum score of 12. The total score ranges from 0 to 144 points, with a higher score indicating greater severity of symptoms.
The nursing care level was also assessed. In the Japanese long-term insurance system, nursing care requirement is classified into seven levels: Levels 1 and 2 for people who need no continuous care, but need some help in daily life, such as support for dressing; and Levels 3–7 for those who need continuous care because of dementia and being bedridden. A higher level indicates a more severe mental and physical status. The level is determined based on the findings of the care need certification committee, which comprises specialists in public health, medical care, and welfare.
If possible, the WCST (Keio version: KWCST) (Kato, Reference Kato1988) was also administered. The KWCST was developed by reducing the number of cards to 48 from 128. The participants were instructed that “this is a test of classification based on any of the three categories of color, shape, or number.” The KWCST was performed with the KWCST FS version software on a computer display. In the KWCST, we recorded the number of categories achieved (CA) and the number of perseverative errors of the Nelson type (PEN).
Analysis of concurrent validity
First, we examined the concurrent validity and reliability of J-EXIT25 and J-CLOX in all the participants. Concurrent validity was assessed based on the correlation between scores in the two scales and those in the Japanese FAB, a standard scale for the assessment of executive functions that has satisfactory concurrent validity and good internal and external reliability (Nakaaki et al., Reference Nakaaki2007). For the participants who were able to undergo the KWCST, correlations between the KWCST scores and the J-EXIT25 and J-CLOX scores were also assessed.
Analysis of reliability
Inter-rater reliability was examined by calculation of intraclass correlation coefficients (ICCs) based on scores for ten participants (5.0% of all the participants; given by two raters: Teruyuki Matsuoka and Shogo Taniguchi) who were blinded to each other's assessments. One of the raters (Teruyuki Matsuoka) evaluated actively and the other (Shogo Taniguchi) observed passively. Another ten participants (5.0% of all the participants) were assessed four weeks later by the same raters and the test-retest reliability was examined by the calculation of ICCs. The internal consistency reliability was examined using Cronbach's coefficient α.
Analyses in healthy, MCI, and dementia participants
Next, we examined the J-EXIT25 and J-CLOX scores in healthy participants and those with MCI and dementia. Demographic and neuropsychological data for each CDR group (CDR 0, 0.5, 1, and 2) were compared using one-way analysis of variance (ANOVA) with the Bonferroni post hoc tests and χ2 tests. Independent-group tests were used to compare between groups and the Pearson correlation tests were used to evaluate associations between data. ROC analysis was used to compare the diagnostic performance of J-EXIT25 with those of J-CLOX1, J-CLOX2, MMSE, and FAB for discriminating between non-dementia (healthy and MCI) and dementia participants. Stepwise regression analyses were used to identify predictor variables associated with ADL (IADL and PSMS) and the care level. Age and J-EXIT25, J-CLOX1, J-CLOX2, MMSE, and FAB scores were used as independent variables in stepwise regression analysis of IADL scores, and these variables and sex were used as independent variables in similar analyses of the PSMS and care level. A backward method in the stepwise regression with p = 0.05 for entry and p = 0.10 for removal were used. Data were analyzed using SPSS 22 for Windows (IBM Corp., Armonk, NY, USA); and p < 0.05 was considered statistically significant.
Results
Characteristics of all participants
The mean scores ± SD were 23.0 ± 5.2 on MMSE, 13.6 ± 6.1 for J-EXIT25, 10.2 ± 3.5 for J-CLOX1, 13.3 ± 2.0 for J-CLOX2, and 12.8 ± 2.8 on FAB. The F0 category (n = 131) was the most common diagnosis, followed by healthy subjects (n = 45), major depressive disorder (n = 11), adjustment disorders (n = 4), delusional disorder (n = 2), chronic hallucinatory psychosis (n = 2), schizophrenia (n = 1), bipolar affective disorder (n = 1), insomnia (n = 1), somatoform disorder (n = 1), epilepsy (n = 1), and mental disorder, not otherwise specified (F99) (n = 1). In the F0 category, Alzheimer's disease (AD; n = 81) was the most common diagnosis, followed by MCI (n = 36), unspecified dementia (n = 8), vascular dementia (VD; n = 4), and DLB (n = 2).
Concurrent validity
The J-EXIT25 scores were significantly correlated with the FAB scores (r = –0.749, p < 0.001). There were also significant correlations between scores for the short versions of EXIT25 (Quick EXIT and EXIT15) and FAB scores (r = –0.758, p < 0.001; r = –0.734, p < 0.001). The J-CLOX1 and J-CLOX2 scores were also significantly correlated with the FAB scores (r = 0.598, p < 0.001; r = 0.578, p < 0.001). The KWCST evaluation was performed in 112 participants. Some of the participants were reluctant to receive the KWCST because they were not familiar with computer, and some were not able to understand the instructions of the KWCST; thus, it was not possible for all the participants to undergo the KWCST. J-EXIT25 scores were significantly correlated with those for CA (r = –0.197, p = 0.037) and PEN (r = 0.336, p < 0.001), whereas J-CLOX1 and J-CLOX2 scores did not show these correlations. Quick EXIT and EXIT15 scores were significantly correlated with the CA scores (r = –0.242, p = 0.010; r = –0.264, p = 0.005) and PEN (r = 0.410, p < 0.001; r = 0.415, p < 0.001).
Reliability
The inter-rater ICCs for J-EXIT25, J-CLOX1, and J-CLOX2 were 0.952 (confidence interval (CI) = 0.805–0.988), 0.970 (CI = 0.880–0.993), and 0.969 (CI = 0.877–0.992) (all p < 0.001) respectively, and the test-retest ICCs were 0.911 (CI = 0.663–0.978), 0.903 (CI = 0.633–0.975), and 0.916 (CI = 0.684–0.979) (all p < 0.001) respectively. Cronbach's coefficient α was 0.718 for the 25 items on J-EXIT25, 0.822 for the 15 items on J-CLOX1, and 0.671 for the 15 items on J-CLOX2. The inter-rater ICCs for Quick EXIT and EXIT15 were 0.982 (CI = 0.929–0.996) and 0.974 (CI = 0.896–0.994) respectively, and the test-retest ICCs were 0.965 (CI = 0.866–0.991) and 0.963 (CI = 0.860–0.991) respectively (all p < 0.001). Cronbach's coefficient α was 0.749 for the 14 items on Quick EXIT and 0.730 for the 15 items on EXIT15.
Analyses in healthy, MCI, and dementia participants
Demographic and neuropsychological data for each CDR group (CDR 0, 0.5, 1, and 2) in healthy, MCI, and dementia participants (n = 176) are shown in Table 1. For J-EXIT25, J-CLOX1, J-CLOX2, MMSE, and FAB, one way ANOVA with the Bonferroni post hoc tests mostly showed significant differences among the CDR groups, although there was no significant difference between CDR 0 and CDR 0.5 for J-CLOX2. Since there were significant differences in age and education, analysis of covariance (ANCOVA) with the Bonferroni post hoc tests using age and education as covariates was also performed. This analysis also showed significant differences among the CDR groups for J-EXIT25 (Figure 1). For J-CLOX1, age was only a significant covariate, and ANCOVA with the Bonferroni post hoc tests using age as a covariate showed that there were significant differences among the CDR groups (Figure 1). For J-CLOX2, age and education were not significant covariates.
Clinical characteristics of participants (n = 176) in different CDR groups
Notes: Data are shown as the mean ± standard deviation, except for sex.
ap < 0.001 vs. CDR 0; bp < 0.05 vs. CDR 0; cp < 0.001 vs. CDR 0.5; dp < 0.05 vs. CDR 0.5; ep < 0.001 vs. CDR 1; fp < 0.05 vs. CDR 1.
CDR: Clinical Dementia Rating; IADL: Instrumental Activities of Daily Living; J-EXIT25: Japanese version of Executive Interview; J-CLOX: Japanese version of Executive Clock Drawing Task; MMSE: Mini-Mental State Examination; FAB: Frontal Assessment Battery; PSMS: Physical Self-Maintenance Scale.
Results of analysis of covariance (ANCOVA) with the Bonferroni post hoc tests. In J-EXIT25 and J-CLOX1, ANCOVA showed significant differences among CDR groups. *p < 0.001, **p < 0.05.
Receiver operating characteristic analysis
Receiver operating characteristic curves for J-EXIT25, J-CLOX1, J-CLOX2, MMSE, and FAB in healthy (n = 45), MCI (n = 36) and dementia participants (n = 95) are shown in Figure 2. The areas under the ROC curve (AUC) for comparing the diagnostic performance for discriminating between non-dementia (n = 81) and dementia (n = 95) participants were 0.916 (CI = 0.877–0.956) for MMSE (p < 0.001), 0.878 (CI = 0.827–0.930) for J-EXIT25, 0.818 (CI = 0.756–0.880) for J-CLOX1 (p < 0.001), 0.789 (CI = 0.721–0.856) for FAB (p < 0.001), and 0.730 (CI = 0.657–0.803) for J-CLOX2.
Results of receiver operating characteristic (ROC) analysis. For comparison of diagnostic performance in discriminating between non-dementia and dementia participants, the area under the ROC curve (AUC) of MMSE was the highest, followed by J-EXIT25, J-CLOX1, FAB, and J-CLOX2.
Scores below the J-EXIT25 cut-off score of 15/16 were obtained for 100% of healthy participants, 93% of MCI participants, and 35% of dementia participants, giving a sensitivity of 65% and specificity of 93% for discriminating between non-dementia and dementia (Youden's index: 0.58). The highest Youden's index was 0.63 with the J-EXIT25 cut-off score of 13/14 (sensitivity: 78%, specificity: 85%). The NPI total score was significantly lower in the participants with the J-EXIT25 score below the cut-off score of 15/16 (n = 108) compared with those with the J-EXIT25 score above the cut-off score (n = 68) (3.3 ± 6.8 vs. 11.1 ± 12.6, p < 0.001). On subscales of the NPI, delusions, agitation, and irritability, scores in the participants with J-EXIT25 scores below the cut-off were significantly lower than for the participants with J-EXIT25 scores above the cut-off (the Bonferroni–Holm corrected p < 0.05).
Scores above the J-CLOX1 cut-off of 9/10 were obtained in 89%, 81%, and 39% of healthy, MCI, and dementia participants respectively, giving a sensitivity of 61% and specificity of 85% for discriminating between non-dementia and dementia (the highest Youden's index (0.46)). The NPI total score was significantly lower in the participants with a J-CLOX1 score above the cut-off (n = 106) compared with those with a J-CLOX1 score below the cut-off (n = 70) (4.8 ± 8.2 vs. 8.8 ± 12.2, p = 0.018). No NPI subscales differed significantly between these groups after the Bonferroni–holm correction. For J-CLOX2, scores above the cut-off of 11/12 were obtained for 100%, 97%, and 74% of healthy, MCI, and dementia participants respectively, with a sensitivity of 26% and a specificity of 99% for discriminating between non-dementia and dementia (Youden's index: 0.25). The highest Youden's index was 0.35 with the J-CLOX2 cut-off score of 13/14 (sensitivity: 63%, specificity: 72%).
Correlation analyses
The J-EXIT25 scores were significantly correlated with age (r = 0.448, p < 0.001), education (r = –0.364, p < 0.001), and scores on the MMSE (r = –0.691, p < 0.001), J-CLOX1 (r = –0.605, p < 0.001), J-CLOX2 (r = –0.595, p < 0.001), and PSMS (r = –0.487, p < 0.001) in healthy, MCI, and dementia participants. These correlations remained significant after the Bonferroni–Holm corrections. Among the 12 items of the NPI, there were significant correlations between J-EXIT25 scores and those for delusions (r = 0.276, p < 0.001), hallucinations (r = 0.206, p = 0.006), agitation (r = 0.299, p < 0.001), apathy (r = 0.162, p = 0.032), and irritability (r = 0.241, p < 0.001); however, the significant relationships with hallucinations and apathy disappeared after the Bonferroni–Holm corrections.
J-CLOX1 scores were significantly correlated with age (r = –0.328, p < 0.001), education (r = 0.157, p = 0.038), and scores on the MMSE (r = 0.636, p < 0.001), J-CLOX2 (r = 0.607, p < 0.001), and PSMS (r = 0.437, p < 0.001) in healthy, MCI, and dementia participants. These correlations remained significant after the Bonferroni–Holm corrections. On the NPI, there were significant correlations between J-CLOX1 scores and delusions (r = –0.184, p = 0.015), agitation (r = –0.211, p = 0.005), and irritability (r = –0.170, p = 0.024), but these correlations were not significant after the Bonferroni–Holm corrections.
J-CLOX2 scores were significantly correlated with age (r = –0.218, p = 0.004) and scores on the MMSE (r = 0.524, p < 0.001) and PSMS (r = 0.385, p < 0.001), but not with education (r = 0.05, p = 0.454), in healthy, MCI, and dementia participants. The correlations were also significant after the Bonferroni–Holm corrections. On the NPI, there were significant correlations between J-CLOX2 scores and delusions (r = –0.153, p = 0.043), agitation (r = –0.241, p = 0.001), and irritability (r = –0.223, p = 0.003), but the correlation with delusions was not significant after the Bonferroni–Holm corrections.
IADL scores for men (n = 52) were significantly correlated with J-EXIT25 (r = –0.662, p < 0.001), J-CLOX1 (r = 0.492, p < 0.001), and J-CLOX2 (r = 0.485, p = <0.001) scores. IADL scores for women (n = 124) were also significantly correlated with J-EXIT25 (r = –0.580, p < 0.001), J-CLOX1 (r = 0.548, p < 0.001), and J-CLOX2 (r = 0.398, p < 0.001) scores. In 61 participants who received Japanese long-term insurance, the care level was significantly correlated with J-EXIT25 (r = 0.479, p < 0.001), J-CLOX1 (r = –0.414, p = 0.001), and J-CLOX2 (r = –0.353, p = 0.005) scores.
The relationship between J-EXIT25 and J-CLOX1 scores is shown in Figure 3. We classified the participants into the following four groups based on their performance on the two tests using the recommended cut-off scores: Group 1: those with high performance on both tests (n = 82); Group 2: high performance on J-EXIT25 and low performance on J-CLOX1 (n = 26); Group 3: low performance on J-EXIT25 and high performance on J-CLOX1 (n = 24); and Group 4: low performance on both tests (n = 44).
Relationship between J-EXIT25 and J-CLOX1. Participants were classified as those with high performance on both tests (Group 1); high performance on J-EXIT25 and low performance on J-CLOX1 (Group 2); low performance on J-EXIT25 and high performance on J-CLOX1 (Group 3); and low performance on both tests (Group 4).
Regression analyses
The results of stepwise regression analyses in healthy, MCI, and dementia participants are shown in Table 2. In stepwise regression analysis of IADL scores, MMSE and J-EXIT25 scores were significantly independent predictors in men (adjusted R2 = 0.565, F = 34.058, p < 0.001), and the MMSE score, age, and the J-CLOX1 score were significantly independent predictors in women (adjusted R2 = 0.419, F = 30.573, p < 0.001). In stepwise regression analysis of PSMS scores, J-EXIT25, MMSE, and J-CLOX1 scores were significantly independent predictors (adjusted R2 = 0.281, F = 23.796, p < 0.001). In stepwise regression analysis of the care level, the J-EXIT25 score was the only significantly independent predictor (adjusted R2 = 0.216, F = 17.553, p < 0.001).
Results of regression analyses
Notes: IADL: Instrumental Activities of Daily Living; J-EXIT25: Japanese version of Executive Interview; J-CLOX: Japanese version of Executive Clock Drawing Task; MMSE: Mini-Mental State Examination; PSMS: Physical Self-Maintenance Scale; VIF: Variance Inflation Factor.
Discussion
The results of the study show the satisfactory concurrent validity and reliability of J-EXIT25 and J-CLOX, and of the short versions of EXIT25 (Quick EXIT and EXIT15) in older people. The J-EXIT25 and J-CLOX scores were also associated with ADL, care level, and neuropsychiatric symptoms, which are thought to reflect executive function. These results suggest that J-EXIT25 and J-CLOX are useful for the assessment of executive function in clinical settings. While the J-EXIT25 score was involved specially in IADL in men, PSMS, care level, and delusions, the J-CLOX1 score was specially related to IADL in women and PSMS. Therefore, J-EXIT25 and J-CLOX1 might evaluate different aspects of executive function.
The J-EXIT25 and J-CLOX1 scores showed significant differences among CDR groups, whereas the J-CLOX2 scores did not differ significantly between the CDR 0 and CDR 0.5 participants. These three scales are thought to assess different cognitive domains: EXIT25 and CLOX1 for executive function and CLOX2 for visuospatial praxis (Royall et al., Reference Royall, Cordes and Polk1998.). Thus, our findings may indicate that executive function is mildly impaired in the pre-dementia stage.
All the healthy participants in the study had J-EXIT25 scores below the cut-off of 15/16. In older people with cognitive dysfunction with scores below the cut-off score, neuropsychiatric symptoms, including delusions, agitation, and irritability, were significantly milder than in those with scores above the cut-off score. The J-EXIT25 scores were correlated with the neuropsychiatric symptoms measured by the NPI. Previous study showed that the executive function measured by EXIT25 was associated with disruptive behaviors in older people (Royall et al., Reference Royall, Mahurin and Gray1992; Stokholm et al., Reference Stokholm, Vogel, Gade and Waldemar2005). Using the original EXIT25, essentially no disruptive behavior was found in older participants below the cut-off score (Royall et al., Reference Royall, Mahurin and Gray1992). In the Dutch (Stokholm et al., Reference Stokholm, Vogel, Gade and Waldemar2005) and Chinese (Chan et al., Reference Chan, Chiu and Lam2006) versions of EXIT25, scores for normal participants were also below the cut-off of 15/16. The consistent findings among these studies indicate that the cut-off score on the J-EXIT25 is useful for the screening of impairment of executive function.
In discriminating between non-dementia and dementia, the specificities and sensitivities of J-EXIT25 and J-CLOX were high and low respectively. In this study, the mean MMSE score was relatively high and the mean J-EXIT25 and J-CLOX scores were within normal ranges when the recommended cut-off scores were used. The severity in most of the participants with dementia was relatively mild and their executive dysfunction was also suggested to be mild. Therefore, the number of dementia participants with executive dysfunction was relatively low, and this might have caused the low sensitivities of J-EXIT25 and J-CLOX. Previous studies have also shown high specificity and low sensitivity of CLOX in MCI (Forti et al., Reference Forti, Olivelli, Rietti, Maltoni and Ravaglia2010) and subcortical ischemic vascular disease (Wong et al., Reference Wong2004). We also found that the CDT-copy task had high specificity and low sensitivity in differentiating AD from MCI or healthy participants (Kato et al., Reference Kato2013). The J-EXIT25 and J-CLOX cut-off scores might be insufficient to distinguish between non-dementia and dementia, especially mild dementia. However, the J-EXIT25 and J-CLOX1 scores for executive function were superior to the J-CLOX2 score for visuospatial praxis in discriminating between non-dementia and dementia. Screening for executive function may be more useful than for visuospatial praxis in discriminating between non-dementia and dementia.
EXIT25 and CLOX1 have been used to evaluate executive function (Royall et al., Reference Royall, Mahurin and Gray1992; Reference Royall, Cordes and Polk1998). Both J-EXIT25 and J-CLOX1 scores were associated with age, scores on the MMSE, IADL, and PSMS and care level. However, differences between J-EXIT25 and J-CLOX1 should be noted. For example, J-EXIT25 differentiated the severity of neuropsychiatric symptoms, including delusions, agitation, and irritability, when the recommended cut-off score was used, while J-CLOX1 did not do so. The J-EXIT25 scores were also correlated with severity in delusions, which are considered to be associated with the frontal lobe dysfunction (Matsuoka et al., Reference Matsuoka2010). In regression analyses, J-EXIT25 was a predictor of IADL in men, PSMS, and care level, and J-CLOX1 was a predictor of IADL in women and PSMS. These findings suggest that J-EXIT25 can better assess tool use and care level, while J-CLOX1 is a good scale to evaluate capacity to perform housework, which is usually done by older women in Japan.
These results are consistent with previous studies showing that the EXIT25 scores are good indicators for inhaler techniques (Allen et al., Reference Allen, Jain, Ragab and Malik2003), functional status (Pereira et al., Reference Pereira, Yassuda, Oliveira and Forlenza2008), problems in self-care (Thabit et al., Reference Thabit2009), and care level (Royall et al., Reference Royall, Mahurin and Gray1992). A previous study also showed that EXIT25 was effective for evaluating decision-making capacity, in contrast to CLOX and MMSE (Schillerstrom et al., Reference Schillerstrom, Rickenbacker, Joshi and Royall2007). Another study reported that IADL was associated with executive impairment, and that basic ADL was related to apathy, which suggests neuroanatomical differences between these behaviors (Stout et al., Reference Stout, Wyman, Johnson, Peavy and Salmon2003). As shown in Figure 3, both J-EXIT25 and J-CLOX1 have common and distinct psychometric properties. Thus, these two scales may assess different aspects of executive function. Since some regions, including the frontal, temporal, and parietal lobes, are involved in CDT (Matsuoka et al., Reference Matsuoka2013), J-CLOX1 may reflect executive function and other cognitive functions such as visuospatial ability and comprehension. There is a lack of data on this important issue and further studies, including imaging that focuses on differences between EXIT25 and CLOX1, are clearly warranted.
The current study has several limitations despite the inclusion of a large number of participants. First, we could not administer the KWCST in some of the participants, and information on care level was only available for the participants who received long-term care insurance. Since we excluded some of the participants who were reluctant to receive the KWCST and who were not able to understand the instruction of the KWCST in correlation analysis, the J-EXIT25 might be weakly correlated to the KWCST, and the J-CLOX might not be significantly correlated to the KWCST. Second, the dementia participants were heterogeneous, although most had AD. This may have led to relatively low sensitivities of J-EXIT25 and J-CLOX for discriminating non-dementia from dementia. Third, the CDR rater was aware of the MMSE score and this may have caused relatively high AUC for MMSE. Fourth, we did not examine the divergent validity. These limitations should be noted in the interpretation of our results.
In summary, J-EXIT25 and J-CLOX were found to be valid and reliable instruments for the assessment of executive function in older people. J-EXIT25 and J-CLOX have common and distinct psychometric properties in the assessment of executive function. These scales have the potential to contribute to capacity assessment in various clinical settings in Japan as reported in other countries.
Conflict of interest
None.
Description of authors’ roles
T. Matsuoka translated J-EXIT25 and J-CLOX, designed the study, collected and analyzed the data, and wrote the paper. J. Narumoto, H. Koumi, and Y. Kato translated J-EXIT25 and J-CLOX, designed the study, collected the data, and assisted in writing the paper. S. Taniguchi examined the inter-rater reliability, collected the data, and assisted in writing the paper. M. Ogawa, H. Fujimoto, A. Okamura, K. Shibata, and K. Nakamura collected the data and assisted in writing the paper. H. Uchida, S Nakaaki, M. Mimura, and K. Fukui supervised the study and writing of the paper.
Acknowledgments
We would like to thank Professor Donald Royall for his contribution to the translation process for the Japanese versions of EXIT25 and CLOX. This research is supported by the “Redesigning Communities for Aged Society” project (Research Institute of Science and Technology for Society, Japan Science and Technology Agency: RISTEX, JST).
