BACKGROUND
Multiple sclerosis (MS) is an inflammatory demyelinating and neurodegenerative disease of the central nervous system (CNS). The prevalence of cognitive impairment in MS patients is up to 65% (Jongen, Ter Horst, & Brands, Reference Jongen, Ter Horst and Brands2012). Cognitive dysfunction may have a strong influence on working ability and a negative impact on the health-related quality of life of MS patients (Benito-León, Morales, Rivera-Navarro, & Mitchell, Reference Benito-León, Morales, Rivera-Navarro and Mitchell2003; Rao, Leo, Bernardin, & Unverzagt, Reference Rao, Leo, Bernardin and Unverzagt1991). The most commonly affected cognitive domains are information-processing speed, memory, attention, and verbal and visual-spatial learning (Ferreira, Reference Ferreira2010; Guimarães & Sá, Reference Guimarães and Sá2012). Patients may exhibit cognitive deficits regardless of their disease type and level of disability; however, patients presenting with severe disability may more often be found impaired (Amato et al., Reference Amato, Langdon, Montalban, Benedict, Deluca, Krupp and Comi2013; Bakirtzis et al., Reference Bakirtzis, Ioannidis, Messinis, Nasios, Konstantinopoulou, Papathanasopoulos and Grigoriadis2018). Longitudinal studies have demonstrated that early cognitive impairment may predict future physical disability (Deloire, Ruet, Hamel, Bonnet, & Brochet, Reference Deloire, Ruet, Hamel, Bonnet and Brochet2010; Pitteri, Romualdi, Magliozzi, Monaco, & Calabrese, Reference Pitteri, Romualdi, Magliozzi, Monaco and Calabrese2017). Failure in information-processing speed tests was also found to be a good predictor of long-term cognitive impairment in MS (Trapp & Nave, Reference Trapp and Nave2008).
The Brief International Cognitive Assessment for MS (BICAMS) battery has been proposed as an easily administered tool for the assessment of cognitive functions in MS, that can be used in everyday clinical practice (Benedict, Amato, et al., Reference Benedict, Amato, Boringa, Brochet, Foley, Fredrikson and Langdon2012). This battery lasts about 15 min and consists of three tests: the Symbol Digit Modalities Test (SDMT) as a measure of information-processing speed, initial learning trials of the second edition of the California Verbal Learning Test (CVLT-II) as a measure of verbal learning and memory, and the Brief Visuospatial Memory Test – Revised (BVMT-R) as a measure of visual learning and memory (Benedict, Reference Benedict1997; Delis, Kramer, Kaplan, & Ober, Reference Delis, Kramer, Kaplan and Ober2000; Smith, Reference Smith1982). Performance in this battery may be influenced by nationality, therefore it has been validated in several languages including non-English-speaking populations (Filser et al., Reference Filser, Schreiber, Pöttgen, Ullrich, Lang and Penner2018; Giedraitienė, Kizlaitienė, & Kaubrys, Reference Giedraitienė, Kizlaitienė and Kaubrys2015; Ozakbas et al., Reference Ozakbas, Yigit, Cinar, Limoncu, Kahraman and Kösehasanogullari2017; Polychroniadou et al., Reference Polychroniadou, Bakirtzis, Langdon, Lagoudaki, Kesidou, Theotokis and Grigoriadis2016; Smerbeck et al., Reference Smerbeck, Benedict, Eshaghi, Vanotti, Spedo, Blahova Dusankova and Langdon2018). When administered, a more accurate Expanded Disability Status Scale (EDSS) calculation may be performed in up to 25% of patients with an EDSS score ≤ 4.0 (Saccà et al., Reference Saccà, Costabile, Carotenuto, Lanzillo, Moccia, Pane and Brescia Morra2017).
BICAMS can be used by clinicians without the need of trained neuropsychologists while it enables the longitudinal monitoring of MS patients using alternate forms (Benedict, Smerbeck, et al., Reference Benedict, Smerbeck, Parikh, Rodgers, Cadavid and Erlanger2012; Langdon et al., Reference Langdon, Amato, Boringa, Brochet, Foley, Fredrikson and Benedict2012).
No cognitive tests have been so far validated for MS in the Russian Federation. This study aims to validate the BICAMS battery in the Russian population of MS patients.
METHODS
Participants
The study was held in the City Center of Multiple Sclerosis, SBIH City Clinical Hospital #31. The study was approved by the local ethical committee of the SBIH City Clinical Hospital #31, St. Petersburg, Russia and was conducted in accordance with the ethical standards of the Declaration of Helsinki. All participants provided informed consent before their inclusion in this study.
To provide a representative study sample according to age, the distribution of patients in the whole population of MS patients visiting the City Center of Multiple Sclerosis was calculated using an electronic database. Patients were divided into groups of 10-year intervals: 18–20, 21–30, 31–40, 41–50, 51–60, and 61 years and older. The distribution for every age was calculated based on the assumption of including approximately 100 patients: 18–20 (2%), 21–30 (23%), 31–40 (32%), 41–50 (23%), 51–60 (14%), and 61 years and older (6%). Participants from the control group were enrolled to keep the same distribution.
Inclusion criteria for MS population were as follows: (1) MS diagnosis according to the McDonald 2017 criteria (any MS subtype); (2) age from 18 to 65 years; (3) EDSS score of 0–7.5 inclusive; and (4) written informed consent provided. Exclusion criteria were (1) any current or previous neurological disorder of CNS other than MS; (2) presence of psychiatric disorders, not related to MS; (3) drug or alcohol abuse; (4) severe visual, motor, or sensory impairment that might interfere with cognitive test performance; (5) ongoing MS exacerbation, corticosteroid, or immunosuppressive treatment within 30 days before enrolment; and (6) MS patients with a Hospital Anxiety and Depression Scale (HADS) ≥11 points in at least one scale (Ivanets, Kinkulkina, Avdeeva, & Sysoeva, Reference Ivanets, Kinkulkina, Avdeeva and Sysoeva2016; Zigmond & Snaith, Reference Zigmond and Snaith1983). Participants from two groups were matched on age and sex. Inclusion criteria for the control group were (1) age from 18 to 65 years; and (2) written informed consent provided. Exclusion criteria were (1) people with a neuropsychiatric diagnosis; (2) the presence of known relative with MS; (3) drug or alcohol abuse; and (4) people with a HADS score ≥11 points on at least one scale.
Procedures
According to the international guidelines for the BICAMS validation, test instructions were translated back and forth by two blinded to each other professional translators (Benedict, Amato, et al., Reference Benedict, Amato, Boringa, Brochet, Foley, Fredrikson and Langdon2012). All necessary permissions from copyright holders were obtained. Test stimuli of the SDMT and the BVMT-R are independent of language and original tests’ versions were applied. For the CVLT-II, 16 words from the original versions were translated into Russian and culturally adapted for the Russian population. All participants underwent neuropsychological assessment using the BICAMS battery in a standardized manner with a fixed order: (1) SDMT, (2) CVLT-II, and (3) BVMT-R. Also, MS patients were previously administered the Multidimensional Fatigue Inventory (MFI-20) for the evaluation of their self-reported levels of fatigue (Pichkaleva, Baidina, Trushnikova, & Danilova, Reference Pichkaleva, Baidina, Trushnikova and Danilova2019; Smets, Garssen, Bonke, & De Haes, Reference Smets, Garssen, Bonke and De Haes1995). All participants were screened with the HADS for their levels of depression and anxiety.
The SDMT measures information-processing speed. It consists of a pseudorandomized series of nine abstract geometrical symbols, which correspond to numbers from 1 to 9. In an oral version of SDMT, participants enunciate related numbers as rapidly as they can in 90 s. The total score corresponds to the number of correct answers (Smith, Reference Smith1982). We used the oral versions of the SDMT alternate forms #1 for the first test session and #2 for the retest session. This test can be completed within 5 min including instructing and testing. The SDMT is the most widely used test in cognitive studies regarding MS, due to its excellent psychometric properties, ecological validity, and significant correlation with other disease measures (Bakirtzis et al., Reference Bakirtzis, Ioannidis, Messinis, Nasios, Konstantinopoulou, Papathanasopoulos and Grigoriadis2018; Van Schependom et al., Reference Van Schependom, D’hooghe, Cleynhens, D’hooge, Haelewyck, De Keyser and Nagels2014).
The CVLT-II is a measure of verbal memory. A list of 16 words is presented for five sequential trials. These words are grouped into four semantic categories, arranged pseudorandomly (Delis et al., Reference Delis, Kramer, Kaplan and Ober2000). We used words from the original test versions which were translated into Russian and culturally adapted for the Russian population (e.g., see Table 1). According to BICAMS recommendations, we used only the immediate recall part of the test, which is as follows: an examiner reads aloud the list of words in approximately 20 s (Langdon et al., Reference Langdon, Amato, Boringa, Brochet, Foley, Fredrikson and Benedict2012). The participant listens to the complete list and then is asked to report as many words as possible, in any order. Five trials in total should be performed and the sum score is the total number of recalled items across all five trials. The total time to administer the CVLT-II five recall trials is 5–10 min including instructions, testing, and responses.
Example of the Russian translation of the CVLT-II forms 1 and 2
Note. CVLT-II, California Verbal Learning Test-II.
The BVMT-R is an evaluation of visuospatial learning and memory. The participants are asked to memorize six simple abstract geometric figures for 10 s. Immediately afterward, the participant is asked to draw these figures in the correct position using paper and pencil, taking as much time as needed for reproduction. The procedure is repeated three times with the same stimuli and in the same manner. Each figure is scored from zero to two points, depending on accuracy and location. The total recall score is composed of the total stimulus scores of all three trials (Benedict, Reference Benedict1997). We used forms #1 for the first test session and form #2 for the retest session, which are equivalent to the original version and provide good test–retest reliability (Benedict, Groninger, Schretlen, Dobraski, & Shpritz, Reference Benedict, Groninger, Schretlen, Dobraski and Shpritz1996). The test was completed within 5–10 min including instructing, testing and responses.
Finally, 44 MS patients were re-examined after four weeks to assess the BICAMS test–retest reliability. Retest was done for patients who fulfilled all inclusion criteria, were able to complete all three tests, agreed, and had a possibility for another visit to the center within four weeks. All three tests were repeated in the same manner but using alternate forms to reduce practice effects.
Statistical Analysis
Statistical analysis was performed with Statistica 13 software. The Kolmogorov–Smirnov test was used to account for normality in groups. Group differences in demographic characteristics were examined with t-test and Mann–Whitney, depending on the type of data distribution. The test–retest reliability in the MS population was evaluated with Spearman’s correlation analysis, where r-values for test–retest correlation were considered adequate if > .70 and good if > .80 (Cohen, Reference Cohen1988). Cronbach’s alpha was determined to define the internal consistency of BICAMS subtests. The effect size was calculated with Cohen’s d statistic. Raw scores of tests were compared using Mann–Whitney’s U-test. To evaluate how many MS patients were found impaired on each of the three components of BICAMS, z scores were generated. The performance of participants in the control group was used to generate a composite z score and individual z scores for each subtest. The score was counted using a formula: Z score = (Scale score in the MS group − Mean of the scale score in the control group)/(Standard deviation of the mean score in the control group)
Predictive significance of neurological impairment and fatigue levels on BICAMS tests were counted using Spearman’s correlation coefficient. Multifactorial linear regression models were built to evaluate the impact of several baseline characteristics of results of patients in the MS group. To define if disability level can impact the sensitivity of the battery, we divided the MS group into two subgroups based on EDSS score: ≥3.5 and ≤3.0. Results of their tests were compared with the HC group.
RESULTS
Demographic Characteristics
A total of 195 subjects [108 MS patients and 87 healthy control (HC)] entered the study at the screening. Ten MS patients and one control subject had an abnormal HADS score (≥11 points in at least one scale) and were excluded. Finally, 98 MS patients and 86 controls were eligible to enter the study. Demographic and main disease characteristics are presented in Table 2.
Main baseline characteristics of study participants
Note. MS, multiple sclerosis; HC, healthy control; EDSS, Expanded Disability Status Scale; HADS, Hospital Anxiety and Depression scale; RRMS, relapsing-remitting multiple sclerosis; SPMS, secondary progressive multiple sclerosis; all data are presented as mean (SD) for normal distribution and as median (25th percentile; 75th percentile) for nonnormal distribution.
We did not find any significant difference in demographic characteristics between MS patients and HC: mean age was 38.44 ± 11.47 for the MS group and 38.17 ± 13.29 for the control group (HC) (p = .65). Mean years of education were 15.12 ± 2.79 for MS population and 16.26 ± 3.02 for HCs (p = .01). The Chi-square test demonstrated that groups did not differ in the proportion of males/females (70.4% women for MS and 63.95% women for controls). Median EDSS scores of the MS patients’ group were 3.0 (range 1–6.5), with mean disease duration of 9.5 ± 7.44 years. MS group presented higher scores of fatigue and more pronounced depression scores than HCs according to HADS and MFI-20 tests: (1) median depression level – 3.0 (0; 9.0) versus 2.0 (0; 8.0), p = .01; and (2) median fatigue level – 48.0 (23.0; 77.0) versus 41.5 (23.0; 69.0), p = .02. Anxiety levels did not significantly differ between groups (p = .63).
Validity
Groups’ comparisons (MS vs HCs) in BICAMS tests at baseline demonstrated that MS patients had significantly lower SDMT test score (p < .0001), CVLT-II scores (p = .02), and BVMT-R scores (p = .03) as compared to HCs. Groups showed a moderate effect size of differences for all tests. Results are demonstrated in Table 3.
Differences between MS patients and control group in raw test scores
Note. BVMT-R, Brief Visuospatial Memory Test-Revised; SDMT, Symbol Digit Modality Test; CVLT-II, California Verbal Learning Test-II; MS, multiple sclerosis; HC, healthy controls. All data are presented as mean (SD) for SDMT and as median (25th percentile; 75th percentile) for BVMT-R and CVLT.
The mean (SD) composite z score was −.42 (1.17) in the MS group. Mean z scores of BICAMS subtests showed a significant difference between groups for all three tests (see Table 4).
Comparison of mean z scores for tests in BICAMS battery
Note. BICAMS, Brief International Cognitive Assessment for Multiple Sclerosis; MS, multiple sclerosis; HC, healthy controls; SDMT, Symbol Digit Modality Test; CVLT-II, California Verbal Learning Test-II; BVMT-R, Brief Visuospatial Memory Test-Revised.
A z score ≤ −1.5 was considered as “impaired.” When these variables were categorized as “impaired” and “not impaired,” the results were as follows: (1) SDMT: 23/98 (23.47%) for MS patients, 7/86 (8.14%) for controls; (2) BVMT-R: 19/98 (19.39%) in the MS group, 7/86 (8.14%) in the control group; and (3) CVLT-II: 17/98 (17.35%) in MS patients and 7/86 (8.14%) in the control group. Impairment in at least one test was seen in 34/98 (34.69%) MS patients and 14/86 (16.28%) of controls.
Reliability
Forty-four MS patients returned to the center after 3–5 weeks to assess test–retest reliability and all three tests were repeated in the same manner but using alternate forms to reduce practice effects. The analysis showed good test–retest correlation coefficient for SDMT and CVLT-II (r = .82 and r = .85, respectively) and adequate correlation for BVMT-R (r = .70). Data for reliability analysis are presented in Table 5.
Retest reliability of the overall sample
Note. BVMT-R, Brief Visuospatial Memory Test-Revised; SDMT, Symbol Digit Modality Test; CVLT-II, California Verbal Learning Test-II. All data are presented as mean (SD) for SDMT and as median (25th percentile; 75th percentile) for BVMT-R and CVLT.
Cronbach’s alpha for baseline and retest revealed good levels of internal consistency for three BICAMS tests at BL (α = .81) and RT (α = .83). Cronbach’s alpha was also good for individual tests within the BICAMS battery when measured at baseline: BVMT-R (.77), SDMT (.83), and CVLT-II (.79).
Impact of Cofactors and Disability
MS patients showed a significantly higher level of depression based on HADS and fatigue (MFI-20). Correlation analysis was performed to evaluate the impact of depression and fatigue on results of tests. HADS depression scores showed the only significant negative correlation with SDMT score in MS group (r = −.28, p = .005). There was also a significant negative correlation between SDMT scores and MFI-20 in MS patients (Spearman’s r = .28, p = .006). In HC group, we found significant negative correlation between BVMT-R scores and MFI-20 (Spearman’s r = −.24, p = .02).
Since all three factors (education, depression, and fatigue) showed differences in MS patients, with some of them showing correlations with results of BICAMS subtests, multifactorial models including all possible confounding covariates were built for each of three tests. All models were of adequate quality. There was a significant impact of the duration of education on results of SDMT, while for CVLT-II tests there was only a trend for significance. No significant impact of depression or fatigue was revealed in our model. Results of the analysis are presented in Table 6.
Results of the multifactorial analysis of covariance
Note. BVMT-R, Brief Visuospatial Memory Test-Revised; SDMT, Symbol Digit Modality Test; CVLT-II, California Verbal Learning Test-II; HADS, Hospital Anxiety and Depression scale.
To study the effect of disability, MS group was divided into two subgroups defined with EDSS score ≥3.5 (n = 44) and ≤3.0 (n = 54). Results of all three tests were compared to the results of HC groups. The comparison showed that only in case of EDSS score ≥3.5 there was a significant difference between MS and HC. In patients with lower disability level, no significant difference was detected. For more details, see Table 7.
The impact of disability level on BICAMS subtests
Note. *p = .0004; **p < .0001; ***p = .003.
DISCUSSION
Cognitive impairment in MS is common, therefore the routine assessment of cognitive functions should be implemented in clinical practice (Bakirtzis et al., Reference Bakirtzis, Ioannidis, Messinis, Nasios, Konstantinopoulou, Papathanasopoulos and Grigoriadis2018). There are several neuropsychological batteries of tests for MS such as the Minimal Assessment of Cognitive Function in MS (MACFIMS), the Brief Repeatable Battery of Neuropsychological tests (BRB-N), and BICAMS (Benedict, Amato, et al., Reference Benedict, Amato, Boringa, Brochet, Foley, Fredrikson and Langdon2012; Benedict et al., Reference Benedict, Cookfair, Gavett, Gunther, Munschauer, Garg and Weinstock-Guttman2006; de Caneda & de Vecino, Reference de Caneda and de Vecino2016). The BICAMS is a reliable instrument to evaluate cognitive impairment in MS, that allows monitoring of cognition over time. The BICAMS is relatively easily administered in 15–20 min compared to 45 min in BRB-N and 90 min in MACFIMS (Polychroniadou et al., Reference Polychroniadou, Bakirtzis, Langdon, Lagoudaki, Kesidou, Theotokis and Grigoriadis2016).
This study aimed to validate the BICAMS in the Russian-speaking population and we followed the recommendations and standards according to the international guidelines for validation (Benedict, Amato, et al., Reference Benedict, Amato, Boringa, Brochet, Foley, Fredrikson and Langdon2012). Our results showed that the MS population performed worse than the HCs group in all tests (SDMT, CVLT-II, and BVMT-R,). In this study, based on the criteria of impairment in at least one test, approximately 35% of our patients presented with cognitive dysfunction. These data are in line with other validation studies, showing good characteristics of the battery; however, the percentage is lower than in other studies (Filser et al., Reference Filser, Schreiber, Pöttgen, Ullrich, Lang and Penner2018; Ozakbas et al., Reference Ozakbas, Yigit, Cinar, Limoncu, Kahraman and Kösehasanogullari2017; Polychroniadou et al., Reference Polychroniadou, Bakirtzis, Langdon, Lagoudaki, Kesidou, Theotokis and Grigoriadis2016). This is, apparently due to the inclusion of a significant number of patients in the early stages of the disease, with worse results in patients with EDSS score ≥3.5. This fact may reflect the impact of more efficient compensatory mechanisms of cognitive functions and higher cognitive reserve in the early stages (Artemiadis et al., Reference Artemiadis, Bakirtzis, Ifantopoulou, Zis, Bargiotas, Grigoriadis and Hadjigeorgiou2020; Sumowski et al., Reference Sumowski, Rocca, Leavitt, Dackovic, Mesaros, Drulovic and Filippi2014; Uher et al., Reference Uher, Krasensky, Sobisek, Blahova Dusankova, Seidl, Kubala Havrdova and Vaneckova2018). However, the study by Harel et.al. did not show a prevalence of cognitive dysfunction in severely disabled patients which suggests that other mechanisms (e.g., high cognitive reserve or efficient compensatory strategies) may also be in charge of our findings (Harel et al., Reference Harel, Kalron, Menascu, Magalashvili, Dolev, Doniger and Achiron2019).
This study had several findings, that may have had influenced the results. Fatigue and depression levels were higher in the MS group and the total duration of education was lower in the MS group. This may have appeared due to no specific control for education level at inclusion or due to the inclusion of more educated people in the control group. However, fatigue and depression score only partially explained lower results of the MS cohort, especially in the task requiring information-processing speed. The assessment of the influence of these factors on BICAMS results using complex statistical models showed only that the duration of education had a statistically significant impact on the results of SDMT. Longer duration of education was associated with better performance of cognitively demanding tasks in MS in several studies, probably, due to higher cognitive reserve (Benedict, Morrow, Weinstock Guttman, Cookfair, & Schretlen, Reference Benedict, Morrow, Weinstock Guttman, Cookfair and Schretlen2010; Carotenuto et al., Reference Carotenuto, Moccia, Costabile, Signoriello, Paolicelli, Simone and Lauro2019; Da Silva et al., Reference Da Silva, Cavaco, Moreira, Bettencourt, Santos, Pinto and Montalban2015; Luerding, Gebel, Gebel, Schwab-Malek, & Weissert, Reference Luerding, Gebel, Gebel, Schwab-Malek and Weissert2016). Our findings of association of SDMT results with a duration of education may indicate on higher cognitive reserve in more educated MS patients, which may be an interesting result of the study. However, proper planning of validation studies should include a strict control for the duration of education at baseline.
Test–retest reliability in our population fits the recommended international standards for BICAMS validation and was good for SDMT and CVLT-II and adequate for BVMT-R, that is in line with prior findings (Benedict, Amato, et al., Reference Benedict, Amato, Boringa, Brochet, Foley, Fredrikson and Langdon2012; Sousa et al., Reference Sousa, Rigueiro-Neves, Miranda, Alegria, Vale, Passos and Sá2018; Walker et al., Reference Walker, Osman, Berard, Rees, Freedman, MacLean and Cousineau2016).
This study had several limitations. Here, we did not take into account possible beneficial effects of disease-modifying therapies and symptomatic treatment on cognitive functions; however, this issue is not yet fully defined (Chen, Goverover, Genova, & DeLuca, Reference Chen, Goverover, Genova and DeLuca2020; Gudesblatt et al., Reference Gudesblatt, Wissemann, Zarif, Bumstead, Fafard, Wilken and Lee2018; Penner et al., Reference Penner, Stemper, Calabrese, Freedman, Polman, Edan and Sandbrink2012). In this study, we also did not assess the impact of daytime sleepiness and daily variations of fatigue on the results of the test. We also did not evaluate the impact of physiotherapy and cognitive rehabilitation the patients may have received. The data were collected from one MS center, so further studies are needed to evaluate a broader prevalence of cognitive dysfunction in the Russian population of MS patients. Although the findings were significant with adequate effect sizes, the study sample size was minimally adequate to detect changes and provide a good quality of validation. Studies with larger Russian samples should be done to replicate our findings. Despite the fact, that the amount of retested patients was sufficient to perform the test–retest analysis, further studies are needed to assess the long-term reproducibility of the battery.
This study shows that the BICAMS battery is a reliable and valid cognitive instrument that can be used for evaluation in Russia in MS patients. Using BICAMS in everyday clinical practice could help determine cognitive impairment in MS patients to establish a prognosis and start rehabilitation and active disease-modifying therapy. At the same time, the BICAMS battery represents a minimal assessment that is most useful for screening and monitoring patients over time, and that it cannot substitute for a comprehensive neuropsychological assessment that includes numerous other cognitive domains that are typically impacted in MS.
CONCLUSIONS
Despite previous limitations, this study results support the reliability and validity of the BICAMS battery in the Russian population of patients. We consider that it can be recommended as part of the neurological assessment for MS for routine use in Russia.
CONFLICT OF INTEREST
The authors have nothing to disclose.
