Introduction
Multiple sclerosis (MS) is an autoimmune disorder characterized by demyelination in the central nervous system (CNS). MS is becoming increasingly common across the globe, with approximately 2.8 million people living with MS worldwide (Walton et al., Reference Walton, King, Rechtman, Kaye, Leray, Marrie, Robertson, La Rocca, Uitdehaag, van der Mei, Wallin, Helme, Napier, Rijke and Baneke2020). Individuals are typically diagnosed between 20 and 40 years old (Huang et al., Reference Huang, Chen and Zhang2017). In adult individuals with MS, cognitive deficits have been described in approximately 40% of individuals and are thought to be associated with the formation of both white and gray matter lesions in the brain, as well as gray matter degeneration (Huang et al., Reference Huang, Chen and Zhang2017; Klaver Reference Klaver, De Vries, Schenk and Geurts2013; Portaccio et al., Reference Portaccio, Goretti, Lori, Zipoli, Centorrino, Ghezzi, Patti, Bianchi, Comi, Trojano and Amato2009; Rao et al., Reference Rao, Leo, Bernardin and Unverzagt1991; Staff et al., Reference Staff, Lucchinetti and Keegan2009; Suppiej & Cainelli, Reference Suppiej and Cainelli2014). Compared to healthy control groups, neuropsychological research examining the cognitive involvement of MS has indicated an increased risk for neurocognitive deficits in several areas.
Pediatric-onset multiple sclerosis (POMS) occurs in approximately 2 to 5% of all individuals with MS (Ekmekci, Reference Ekmekci2017), with clinical symptoms first appearing before 18 years of age. Approximately 30 to 40% of all patients with POMS display cognitive impairments, defined as having shown an intelligence quotient lower than healthy controls (Ekmekci, Reference Ekmekci2017; MacAllister et al., Reference MacAllister, Belman, Milazzo, Weisbrot, Christodoulou, Scherl, Preston, Cianciulli and Krupp2005; Portaccio et al., Reference Portaccio, Goretti, Lori, Zipoli, Centorrino, Ghezzi, Patti, Bianchi, Comi, Trojano and Amato2009). In addition to intellectual abilities, the areas at increased risk for impairment include complex attention, processing speed, executive functions, language, memory, and visuomotor and visuospatial abilities in children (Ekmekci, Reference Ekmekci2017; Portaccio et al., Reference Portaccio, Goretti, Lori, Zipoli, Centorrino, Ghezzi, Patti, Bianchi, Comi, Trojano and Amato2009; Tan et al., Reference Tan, Hague, Greenberg and Harder2018). In a longitudinal study by MacAllister et al. (Reference MacAllister, Christodoulou, Milazzo and Krupp2007), MS patients 16 years and younger who were within two years of MS onset showed cognitive decline between an initial evaluation and a second evaluation approximately a year later. It is important to note that cognitive outcomes of individuals with POMS vary significantly across the heterogeneous group.
The standard of care for monitoring potential cognitive impairment in individuals with MS, as set out by the National MS Society, includes early baseline screening with validated measures, annual reevaluations, and psychoeducation regarding potential cognitive impact. This standard has previously been established in the adult MS population, and cognitive screening has been carried out using neuropsychological batteries such as the Minimal Assessment of Cognitive Function in MS and the Brief International Cognitive Assessment for MS (Benedict et al., Reference Benedict, Fischer, Archibald, Arnett, Beatty, Bobholz, Chelune, Fisk, Langdon, Caruso, Foley, LaRocca, Vowels, Weinstein, DeLuca, Rao and Munschauer2002; Langdon, Reference Langdon, Amato, Boringa, Brochet, Foley, Fredrikson, Hämäläinen, Hartung, Krupp, Penner, Reder and Benedict2012). These neuropsychological batteries contain targeted assessments to examine vulnerable areas of cognitive functioning for individuals with MS, which may be utilized to assess for treatment effects, evaluate the progression of cognitive impairment, and screen for new-onset cognitive problems (Benedict et al., Reference Benedict, Fischer, Archibald, Arnett, Beatty, Bobholz, Chelune, Fisk, Langdon, Caruso, Foley, LaRocca, Vowels, Weinstein, DeLuca, Rao and Munschauer2002; Benedict Reference Benedict, Smerbeck, Parikh, Rodgers, Cadavid and Erlanger2012; Ekmekci, Reference Ekmekci2017; Kalb et al., Reference Kalb, Beier, Benedict, Charvet, Costello, Feinstein, Gingold, Goverover, Halper, Harris, Kostich, Krupp, Lathi, LaRocca, Thrower and DeLuca2018; Langdon et al., Reference Langdon, Amato, Boringa, Brochet, Foley, Fredrikson, Hämäläinen, Hartung, Krupp, Penner, Reder and Benedict2012). In general, these targeted neuropsychological screening batteries are abbreviated batteries (usually 1 to 3 hours) that are established specifically to monitor those who present with higher risk status based on their medical conditions or treatments (Hardy et al., Reference Hardy, Olson, Cox, Kennedy and Walsh2017). In POMS, a targeted screening approach also makes sense as only a third of the population will demonstrate cognitive challenges, which is also helpful when considering resource allocation.
Given that the care of individuals with complex medical conditions requires multiple medical specialties, the establishment of multidisciplinary clinics (MDCs) has shown to be an effective use of resources to allow various providers to monitor and screen patients at one medical appointment routinely and to facilitate interdisciplinary communication regarding care. This also helps reduce the burden of the individuals needing to attend different subspecialty appointments (Schaaf et al., Reference Schaaf, Ley, Riegler, Poetker, Xanthakos, Sizemore, Crimmins, Helmrath, Tracy, Arce-Clachar, Crail, Morwessel, Frenck, Tariq and Shah2023). For a neuropsychologist, administering a full standardized assessment battery is not feasible within the setting of an MDC appointment, as this typically requires multiple hours of individualized testing with trained personnel (Wilcutt et al., Reference Willcutt, Boada, Riddle, Chhabildas, DeFries and Pennington2012). Thus, during an MDC appointment, the neuropsychologist’s role may focus more on clinical interviews, consultations, and/or targeted neuropsychological screening evaluations to efficiently gather the necessary information to assist with clinical management.
To increase efficiency in these MDC settings, pediatric neuropsychologists may use caregiver-report screening questionnaires to help structure the interview or identify aspects of cognitive concern to closely monitor. Thus far, these caregiver-report screening questionnaires have been found to effectively screen several pediatric medical populations, including oncology, metabolic, and cardiac populations (Schaaf et al., Reference Schaaf, Ley, Riegler, Poetker, Xanthakos, Sizemore, Crimmins, Helmrath, Tracy, Arce-Clachar, Crail, Morwessel, Frenck, Tariq and Shah2023; Wolfe et al., Reference Wolfe, Hutaff-Lee and Wilkening2022). Cognitive functioning in the POMS group may fluctuate over time (Amato et al., Reference Amato, Goretti, Ghezzi, Lori, Zipoli, Moiola, Falautano, De Caro, Viterbo, Patti, Vecchio, Pozzilli, Bianchi, Roscio, Martinelli, Comi, Portaccio and Trojano2010, Reference Amato, Goretti, Ghezzi, Hakiki, Niccolai, Lori, Moiola, Falautano, Viterbo, Patti, Cilia, Pozzilli, Bianchi, Roscio, Martinelli, Comi, Portaccio and Trojano2014); therefore, being able to utilize screening questionnaires may also help to track these changes. That said, research examining the use of these caregiver questionnaires for the POMS population is scant and has been more focused on the adult group (Nauta et al., Reference Nauta, Balk, Sonder, Hulst, Uitdehaag, Fasotti and de Jong2019; O’Brien et al., Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007). Studies that have utilized screening questionnaires, have focused primarily on the impact of the disease on Quality of Life rather than screening for potential cognitive impairments that warrant further evaluation (Mrosková et al., Reference Mrosková, Klímová, Majerníková and Tkáčová2021).
A few caregiver-report questionnaires have been examined for their utility in triaging patients in pediatric MDC settings. The Pediatric Perceived Cognitive Functioning (Peds PCF; Lai et al., Reference Lai, Butt, Zelko, Cella, Krull, Kieran and Goldman2011) and Colorado Learning Difficulties Questionnaire (CLDQ; Patrick et al., Reference Patrick, McCurdy, Chute, Mahone, Zabel and Jacobson2013) are two such questionnaires. In a study by Lai et al. (Reference Lai, Butt, Zelko, Cella, Krull, Kieran and Goldman2011), the Peds PCF showed sensitivity to changes in mental status related to neurological conditions in pediatric populations, with the ability to distinguish between children with and without neurological diagnoses. Research on the CLDQ suggests it may be a helpful screening tool for concerns about learning disabilities, specifically regarding reading and mathematics skills (Patrick et al., Reference Patrick, McCurdy, Chute, Mahone, Zabel and Jacobson2013). A study examining both questionnaires by Wolfe et al. (Reference Wolfe, Hutaff-Lee and Wilkening2022) showed that the Peds PCF and CLDQ were predictive of neuropsychological test performance in brain tumor, non-central nervous system cancer, and Fontan circulation pediatric populations (Wolfe et al., Reference Wolfe, Hutaff-Lee and Wilkening2022).
This study assessed the utility of screening questionnaires, the Peds PCF and CLDQ, in the context of a targeted standardized neuropsychological assessment for individuals with POMS. We explored the sensitivity and specificity of these screeners' ability to identify clinical concerns in a MDC setting and described their utility as a guiding tool for identifying general cognitive difficulties. We hypothesized that the Peds PCF and the CLDQ would be able to identify areas of cognitive challenges for this population that are commensurate with their performance on the brief targeted neuropsychological assessment.
Materials and methods
Participants
Retrospective data was gathered from electronic medical records at a single pediatric hospital in the Mountain West region of the United States from 2018 to 2023. This study was approved by the University of Colorado IRB and was completed in accordance with the Helsinki Declaration. Participants included those aged 9 to 20 years old diagnosed with relapsing and remitting POMS. Participants were seen for screening and completed a targeted neuropsychological evaluation as part of routine clinical care for POMS in the neuroimmunology MDC. Exclusion criteria were severe developmental delay (e.g., if the child were nonverbal, the rating scale questions would not be applicable), caregivers who self-identified as unable to read the English questionnaires, or young adults who attended the appointment without a caregiver. Because this screening was conducted as part of routine clinical care, informed consent was waived by the institutional review board.
Measures
Two screening measures were administered in the MDCs: the Peds PCF 10-item parent report measure and the CLDQ. The Peds PCF is part of the National Institute of Health’s Patient-Reported Outcomes Measurement Information System (PROMIS; Lai et al., Reference Lai, Butt, Zelko, Cella, Krull, Kieran and Goldman2011). It assesses parent/caregiver ratings of their child’s cognitive functioning, including attention, memory, and processing speed, using a Likert scale (Lai et al., Reference Lai, Butt, Zelko, Cella, Krull, Kieran and Goldman2011). One total score is obtained and subsequently converted to a T-score based on normative data from Lai and colleagues, with higher T-scores indicating better functioning. The Peds PCF has previously been validated in children with pediatric oncology patients (Lai et al., Reference Lai, Wagner, Jacobsen and Cella2014) and healthy controls, as well as other pediatric populations at risk of neurocognitive impairment (Ilik et al., Reference Ilik, IJsselstijn, Gischler, van Gils-Frijters, Schnater and Rietman2022; Wolfe et al., Reference Wolfe, Hutaff-Lee and Wilkening2022).
The CLDQ is a 22-item Likert scale parent report measure assessing Reading (6 items), Math (5 items), Spatial Organization (4 items), Social Cognition (4 items), and Social Anxiety (3 items). Total scores in each of the five domains are obtained and converted to z-scores based on population norms, with higher z-scores indicating higher concern for problems in that area. The CLDQ has been validated in children with neurodevelopmental disorders (Willcutt et al., Reference Willcutt, Boada, Riddle, Chhabildas, DeFries and Pennington2011) and medical conditions (Patrick et al., Reference Patrick, McCurdy, Chute, Mahone, Zabel and Jacobson2013).
Procedure
The Peds PCF and CLDQ were provided to caregivers during each MDC appointment. After completing the forms, the neuropsychologist scored the questionnaires and reviewed the results before an interview with the patient and family to allow the family to explain their concerns further or correct any misperceptions of their questionnaire responses. Every patient subsequently completed the same targeted neuropsychological evaluation at the appointment, administered by the licensed pediatric clinical neuropsychologist, with assistance from pediatric neuropsychology learners and psychometrists. For the purposes of this study, we included performance on measures that represented neurocognitive domains that are often impacted by POMS (e.g., processing speed and working memory), as well as one measure that is typically less susceptible to change (e.g., vocabulary).
The Reading, Math, and Spatial Organization scores from the CLDQ were examined in addition to the Peds PCF during the current study. Neuropsychological test scores included in this study were the age-based standard scores from the Vocabulary Subtest from the Wechsler Abbreviated Scale of Intelligence Second Edition (WASI-II; Wechsler, Reference Wechsler2011); Digit Span and Symbol Search from the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V; Wechsler, Reference Wechsler2014) or the Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV; Wechsler, Reference Wechsler2008); Symbol Digit Modalities Test (SDMT; Smith, Reference Smith1982); Math Fluencies from the Wechsler Individual Achievement Test, Fourth Edition (WIAT-4; Wechsler, 2020); and Sight Word Efficiency and Phonemic Decoding from the Test of Word Reading Efficiency, Second Edition (TOWRE-2; Torgesen, et al., Reference Torgesen, Wagner and Rashotte2012).
Data abstraction and analysis
Retrospective data, including participant demographics, limited medical and academic history, Peds PCF scores, CLDQ Reading, Math, Spatial Organization scores, and neuropsychological testing scores, were abstracted from the electronic medical record. Race and ethnicity in the medical record are self-identified. The area deprivation index (ADI), which allows for rankings of neighborhoods by socioeconomic disadvantage (Reference Tan, Hague, Greenberg and Harder39), was calculated for each participant using federal information processing standard scores to determine state and national rankings. The state ADI scores ranged from 1 to 10, with higher numbers representing more deprivation. The national ADI scores ranged from 1 to 100, with higher scores representing more deprivation. Descriptive statistics were obtained. Data were examined for normality using the Shapiro–Wilk test. Missingness was treated with pairwise deletion. The alpha level for significance was set at<0.05, and two-tailed hypothesis testing was used throughout.
Correlation coefficients were obtained to investigate relationships between screening and neuropsychological standard scores and assess for any potential sociodemographic confounders. Neuropsychological standard scores were dichotomized accordingly as either greater than 1 standard deviation (SD) below the mean (“at risk”), or less than or equal to 1 SD below the mean (“not at-risk”). While a cutoff of 1 SD is lower than the 1.5 or 2 SD cutoffs commonly used in research investigating cognitive impairments associated with complex medical conditions, a more generous cutoff was utilized for this study on screening measures based on the premise that a score below this level would merit further clinical assessment and consideration for intervention. In order to optimize potential clinical application, the percentage of “at risk” scores out of the total number of neuropsychological test scores were calculated for each participant. The sample was dichotomized around the median percentage of “at risk” scores, and nonparametric receiver operating characteristic (ROC) curve analysis was performed to assess predictive utility. Youden indices were calculated to identify recommended “cut scores” for considering a referral for neuropsychological evaluation based on screening, with “adequate” (>0.70) or “moderate” (>0.50) sensitivity and specificity (Ruopp et al., Reference Ruopp, Perkins, Whitcomb and Schisterman2008).
The final sample included 49 participants. (69% female; 43% Hispanic/Latinx; mean age = 16.14 years, range = 9.90–20.63 years; see Table 1 for Descriptive Statistics).
Sample descriptive data (n = 49)
Note: SD = standard deviation; IEP = Individualized Education Program; PCF = Perceived Cognitive Function; CLDQ = Colorado Learning Difficulties Questionnaire; SDMT = Symbol Digit Modalities Test; TOWRE-2 = Test of Word Reading Efficiency, 2nd Edition. T-scores have a mean of 50, SD of 10. Z-scores have a mean of 0, SD of 1. Standard scores have a mean of 100, SD of 15. Higher scores indicate better functioning except on the CLDQ.
Shapiro–Wilk tests were significant (p-values > .05), indicating non-normal distributions for several variables, including Digit Span, Symbol Search, and all Peds PCF and CLDQ scores. As such, nonparametric Spearman’s rho correlations were utilized.
Relationships between sociodemographic variables (i.e., sex, race, ethnicity, and ADI national and state rank) and standardized scores from neuropsychological tests and screening measures were investigated with nonparametric correlations for dichotomous and continuous variables (i.e., sex, ethnicity, and ADI) and univariate analysis of variance for categorical variables (ANOVA; race). There were no relationships between sex, ethnicity, or national ADI with neuropsychological tests or screening measures (all ps > .05). State ADI was related only to Vocabulary score, such that a lower score was related to social disadvantage (p < .05). Race was found to be related only to TOWRE-2 Sight Word Efficiency standardized scores (p < .01). Follow-up analysis revealed that scores in the White race group were significantly lower than in the combined American Indian, Asian, Black/African American Biracial, and Unknown/Other races group.
Bivariate correlations demonstrated relationships between screening measures and some neuropsychological test scores (Table 2). Effect sizes were medium across comparisons for both screening measures (CLDQ: Spearman’s rho = −.321 to −.563; Peds PCF: Spearman’s rho = .308 to .444). The CLDQ Spatial Organization z-score was not correlated with any of the neuropsychological testing scores included in this study. When associations with Vocabulary score were adjusted for State ADI using partial correlations, all relationships that were previously statistically significant, remained so. When associations with TOWRE-2 Sight Word Efficiency were adjusted for race, the relationship with CLDQ Reading score remained statistically significant, but the relationship with CLDQ Math score was no longer significant (p = .11)
Relationships between screening indices and neuropsychological test scores
Note: Spearman’s rho correlation coefficients are presented in this table. *p < .05; **p < .01. PCF = Perceived Cognitive Function; CLDQ = Colorado Learning Difficulties Questionnaire; SDMT = Symbol Digit Modalities Test; TOWRE-2 = Test of Word Reading Efficiency, 2nd Edition. Higher scores indicate better functioning except on the CLDQ.
ROC curve analysis showed that the Peds PCF T-score predicted impaired performance on Symbol Search and math fluency (AUC = 0.748 to 0.796; all ps < .05). The CLDQ Reading z-score predicted timed word reading, as well as vocabulary, working memory, processing speed (oral), visual motor integration, and math fluency test performance (AUC = 0.699−0.847; all ps < .05). The CLDQ Math z-score predicted math fluency as well as vocabulary (AUC = 0.779 to 0.825; all ps < .05).
Next, the percentage of standardized scores measuring in the “at risk” range was calculated for each participant (i.e., the number of tests with scores less than one standard deviation below the mean divided by the total number of tests administered for each person). After this, the median and mean percentages were calculated for the sample. The median percentage of “at-risk” scores was 25%, and the mean percentage of “at-risk” scores was 27%. Given the similarity in percentages, we choose to dichotomize participants into those who had 25% or more scores in the “at risk” range (deemed the “clinical concerns” group) and those who had fewer than 25% scores in the “at risk” range. This is done under the clinical assumption that patients with more than 25% of scores in the “at-risk” range would be recommended for a further comprehensive evaluation, requiring a full-day evaluation and examining all aspects of neurocognitive functioning. ROC curve analysis demonstrated that the Peds PCF, CLDQ Reading, and CLDQ Math scores each predicted membership in the “clinical concerns” group (Figure 1; AUCs = 0.763−0.775; all ps < .01). Youden indices were calculated to reveal cut scores that optimized sensitivity and specificity for predicting the “clinical concerns” group. In this sample, the cut score was measured approximately at the normative mean for each screening measure (Table 3).
Receiver operating characteristic curve demonstrating predictive utility of screening measures for clinical concerns on neuropsychological testing. Note: Clinical concern is defined as having 25% or more of neuropsychological test scores measuring greater than 1 standard deviation below the normative mean. PCF = Perceived Cognitive Function; CLDQ = Colorado Learning Difficulties Questionnaire. Lines in the graph represent the CLDQ, Peds PCF, or Reference lines as indicated in the legend.
Sensitivity and specificity for screening measures predicting clinical concerns on neuropsychological testing
Note: * p < .05; ** p < .01. Clinical concern is defined as having 25% or more of neuropsychological test scores measuring greater than 1 standard deviation below the normative mean. AUC = area under the curve; PCF = Perceived Cognitive Functioning; CLDQ = Colorado Learning Difficulties Questionnaire.
Discussion
This study examined the utility of the Peds PCF and the CLDQ screening questionnaires in an MDC setting for individuals with POMS. Correlation analyses showed strong interrelatedness between ratings on screening measures and testing performance, all in the expected direction, such that more reported challenges were related to lower test performance. Amongst the scales, the CLDQ Reading Scale was the best for identifying areas of academic and cognitive difficulty in neuropsychological evaluation for individuals with POMS. The CLDQ Math Scale and Peds PCF score were also correlated with performance on several aspects of neuropsychological testing, including vocabulary, working memory, simple process speed, and math fluency. In contrast, the CLDQ Spatial Organization Scale was the weakest at predicting cognitive and academic scores on neuropsychological testing. Together, however, our findings suggest that the measures demonstrate clinical utility. Given that scores on the questionnaires had strong interrelatedness with performance-based measures, these questionnaires may be a useful tool for providing more qualitative information to supplement performance on test measures. Additionally, at our institution, these questionnaires are completed by caregivers on a yearly basis, assisting with the comparison of reported concerns across time (e.g., examining increased concerns regarding academic performance).
The Peds PCF and CLDQ Reading and Math scales had around 63 to 68% sensitivity for detecting the likelihood of accurately detecting those who fall in the clinical concerns group. Interestingly, the specificity range is greater than the sensitivity range, falling between 85-90% for the same scales. Clinically, providers may feel reassured that these scales are more accurate at predicting when a youth does not fall into the clinical concerns group. The finding that the CLDQ Reading z-score is the most robust predictor of performance on several neuropsychological tests is consistent with previously published literature (Wolfe et al., Reference Wolfe, Hutaff-Lee and Wilkening2022). The relationship between reading fluency and vocabulary test scores to the CLDQ reading scale is unsurprising given the bidirectional relationship of these variables (e.g., better readers have higher vocabulary, and better reading fluency will contribute to higher reading scale scores). In addition, working memory and processing speed have been identified as cognitive processes that support reading skills (De Weerdt et al., Reference De Weerdt, Desoete and Roeyers2013; McGrath et al., Reference McGrath, Pennington, Shanahan, Santerre-Lemmon, Barnard, Willcutt, Defries and Olson2011; Shanahan et al., Reference Shanahan, Pennington, Yerys, Scott, Boada, Willcutt, Olson and DeFries2006). The finding that parent ratings of reading were also associated with math fluency performance is consistent with cross-domain studies identifying shared cognitive processes between reading and math skills (Ashkenazi et al., Reference Ashkenazi, Black, Abrams, Hoeft and Menon2013; Balhinez & Shaul, Reference Balhinez and Shaul2019). Duncan et al. (Reference Duncan, Dowsett, Claessens, Magnuson, Huston, Klebanov, Pagani, Feinstein, Engel, Brooks-Gunn, Sexton, Duckworth and Japel2007) also suggest that early math skills predicted reading ability even better than reading skills predicted math ability, which is consistent with our finding that math fluencies were significantly associated with the CLDQ Reading and Math scales, whereas timed word reading was only significant associated with the CLDQ Reading scale.
We did not find any relationships between sex, ethnicity, or national ADI with neuropsychological tests or screening measures. We found that a lower vocabulary score was related to social disadvantage, consistent with other well-documented findings examining the relationship between socioeconomic status and language and literacy development (e.g., Hoff, Reference Hoff2013; Spencer et al., Reference Spencer, Clegg and Stackhouse2012). Race was found to be related only to the speed sight word measure, such that performance in the White race group was significantly poorer than in the combined American Indian, Asian, Black/African American Biracial, and Unknown/Other races group. This may be due to the large sample of White participants in this study (77%), which allowed more opportunities to capture individuals with preexisting reading difficulties than in the combined race group. Furthermore, while our sample size was adequate to detect statistically significant relationships between neuropsychological tests and screening measures, it may have been underpowered to detect more subtle associations between indicators of social disadvantage and scores on neuropsychological tests and screening measures. Future research is needed to discern whether social determinants of health are reflected in scores on neuropsychological screeners in particular.
The SDMT is widely used to screen for cognitive change in individuals with MS (Parmenter et al., Reference Parmenter, Weinstock-Guttman, Garg, Munschauer and Benedict2007; Sonder et al., Reference Sonder, Burggraaff, Knol, Polman and Uitdehaag2014). Our study found that only the CLDQ Reading scale correlated with performance on the SDMT oral and that the motor version of the SDMT was not correlated with the CLDQ Reading scale. The oral form of the SDMT has often been used in research without the written form to eliminate the impact of gross or fine motor impairment in MS populations (Brenton et al., Reference Brenton, Koshiya, Woolbright and Goldman2019; Charvet et al., Reference Charvet, Beekman, Amadiume, Belman and Krupp2014). However, we were surprised at the lack of a relationship between the SDMT and the Peds PCF. While the reason for our findings is unclear, the results nonetheless reinforce the possibility that the CLDQ Reading scale may be the most robust screening questionnaire when assessing for cognitive challenges in the POMS population.
The CLDQ Spatial Organization scale did not predict any neuropsychological test results. Upon qualitatively examining the questions on the measure, we suspect that this is because the four questions in the CLDQ that make up the Spatial composite were not related to the neuropsychological areas measured (e.g., vocabulary, working memory, processing speed, math fluency, and reading fluency) in this study. While the CLDQ Spatial scale has been correlated with math performance before (Willcutt et al., Reference Willcutt, Boada, Riddle, Chhabildas, DeFries and Pennington2011; Wolfe et al., Reference Wolfe, Hutaff-Lee and Wilkening2022), we based math performance on math fluencies during the targeted evaluation, which examines timed single-digit basic math skills (i.e., addition, subtraction, and multiplication), which would not necessarily require intact spatial skills to accurately compute as might a more complex math problem. For example, one statement on the CLDQ reads, “When doing arithmetic problems, has difficulty keeping the numbers lined up in columns.”
While the literature suggests the utility of the Peds PCF for several clinical populations, including children who survive neonatal illnesses, giant omphalocele, multiple congenital anomalies or gastroschisis, non-CNS cancers, and post-Fontan procedure (Hijkoop et al., Reference Hijkoop, Rietman, Wijnen, Tibboel, Cohen-Overbeek, van Rosmalen and IJsselstijn2019; Ilik et al., Reference Ilik, IJsselstijn, Gischler, van Gils-Frijters, Schnater and Rietman2022; Wolfe et al., Reference Wolfe, Hutaff-Lee and Wilkening2022), one study examining the use of the Peds PCF, showed nearly no significant correlation for children with minimal hepatic encephalopathy (Ohnemus et al., Reference Ohnemus, Neighbors, Sorensen, Lai and Alonso2019). This study found that results from the Peds PCF correlated well with the Behavior Rating Inventory of Executive Function (BRIEF; also a caregiver-report rating scale) but not with other neurocognitive test measures in a sample of 18 participants. We found that the scores from the Peds PCF only correlated with simple processing speed and math fluency but not several other areas, suggesting the Peds PCF would not be strong as a standalone measure for identifying challenges in the POMS group.
We utilized ROC curve analysis and Youden index calculations to explore potential cut scores on the screening measures for predicting those in the clinical concerns group (those with more than 25% impaired scores) on neuropsychological testing. Together, the cut scores we derived for the Peds PCF, CLDQ Reading, and CLDQ Math scales all measured approximately at the normative mean for each screening measure (T-score = 48.95, z-score = 0.02, z-score-0.37, respectively). This suggests that the Peds PCF and CLDQ scores may be more useful as a qualitative measure for guiding MDC consultations than a quantitative measure for identifying the need for further neuropsychological evaluation. A more conservative but comprehensive approach for referring to further neuropsychological evaluation that considers caregiver ratings on the screening measures, qualitative interview information, and performance on a targeted neuropsychological battery may be most clinically indicated, consistent with the current model for follow-up care with this population.
The questionnaire and its data may also serve as a useful tool in other ways. First, these questionnaires can help providers with structured interview formats, which are often more advantageous than unstructured interviews in clinical settings (Mueller & Segal, Reference Mueller, Segal, Cautin and Lilienfeld2015). Although unstructured interviews have some advantages, including building rapport with patients, structured interviews allow for increased interrater and diagnostic reliability and decrease the chance for discrepancies in patient information, such as how individuals respond to questions or what information they share with the neuropsychologist (Mueller & Segal, Reference Mueller, Segal, Cautin and Lilienfeld2015). In addition, a study conducted by Kim et al. (Reference Kim, Zemon, Rath, Picone, Gromisch, Glubo, Smith-Wexler and Foley2017) compared whether the SDMT or questionnaires (e.g., the Multiple Sclerosis Neuropsychological Screening Questionnaire and the Behavior Rating Inventory of Executive Function) were better predictors of outcomes in an adult MS sample and found that the SDMT was better at predicting neuropsychological outcomes. In their article, they propose that questionnaires can offer complementary information that performance-based measures alone cannot, such as information that helps identify rehabilitation goals and recommendations. This could also be the case with the Peds PCF and CLDQ information since the many questions mostly ask about specific skills in daily life (e.g., “has difficulty with spelling,” “difficulty learning math facts”). In the case that individuals are referred for a comprehensive evaluation, this data may also be used to inform the testing battery.
This study is not without limitations. First, this is a retrospective pilot study without a normal control group, which future studies may consider including. At our center, comprehensive evaluations are only scheduled if a patient exhibits at-risk scores on the targeted neuropsychological battery to clarify diagnostic impressions to guide treatment and recommendations as part of a tiered neuropsychological approach (Hardy et al., Reference Hardy, Olson, Cox, Kennedy and Walsh2017). Therefore, this paper compared the performance of screening measures to that of targeted evaluations instead of a comprehensive evaluation. While the targeted evaluation was mostly designed to pick up potential changes in areas thought to most likely be impacted in the POMS population (e.g., processing speed), comparison to a more comprehensive test battery may also be warranted in future studies to better capture some of the other cognitive areas that are referenced in the questionnaire items (e.g., visual-spatial functioning). There is literature to suggest that time since disease onset impacts neuropsychological outcomes, which we did not include in our analysis. That said, this study examined only the pediatric population, so duration is somewhat limited. Even so, our sample had a mean age of 16, and while this is consistent with the general mean age of onset for youth with POMS, the generalizability of these findings to younger children warrants additional attention. In addition, our center does not regularly use self-report measures as part of these batteries. Therefore, future studies may consider examining self-report measures and caregiver ratings. The Peds PCF, CLDQ, and the performance-based measures were all developed and normed on U.S.-based populations, which is an aspect to consider, given that 43% of our sample identified as Hispanic/Latinx. Furthermore, we were also limited by the fact that the Peds PCF and CLDQ were administered in English because neither of these questionnaires has been adapted for and validated in other languages and cultures; thus, we could not capture the responses of those parents who speak a different primary language, therefore likely limiting our generalizability. Cognitive patterns in the POMS group have been found to fluctuate over time (Amato et al., Reference Amato, Goretti, Ghezzi, Lori, Zipoli, Moiola, Falautano, De Caro, Viterbo, Patti, Vecchio, Pozzilli, Bianchi, Roscio, Martinelli, Comi, Portaccio and Trojano2010, Reference Amato, Goretti, Ghezzi, Hakiki, Niccolai, Lori, Moiola, Falautano, Viterbo, Patti, Cilia, Pozzilli, Bianchi, Roscio, Martinelli, Comi, Portaccio and Trojano2014); therefore, longitudinal studies examining screeners with brief neuropsychological batteries may help understand this relationship over time. It is possible that the cognitive challenges in individuals with MS included in this study were more qualitatively distinct than what the screening questionnaire could pick up; however, it is also certainly possible that our sample size was more stable, given that only 12% of the group had IEPs. In addition, most of our participants were treated with rituximab, and few had reported relapses. Given the small numbers, we did not run an analysis to compare the group treated with a disease-modifying therapy and those that had no treatment. While it is not yet fully understood how this treatment interacts with relapse severity, there are smaller studies that suggest early high-efficacy therapy may protect against cognitive decline for POMS patients (Johnen et al., Reference Johnen, Elpers, Riepl, Landmeyer, Krämer, Polzer, Lohmann, Omran, Wiendl, Göbel and Meuth2019; Kania et al., Reference Kania, Ambrosius, Kozubski and Kalinowska-Łyszczarz2023) and potentially, a decrease in relapse rate with rituximab (Breu et al., Reference Breu, Sandesjö, Milos, Svoboda, Salzer, Schneider, Reichelt, Bertolini, Blaschek, Fink, Höftberger, Lycke, Rostásy, Seidl, Siegert, Wickström and Kornek2024). Lastly, the SDMT has been identified as a robust measure for identifying cognitive changes in this population. Future studies may examine the utility of incorporating this brief cognitive measure alongside screening questionnaires in an MDC setting.
Conclusion
Screening questionnaires are tools often used in medical appointments to identify those needing further specialty care. We examined the utility of the Peds PCF and the CLDQ screening questionnaires in an MDC setting for POMS and overall found that the measures demonstrate clinical utility. Comparison of scores across caregiver rating questionnaires and on a targeted neuropsychological battery suggests that the survey alone may not be sensitive enough to identify cognitive difficulties in children with POMS, but our study indicates that these ratings may still provide qualitatively meaningful information when given in tandem with neuropsychological testing.
Acknowledgements
We would like to thank the neuroimmunology team and patients at Children’s Hospital Colorado.
Funding statement
There is no funding/grant support for this work.
Competing interests
The authors report there are no competing interests to declare.
