Statement of Research Significance
Research Question(s) or Topic(s): Validation of the NIH Toolbox for examination of neurocognitive function in children and adults with early-treated phenylketonuria. Main Findings: The NIH Toolbox demonstrated strong concurrent validity with Wechsler indices and moderate correlations with other established measures. Deficits on the NIH Toolbox’s Fluid Cognition Composite were identified in younger children with phenylketonuria (PKU), indicating the test’s sensitivity to neurocognitive difficulties associated with PKU. Study Contributions: The study presents the NIH Toolbox as a valid and practical measure of neurocognitive function in individuals with PKU across the lifespan, supporting its potential integration into clinical trials and research protocols
Introduction
PKU is a rare autosomal recessive condition characterized by pathogenic variants in the phenylalanine hydroxylase (PAH) gene. The resulting deficiency in PAH enzyme disrupts the metabolism of phenylalanine (Phe) into tyrosine, a precursor for dopamine and other catecholamines that are critical for brain function (Boot et al., Reference Boot, Hollak, Huijbregts, Jahja, van Vliet, Nederveen, Nieman, Bosch, Bour, Bakermans, Abeling, Bassett, van Amelsvoort, van Spronsen and Booij2017; Harding et al., Reference Harding, Winn, Gibson, Arning, Bottiglieri and Grompe2014). Brain levels of monoamine neurotransmitter levels are further adversely impacted by competition at blood-brain barrier between excess Phe and other large neutral amino acids (e.g., tryptophan) as well as Phe-mediated inhibition of tyrosine hydroxylase and tryptophan hydroxylase activities in brain thereby impeding the synthesis of dopamine and serotonin from tyrosine and tryptophan, respectively (Curtius et al., Reference Curtius, Niederwieser, Viscontini, Leimbacher, Wegmann, Blehova, Rey, Schaub and Schmidt1981; Harding et al., Reference Harding, Winn, Gibson, Arning, Bottiglieri and Grompe2014; Winn et al., Reference Winn, Scherer, Thöny, Ying, Martinez, Weber, Raber and Harding2018). Elevated brain Phe may also disrupt other central processes such as glutamatergic synaptic transmission and pyruvate kinase activity (Feksa et al., Reference Feksa, Cornelio, Dutra-Filho, de Souza Wyse, Wajner and Wannmacher2003; Martynyuk et al., Reference Martynyuk, Glushakov, Sumners, Laipis, Dennis and Seubert2005).
If left untreated, PKU is typically associated with severe irreversible cognitive and neurologic impairments during development (Bechar et al., Reference Bechar, Bornstein, Elian and Sandbank1965; Paine, Reference Paine1957). Historically, the treatment of PKU has centered on reducing Phe levels through dietary restrictions, beginning shortly after birth and continuing throughout one’s lifetime (Vockley et al., Reference Vockley, Andersson, Antshel, Braverman, Burton, Frazier, Mitchell, Smith, Thompson and Berry2014). Importantly, even with early and continuous treatment, individuals with PKU are still at risk for significant neurologic and cognitive sequelae. The most prevalent neurologic finding in individuals with early-treated PKU is disruptions in white matter integrity, with higher Phe levels associated with more severe abnormalities (Anderson & Leuzzi, Reference Anderson and Leuzzi2010; Clocksin et al., Reference Clocksin, Hawks, White and Christ2020). Nascent evidence suggests that gray matter structures of the brain (e.g., posterior cortex, putamen, cerebellum) may be impacted as well (Aldridge et al., Reference Aldridge, Cole, Moffitt Gunn, Peck, White and Christ2020; Brown et al., Reference Brown, Clocksin, Abbene, Ursery and Christ2022; Christ et al., Reference Christ, Price, Bodner, Saville, Moffitt and Peck2016).
From a neurocognitive standpoint, early-treated PKU is associated with a slight decrease in overall intellectual ability (O’Sullivan et al., Reference O’Sullivan, Tomazos, van Spronsen, Szabo, Venkataraman, Huria, Smith, Molony, Ingalls, Somera-Molina, Zhang and Harding2025) coupled with more pronounced disruptions in specific domains including executive function, attention, processing speed, and fine motor control (Christ et al., Reference Christ, Huijbregts, de Sonneville and White2010, Reference Christ, Abbene, Clocksin and Wegrzyn2021; Clocksin et al., Reference Clocksin, Abbene and Christ2023; Palermo et al., Reference Palermo, Geberhiwot, MacDonald, Limback, Hall and Romani2017). Individuals with early-treated PKU are also at increased risk for anxiety and depression as well as other psychosocial and psychiatric difficulties (Bilder et al., Reference Bilder, Kobori, Cohen-Pfeffer, Johnson, Jurecki and Grant2017; Boland et al., Reference Boland, Schoen, Singh, Clocksin, Cissne and Christ2024). Poor metabolic control (as reflected by higher Phe levels) is associated with more severe disruption in neurocognitive and psychological functioning for both children and adults (Clocksin et al., Reference Clocksin, Abbene and Christ2023; Romani et al., Reference Romani, Huijbregts, van Spronsen, MacDonald, van Wegberg, Staines and Olson2025).
Strict adherence to the dietary treatment regimen and maintenance of Phe levels within the recommended therapeutic target range is often difficult (Jurecki et al., Reference Jurecki, Cederbaum, Kopesky, Perry, Rohr, Sanchez-Valle, Viau, Sheinin and Cohen-Pfeffer2017) thus contributing to elevated Phe levels, poorer quality of life, and increased neurocognitive difficulties in individuals with early-treated PKU. This reality has fueled recent efforts towards developing novel therapies including tetrahydrobiopterin supplementation (sapropterin, sepiapterin; Harding, Reference Harding2024; Levy et al., Reference Levy, Milanowski, Chakrapani, Cleary, Lee, Trefz, Whitley, Feillet, Feigenbaum, Bebchuk, Christ-Schmidt and Dorenbaum2007; Muntau et al., Reference Muntau, Longo, Ezgu, Schwartz, Lah, Bratkovic, Margvelashvili, Kiykim, Zori, Campistol Plana, Bélanger-Quintana, Lund, Guilder, Chakrapani, Mungan, Guimas, Cabrales Guerra, MacDonald and Ingalls2024), enzyme substitution therapy (phenylalanine ammonia lyase; Harding et al., Reference Harding, Amato, Stuy, Longo, Burton, Posner, Weng, Merilainen, Gu, Jiang and Vockley2018, Reference Harding, Longo, Northrup, Sacharow, Singh, Thomas, Vockley, Zori, Bulloch Whitehall, Lilienstein, Lindstrom, Levy, Jones and Burton2024; Levy et al., Reference Levy, Sarkissian and Scriver2018; Thomas et al., Reference Thomas, Levy, Amato, Vockley, Zori, Dimmock, Harding, Bilder, Weng, Olbertz, Merilainen, Jiang, Larimore, Gupta, Gu and Northrup2018), additional small-molecule therapies such as repinatrabit (Harding et al., Reference Harding, Longo, Viader, Vaughn, Leal-Pardinas, Jurecki, McNutt, Muntau, Singh, Barrish, Vratsanos, Weng and Throup2025), and potential future nucleic acid-based therapies (mRNA therapy, gene therapy, or gene editing; Martinez et al., Reference Martinez, Harding, Schwank and Thöny2024). While clinical trial results and PKU treatment approval by national/international regulatory bodies such as the U.S. Food and Drug Administration (FDA) largely center on decreases in plasma Phe levels as the primary endpoint, the FDA has indicated that functional outcomes associated with a treatment (i.e., how patients’ lives are improved) are also highly important.
Neurocognitive functioning represents an important aspect of functional outcomes for individuals with PKU. Cognitive abilities such as executive function and attention play an essential role not only in typical everyday life (e.g., academic and occupational settings) but also in treatment management for individuals with PKU (e.g., organizing dietary and pharmacological regimens). Consistent with this, in a large survey conducted by the National PKU Alliance (a national PKU patient advocacy group), respondents identified reduced Phe levels, improved executive function, and improved attention as the three most important aspects of new disease management strategies (Brown & Lichter-Konecki, Reference Brown and Lichter-Konecki2016). Improved memory and mental fog were also highly endorsed as desired treatment targets. Within this context, there is a need for neuropsychological measures that are sensitive to the neurocognitive disruptions associated with early-treated PKU and can be readily integrated into clinical trial design. The NIH Toolbox represents one such candidate measure.
The NIH Toolbox is a computer-administered collection of neurobehavioral tools that assess cognitive, emotional, sensory, and motor function. It was created through a NIH Blueprint for Neuroscience Research initiative to support and advance medical studies (Weintraub, Bauer, et al., Reference Weintraub, Bauer, Zelazo, Wallner-Allen, Dikmen, Heaton, Tulsky, Slotkin, Blitz, Carlozzi, Havlik, Beaumont, Mungas, Manly, Borosh, Nowinski and Gershon2013). The NIH Toolbox’s development was motivated by a need for a standardized comprehensive assessment that can be utilized across a wide range of ages, levels of functioning, and clinical conditions (Gershon et al., Reference Gershon, Wagster, Hendrie, Fox, Cook and Nowinski2013). The Cognitive Battery represents the core of the NIH Toolbox and includes seven primary subtests, which take approximately 3–7 minutes each to administer while assessing a range of cognitive ability. The NIH Toolbox captures aspects related to both fluid cognition (e.g., executive function and processing speed) and crystallized cognition (e.g., receptive vocabulary ability).
The NIH Toolbox has several potential advantages for use in PKU clinical trials. From a scientific perspective, the NIH Toolbox: (1) assesses key cognitive domains (e.g., executive function and processing speed) at risk in individuals with PKU, (2) yields standard scores based on a large normative sample, and (3) integrates into a variety of study designs including those with multiple timepoints. The NIH Toolbox normative process was rigorous, involving a total sample of 4,859 participants, including 3,413 children ages 3–17 years stratified by age and gender (Beaumont et al., Reference Beaumont, Havlik, Cook, Hays, Wallner-Allen, Korper, Lai, Nord, Zill, Choi, Yost, Ustsinovich, Brouwers, Hoffman and Gershon2013; Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015). Its validity and reliability have also been well established in both the general pediatric and adult population (Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014; Weintraub, Bauer, et al., Reference Weintraub, Bauer, Zelazo, Wallner-Allen, Dikmen, Heaton, Tulsky, Slotkin, Blitz, Carlozzi, Havlik, Beaumont, Mungas, Manly, Borosh, Nowinski and Gershon2013; Weintraub, Dikmen, et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh and Gershon2013) as well as in several clinical populations (Carlozzi et al., Reference Carlozzi, Goodnight, Casaletto, Goldsmith, Heaton, Wong, Baum, Gershon, Heinemann and Tulsky2017; Hessl et al., Reference Hessl, Sansone, Berry-Kravis, Riley, Widaman, Abbeduto, Schneider, Coleman, Oaklander, Rhodes and Gershon2016; Shields et al., Reference Shields, Kaat, McKenzie, Drayton, Sansone, Coleman, Michalak, Riley, Berry-Kravis, Gershon, Widaman and Hessl2020; Tulsky et al., Reference Tulsky, Holdnack, Cohen, Heaton, Carlozzi, Wong, Boulton and Heinemann2017). From a pragmatic perspective, NIH Toolbox administration is standardized and largely automated, thus reducing examiner burden while increasing accessibility and the consistency and reliability of implementation across multiple sites.
Initial findings on the NIH Toolbox and PKU are promising. Results from a study by Christ et al. (Reference Christ, Clocksin, Zalik, Goodlett, Sacharow and Abbene2023) support the sensitivity of the NIH Toolbox to PKU-related neurocognitive impairments. The researchers found that a sample of adults with PKU (age 18–36 years) performed significantly worse on the Fluid Cognition Composite as compared to a demographically matched group of individuals without PKU. In addition, performance on the Fluid Cognition Composite and several contributing subtests strongly correlated with recent and historic blood Phe levels.
The present study was designed to further validate the NIH Toolbox as a potential tool for use in future clinical trials and research studies across the lifespan. Specifically, we sought to build upon the findings of Christ et al. (Reference Christ, Clocksin, Zalik, Goodlett, Sacharow and Abbene2023) and examine the sensitivity of the NIH Toolbox to PKU-related neurocognitive difficulties in a larger age range (8–55 years old). We also evaluated concurrent validity of the NIH Toolbox relative to several well-established measures of neurocognitive function (i.e., Wechsler Abbreviated Scale of Intelligence-II, WASI-II; Cambridge Neuropsychological Test Automated Battery, CANTAB; Conners Continuous Performance Test 3, CPT-3).
Methods
Participants
We examined data from a sample of 91 individuals with PKU aged 8 to 55 years (M = 25.4 yrs) participating in the ongoing Longitudinal Study of Hyperphenylalaninemia, which is a major component of the NIH-funded Phenylalanine Families and Researcher Exploring Evidence Consortium (PHEFREE) (For further details on consortium, see Christ et al., Reference Christ, Arnold, Lichter-Konecki, Berry, Grange, Harding, Jurecki, Levy, Longo, Morotti, Sacharow, Thomas and White2024). The self-described race and ethnicity background of the group was 82 White non-Hispanic, 3 White Hispanic or Latino, 4 Asian, 1 Black or African American, and 1 unknown. [Note that the sample’s race and ethnicity distribution is, in part, reflective of the higher incidence rate of PKU in individuals with European ancestry as compared to those with African, Hispanic, or Asian ancestry.] Additional demographic information for the participant sample is included in Table 1.
Sample characteristics
Table 1. Long description
A table comparing characteristics of individuals with PKU across different age groups. The table has 12 rows and 3 columns. Column headers are Variable, Children (8-17 Yrs), and Adults (18+ Yrs). Row 1: Variable, Children (8-17 Yrs), Adults (18+ Yrs). Row 2: N, 36, 55. Row 3: Age (years), 12.0 (2.8), 34.2 (9.8). Row 4: Sex (M/F), 19/17, 21/34. Row 5: Education (years), 5.8 (3.0), 14.8 (2.6). Row 6: FSIQ, 106 (15), 103 (12). Row 7: Most recent Phenylalanine levels (μmol/L), 369 (265), 503 (423). Row 8: Mean level for previous year (μmol/L), 345 (161), 544 (425). Row 9: Phe-restricted diet, 30 (83 percent), 30 (55 percent). Row 10: Medical food/formula, 26 (72 percent), 27 (49 percent). Row 11: Sapropterin, 23 (64 percent), 21 (38 percent). Row 12: Pegvaliaise, 1 (3 percent), 24 (44 percent).
a Frequency & Incidence rate (%) are shown for relevant variables. All other values represent M (SD).
b Estimated based on the WASI-2 (Wechsler, Reference Wechsler2011).
Inclusion criteria for the present analysis include: (1) a diagnosis of PAH deficiency (i.e., PKU) identified via newborn screening, (2) initiation of dietary treatment to limit Phe intake within eight weeks of birth, (3) age 8–55 years at time of study enrollment, and (4) completion of the NIH Toolbox and at least two or more of the validation measures (see below). Note that the full complement of NIH Toolbox subtests necessary to compute the composite scores of interest are administered beginning at age 8 years hence the lower age limit. The upper age limit corresponds generally to the onset of newborn screening for PKU.
Individuals with learning disorders and/or attention-deficit/hyperactivity disorder were not excluded, as symptoms related to these disorders occur more frequently among individuals with PKU than in the general population (Antshel, Reference Antshel2010). Furthermore, individuals were not excluded based on current or past treatment with pharmacological agents used to treat PKU (e.g., sapropterin dihydrochloride or pegvaliase-pqpz). Data on past blood Phe levels were compiled from available medical records. For purposes of the present investigation, we focused on the Phe level obtained most recently before the neurocognitive evaluation. [If participants did not have a Phe level within one year prior to cognitive testing, they were excluded from Phe analyses.] The mean number of days between Phe level and neurocognitive testing was 36 days (SD = 49, range = 0–303).
Materials and procedure
The present study was approved by the University of Utah Institutional Review Board (Review ID# 138750) acting as the central IRB and was carried out in accordance with the provisions of the World Medical Association Declaration of Helsinki. Informed consent (and assent when applicable) was obtained for all individuals prior to participation. As detailed below, the majority of neuropsychological measures were administered in person while a handful were administered remotely via secure video conference or web-based interfaces (e.g., select CANTAB subtests). In both instances, participants completed the tasks in a quiet room with adequate overhead lighting, and their performance (and environment) were closely monitored throughout by a psychometrician, neuropsychologist, or other qualified administrator.
In conjunction with the PHEFREE longitudinal study, extensive data was collected on participant’s physical, psychological, and cognitive health. A full list of the evaluation battery is detailed in a previous publication (Christ et al., Reference Christ, Arnold, Lichter-Konecki, Berry, Grange, Harding, Jurecki, Levy, Longo, Morotti, Sacharow, Thomas and White2024). For purposes of the present analysis, we focused on data from the NIH Toolbox and other core neuropsychological performance measures. Note that the NIH Toolbox, WASI-II Block Design subtest, and the CPT-3 were administered in-person to ensure standardized administration conditions and precise timing. The remaining WASI-II subtests and the CANTAB were administered remotely via secure video conference (Zoom) and Cambridge Cognition’s web platform, respectively, as these measures are well-suited to digital delivery and allow for reliable administration without the need for physical materials, while also increasing efficiency and accessibility for participants.
NIH toolbox
The NIH Toolbox’s Cognition Battery includes seven core subtests measuring aspects of executive function (cognitive flexibility: Dimensional Change Card Sort subtest; inhibitory control and attention: Flanker subtest; working memory: List Sorting subtest) as well as processing speed (Pattern Comparison subtest), episodic memory (Picture Sequence Memory subtest), expressive vocabulary ability (Picture Vocabulary subtest), and reading skills (Oral Reading Recognition subtest). Additional subtest descriptions are included in the Supplementary Materials.
In addition to generating individual subtest scores, the NIH Toolbox also yields three primary summary scores: (1) a Crystallized Cognition Composite that reflects verbal cognitive ability; (2) a Fluid Cognition Composite Score that reflects overall performance across tests of executive function, processing speed, and episodic memory; and (3) a Total Cognition Composite that reflects overall performance across both the crystallized and fluid cognition domains. For both the individual subtests and composites, the NIH Toolbox generates standard T scores which are corrected for age, gender, race/ethnicity, and education (for ages 3–17, parent’s education is used). As noted earlier, validity and reliability of the NIH Toolbox have been previously established in both the general population (Heaton et al., Reference Heaton, Akshoomoff, Tulsky, Mungas, Weintraub, Dikmen, Beaumont, Casaletto, Conway, Slotkin and Gershon2014) as well as in several clinical populations (Carlozzi et al., Reference Carlozzi, Goodnight, Casaletto, Goldsmith, Heaton, Wong, Baum, Gershon, Heinemann and Tulsky2017; Hessl et al., Reference Hessl, Sansone, Berry-Kravis, Riley, Widaman, Abbeduto, Schneider, Coleman, Oaklander, Rhodes and Gershon2016; Shields et al., Reference Shields, Kaat, McKenzie, Drayton, Sansone, Coleman, Michalak, Riley, Berry-Kravis, Gershon, Widaman and Hessl2020; Tulsky et al., Reference Tulsky, Holdnack, Cohen, Heaton, Carlozzi, Wong, Boulton and Heinemann2017).
Wechsler Abbreviated Scale of Intelligence-II (WASI-II; Wechsler, Reference Wechsler2011). The WASI-II is well-established instrument used to estimate overall intellectual ability and has been used extensively in past studies of PKU and other clinical conditions (e.g., Antenor-Dorsey et al., Reference Antenor-Dorsey, Hershey, Rutlin, Shimony, McKinstry, Grange, Christ and White2013; Clocksin et al., Reference Clocksin, Abbene and Christ2023). It comprises four subtests (Vocabulary, Similarities, Block Design, Matrix Reasoning) that allow for computation of an estimated Full-Scale IQ as well as a Perceptual Reasoning Index (PRI) and Verbal Comprehension Index (VCI). The WASI-II provides age-normed standard scores.
CANTAB (Cambridge Cognition, 2019). The CANTAB is a computer-administered set of neuropsychological measures. It is well-established and has been extensively validated (Levaux et al., Reference Levaux, Potvin, Sepehry, Sablier, Mendrek and Stip2007; Lowe & Rabbitt, Reference Lowe and Rabbitt1998; Luciana, Reference Luciana2003; Zygouris & Tsolaki, Reference Zygouris and Tsolaki2015). Of relevance, a web-based version of the CANTAB was recently developed, and initial research suggests no significant differences in performance across the platforms (supervised in-person vs remote web-based) (Backx et al., Reference Backx, Skirrow, Dente, Barnett and Cormack2020). Participants completed the Intra-Extra Dimensional Set Shift (IED), Rapid Visual Information Processing (RVP), Spatial Span (SSP), and Spatial Working Memory (SWM) subtests remotely via the CANTAB’s web-based portal. These subtests were selected based on their assessment of neurocognitive abilities (e.g., executive function, attention, processing speed) that, based on the extant literature (e.g., Clocksin et al., Reference Clocksin, Abbene and Christ2023), are expected to be impacted in individuals with early-treated PKU. [Additional subtest descriptions are included in the Supplementary Materials.]
Age-normed standard scores are available for a selection of subtests (i.e., IED, RVP, SSP, & SWM) and age ranges (i.e., 18 years and older). Within this context, scores on the four subtests were averaged to generate a composite score, and analysis of CANTAB data was restricted to participants 18 years and older.
Conners Continuous Performance Test 3 (CPT-3; Conners, Reference Conners2014). The CPT-3 is a well-established computerized test of attention that captures aspects of sustained attention, inattentiveness, impulsivity, and vigilance. During the test, participants are showing are shown letter stimuli one-at-a-time and are instructed to press a button as quickly as possible for all stimuli except the letter “X” (non-target). The test takes approximately 14 minutes to complete, and the interstimulus interval and proportion of targets to non-targets are varied systematically over the course of the test session. Age-normed t scores (M = 50, SD = 10) are generated for several performance variables. Discriminability (d’, i.e., the ability to differentiate targets from non-targets), hit reaction time (HRT, the mean response speed for correct responses), and response time variability (HRT SD, standard deviation of mean response speed) were chosen as primary outcome variables based on the cognitive processes that they reflect (i.e., selective attention, processing speed, and sustained attention/vigilance, respectively) and their relevance to the neurocognitive profile associated with early-treated PKU. Scores from these three metrics were averaged to create a composite score.
Analytical approach
An alpha level of .05 was adopted. As all hypotheses and analyses were specified a priori rather than derived from exploratory testing, no correction for multiple comparisons was applied.
To simplify data analysis and interpretation, standard scores for all performance measures were converted to a common scale with M = 100 and SD = 15, with higher scores reflecting better performance. The resulting data were considered separately for children (8–17 years old) and adult (18 yrs and older) subgroups. To evaluate NIH Toolbox sensitivity to cognitive compromise in PKU, one sample t-tests were conducted to determine whether performance of the PKU group differed significantly from that of the normative sample (M = 100) on the NIH Toolbox Composite scores. Chi square tests were also used to evaluate whether a disproportionate number of individuals in the PKU group scored in the “clinically at-risk” range (>1 SD below the normative mean) as compared to norm-based expectations (16th percentile). The relationship between NIH Toolbox scores and other factors (i.e., age and recent Phe levels) was also explored via Pearson correlations.
To demonstrate concurrent validity – defined as the extent to which a measure is associated with other instruments assessing similar constructs, the associations between the NIH Toolbox composite scores (i.e., Total Cognition, Crystallized Cognition, and Fluid Cognition) and scores on other well-established instruments were examined. We anticipated that the Total Cognition Composite would correlate with WASI Full Scale IQ (FSIQ), as both measures capture overall cognitive functioning. Similarly, we expected the Crystallized Cognition Composite to correlate with WASI Verbal Comprehension Index, given that both are designed to index acquired verbal knowledge and language-based reasoning abilities. We anticipated that the Fluid Cognition Composite would correlate with WASI PRI, as both primarily assess nonverbal problem solving and aspects of fluid intelligence. Finally, we expected the Fluid Cognition Composite to also correlate with performance on the Conners CPT-3 and CANTAB, as these measures tap core components of attention and executive control. Linear regression was used to test whether participants’ NIH Toolbox Composite scores were associated with their performance on the corresponding validation instruments noted above.
Results
NIH toolbox
A table with descriptive statistics is included in the supplemental materials. The PKU child group (<18 years of age) performed significantly below expectations on the Fluid Cognition Composite and Total Cognition Composites [M = 90.9 & 91.9, respectively; t(35) > 2.77, p < .01, g > .45 in both instances] but not the Crystallized Cognition Composite [M = 96.5; t(35) = 1.33, p = .19, g = .22]. As can be seen in Figure 1, performance on Flanker inhibitory subtest and Pattern Comparison processing speed subtest showed the largest effects [M = 88.5 & 90.3, respectively; t(35) > 3.00, p < .005, g > 0.49 in both instances]. In addition, the proportion of participants in the clinically at-risk range (<1 SD below mean; 16th percentile) was above expectations for the Fluid Cognition Composite (39%; χ 2 = 12.80, p < .001).
Mean scores for the NIH Toolbox shown separately for each age group. Composite scores are depicted in dark blue, and individual subtest scores in lighter blue. The dashed line reflects the normative mean (score = 100). Error bars represent the standard error of the mean. Scores for each age group were compared to the normative mean, and resulting statistically significant effects (p < .05) are denoted by an asterisk.
The correlation between age and both Fluid Cognition and Total Cognition Composite scores approached significance (r = −.31, p = .066 in both instances), with poorer scores observed for younger as compared to older children (see Figure 2). Interestingly, most recent Phe levels correlated negatively with the Crystallized Cognition Composite (r = −.33, p = .049) but not with the Fluid Cognition or Total Cognition Composites (r = −.10 & −.25, p > .14 in both instances).
Scatterplots showing the relationship between NIH Toolbox composite scores and age (top row) and most recent Phe levels (bottom row) for the child group. The dashed lines represent the corresponding linear regression lines.
For the adult group (18 yrs & older), as shown in Figure 1, performance was consistent with expectations for all three composite scores: Fluid Cognition, Crystallized Cognition, and Total Cognition [M = 97.9, 103.7, & 100.9, respectively; t(54) < 1.70, p > .10, g < .22 in all instances]. The proportion of adult participants in the clinically at-risk range (<1 SD below mean; 16th percentile) for the Fluid Cognition Composite was higher than expected but not significantly so (25%; χ 2 = 3.32, p = .068). In the adult group, age did not correlate significantly with any of the three composite scores [|r| < .09, p > .51 in all instances] (See Figure 3). Most recent Phe levels also did not correlate with any of the composites [|r| < .20, p > .15 in all instances].
Scatterplots showing the relationship between NIH Toolbox composite scores and age (top row) and most recent Phe levels (bottom row) for the adult group. The dashed lines represent the corresponding linear regression lines.
Concurrent validity of NIH toolbox with other measures
Across both the child and adult groups, the NIH Toolbox Total Cognition Composite was a significant predictor of the WASI-II FSIQ [Child: F(1,34) = 28.87, p < .001, R 2 = .46; Adult: F(1,53) = 22.10, p < .001, R 2 = .29], indicating that higher Total Cognition was associated with higher FSIQ (see Figure 4). The NIH Toolbox Crystallized Cognition and Fluid Cognition Composites were also significant predictors of the WASI-II Verbal Comprehension [Child: F(1,34) = 30.03, p < .001, R 2 = .47; Adult: F(1,53) = 20.56, p < .001, R 2 = .28] and Perceptual Reasoning Indices [Child: F(1,34) = 13.14, p < .001, R 2 = .29; Adult: F(1,53) = 5.62, p = .021, R 2 = .10], respectively.
Scatterplots showing the relationship between NIH Toolbox composite scores and corresponding validation instruments. The dashed lines represent the corresponding linear regression lines. CC = Cognition Composite; FSIQ = WASI-II Full Scale IQ; VCI = WASI-II Verbal Comprehension Index; PRI = WASI-II Perceptual Reasoning Index; CANTAB = Cambridge Neuropsychological Test Automated Battery; CPT-3 = Conners Continuous Performance Test 3.
The NIH Toolbox Fluid Cognition Composite, which encompasses aspects of executive function and attention, was also a significant predictor of composite scores from the CANTAB in the adult group [F(1,52) = 5.82, p = .019, R 2 = .10]. (As noted previously, CANTAB analysis was restricted to the adult group due to unavailability of normative scores for children.) The predictive relationship between the Fluid Cognition Composite and the CPT-3 composite score was significant for the child group [F(1,34) = 6.28, p = .017, R 2 = .16] and approached significance for the adult group [F(1,51) = 3.24, p = .078, R 2 = .06].
Discussion
In light of a recent surge in novel treatment development for PKU, there is a need for assessment measures that are sensitive to PKU-related neurocognitive disruptions and can be readily integrated into clinical trial and research study design. Findings from an initial study of the NIH Toolbox were promising and demonstrated its sensitivity to neurocognitive compromise in a sample of young adults with PKU (Christ et al., Reference Christ, Clocksin, Zalik, Goodlett, Sacharow and Abbene2023). The current study aimed to examine the NIH Toolbox in a larger age range (8–55 years old) of individuals with early-treated PKU and evaluate the concurrent validity of the NIH Toolbox relative to several more established measures of neurocognitive function.
Presently, we found that both children and adults with PKU performed well within expectations on the Crystallized Cognition Composite (M = 97 & 104, respectively), which is consistent with past findings suggesting that basic verbal ability and knowledge are unaffected in individuals with early-treated PKU (Clocksin et al., Reference Clocksin, Abbene and Christ2023; Palermo et al., Reference Palermo, Geberhiwot, MacDonald, Limback, Hall and Romani2017). In terms of the Fluid Cognition Composite, children with PKU performed below norm-based expectations (M = 91), demonstrating particular difficulty on tests of inhibitory control and processing speed. Poor performance on the Fluid Cognition subtests contributed to the pediatric group also scoring below expectations on the Total Cognition Composite (M = 92). This finding contrasts with the FSIQ estimates derived from the WASI-II, which were within the expected range (M = 106). This apparent discrepancy likely reflects limitations of the WASI-II rather than the NIH Toolbox. Specifically, the WASI-II is based on a limited number of subtests and does not assess executive functioning or attention to the same extent as more comprehensive measures of intellectual ability, such as the full Wechsler Intelligence Scale for Children (or the NIH Toolbox). Therefore, the pediatric group’s lower scores on the Total Cognition Composite relative to WASI-II FSIQ may reflect deficits in cognitive domains (i.e., executive functioning, attention, and processing speed) that are commonly affected in individuals with PKU but are not adequately assessed by the WASI-II. Consistent with this interpretation, the pediatric group also performed below expectations on the CPT-3 (M = 87), a measure that assesses attention and processing speed.
In contrast to findings for the child group, NIH Toolbox Fluid Cognition scores – as well as CANTAB and CPT-3 scores – for the adult group were generally average and within expectations (M = 98, 101, and 100, respectively). This finding is inconsistent with those of the aforementioned Christ et al. (Reference Christ, Clocksin, Zalik, Goodlett, Sacharow and Abbene2023) study, which found that a sample of adults with PKU performed more poorly as compared to a non-PKU group on the Fluid Cognition composite. It is also somewhat unexpected given previous reports of difficulties with executive function, attention, and processing speed (i.e., aspects of fluid cognition) in both children and adults with PKU (Christ et al., Reference Christ, Huijbregts, de Sonneville and White2010; Clocksin et al., Reference Clocksin, Abbene and Christ2023; Romani et al., Reference Romani, Huijbregts, van Spronsen, MacDonald, van Wegberg, Staines and Olson2025). Of note, past research has also found that poorer metabolic control (as reflected by higher Phe levels) is associated with more cognitive difficulties (Clocksin et al., Reference Clocksin, Abbene and Christ2023; Romani et al., Reference Romani, Palermo, MacDonald, Limback, Hall and Geberhiwot2017). Within this context, the present adult participant sample is characterized by relatively good metabolic control (recent M Phe = 503 μmol/L; Mdn Phe = 370 mmol/L) in comparison to the more poorly-controlled cohorts included in many older studies, with over 80% of the current adult group utilizing pharmacological treatment to help manage their metabolic control.
For the child group, fluid composite scores were also found to correlate with age. Performance difficulties were more apparent for younger as compared to older children even after accounting for age-related differences in the general population (per the age-corrected normative scores). To the extent that older children and adults in the present sample performed “within expectations,” the observed pattern is generally consistent with a resolving deficiency. Prior work on age-related differences in children with PKU has been mixed, with some findings suggesting a static deficit across ages (Romani et al., Reference Romani, Huijbregts, van Spronsen, MacDonald, van Wegberg, Staines and Olson2025), and others indicative of an emerging deficit with more pronounced impairment for older as compared to younger children (e.g., White et al., Reference White, Nortz, Mandernach, Huntington and Steiner2002). To the extent that the present data is only cross-sectional (and thus captures age-related differences), collection and analysis of future longitudinal data via the PHEFREE consortium and other similar efforts will be critical in fully evaluating the presence of potential age-related changes in neurocognitive performance.
Results from the psychometric evaluation of the NIH Toolbox in the present sample of individuals with PKU were promising. All three NIH Toolbox composite scores (i.e., Total Cognition, Fluid Cognition, and Crystallized Cognition) demonstrated good concurrent validity with their corresponding validation indices from the WASI-II (i.e., Full Scale IQ, Perceptual Reasoning Index, and Verbal Comprehension Index, respectively). The Fluid Cognition Composite showed more modest correlations with the CANTAB and CPT-3 outcome variables. While there is significant conceptual overlap in the cognitive domains assessed by the NTB Fluid Cognition Composite and each of these measures (e.g., processing speed), there are also likely aspects of cognition that uniquely contribute to each measure as well. For example, the CPT-3 is designed to assess vigilance and sustained attention, abilities that are not optimally captured with the NIH Toolbox given the relatively brief length of its subtests. Conversely, the NIH Toolbox captures other aspects of cognition (e.g., working memory and cognitive flexibility) more thoroughly than the CPT-3 does. It follows that a combination of the NIH Toolbox and CPT-3 (or other measure of sustained attention) may provide a more comprehensive neurocognitive profile than relying on only one of these measures in isolation.
Limitations & future directions
A handful of limitations for the present study are noted. Christ et al. (Reference Christ, Clocksin, Zalik, Goodlett, Sacharow and Abbene2023) reported a robust correlation between concurrent Phe levels and NTB Fluid Cognition composite scores in a sample of adults with PKU. A similar relationship was not observed in the present adult dataset for the Fluid Cognition composite nor the other composite scores. For the child group, a significant correlation was detected between Phe levels and only Crystallized Cognition composite scores (not Total or Fluid Cognition scores). Of note, blood Phe measurements were generally not assessed on the same day that the neurocognitive assessments were obtained, with an average interval of over 30 days between collection of Phe data and neurocognitive testing. This limitation likely hindered the present evaluation of the relationship between metabolic control (as reflected by Phe levels) and neurocognitive performance. Further efforts to collect and analyze historical blood Phe concentrations over the participants’ lifetimes are planned.
A notable feature of the present cohort is the high prevalence of pharmacological Phe-lowering treatment use (greater than 60% and 80% in the child and adult groups, respectively). This reflects a contemporary PKU population and distinguishes our sample from earlier studies, but also introduces important considerations for interpretation. While the present sample size was large within the context of research on rare diseases such as PKU, it was insufficient to examine potential subgroup differences related to factors such as treatment modality (e.g., dietary restriction, sapropterin, pegvaliase) and amount of natural protein intake. Accordingly, findings should be interpreted in the context of widespread and heterogeneous treatment use. Ongoing recruitment and longitudinal follow-up will be essential to better characterize treatment-specific effects and neurocognitive trajectories over time.
As detailed above, the current cross-sectional dataset provides evidence of concurrent validity for the NIH Toolbox within the context of PKU. A next step in the validation of the NIH Toolbox for potential use in intervention-related clinical trials is the collection of longitudinal data, which will allow for the evaluation of other psychometric properties including test–retest reliability and sensitivity-to-change. Data collection efforts are currently underway on this front as part of the PHEFREE longitudinal study (Christ et al., Reference Christ, Arnold, Lichter-Konecki, Berry, Grange, Harding, Jurecki, Levy, Longo, Morotti, Sacharow, Thomas and White2024).
Summary & conclusions
The present study evaluated the use of the NIH Toolbox as a tool for assessing neurocognitive function in individuals with early-treated PKU across a broad age range (8–55 years old). Consistent with prior research, crystallized cognition was intact in both children and adults. In contrast, fluid cognition deficits were observed primarily in younger children while older children and adults generally performed within expectations. NIH Toolbox composite scores demonstrated strong concurrent validity with Wechsler indices and moderate correlations with other established measures, supporting its utility in PKU research. Future work should focus on longitudinal data collection to assess test–retest reliability, sensitivity to change, and potential age-related trajectories.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355617726102264.
Acknowledgments
None.
Funding statement
This research was funded by a research grant from the US National Institutes of Health (NIH). The PHEFREE consortium (U54HD100982) is a part of the National Institutes of Health (NIH) Rare Disease Clinical Research Network (RDCRN), supported through collaboration between the National Center for Advancing Translational Science (NCATS), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), the National Institute of Neurological Disorders and Stroke (NINDS), and the Office of Dietary Supplements. Additional philanthropic support of some investigators is provided by the National PKU Alliance (NPKUA). The authors confirm independence from the funders; the content of the article has not been influenced by the funders.
Competing interests
COH has received consulting fees from BioMarin, Jnana, PTC, Sanofi, Pfizer, Synlogic, Tessera Therapeutics, and Tome Biosciences and conducted clinical research trials for BioMarin, Jnana, Sanofi, and Synlogic related to treatment of PAH deficiency. SEC has received consulting fees from BioMarin, Jnana, Otsuka, and PTC; and BioMarin has funded past and current research. DAW has conducted clinical research trials for BioMarin. MMD has received consulting fees from AMO Pharma and Calico Life Sciences. The remaining authors report no competing interests.
Additional contributors
All PHEFREE Consortium sites contributed to the longitudinal study dataset. Site principal investigators and other additional contributors (to be listed in the PubMed, in alphabetical order): Georgianne Arnold, University of Pittsburgh, Pittsburgh, PA; Gerard T. Berry, Boston Children’s Hospital, Harvard Medical School, Boston, MA; Suzin Blakenship, Washington University, St Louis, MO; Dorothy K. Grange, Washington University School of Medicine, St Louis, MO; Elaina Jurecki, National PKU Alliance, San Ramon, CA; Harvey Levy, Boston Children’s Hospital, Harvard Medical School, Boston, MA; Uta Lichter-Konecki, University of Pittsburgh, Pittsburgh, PA; Nicola Longo, University of Utah, Salt Lake City, UT; Hadley Morotti, Oregon Health & Science University, Portland, OR; Stephanie Sacharow, Boston Children’s Hospital, Harvard Medical School, Boston, MA; Brian J. Shayota, University of Utah, Salt Lake City, UT; Janet Thomas, Children’s Hospital of Colorado, University of Colorado School of Medicine, Aurora, CO.
Open access
