Overview of included studies

The systematic search was conducted in two separate runs. In the first run, 2,140 records were identified. After removal of duplicates (N = 938), 1,202 records were screened. Of these, 935 were excluded primarily because they consisted of gray literature or review articles, involved non-validated, non-digital, or noncognitive assessment tools, focused on pediatric populations, described rehabilitation interventions rather than assessment tools, or did not assess attention or EF. Additional exclusions were due to telephone-based assessments or publications not in English (Figure 1). Following full-text assessment and further eligibility screening, 209 studies from the first run met the inclusion criteria.

Figure 1.

PRISMA 2020 flow diagram of the study selection process. Two independent database searches (Embase, Scopus, PubMed) were conducted. Following duplicate removal, screening, and eligibility assessment, a total of 236 studies were included in the final review (209 from the first run, 27 from the second).

A second search run identified 443 additional records. After the removal of duplicates (N = 194), 249 records were screened, of which 193 were excluded, mainly because they had been identified in the previous run or did not meet the eligibility criteria. After full-text assessment, 27 additional studies were included.

Overall, the final sample comprised 236 studies (209 from the first run and 27 from the second run), spanning from randomized controlled trials to observational and longitudinal designs, to validation studies. The studies were rather heterogeneous also when considering population characteristics, ranging from healthy adults and aging cohorts to clinical groups with neurological or psychiatric conditions. The methodological quality of the included studies was generally good. According to the JBI categories, 123 studies (52.1%) had a low risk of bias, 54 (22.9%) a low-to-moderate risk of bias, 43 (18.2%) a moderate risk of bias, 15 (6.4%) a moderate-to-high risk of bias, and 1 study (0.4%) a high risk of bias. Different JBI critical appraisal checklists were applied according to study design, with the Analytical Cross-Sectional Studies checklist being the most frequently used (N = 123, 52.1%), followed by the Diagnostic Test Accuracy checklist (N = 57, 24.2%) and the Cohort Studies checklist (N = 31, 13.1%). Methodological quality varied across study designs. Most analytical cross-sectional studies were rated as having a low risk of bias (78.9%), whereas diagnostic accuracy studies were more frequently classified as moderate risk (42.1%) or moderate-to-high risk of bias (21.1%). Among cohort studies, ratings were more evenly distributed, with 32.3% judged as low risk and 32.3% as low-to-moderate risk of bias. Nearly all case–control studies were classified as low or low-to-moderate risk of bias (83.3%). The following subsections present these findings in relation to each research question.

The technological continuum

Among the 236 included studies 107 distinct digital tools were identified. Despite their heterogeneity, these tools can be broadly categorized following a gradient of technological sophistication and complexity (Chen et al., Reference Chen, Zhen and Peng2023).

At the lower end of the continuum there are purely digitized versions of traditional paper-and-pencil tests, which were reported either as standalone tools (N = 10) or as components within larger test batteries (N = 6). The most used among these were the Trail Making Test, the Wisconsin Card Sorting Test, and the Stroop task. Moving along the gradient, native computerized test batteries can be found. Among these, NeuroTrax (N = 27), Cognitive Drug Research (CDR) System (N = 17), and CANTAB (N = 13) were those most frequently reported. While many subtests are intrinsically digital, some retain conceptual continuity with traditional measures. For example, NeuroTrax incorporates digital adaptations of the Trail Making Test, Stroop task, and Digit Span. In contrast, batteries such as CANTAB, Cogstate, CDR, and ANAM include tasks modeled on established constructs (e.g., memory, attention, processing speed) which however are not digital versions of well-established paper-and-pencil tests. Overall, these platforms occupy an intermediate position.

At the higher end of the technological gradient there are emerging and immersive technologies, including VR platforms, serious games, and hybrid systems integrating eye-tracking. These tools capitalize on the digital medium itself to enable assessments that cannot be replicated on paper nor on a monitor, offering interactive, immersive, and ecologically valid testing environments. VR has been applied to the assessment of multiple cognitive domains across clinical and nonclinical populations. For instance, two studies specifically investigated attention and executive dysfunctions using VR-based paradigms, focusing on attention deficits following traumatic brain injury and on the ecological evaluation of EF through immersive multitasking scenarios (Jansari et al., Reference Jansari, Devlin, Agnew, Akesson, Murphy and Leadbetter2014; Malegiannaki et al., Reference Malegiannaki, Garefalaki, Pellas and Kosmidis2024). One study examined the characterization and assessment of EF through a realistic virtual cooking task in euthymic patients with bipolar disorder (Cañada et al., Reference Cañada, Torres, Andreu-Martinez, Cristancho, Chicchi Giglioli, Garcia-Blanco, Adriasola, Navalón, Sierra and Alcañiz2024). Two validation studies addressed the psychometric properties of established neuropsychological measures adapted to VR environments, namely the Virtual Reality Paced Auditory Serial Addition Test (VR-PASAT) in a college sample and the Virtual Reality Stroop Task (VRST) in active-duty military personnel (Armstrong et al., Reference Armstrong, Reger, Edwards, Rizzo, Courtney and Parsons2013; Parsons & Courtney, Reference Parsons and Courtney2014). In addition, one experimental study introduced a gamified VR paradigm to assess visual search, response inhibition, and visual short-term memory (Hadjiaros et al., Reference Hadjiaros, Shimi, Neokleous, Pattichis and Avraamides2025). Eye-tracking has been used to examine cognitive functioning through the analysis of eye movement behavior in clinical populations. One study investigated the relationship between eye movement parameters and cognitive domains in non-demented patients with Parkinson’s disease (Wong et al., Reference Wong, Fung and Chan2019). One randomized controlled trial assessed the effectiveness of an eye movement–based intervention to improve EF in stroke patients (He et al., Reference He, Ji, Wei, Wang, Xu, Lu, Ma and Wang2021). One study validated eye-tracking measures embedded in the computerized Wisconsin Card Sorting Test by comparing stroke patients with healthy controls (Leitner et al., Reference Leitner, Coady, Miller and Libben2023). These platforms enable the measurement of multitasking, attention, visuospatial skills, and EF in dynamic, immersive, and interactive environments, providing richer, more behaviorally relevant data than conventional digital or digitized paper-and-pencil batteries (Kirkham et al., Reference Kirkham, Kooijman, Albertella, Myles, Yücel and Rotaru2024). These “emerging technology” tools aim to create more ecologically valid assessments of cognitive function, capturing performance in contexts that closely resemble real-life situations rather than structured, artificial tests. While still less frequently used than standardized computerized assessments, their adoption is increasing, reflecting a trend toward leveraging digital technology to move beyond screen-based testing (Harris et al., Reference Harris, Tang, Birnbaum, Cherian, Mendhe and Chen2024).

Administration modalities

Administration modality can be decomposed along two independent dimensions: (i) the location of the examiner, ranging from fully in-person to fully remote, and (ii) the type of software installation of the test, distinguishing between locally installed software and web- or app-based platforms. As these dimensions are independent, their combinations give rise to different ways of administering the test. For instance, a web-based tool may still be administered in person. Yet this was not the case for most studies in this review which almost invariably described conventional administration setups.

The vast majority of administrations were conducted in person only, accounting for 200 out of 236 studies (85%). In terms of software installation, most of the studies relied on locally installed software administered via standard computers (N = 153), tablets (N = 27), or multi-device platforms (N = 13), whereas only 2 studies used explicitly web-based platforms, both in the context of remote unsupervised administration. Remote, unsupervised administration accounted for 28 out of 236 studies (11.9%). Remote, supervised sessions were reported in 5 out of 236 studies (2.1%). Hybrid approaches combining in-person and remote modalities were reported in 4 studies: in two cases the remote component was unsupervised, in one case it was supervised, and in one case it involved videoconference-based administration (1.7%). Importantly, several web- or app-based platforms that in principle are technically suitable for self-administration, were only delivered under supervision, either in person or via videoconference, likely to better ensure protocol adherence. Their usability and operational requirements, including the need for guidance, clear instructions, and support for digital literacy, are even more critical for certain populations such as older adults, whereby sensory or motor limitations and accessibility needs can strongly limit the feasibility of independent use (Wallace et al., Reference Wallace, Donoso Brown, Simpson, D’Acunto, Kranjec, Rodgers and Agostino2019).

Licensing, technical requirements, and open access

Only 37 out of the 107 identified digital tools (34.9%) were presented with relevant information on licensing or normative data availability, even after tracing each tool to at least one external validation publication identified through the reference lists of the included studies. Among these, the majority were available exclusively through commercial license (N = 20; 18.9% of all tools), including widely adopted platforms such as CANTAB (Kuzmickienė & Kaubrys, Reference Kuzmickienė and Kaubrys2016), CDR (McKeith et al., Reference McKeith, Del Ser, Spano, Emre, Wesnes, Anand, Cicin-Sain, Ferrara and Spiegel2000), CogState (Cairney et al., Reference Cairney, Clough, Jaragba and Maruff2007), NeuroTrax (Dwolatzky et al., Reference Dwolatzky, Whitehead, Doniger, Simon, Schweiger, Jaffe and Chertkow2003), and TAP (Schulz et al., Reference Schulz, Kopp, Kunkel and Faiss2006). Two additional tools (1.9%) offered free versions with limited features, typically requiring payment for enabling full functionality such as scoring (LUMOSITY, Kassam et al., Reference Kassam, Chen, Nosheny, McGirr, Williams, Ng, Camacho, Mackin, Weiner and Ismail2023; NeuroCognitive Performance Test, Jaffe et al., Reference Jaffe, Kaluszka, Ng and Schafer2022), and two (1.9%) provided demo-only versions (NIH-TB-CB, O’Shea et al., Reference O’Shea, Cohen, Porges, Nissim and Woods2016; OCS-Plus, Demeyere et al., Reference Demeyere, Haupt, Webb, Strobel, Milosevich, Moore, Wright, Finke and Duta2021).

Seven tools (6.5%) could be accessed without requiring payment: two usable directly online without installation (Cognitron, Shibata et al., Reference Shibata, Attaallah, Tai, Trender, Hellyer, Hampshire, Irani, Manohar and Husain2024; VIRTRAEL, Rute-Perez et al., Reference Rute-Perez, Rodriguez-Dominguez, Saez-Sanz, Perez-Garcia and Caracuel2025), and five available via free download or dedicated software (THINC-it, Shibaoka et al., Reference Shibaoka, Masuda, Iwasawa, Ikezawa, Eguchi and Nakagome2023; ABBA, Loring et al., Reference Loring, Hewitt, Drane, Zhao, Xu, Lah and Goldstein2025; ACE-X, O’Laughlin et al., Reference O’Laughlin, Cheng, Volponi, Lorentz, Obregon, Younger, Gazzaley and Uncapher2025; BoxTask, Foxe et al., Reference Foxe, Irish, Carrick, Cheung, Teng, Burrell, Kessels and Piguet2024; CNBC, Zhang et al., Reference Zhang, Hong, Yang, Li and Ye2024). A further tool, CALCAP (Waterloo et al., Reference Waterloo, Ingebrigtsen and Romner1997), resulted available for free download yet requiring obsolete hardware, substantially limiting its practical usability. None of these tools provided public access to source code or normative data. Only one tool, SATURN (Self-Administered Tasks Uncovering Risk of Neurodegeneration; Bissig et al., Reference Bissig, Kaye and Erten-Lyons2020), met the stricter criterion of genuine open-source release: its source code is publicly available, and an independent research group has already used it to derive normative data (e.g., for Italy see Giaquinto et al., Reference Giaquinto, Assecondi, Leccese, Romano and Angelelli2024). Finally, three tools (2.8%) were not publicly available or were restricted to academic research contexts, and none reported normative data, limiting their current clinical applicability (CoCoSc, Wong et al., Reference Wong, Fong, Mok, Leung and Tong2017; CoGNIT, Behrens et al., Reference Behrens, Elgh, Leijon, Kristensen, Eklund and Malm2020; KeyStrokes, Lopez et al., Reference Lopez, Fulton, Kristinsson, Kim, Stuart, Chen, Thomas, Hussey-Zommers, Roy, Kapoor and Conrad2025).

Regarding technical requirements, most tools (N = 56; 52.8%) could be administered using standard computers. An additional 20 tools (18.9%) were designed for tablets (e.g., TabCAT, Tsoy et al., Reference Tsoy, Possin, Thompson, Patel, Garrigues, Maravilla, Erlhoff and Ritchie2020; THINC-it, Shibaoka et al., Reference Shibaoka, Masuda, Iwasawa, Ikezawa, Eguchi and Nakagome2023; OCS-Plus, Demeyere et al., Reference Demeyere, Haupt, Webb, Strobel, Milosevich, Moore, Wright, Finke and Duta2021), and one (0.9%) for smartphones (EMCA, Campbell et al., Reference Campbell, Paolillo, Heaton, Tang, Depp, Granholm, Heaton, Swendsen, Moore and Moore2020). Twenty tools (18.9%) operated across multiple device types. Specialized hardware was required for the six VR-based platforms reported above. Two tools (1.9%) were exclusively web-based, requiring only a browser without dedicated software installation (NIH-TB-CB, O’Shea et al., Reference O’Shea, Cohen, Porges, Nissim and Woods2016; ONAs, Biagianti et al., Reference Biagianti, Fisher, Brandrett, Schlosser, Loewy, Nahum and Vinogradov2019). Finally, two tools (1.9%) did not specify the device used: CADTS (Walia et al., Reference Walia, Kumar, Khosla and Grover2023) and an unnamed computerized battery (Lorenzo-Lopez et al., Reference Lorenzo-Lopez, Cibeira, Hemadeh, Lopez-Lopez, Lema-Arranz, Maseda, Fernandez-Bertolez, Costa, Pasaro, Valdiglesias, Millan-Calenti and Laffon2025).

This heterogeneity may directly influence their implementation, scalability, and adoption in clinical and research contexts, independently of their technological sophistication or psychometric coverage. However, these results align with a growing shift in recent years toward tablet‑based and cross‑platform digital tools, which offer greater portability and ease of deployment compared with desktop‑only solutions (Asensio & Duñabeitia, Reference Asensio and Duñabeitia2023; Koo & Vizer, Reference Koo and Vizer2019).

Psychometric characteristics

For each digital tool we tried to quantify the extent to which it has been validated from a psychometric point of view. To this aim, we calculated a psychometric coverage index (PCI) on the number of psychometric properties evaluated across validation-related studies only (N = 122). The set of psychometric properties included in the PCI was not defined a priori based on a specific psychometric framework. Instead, a data-driven approach was adopted. All psychometric characteristics assessed across the eligible validation studies were extracted during data collection. The final set of 11 psychometric properties corresponded to all the psychometric outcomes reported in at least one of the included studies. These were: criterion validity, concurrent validity, ecological validity, factorial/construct validity, discriminant validity, internal consistency, test–retest reliability, convergent validity, inter-setting reliability, practice effects, and intra-rater reliability. The PCI was then calculated as a proportion based on the presence or absence of each of these properties. Since studies that used previously validated tools without novel psychometric data were excluded from this counting, the PCI reflects the extent of psychometric evaluation for each tool, rather than their number of citations or the number of studies. This approach ensures that those tools that have been thoroughly validated even in a small number of studies still receive a high PCI, also when they are then widely used without additional validation. For example, a tool with a single validation study assessing five properties would be characterized by a PCI = 5/11≈ 0.45, whereas a widely used tool evaluated in multiple studies would not achieve a higher PCI unless new properties were assessed. Note that the PCI reflects psychometric coverage only within the attention/EF literature captured by our search; tools extensively validated for other domains may therefore show a lower PCI than their overall validation record would suggest.

Across validation-focused studies (N = 122), certain psychometric properties were examined more frequently than others (Table 2). Concurrent validity was the most reported psychometric property (45.1%), followed by discriminant validity and convergent validity (33.6% each), and factorial/construct validity (32.8%). Test–retest reliability and criterion validity were reported in approximately 30% and 27% of studies, respectively. In contrast, internal consistency, practice effects, and ecological validity were less frequently evaluated, ranging from 10.7% to 15.6%. Inter-setting reliability and intra-rater reliability were rarely reported (4.9% each). While some core psychometric properties are widely assessed, several important dimensions of reliability and validity remain underexplored across digital tools, indicating potential gaps in comprehensive psychometric evaluation.

Importantly, a comparison between psychometric coverage, number of validation studies, and absolute number of citations revealed a partial mismatch. Tools that were most frequently used were not consistently those with the highest number of validation studies or the broadest psychometric coverage. Only a few tools simultaneously ranked highly across all three indicators. This pattern, illustrated in Figure 2, highlights a gap between empirical adoption and depth of psychometric evaluation: many tools that are widely used may nonetheless show limited psychometric coverage, reflecting a predominance of passive use over active psychometric validation. To examine this relationship more systematically, tools were cross-tabulated by licensing model, the primary determinant of accessibility in our dataset. Table 1 summarizes the 88 validated digital neuropsychological tools according to licensing models, reporting the number of tools, mean PCI, PCI distribution across three bands, and median and maximum number of citations.

Figure 2.

Most-used digital cognitive assessment tools by number of citations, number of validation studies, and psychometric coverage index. Tools appearing in the top 25% for number of citations across included studies (cutoff ≥ 2 uses) are displayed, sorted by number of citations. The left panel shows overall number of citations across all included studies. The center panel shows the number of dedicated validation studies identified for each tool. The right panel shows the psychometric coverage index, computed exclusively from validation studies and defined as the proportion of psychometric properties assessed across validation studies relative to the total number of psychometric properties considered.

Table 1.

Psychometric coverage and number of citations of digital neuropsychological tools by licensing model

Table 1. Long description

A table summarizing 88 validated digital neuropsychological tools according to licensing models. The table has 5 rows and 8 columns. Column headers are Licensing model, N, Mean PCI, PCI < 33% N (%), PCI 33–66% N (%), PCI > 66% N (%), Median number of citations, and Max number of citations. Row labels are Pay to use, Free to use/Open access, NS, Other, and Total. Row 1: Pay to use, 17, 0.33, 7 (41%), 9 (53%), 1 (6%), 2, 27. Row 2: Free to use/Open access, 8, 0.33, 3 (38%), 4 (50%), 1 (12%), 1, 5. Row 3: NS, 57, 0.25, 40 (70%), 17 (30%), 0 (0%), 1, 7. Row 4: Other, 6, 0.30, 3 (50%), 3 (50%), 0 (0%), 1, 5. Row 5: Total, 88, 0.28, 53 (60%), 33 (38%), 2 (2%), 1, 27.

Note: N = 88 tools with at least one validation study (total validation studies N = 122). PCI = psychometric coverage index (proportion of 11 psychometric properties reported in validation studies only). Number of Citations = number of independent studies employing each tool among the 236 included. “Other” includes free demo, academic-use-only, limited free version, and not-public licensing. NS = not specified in any associated publication.

Table 2.

Frequency and percentage of psychometric properties examined across validation studies

Table 2. Long description

A table with two columns and twelve rows. The first column lists psychometric properties, and the second column shows their frequency in percentage. Row 1: Concurrent validity, 55 (45.1 percent). Row 2: Discriminant validity, 41 (33.6 percent). Row 3: Convergent validity, 41 (33.6 percent). Row 4: Factorial validity/construct validity, 40 (32.8 percent). Row 5: Test-retest reliability, 37 (30.3 percent). Row 6: Criterion validity, 33 (27 percent). Row 7: Internal consistency/reliability, 19 (15.6 percent). Row 8: Practice effect, 17 (13.9 percent). Row 9: Ecological validity, 13 (10.7 percent). Row 10: Inter-setting reliability, 6 (4.9 percent). Row 11: Intra-rater reliability, 6 (4.9 percent).

Note: N = 122 validation studies. Percentages reflect the proportion of studies examining each psychometric property; values do not sum to 100% as individual studies could examine multiple properties.

The mismatch between number of citations and psychometric coverage hypothesized above becomes clearer when tools are examined by their licensing model in more detail. The highest number of citations was concentrated in a small cluster of well-established platforms (e.g., NeuroTrax, Cogstate, CDR). These platforms benefit from decades of refinement, broad normative datasets, and relatively robust psychometric properties, offering both remote and unsupervised administration (e.g., Smith et al., Reference Smith, Need, Cirulli, Chiba-Falek and Attix2013; Wärn et al., Reference Wärn, Andersson and Berginström2025; Wragg et al., Reference Wragg, Skirrow, Dente, Cotter, Annas, Lowther, Backx, Barnett, Cree, Kroll and Cormack2024). The psychometric evidence supporting these tools includes studies of reliability, validity, and normative standardization across multiple populations (CogState: Hammers et al., Reference Hammers, Spurgeon, Ryan, Persad, Barbas, Heidebrink, Darby and Giordani2012; Kataja et al., Reference Kataja, Karlsson, Tolvanen, Parsons, Schembri, Kiiski-Mäki and Karlsson2017; Maruff et al., Reference Maruff, Thomas, Cysique, Brew, Collie, Snyder and Pietrzak2009; CDR: Saxby et al., Reference Saxby, Harrington, McKeith, Wesnes and Ford2003; Wesnes et al., Reference Wesnes, Edgar, Dean and Wroe2009; CANTAB: Backx et al., Reference Backx, Skirrow, Dente, Barnett and Cormack2020; NeuroTrax: Doniger et al., Reference Doniger, Zucker, Schweiger, Dwolatzky, Chertkow, Crystal and Simon2005; Golan et al., Reference Golan, Wilken, Doniger, Fratto, Kane, Srinivasan, Zarif, Bumstead, Buhse, Fafard, Topalli and Gudesblatt2019). Their structured administration and well-defined scoring procedures make them particularly suitable for clinical diagnostics, longitudinal monitoring, and multicenter research, where comparability and methodological rigor are essential (Parsons & Duffield, Reference Parsons and Duffield2020). However, such tools are often proprietary and costly, with closed algorithms and restricted access to raw data and scoring procedures. Second, and more critically, despite their wide diffusion not all the tools within the category can be considered robust from a psychometric point of view: the validity of some widely used, well-established tools is sometimes assumed rather than empirically tested. For instance, CANTAB, despite ranking fourth in number of citations, shows a PCI of only 0.27 (3 out of 11 properties evaluated). NIH-TB-CB has been applied across different populations including aging cohorts, COVID-19 long haulers, and metabolic disorders (Christ et al., Reference Christ, Clocksin, Zalik, Goodlett, Sacharow and Abbene2023; Graham et al., Reference Graham, Clark, Orban, Lim, Szymanski, Taylor, DiBiase, Jia, Balabanov, Ho, Batra, Liotta and Koralnik2021; O’Shea et al., Reference O’Shea, Cohen, Porges, Nissim and Woods2016; Yassi et al., Reference Yassi, Pase, Buckley, Rosenich, Watson, Maruff and Lim2022), yet it only achieves a PCI of 0.09, because only its construct validity has been formally evaluated, in a single study (Ott et al., Reference Ott, Schantell, Willett, Johnson, Eastman, Okelberry, Wilson, Taylor and May2022). Conversely, Cognivue®, despite its comparatively limited frequency of use (2 citations), achieved a considerably higher PCI (0.55, 6 out of 11 properties evaluated).

Open-access and freely available tools, which represent the smallest (N = 6) and empirically least-developed portion of the identified instruments showed limited psychometric coverage, a pattern likely reflecting the small number of independent validation studies available for each instrument. As a notable exception within the broader category of freely available tools Cognitron achieved a high PCI (0.73) alongside moderate number of citations (N = 5). Its psychometric properties have been evaluated across diverse clinical and community populations, including COVID-19 (Hampshire et al., Reference Hampshire, Chatfield, MPhil, Jolly, Trender, Hellyer, Giovane, Newcombe, Outtrim, Warne, Bhatti, Pointon, Elmer, Sithole, Bradley, Kingston, Sawcer, Bullmore and Rowe2022, Reference Hampshire, Azor, Atchison, Trender, Hellyer, Giunchiglia, Husain, Cooke, Cooper, Lound, Donnelly, Chadeau-Hyam, Ward and Elliott2024), traumatic brain injury (Del Giovane et al., Reference Del Giovane, Trender, Bălăeţ, Mallas, Jolly, Bourke, Zimmermann, Graham, Lai, Losty, Oiarbide, Hellyer, Faiman, Daniels, Batey, Harrison, Giunchiglia, Kolanko, David and Hampshire2023), Parkinson’s disease (Bălăeț et al., Reference Bălăeţ, Alhajraf, Zerenner, Welch, Razzaque, Lo, Giunchiglia, Trender, Lerede, Hellyer, Manohar, Malhotra, Hu and Hampshire2024), and autoimmune limbic encephalitis (Shibata et al., Reference Shibata, Attaallah, Tai, Trender, Hellyer, Hampshire, Irani, Manohar and Husain2024). Cognitron thus represented an exception in which high psychometric coverage co-occurred with free accessibility, a combination otherwise uncommon among these tools.

Tools relying on emerging and immersive technologies such as VR platforms, serious games, and eye-tracking hybrid systems were often characterized by a lack of licensing information and psychometric validation that remained the most limited among the included tools. Immersive environments enable researchers to assess attention in settings that more closely resemble the multitasking demands of everyday life. They also generate multimodal data, including gaze patterns and real-time behavioral adaptations. Together, these measures can capture aspects of performance that are often missed by conventional, computer-based, tasks (Kirkham et al., Reference Kirkham, Kooijman, Albertella, Myles, Yücel and Rotaru2024). However, we observed that available evidence derived from pilot studies conducted in small, often clinical, samples (VRST, Armstrong et al., Reference Armstrong, Reger, Edwards, Rizzo, Courtney and Parsons2013; JEF VR, Jansari et al., Reference Jansari, Devlin, Agnew, Akesson, Murphy and Leadbetter2014; VR-PASAT, Parsons & Courtney, Reference Parsons and Courtney2014; WCST + Eye-tracking, Leitner et al., Reference Leitner, Coady, Miller and Libben2023; CBAAD, Malegiannaki et al., Reference Malegiannaki, Garefalaki, Pellas and Kosmidis2024; VirtualCookingTask, Cañada et al., Reference Cañada, Torres, Andreu-Martinez, Cristancho, Chicchi Giglioli, Garcia-Blanco, Adriasola, Navalón, Sierra and Alcañiz2024; VR traditional tests, Hadjiaros et al., Reference Hadjiaros, Shimi, Neokleous, Pattichis and Avraamides2025). Licensing information was not specified for seven of the eight tools in this group, with only one tool explicitly released under an open-access model (VIRTRAEL, Rute-Perez et al., Reference Rute-Perez, Rodriguez-Dominguez, Saez-Sanz, Perez-Garcia and Caracuel2025).

Cognitive domains measured

In our sample of 107 unique cognitive assessment tools, most instruments assessed attention and EF as part of broader cognitive batteries rather than in isolation. Specifically, 67 tools measured both attention and EF together with additional domains such as memory, language, arithmetic, or orientation. Nine tools combined EF but not attention with other cognitive domains (C-TOC, Jacova et al., Reference Jacova, McGrenere, Lee, Wang, Le Huray, Corenblith, Brehmer, Tang, Hayden, Beattie and Hsiung2015; CANS-MCI, Memória et al., Reference Memória, Yassuda, Nakano and Forlenza2014; CCAB, Woods et al., Reference Woods, Pebler, Johnson, Herron, Hall, Blank, Geraci, Williams, Chok, Lwi, Curran, Schendel, Spinelli and Baldo2024; CFT, Trustram Eve & de Jager, Reference Trustram Eve and de Jager2014; CoCoSc, Wong et al., Reference Wong, Fong, Mok, Leung and Tong2017; EMCA, Campbell et al., Reference Campbell, Paolillo, Heaton, Tang, Depp, Granholm, Heaton, Swendsen, Moore and Moore2020; GrayMatters®, Brinkman et al., Reference Brinkman, Reese, Norsworthy, Dellaria, Kinkade, Benge, Brown, Ratka and Simpkins2014; ICAT, Zarp et al., Reference Zarp, Bruun, Christiansen, Krogh, Kuchinke, Bernsen, Jespersen, Bardram, Kessing and Miskowiak2025; MOST, Clionsky & Clionsky, Reference Clionsky and Clionsky2014), while 6 combined attention but not EF with other domains (BOCA, Vyshedskiy et al., Reference Vyshedskiy, Netson, Fridberg, Jagadeesan, Arnold, Barnett, Gondalia, Maslova, de Torres, Ostrovsky, Durakovic, Savchenko, McNett, Kogan, Piryatinsky and Gold2022; CCTE, Kurzbuch et al., Reference Kurzbuch, Pauli, Gaál, Kerling, Kasper, Stefan, Hamer and Graf2013; FePsy, Odiase et al., Reference Odiase, Ogunrin and Ogunniyi2007; HK-VMT, Fung et al., Reference Fung, Lee, Ma and Lam2024; Headminder, Cheung et al., Reference Cheung, Ng, Shwe, Ho, Foo, Cham, Lee, Fan, Tan, Yong, Madhukumar, Loo, Ang, Wong, Chay, Ooi, Dent, Yap, Ng and Chan2015; Neurit. Space, Massetti et al., Reference Massetti, Albini, Casati, Toneatto, Terruzzi, Etzi, Tesio, Gallace and Vallar2023). In contrast, 18 tools measured attention and EF without including these broader domains (e.g., PASAT, Aliyari et al., Reference Aliyari, Golabi, Sahraei, Daliri, Minaei-Bidgoli, Tadayyoni and Kazemi2022; TMT, Park & Schott, Reference Park and Schott2022; THINC-it, Shibaoka et al., Reference Shibaoka, Masuda, Iwasawa, Ikezawa, Eguchi and Nakagome2023), 3 assessed EF only (JEF VR, Jansari et al., Reference Jansari, Devlin, Agnew, Akesson, Murphy and Leadbetter2014; Koski et al., Reference Koski, Brouillette, Lalonde, Hello, Wong, Tsuchida and Fellows2011; Leitner et al., Reference Leitner, Coady, Miller and Libben2023), and 2 assessed attention only (CADTS, Walia et al., Reference Walia, Kumar, Khosla and Grover2023; CBAAD, Malegiannaki et al., Reference Malegiannaki, Garefalaki, Pellas and Kosmidis2024). Notably, even among tools with a narrower cognitive scope, the majority explicitly incorporated working memory and/or processing speed alongside their primary attentional or EF measures. Overall, these findings indicate that even among tools specifically selected for their focus on attention and EF, these functions are normally assessed as part of broader multidimensional batteries, with relatively few tools targeting specifically attention or EF.

Table 3 provides an overview of the key characteristics of all digital neuropsychological tools described in the review, including device requirements, administration modality, licensing costs, and type of behavioral measures collected. Additionally, for each tool the table reports the administration languages and the number of studies referring to each tool alongside the year of first and last publication (see OSF for details: osf.io/guz9t).

Table 3.

Main characteristics of the digital neuropsychological tools included in the review

Table 3. Long description

The table compares digital neuropsychological tools based on first publication year, last publication year, languages, compatible devices, administration, licensing costs, and behavioral measures. It includes 100 rows and 11 columns. Column headers are Tool name, First pub. year, Last pub. year, Languages, Compatible devices, Administration, Licensing costs, Behavioral measures, N studies, and Notes. Each row lists a different tool with corresponding details. For example, the first row describes the ABSA System, first published in 2014, compatible with computers, administered remotely, with free-to-use licensing, and measuring accuracy.

Note: The table reports all digital tools identified in the systematic review for the domains of attention and executive functions, including year of first and last publication, available languages, required devices, administration modality, licensing models, behavioral measures, and number of studies per tool. NS = not specified; N studies = number of studies using that tool. The entries “computerized tasks (a),” “computerized tasks (b),” and “computerized tasks (c)” refer to digital tools that were not assigned a specific name in the original studies.

Regarding administration language, 45 tools (42.1%) were reported exclusively in English, while 35 (32.7%) were reported in a single non-English language (spanning 19 different languages, most frequently German, Chinese, Italian, Spanish, and Japanese). Only 20 tools (18.7%) were available in more than one language, and multi-language availability was concentrated in a small subset of established commercial batteries (e.g., CANTAB, CDR, NeuroTrax). Notably, of the 29 distinct languages identified overall, 14 (48.3%) were represented by only a single tool (Figure 3).

Figure 3.

Number of digital tools by administration language. Bar chart showing the number of identified tools available in each administration language, based on the languages reported across the included validation and usage studies. Only languages available in at least two tools are displayed for readability (out of 107 total tools).