Research Significance
Research Question(s) or Topic(s): The accurate and efficient measurement of executive functioning is important for a range of clinical conditions including neurodegenerative diseases such as Alzheimer’s disease and frontotemporal dementia. Tablet-based assessments provide many advantages over traditional and computer-based testing. A highly utilized tablet-based cognitive battery, TabCAT, includes measures of executive functioning, but lacks a single outcome measure. Here we develop and validate a single tablet-based summary measure of executive functioning. Main Findings: Modern statistical procedures were used to create an executive functioning composite score. We show that this score is reliable and associated as expected with other cognitive tasks. We also demonstrate that participants with greater clinical severity perform worse on this measure than individuals with less impairment and show faster rates of longitudinal decline. Study Contributions: The TabCAT-EXAMINER will aid assessment of executive functioning in observational research and clinical trials.
Introduction
>Executive functioning is an umbrella term for a multidimensional set of cognitive processes that influence goal-directed behavior (Karr et al., Reference Karr, Areshenkoff, Rast, Hofer, Iverson and Garcia-Barrera2018; Rabinovici et al., Reference Rabinovici, Stephens and Possin2015). It is supported by a dispersed but connected set of brain regions (Alvarez & Emory, Reference Alvarez and Emory2006; Bettcher et al., Reference Bettcher, Mungas, Patel, Elofson, Dutt, Wynn, Watson, Stephens, Walsh and Kramer2016; Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014; Niendam et al., Reference Niendam, Laird, Ray, Dean, Glahn and Carter2012; Reineberg & Banich, Reference Reineberg and Banich2016) and is impacted by neurological conditions including epilepsy (Arrotta et al., Reference Arrotta, Reyes, Kaestner, McDonald, Hermann, Barr, Sarmey, Sundar, Kondylis, Najm, Bingaman and Busch2022; Longo et al., Reference Longo, Kerr and Smith2013), multiple sclerosis (Foong et al., Reference Foong, Rozewicz, Quaghebeur, Davie, Kartsounis, Thompson, Miller and Ron1997; Roman & Arnett, Reference Roman and Arnett2016), traumatic brain injury (Azouvi et al., Reference Azouvi, Arnould, Dromer and Vallat-Azouvi2017; Broadway et al., Reference Broadway, Rieger, Campbell, Quinn, Mayer, Yeo, Wilson, Gill, Fratzke and Cavanagh2019), and neurodegenerative diseases such as Alzheimer’s disease and frontotemporal dementia (FTD) (Guarino et al., Reference Guarino, Favieri, Boncompagni, Agostini, Cantone and Casagrande2019; Rabinovici et al., Reference Rabinovici, Stephens and Possin2015; Staffaroni, Bajorek, et al., Reference Staffaroni, Bajorek, Casaletto, Cobigo, Goh, Wolf, Heuer, Elahi, Ljubenkov, Dever, Kornak, Appleby, Bove, Bordelon, Brannelly, Brushaber, Caso, Coppola and Dheel2020; Stopford et al., Reference Stopford, Thompson, Neary, Richardson and Snowden2012). The parcellation of executive functioning into subdomains varies by study, but cognitive neuroscience and statistical approaches commonly identify subdomains such as inhibition, set-shifting, fluency/generativity, and working memory (Kramer, Reference Kramer2014; Miyake et al., Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000). The assessment of executive functioning is complicated by the breadth of cognitive tests used to measure these domains, many of which focus on a specific subcomponent (e.g., inhibition) (Faria et al., Reference Faria, Alves and Charchat-Fichman2015). As a result, clinics and research studies differ in what aspect of executive functioning they are describing, which may attenuate reproducibility and hamper cross-study comparisons. To address these challenges, the National Institute of Health (NIH) funded the Executive Abilities: Measures and Instruments for Neurobehavioral Evaluation and Research (EXAMINER) project to develop a laptop-based battery of executive functioning tests that was informed by cognitive neuroscience, and could enable measurement of executive functioning in clinical trials and observational research (Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014; Kramer, Reference Kramer2014). The project aimed to create a modular, flexible battery that captured the subcomponents of executive functioning. NIH-EXAMINER tasks of working memory, verbal fluency, and cognitive control were modeled using confirmatory factor analysis and item response theory (IRT) to create an Executive Composite score that summarized executive functioning into a single pragmatic and psychometrically robust outcome (Bott et al., Reference Bott, Johnson, Schuff, Galifianakis, Subas, Pollock, Pressman, Kramer and Possin2014; Possin et al., Reference Possin, LaMarre, Wood, Mungas and Kramer2014).
Over the last decade, the NIH-EXAMINER and the Executive Composite score have been implemented and validated in myriad studies. For example, scores on the NIH-EXAMINER are strongly associated with real-world executive behavior and with volumes of frontal, parietal, and subcortical brain regions known to support executive functioning (Possin et al., Reference Possin, LaMarre, Wood, Mungas and Kramer2014). The NIH-EXAMINER has also shown utility for detecting and monitoring symptoms of neurological conditions including traumatic brain injury (Broadway et al., Reference Broadway, Rieger, Campbell, Quinn, Mayer, Yeo, Wilson, Gill, Fratzke and Cavanagh2019), attention deficit disorder (Schreiber et al., Reference Schreiber, Possin, Girard and Rey-Casserly2014), and pediatric sickle cell disease (Schatz et al., Reference Schatz, Stancil, Katz and Sanchez2014). The NIH-EXAMINER is sensitive to cognitive changes across a range of neurodegenerative diseases (You et al., Reference You, Geschwind, Sha, Apple, Satris, Wood, Johnson, Gooblar, Feuerstein, Finkbeiner, Kang, Miller, Hess, Kramer and Possin2014), and worse performance has been associated with greater disease severity, suggesting it measures clinically meaningful symptoms (Possin et al., Reference Possin, LaMarre, Wood, Mungas and Kramer2014; Staffaroni, Bajorek, et al., Reference Staffaroni, Bajorek, Casaletto, Cobigo, Goh, Wolf, Heuer, Elahi, Ljubenkov, Dever, Kornak, Appleby, Bove, Bordelon, Brannelly, Brushaber, Caso, Coppola and Dheel2020). Moreover, the Executive Composite score is a sensitive marker for early disease manifestations, as demonstrated by its ability to detect cross-sectional and longitudinal changes in presymptomatic FTD mutation carriers (Staffaroni, Bajorek, et al., Reference Staffaroni, Bajorek, Casaletto, Cobigo, Goh, Wolf, Heuer, Elahi, Ljubenkov, Dever, Kornak, Appleby, Bove, Bordelon, Brannelly, Brushaber, Caso, Coppola and Dheel2020). The NIH-EXAMINER has been used to measure treatment effects on cognitive performance in pharmacological treatment trials (Assogna et al., Reference Assogna, Sprugnoli, Press, Dickerson, Macone, Bonnì, Borghi, Connor, Hoffman, Grover, Wong, Shen, Martorana, O’Reilly, Ruffini, El Fakhri, Koch and Santarnecchi2021; Leite et al., Reference Leite, Gonçalves and Carvalho2022; Miller et al., Reference Miller, Banks, Léger and Cummings2014; Yu et al., Reference Yu, Lin, Salisbury, Shah, Chow, Vock, Nelson, Porsteinsson and Jack2018), including those targeting AD and other causes of mild cognitive impairment and mild dementia (Hajjar et al., Reference Hajjar, Okafor, McDaniel, Obideen, Dee, Shokouhi, Quyyumi, Levey and Goldstein2020; Vossel et al., Reference Vossel, Ranasinghe, Beagle, La, Ah Pook, Castro, Mizuiri, Honma, Venkateswaran, Koestler, Zhang, Mucke, Howell, Possin, Kramer, Boxer, Miller, Nagarajan and Kirsch2021), as well as non-pharmacological trials of mindfulness, cognitive training, and transcranial direct current stimulation (Assogna et al., Reference Assogna, Sprugnoli, Press, Dickerson, Macone, Bonnì, Borghi, Connor, Hoffman, Grover, Wong, Shen, Martorana, O’Reilly, Ruffini, El Fakhri, Koch and Santarnecchi2021; Clark et al., Reference Clark, Chatterjee, Skinner, Lysne, Sumonthee, Wu, Cohen, Rose and Woods2021; Leite et al., Reference Leite, Gonçalves and Carvalho2022; Yu et al., Reference Yu, Lin, Salisbury, Shah, Chow, Vock, Nelson, Porsteinsson and Jack2018).
Since the development of the laptop-based NIH-EXAMINER, tablets have become a frequently used alternative to paper-and-pencil testing or laptops for cognitive assessments. Tablets offer similar benefits as computer-based testing (e.g., accurate logging of reaction time, automated scoring), but offer additional advantages with respect to portability, affordability, and user friendliness (Canini et al., Reference Canini, Battista, Della Rosa, Catricalà, Salvatore, Gilardi and Castiglioni2014; Vaportzis et al., Reference Vaportzis, Giatsi Clausen and Gow2017; Wilson et al., Reference Wilson, Byrne, Rodgers and Maden2022). Staffaroni et al., (Reference Staffaroni, Bajorek, Casaletto, Cobigo, Goh, Wolf, Heuer, Elahi, Ljubenkov, Dever, Kornak, Appleby, Bove, Bordelon, Brannelly, Brushaber, Caso, Coppola and Dheel2020) and Wilson et al., (Reference Wilson, Byrne, Rodgers and Maden2022) provide brief reviews on the benefits of tablet-based assessments over computer-based assessments. One such tablet-based testing platform is the UCSF Tablet-based Cognitive Assessment Tool (TabCAT; https://tabcathealth.com/). A subset of the validated NIH-EXAMINER tasks was programmed into the TabCAT platform (TabCAT-EXAMINER). Based on user feedback and data analyses of original NIH-EXAMINER tasks, slight modifications were made to some tasks (e.g., reducing the number of practice trials), and a new task, Running Dots, was added in place of the N-Back task (see Methods: TabCAT-EXAMINER). Prior studies have demonstrated that changing the administration platform from a computer to tablet can impact scores. For example, the NIH Toolbox is a set of cognitive tasks that was adapted from computer to tablet, with subtle alterations to the tasks. These alterations led to critical differences in performance for the majority of tasks, necessitating the re-creation of validation and normative data (Brearly et al., Reference Brearly, Rowland, Martindale, Shura, Curry and Taber2018; Germine et al., Reference Germine, Reinecke and Chaytor2019). We therefore set out to recreate and validate an EXAMINER executive functioning composite for tablets.
Here we describe the development and validation of the TabCAT-EXAMINER score, a measure of executive functioning. The precision of a composite score across a range of clinical presentations is improved when that score is developed in a heterogenous testing cohort (Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014). As such, model development and initial validation analyses were conducted in a large, diagnostically heterogenous cohort of individuals with AD and related disorders, including FTD. Given the high frequency of executive dysfunction in FTD and the reported usefulness of the original NIH-EXAMINER in monitoring FTD (Miller et al., Reference Miller, Banks, Léger and Cummings2014), we selected participants from the ARTFL-LEFFTDS Longitudinal Frontotemporal Lobar Degeneration (ALLFTD) consortium with longitudinal TabCAT-EXAMINER data as a validation cohort. In addition to replicating findings observed in the development cohort, we conducted analyses in this validation cohort to further understand longitudinal performances, including test-retest reliability. We hypothesized that a single, general TabCAT-EXAMINER score will 1) fit the data well; 2) demonstrate convergent and divergent validity with other independent cognitive tasks; 3) be sensitive to a variety of neurodegenerative syndromes and associated with worsening disease severity; and 4) be sensitive to subtle cognitive changes in presymptomatic carriers of autosomal dominant FTD mutationsReference Staffaroni, Quintana, Wendelberger, Heuer, Russell, Cobigo, Wolf, Goh, Petrucelli, Gendron, Heller, Clark, Taylor, Wise, Ong, Forsberg, Brushaber, Rojas and VandeVrede.
Materials and Methods
Participants
All participants completed at least one of the five tablet-based TabCAT-EXAMINER measures. Some participants were previously exposed to the verbal fluency measures which are administered as part of the standard Uniform Data Set (UDS) Version 3.0 neuropsychological battery, but baseline data represent the first time that participants completed one of the TabCAT-EXAMINER measures. Inclusion criteria required English as the primary language to remove variance associated with language that may impact performance on these tasks. An associated study is underway to evaluate the TabCAT-EXAMINER in a Spanish speaking cohort (see future directions). Written informed consent was obtained for all participants or their legal representative. Studies were approved by the Institutional Review Boards at University of California, San Francisco (UCSF) and Johns Hopkins University (Trial Innovation Network) in accordance with university institutional guidelines and the Helsinki Declaration.
The total sample comprised 2135 participants. A subset of these participants were used as the developmental sample (n = 1793; Table 1), which included individuals recruited through UCSF (n = 509) and the ALLFTD research consortium (n = 1284), a multi-center study of FTD (Boeve et al., Reference Boeve, Boxer, Rosen, Forsberg, Heuer, Brushaber, Appleby, Biernacka, Bordelon, Botha, Brannelly, Dickerson, Dickson, Kimiko, Faber, Fagan, Fields, Fishman, Foroud and Wszolek2020). Similar to the original NIH-EXAMINER development sample (Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014; Possin et al., Reference Possin, LaMarre, Wood, Mungas and Kramer2014), this development subsample was symptomatically heterogeneous and syndromic diagnoses were made in consensus conferences using published criteria (Barker et al., Reference Barker, Gottesman, Manoochehri, Chapman, Appleby, Brushaber, Devick, Dickerson, Domoto-Reilly, Fields, Forsberg, Galasko, Ghoshal, Goldman, Graff-Radford, Grossman, Heuer, Hsiung and Knopman2022; Brooks et al., Reference Brooks, Miller, Swash and Munsat2000; Dubois et al., Reference Dubois, Feldman, Jacova, DeKosky, Barberger-Gateau, Cummings, Delacourte, Galasko, Gauthier, Jicha, Meguro, O’Brien, Pasquier, Robert, Rossor, Salloway, Stern, Visser and Scheltens2007; Gorno-Tempini et al., Reference Gorno-Tempini, Hillis, Weintraub, Kertesz, Mendez, Cappa, Ogar, Rohrer, Black, Boeve, Manes, Dronkers, Vandenberghe, Rascovsky, Patterson, Miller, Knopman, Hodges, Mesulam and Grossman2011; Jack Jr et al., Reference Jack, Bennett, Blennow, Carrillo, Dunn, Haeberlein, Holtzman, Jagust, Jessen, Karlawish, Liu, Molinuevo, Montine, Phelps, Rankin, Rowe, Scheltens, Siemers, Snyder and Silverberg2018; Litvan et al., Reference Litvan, MacIntyre, Goetz, Wenning, Jellinger, Verny, Bartko, Jankovic, McKee, Brandel, Chaudhuri, Lai, D’Olhaberriague, Pearce and Agid1998; McKeith et al., Reference McKeith, Boeve, Dickson, Halliday, Taylor, Weintraub, Aarsland, Galvin, Attems, Ballard, Bayston, Beach, Blanc, Bohnen, Bonanni, Bras, Brundin, Burn, Chen-Plotkin and Kosaka2017; Neary et al., Reference Neary, Snowden, Gustafson, Passant, Stuss, Black, Freedman, Kertesz, Robert, Albert, Boone, Miller, Cummings and Benson1998; Petersen et al., Reference Petersen, Aisen, Beckett, Donohue, Gamst, Harvey, Jack, Jagust, Shaw, Toga, Trojanowski and Weiner2010; Petersen et al., Reference Petersen, Aisen, Boeve, Geda, Ivnik, Knopman, Mielke, Pankratz, Roberts, Rocca, Weigand, Weiner, Wiste and Jack2013; Petersen et al., Reference Petersen, Caracciolo, Brayne, Gauthier, Jelic and Fratiglioni2014; Postuma et al., Reference Postuma, Berg, Stern, Poewe, Olanow, Oertel, Obeso, Marek, Litvan, Lang, Halliday, Goetz, Gasser, Dubois, Chan, Bloem, Adler and Deuschl2015; Rascovsky et al., Reference Rascovsky, Hodges, Knopman, Mendez, Kramer, Neuhaus, van Swieten, Seelaar, Dopper, Onyike, Hillis, Josephs, Boeve, Kertesz, Seeley, Rankin, Johnson, Gorno-Tempini, Rosen and Miller2011; Sachdev et al., Reference Sachdev, Kalaria, O’Brien, Skoog, Alladi, Black, Blacker, Blazer, Chen, Chui, Ganguli, Jellinger, Jeste, Pasquier, Paulsen, Prins, Rockwood, Roman and Scheltens2014). The sample included participants from UCSF and ALLFTD diagnosed with/as cognitively unimpaired (CU; n = 786), mild cognitive impairment (MCI; n = 237), Alzheimer’s dementia (AD; n = 115), behavioral variant FTD (bvFTD; n = 167), corticobasal syndrome (CBS; n = 29), logopenic variant primary progressive aphasia (lvPPA; n = 28), nonfluent variant PPA (nvPPA; n = 44), progressive supranuclear palsy (PSP; n = 37), and semantic variant PPA (svPPA; n = 49). There were 112 participants with non-neurodegenerative diagnoses (e.g., primary mood etiologies) and/or small subsamples (e.g., amyotrophic lateral sclerosis: n < 10) labeled as “other.” There were 184 participants who were missing diagnosis. The MCI group included those with mild cognitive deficits and/or mild behavioral changes irrespective of suspected etiology (Barker et al., Reference Barker, Gottesman, Manoochehri, Chapman, Appleby, Brushaber, Devick, Dickerson, Domoto-Reilly, Fields, Forsberg, Galasko, Ghoshal, Goldman, Graff-Radford, Grossman, Heuer, Hsiung and Knopman2022).
Sample description and TabCAT-EXAMINER scores by CDR®+NACC-FTLD Global Score subgroups
Note: The “Comparison” column indicates significance of comparisons (T-test or Chi2 test) between CDR®+NACC-FTLD Global score groups. Statistical comparisons involving the CDR®+NACC-FTLD Global Score = 3 group were not computed due to small sample size. Means, (standard deviations), and, when applicable, sample size [n], are presented. There were 368 participants missing CDR®+NACC-FTLD Global Scores.
CU = cognitively unimpaired, MCI = mild cognitive impairment, AD = Alzheimer’s dementia, bvFTD = behavioral variant Frontotemporal dementia, CBS = corticobasal syndrome, lvPPA = logopenic primary progressive aphasia, nvPPA = nonfluent variant PPA, PSP = progressive supranuclear palsy, svPPA = semantic variant PPA.
Other: participants with non-neurodegenerative diagnoses (e.g., primary mood etiologies) and/or small subsamples (e.g., amyotrophic lateral sclerosis: n < 10).
A different subset of ALLFTD participants (n = 342) not included in the development cohort comprised the validation cohort. This subsample had longitudinal TabCAT-EXAMINER data, completed 6–24 months after their baseline (mean follow-up length: 368 days; SD = 64). This sample included those diagnosed as: CU (n = 175), MCI (n = 26), AD (n = 10), bvFTD (n = 46), CBS (n = 5), lvPPA (n = 1), nvPPA (n = 15), PSP (n = 8), and svPPA (n = 14). There were 127 participants who fell into the categories labeled as “other” in the development sample (e.g., primary etiology mood, n = 3; Parkinson’s disease, n = 1). There were 85 participants who were missing diagnosis.
A description of the sample with genetic testing is presented in Supplemental Tables 1 and 2. DNA samples were screened using targeted sequencing of a custom panel of genes previously implicated in neurodegenerative diseases, including GRN and MAPT. Hexanucleotide repeat expansions in C9orf72 were detected using both fluorescent and repeat-primed PCR (Ramos et al., Reference Ramos, Dokuru, Van Berlo, Wojta, Wang and Huang2020). The developmental cohort included 79 participants who carried pathogenic mutations (MAPT, n = 23; C9orf72, n = 43; and GRN, n = 13) known to cause familial frontotemporal lobar degeneration (f-FTLD). There were also 132 non-carriers from families with an identified pathogenic f-FTLD mutation (i.e., family controls). The validation cohort included 80 participants who carried f-FTLD genetic mutations (MAPT, n = 24; C9orf72, n = 44; and GRN, n = 12). There were also 101 non-carriers from families with an identified pathogenic f-FTLD mutation in the validation cohort.
TabCAT-EXAMINER
The TabCAT-EXAMINER comprises nine executive functioning tasks, including five tablet-based measures and four measures of verbal fluency. Tablet-based measures included tasks of inhibition (Flanker), set-shifting (Set Shifting), processing speed/working memory (Match), and working memory (Dot Counting, Running Dots) (Mullady et al., Reference Mullady, Castellanos, Lopez, Aguirre, Weeks, King, Valle, Goode, Tsoy, Possin, Miller, Kushel and Lanata2022; Possin et al., Reference Possin, Moskowitz, Erlhoff, Rogers, Johnson, Steele, Higgins, Stiver, Alioto, Farias, Miller and Rankin2018; Weintraub et al., Reference Weintraub, Salmon, Mercaldo, Ferris, Graff-Radford, Chui, Cummings, DeCarli, Foster, Galasko, Peskind, Dietrich, Beekly, Kukull and Morris2009). These tasks are also influenced by processing speed. Running Dots replaced the N-back task in the NIH-EXAMINER in response to user feedback. Match, a brief, well-validated assessment of executive functioning was also added to the battery to expand upon tasks modified from the NIH-EXAMINER. Other differences between the TabCAT-EXAMINER and the original include alterations due to the change in format (computer vs tablet) such as screen size, instructions, stimuli (i.e., size and shape) and user interface. Additional task-specific alterations are described below and in more detail at https://tabcathealth.com/. Images of example TabCAT tasks are presented in Supplemental Figure 1.
Flanker
The participant is presented with a display of arrows and asked to indicate which direction the central arrow is pointing. There are two conditions: in the congruent condition the non-target stimuli point in the same direction as the central arrow; in the non-congruent condition, the non-target stimuli point in the opposite direction of the central arrow. This task was modeled after the original NIH-EXAMINER Flanker task (Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014). Alterations included a reduction in the total number of practice and response trials, as well as slight changes in the minimum and maximum response times allowed. The primary outcome for this task is a score that combines speed and accuracy (range: 0–10); see (Mullady et al., Reference Mullady, Castellanos, Lopez, Aguirre, Weeks, King, Valle, Goode, Tsoy, Possin, Miller, Kushel and Lanata2022; Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014; Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer, Carlozzi, Slotkin, Blitz, Wallner-Allen, Fox, Beaumont, Mungas, Nowinski, Richler, Deocampo, Anderson, Manly, Borosh and Gershon2013) for details.
Set shifting
Participants are asked to match a stimulus presented at the top of the screen to one of two stimuli at the bottom of the screen. In the task-homogenous condition, participants are asked to match on shape or color. In the task-heterogenous condition, participants are asked to alternate between matching color and shape, pseudo-randomly. Similar to Flanker, this task was modeled after the original NIH-EXAMINER Set Shifting task and underwent minor alterations for the TabCAT system: reduction in practice trials, slight alterations to stimuli size and shape, and a slight change in the minimum and maximum allowed response times. Similar to Flanker, the outcome variable combines accuracy and reaction time across all task-heterogenous conditions (range: 0–10) (Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014).
Match
Participants are shown a set of numbers that are each matched to a simple shape in a key at the top of the screen. At the bottom of the screen the same set of shapes are presented. During each trial, a number appears at the center of the screen. Participants are asked to tap the corresponding shape at the bottom of a screen as quickly as possible for two minutes. Like the classic Digit Symbol Substitution Test paradigm on which it was based, Match is a highly sensitive but not specific measure of executive functioning (Possin et al., Reference Possin, Moskowitz, Erlhoff, Rogers, Johnson, Steele, Higgins, Stiver, Alioto, Farias, Miller and Rankin2018) that relies on processing speed, attention, visual-motor coordination, and working memory (Jaeger, Reference Jaeger2018). Although Digit Symbol Substitution Tests are affected by processing speed, performance on these tasks is also affected by working memory and other executive functioning abilities (Baudouin et al., Reference Baudouin, Clarys, Vanneste and Isingrini2009; Jaeger, Reference Jaeger2018; Piccinin & Rabbitt, Reference Piccinin and Rabbitt1999; Salthouse, Reference Salthouse1992).
Dot counting
Participants are asked to count the total number of blue circles on a display. The number of circles ranges from two to nine per display, with additional distractors (e.g., squares). The number of displays presented increases each trial, and at the end of each trial, the participant is asked to recall the total number of blue circles per display. There are six total trials. This task is modeled after the original NIH-EXAMINER Dot Counting task and assesses verbal working memory (Kramer et al., Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer2014). Compared to the NIH-EXAMINER, the TabCAT-EXAMINER version altered the number and size of stimuli at each trial. The outcome variable is the total number of correct responses across all trials (range: 0–27) (Mullady et al., Reference Mullady, Castellanos, Lopez, Aguirre, Weeks, King, Valle, Goode, Tsoy, Possin, Miller, Kushel and Lanata2022).
Running dots
Participants are shown a 16-square grid where a series of dots appear, one by one. They are asked to recall the last two to five locations where a dot appeared, after the dots disappear from the screen. The condition defining the target dots varies (e.g., the last two dots vs last four dots), as does the number of dots presented before the target dots. Participants are awarded a point for each dot they recall in the correct location and order; they are awarded half a point for recalling the correct location of a dot, but in the incorrect order. This task assessing spatial working memory was designed and implemented only in the TabCAT system.Reference Kramer, Mungas, Possin, Rankin, Boxer, Rosen, Bostrom, Sinha, Berhel and Widmeyer The primary outcome variable is the sum of total points across all trials (range: 0–42).
Verbal fluency
Measures included semantic fluency (Animals, Vegetables) and phonemic fluency (words [L], words [F]) collected as part of the UDS version 3.0 Neuropsychological Battery (Weintraub et al., Reference Weintraub, Besser, Dodge, Teylan, Ferris, Goldstein, Giordani, Kramer, Loewenstein, Marson, Mungas, Salmon, Welsh-Bohmer, Zhou, Shirk, Atri, Kukull, Phelps and Morris2018). Although these tasks are not administered on the tablet, the TabCAT system allows the user to input verbal fluency scores.
Other assessments
Additional cognitive tasks
In addition to the TabCAT-EXAMINER, participants completed a battery of standard paper-and-pencil neuropsychological measures within 180 days of tablet testing (supplemental methods). Measures included four UDS executive functioning tasks (Lezak et al., Reference Lezak, Howieson, Loring and Fischer2004; Possin et al., Reference Possin, Laluz, Alcantar, Miller and Kramer2011; Weintraub et al., Reference Weintraub, Salmon, Mercaldo, Ferris, Graff-Radford, Chui, Cummings, DeCarli, Foster, Galasko, Peskind, Dietrich, Beekly, Kukull and Morris2009). Participants also completed tasks of visuospatial ability (Benson Figure Copy), visual memory (Benson Figure Recall; Possin et al., Reference Possin, Laluz, Alcantar, Miller and Kramer2011), verbal memory (Craft Story Recall (Craft et al., Reference Craft, Newcomer, Kanne, Dagogo-Jack, Cryer, Sheline, Luby, Dagogo-Jack and Alderson1996) and California Verbal Learning Test-II, Short Form (Delis et al., Reference Delis, Kramer, Kaplan and Ober2000), and naming (Boston Naming test; Weintraub et al., Reference Weintraub, Besser, Dodge, Teylan, Ferris, Goldstein, Giordani, Kramer, Loewenstein, Marson, Mungas, Salmon, Welsh-Bohmer, Zhou, Shirk, Atri, Kukull, Phelps and Morris2018 or Multilingual Naming Test; Stasenko et al., Reference Stasenko, Jacobs, Salmon and Gollan2019). Participants scores on the Multilingual Naming Test were converted to Boston Naming Test scores based on a prior crosswalk analysis (Monsell et al., Reference Monsell, Dodge, Zhou, Bu, Besser, Mock, Hawes, Kukull and Weintraub2016). Finally, participants completed two additional measures on the TabCAT platform that are not part of the EXAMINER: Line Orientation, a measure of visuospatial perception, and Favorites, a measure of associative memory (Possin et al., Reference Possin, Moskowitz, Erlhoff, Rogers, Johnson, Steele, Higgins, Stiver, Alioto, Farias, Miller and Rankin2018).
Clinical Dementia Rating Scale Dementia Staging Instrument plus NACC Behavior and Language Domains (CDR®+NACC-FTLD)
The CDR®+NACC-FTLD (Knopman et al., Reference Knopman, Kramer, Boeve, Caselli, Graff-Radford, Mendez, Miller and Mercaldo2008) is a clinician-administered, semi-structured interview with participants and informants that adds language and behavior domains to the original CDR (Morris, Reference Morris1997) to account for core impairments of FTD-spectrum disorders. An algorithm (Miyagawa et al., Reference Miyagawa, Brushaber, Syrjanen, Kremers, Fields, Forsberg, Heuer, Knopman, Kornak, Boxer, Rosen, Boeve, Appleby, Bordelon, Bove, Brannelly, Caso, Coppola, Dever and Wszolek2020) was used to calculate a Global score that can be used to characterize disease severity as asymptomatic (0), prodromal (0.5), or symptomatic (1, 2, or 3). Within the symptomatic group a score of 1, 2, and 3, refers to mild, moderate, and severe dementia, respectively. A sum of the eight domain Box Scores (CDR®+NACC-FTLD sum of boxes) was also calculated (max score = 24). Greater Global and Box Scores indicate more functional impairment (Miyagawa et al., Reference Miyagawa, Brushaber, Syrjanen, Kremers, Fields, Forsberg, Heuer, Knopman, Kornak and Boxer2019, Reference Miyagawa, Brushaber, Syrjanen, Kremers, Fields, Forsberg, Heuer, Knopman, Kornak, Boxer, Rosen, Boeve, Appleby, Bordelon, Bove, Brannelly, Caso, Coppola, Dever and Wszolek2020). This measure was administered to participants with and without FTD.
Factor score modeling
Model building was conducted using the statistical software R (Team, Reference Team2021) and is fully described in the supplement. Briefly, exploratory factor analysis was used to determine the number of factors. A confirmatory model was then fit, followed by a multilevel IRT model using the MIRT package (Chalmers, Reference Chalmers2012).
IRT is a form of latent variable modeling, similar to confirmatory factor analysis. This method differs from traditional factor analysis in that it models item difficulty as well as the item’s ability to discriminate between participants with different abilities. As a result, items in IRT models are weighted in how effectively they assess the target ability at varying levels along that ability continuum (Mungas et al., Reference Mungas, Reed and Kramer2003). In the present analyses, we utilized a graded response model that allows for ordinal variables (Samejima, Reference Samejima2011) and recoded continuous variables into ordinal scores with up to 12 response categories and at least five observations in each category. This method retained the overall distribution of the raw, continuous data. The IRT model set the loadings within each fluency factor to be equal; this approach in MIRT is equivalent to specifying correlated measurement error in a CFA model. Acceptable model fit was determined via factor loadings (>.4 retained; Matsunaga, Reference Matsunaga2010; Williams et al., Reference Williams, Onsman and Brown2010), the Tucker Lewis Index (TLI; >0.95; Schumacker & Lomax, Reference Schumacker and Lomax2004), the Comparative Fit Index (CFI; <0.90; Savalei & Bentler, Reference Savalei and Bentler2006), and the Root Mean Square Error of Approximation (RMSEA; <.08; Byrne, Reference Byrne2001).
A series of analyses were conducted to determine the reliability and utility of the TabCAT-EXAMINER score when calculated with fewer components, mimicking scenarios in which not all tests were administered. We retained the thresholds and loadings from the full model (described above) in a series of sub-models that retained two to seven components; all possible 502 combinations were evaluated. A separate factor score was generated by each model. Models were then grouped by number of components (e.g., two-component model, three-component model) and by requiring a specific component be included (e.g., two-factor models that included Match). To evaluate the reliability and internal consistency of the TabCAT-EXAMINER score, each reduced factor score was correlated with the full, nine-component TabCAT-EXAMINER. To evaluate the utility of a factor score generated with fewer than nine components, each reduced factor score was also correlated with the CDR®+NACC-FTLD Sum of Boxes. Results were separated by the number of components included in the model and by specific components.
Convergent and divergent validity
To evaluate construct validity, we tested the association of the TabCAT-EXAMINER score with independent neuropsychological measures and scales of disease severity. First, a linear regression examined the association between the CDR®+NACC-FTLD Sum of Boxes and the TabCAT-EXAMINER score. Next, separate linear regressions were fit to assess the association between the TabCAT-EXAMINER score and age, education, sex, and neuropsychological measures while controlling for the CDR®+NACC-FTLD Sum of Boxes. Correlations did not adjust for demographic variables, as the goal was to understand the relationship between the TabCAT-EXAMINER score and paper-and-pencil measures rather than to evaluate the scores incremental utility above and beyond demographics. In a second set of analyses, linear regression was used to evaluate differences in TabCAT-EXAMINER scores across levels of clinical severity (CDR®+NACC-FTLD Global score), controlling for age, education, and sex. As we would expect executive functioning to differ between syndromic diagnostic groups and controls, we then demonstrated known group validity by assessing differences in TabCAT-EXAMINER scores across diagnostic syndromes through linear regressions, controlling for age, sex, and education. Based on prior studies (Possin et al., Reference Possin, LaMarre, Wood, Mungas and Kramer2014; Staffaroni et al., Reference Staffaroni, Asken, Casaletto, Fonseca, You, Rosen, Boxer, Elahi, Kornak, Mungas and Kramer2021), we hypothesized a similar degree of executive impairment across diagnostic groups. We next evaluated the ability of the TabCAT-EXAMINER to differentiate asymptomatic and prodromal f-FTLD mutation carriers (CDR®+NACC-FTLD Global score <1; n = 33) from family controls (CDR®+NACC-FTLD Global score = 0; n = 56) using regression. Covariates included education, age, and sex. This analysis was restricted to those over the age of 45 to remove participants likely to be many years from symptom onset based on natural history studies (Staffaroni et al., Reference Staffaroni, Quintana, Wendelberger, Heuer, Russell, Cobigo, Wolf, Goh, Petrucelli, Gendron, Heller, Clark, Taylor, Wise, Ong, Forsberg, Brushaber, Rojas and VandeVrede2022).
Validation cohort and longitudinal analyses
Cross-sectional analyses
Mirroring analyses in the development sample, regressions assessed the association between the TabCAT-EXAMINER score and age, education, sex, CDR®+NACC-FTLD Sum of Boxes, and neuropsychological measures. Regressions next compared TabCAT-EXAMINER scores across diagnostic syndromes. We then replicated comparisons between asymptomatic and prodromal f-FTLD mutation carriers (n = 37) from family controls (n = 31) using regressions in a sample over the age of 45.
Longitudinal analyses
First, in a sample defined as functionally intact (CDR®+NACC-FTLD Global score = 0) at baseline and follow-up, test-retest reliability was assessed as the correlation between baseline and longitudinal scores. Second, to test the hypothesis that rates of decline in TabCAT-EXAMINER scores hasten as disease severity increases, we fitted linear regressions with TabCAT-EXAMINER annualized change scores ( [visit 2 – visit 1] / retest-interval ) as the outcome and baseline disease severity (CDR®+NACC-FTLD Global score group) as the predictor. Third, we fitted linear regressions to test whether those who converted to a higher CDR®+NACC-FTLD Global score at follow up showed more rapid TabCAT-EXAMINER declines than those who remained stable at follow up. Covariates included education, sex, and retest interval.
Results
Model building
The exploratory factor analysis scree plot (Supplemental Figure 2) suggested a one factor model accounted for the majority of variance (84%): eigenvalue1-factor: 5.53; eigenvalue2-factor: 1.0; eigenvalue>2-factors: mean = .35 (SD = .23). The CFA model, which accounted for shared measurement variance of the verbal fluency tasks, fit the data well (CFI = 0.996, TLI = 0.995, RMSEA = 0.028) with moderate-to-strong factor loadings. This final model (Figure 1) was specified as a multilevel IRT model to generate factor scores.
TabCAT-EXAMINER confirmatory factor analysis. Note. Standardized factor loadings are presented for the final confirmatory factor analysis model. Latent factors are represented by circles/ovals. Rectangles represent measured items.
Factor score distribution and reliability are detailed in the supplement (Supplemental Figures 3–5). Briefly, the TabCAT-EXAMINER score was normally distributed. A series of models calculated the TabCAT-EXAMINER with fewer than nine components and correlated these models with the full, nine component TabCAT-EXAMINER. As the number of components included in a model decreased, there was a reduction in the average correlation and an increase in uncertainty of the estimated association. However, the average correlation with the full TabCAT-EXAMINER remained strong, ranging from .84 to .90 across models. These results are presented in Supplemental Figure 4. The association of the reduced models and the full TabCAT-EXAMINER varied somewhat by specific component (Supplemental Figure 5), particularly when fewer than four components were included. There was also variability in the association between the reduced factor score and the CDR®+NACC-FTLD Sum of Boxes when fewer components were included (Supplemental Figure 6).
Developmental sample: convergent and divergent validity
Associations between the TabCAT-EXAMINER score and paper-and-pencil neuropsychological tasks in the developmental sample are presented in Figure 2 (circles). Higher TabCAT-EXAMINER scores (better performances) were associated with more years of education (β =.09, [95%CI: .05, .13]; p < .001; partial-eta 2 = .07) and younger age (β =−.35, [95%CI: −.38, −.31], p < .001; partial-eta 2 = .27). Additionally, paper-and-pencil executive functioning tasks were significantly associated with the TabCAT-EXAMINER scores in the expected direction, and standardized betas suggest large effects (β’s range; .37 to .52; p’s < .001; partial-eta 2 range: .45 to .59). The tasks of non-executive domains (i.e., language, memory, visuospatial) were also significantly associated with the TabCAT-EXAMINER score, with modest reductions in magnitude (β’s range: .13 to .42; partial-eta 2 range: .25 to .47) compared to the magnitude of associations with tasks of executive functioning.
Associations of the TabCAT-EXAMINER score with demographics and clinical measures. Forest plot presenting standardized regression betas in the developmental (circle) and validation cohorts (triangle). The first row presents the association between CDR®+NACC-FTLD Sum of Boxes (reverse scored) and the TabCAT-EXAMINER. All other regression models control for CDR®+NACC-FTLD Sum of Boxes. Confidence intervals are shown via the horizontal lines. BNT = Boston Naming Test, MINT = Multilingual Naming Task.
Lower TabCAT-EXAMINER scores were strongly correlated with greater disease severity as measured by the CDR®+NACC-FTLD Box Score (β=−.65, [95%CI: −.67, −.61], p < .001; partial-eta 2 = .43), and remained associated after adjusting for covariates (β = −.16, [95%CI: −.16, −.14], p < .001 partial-eta 2 = .47). There was a stepwise decline in factor score with increasing CDR®+NACC-FTLD Global score (0 > 0.5 > 1 > 2; p < 0.001). Table 1 provides summary scores by CDR®+NACC-FTLD Global scores across all TabCAT-EXAMINER components and factor score.
The functionally intact control group had a significantly greater TabCAT-EXAMINER score than any of the symptomatic groups (Figure 3; p’s < .006). As a follow-up analysis, the control group was compared to symptomatic groups with at least 1 positive AD-biomarker. AD-biomarker positivity was based on PET-imaging, CSF, and/or plasma and diagnoses were determined in a consensus conferences using published criteria (Jack Jr et al., Reference Jack, Bennett, Blennow, Carrillo, Dunn, Haeberlein, Holtzman, Jagust, Jessen, Karlawish, Liu, Molinuevo, Montine, Phelps, Rankin, Rowe, Scheltens, Siemers, Snyder and Silverberg2018). Results supported a stepwise decline in factor scores (p’s < .001) with increasing severity: cognitively unimpaired (n = 647; mean = .53, SD = .63), mild cognitive impairment likely due to Alzheimer’s disease (1+ AD biomarker; n = 40; mean = −.4; SD = .54), and Alzheimer’s disease dementia (1+ AD biomarker; n = 37; mean = −1.13, SD = .74) groups.
TabCAT-EXAMINER performance by clinical syndrome within the developmental sample. CU = cognitively unimpaired, MCI = mild cognitive impairment, lvPPA = logopenic primary progressive aphasia, AD = Alzheimer’s disease dementia, CBS = corticobasal syndrome, PSP = progressive supranuclear palsy, nvPPA = nonfluent-variant primary progressive aphasia, svPPA = semantic-variant primary progressive aphasia, bvFTD = behavioral-variant frontotemporal dementia.
In a subsample of individuals over the age of 45 (n = 89), the average score for asymptomatic/prodromal f-FTLD mutation carriers was worse than family controls, although this difference was not statistically significant (β =−.21, [95%CI: −.43, −.01], p = .06).
Validation sample and longitudinal analyses
Figure 2 provides associations among the TabCAT-EXAMINER score, demographics, and paper-and-pencil neuropsychological tasks in the ALLFTD validation sample (triangles). This figure demonstrates the similarities between standardized regression coefficients in the validation and development sample (e.g., association with Trail Making Test B: ALLFTD Validation: β = .54; Development: β = .52). The TabCAT-EXAMINER score was correlated with baseline CDR®+NACC-FTLD Box Score (β =−.74, [95%CI: −.78, −.70], p < .001; partial-eta 2 = .43), and this association remained significant while controlling for age, education, and sex, (β =−.16, [95%CI: −.18, −.14], p < .001 partial-eta 2 = .65). Similar to findings in the developmental sample, a stepwise decline in factor score was observed with increasing CDR®+NACC-FTLD Global score (0 > 0.5 > 1 > 2; p’s < .01). As found in developmental cohort, the majority of diagnostic groups performed significantly worse on the TabCAT-EXAMINER score than the cognitively unimpaired sample (i.e., CU > CBS, bvFTD, PPA, PSP, svPPA). Details are presented in Supplemental Figure 7. Within a small sample of asymptomatic/prodromal individuals from families known to carry f-FTLD mutations (n = 68), those with mutations showed a trend towards worse performance on average than noncarrier family members on the baseline TabCAT-EXAMINER (β = −.17; 95%CI: −.38, .05; p = .13), although this difference was not statistically significant. Of note, this association was similar in magnitude to that observed in the development sample (β = −.17 vs −.21).
In those who remained unimpaired at follow up (n = 146, mean interval = 383 days; SD = 52), test-retest reliability of the factor score was 0.86 (95% CI: .81, .90). Supplemental figure 8 provides information on test-retest reliability for all TabCAT-EXAMINER subtests. Annualized change scores were calculated for all participants. TabCAT-EXAMINER scores tended to decline more rapidly with more advanced disease progression (Figure 4). Examining the between-group differences, the asymptomatic group demonstrated significantly less annualized change than the prodromal group (β = −.16; 95% CI: −.29, −.03; p = .02) and the symptomatic group (β = −.22; 95% CI: −.32, −.15; p < .001). The comparison between prodromal and symptomatic groups was in the expected direction but the effect was small and not statistically significant (β = −.11; 95% CI: −.29, −.07; p = .23).
Longitudinal changes in TabCAT-EXAMINER scores by baseline disease severity. Presents the average TabCAT-EXAMINER score at baseline and follow-up by baseline CDR®+NACC-FTLD Global score (0 vs. 0.5 vs. 1+). Error bars represent standard deviation at each timepoint. Scores are in raw units.
Asymptomatic participants (CDR®+NACC-FTLD = 0) who progressed to show at least mild symptoms(CDR®+NACC-FTLD≥0.5) at follow-up (n = 14) showed significantly greater annualized decline (moderate effect size) in TabCAT-EXAMINER scores (β = −.34; 95% CI: −.52, −.16; p < .001) than those who were unimpaired at both time points.
Discussion
We present the development and validation of the TabCAT-EXAMINER score. Fit statistics indicated the model fit the data well, with moderate-to-strong factor loadings for all subtests, and the normally distributed factor score showed high test-retest reliability. Our data support the construct validity and psychometrics of the TabCAT-EXAMINER score through strong associations with other tasks of executive functioning and processing speed. We also show known-group validity in that lower scores differentiated controls from multiple diagnostic syndromes and showed a stepwise decline with increasing disease severity. These results were replicated in a subsample of ALLFTD participants. Taken together, these data suggest the single and pragmatic TabCAT-EXAMINER score is a robust and valid measure of executive functioning across a range of neurodegenerative syndromes.
Several steps were taken to maximize the relevance of the TabCAT-EXAMINER score to a broad research community interested in studying executive functioning and processing speed. To improve the generalizability of the factor score for use across a range of clinical conditions, the development of the score was conducted within a diagnostically heterogeneous sample (i.e., disease severity, syndrome). Indeed, our results underscore the sensitivity of the TabCAT-EXAMINER score to executive deficits across a wide range of clinical syndromes. Additionally, the factor score was computed using IRT methodology to enhance its use by a variety of researchers with different goals. An important value of IRT is that valid and reliable scores on the same scale can be created using IRT regardless of which tests were administered, enabling researchers to tailor their battery to include only subsets that are pertinent to their study. The TabCAT-EXAMINER demonstrated this feature of IRT through several supplemental analyses that studied factor scores calculated in hypothetical scenarios in which only a subset of the tasks were administered. First, factor scores calculated with the full set of TabCAT-EXAMINER tasks were highly correlated with scores calculated with only a subset of test items. The variance around that association, however, shrank as more tests were included. Second, we investigated how the inclusion of specific items affected the association of the TabCAT-EXAMINER and the CDR®+NACC-FTLD Sum of Boxes. Once five of nine tasks were used to estimate the composite score, all generated scores showed similar associations with the CDR®+NACC-FTLD Sum of Boxes. Finally, the TabCAT-EXAMINER demonstrated strong test-retest reliability across time. Taken together, these analyses support use of the TabCAT-EXAMINER, even if data for all the subtasks are not available.
We also conducted a series of analyses to validate the utility of these tests for FTD participants due to the importance of executive functioning within these syndromes. We first replicated prior investigations with the laptop-based NIH-EXAMINER, finding that the composite score captured executive functioning deficits of FTD, was sensitive to early stages of the disease, and was inversely related to disease severity as indexed by the negative correlation with the CDR®+NACC-FTLD, a measure developed specifically to assess clinical functioning in FTD. In a series of longitudinal analyses, faster rates of decline were observed in those with more advanced clinical severity and participants that showed clinical progression over 6 to 24 months exhibited greater declines in TabCAT-EXAMINER scores than those who remained clinically stable. Together, these results support this score as a tool for monitoring disease progression. Across both developmental and validation cohorts, asymptomatic f-FTLD mutation carriers performed worse on average than non-carrier family controls. Although these differences were non-significant and had small effect sizes, that is somewhat expected due to the small number of participants and the heterogeneity of this sample, which comprised three different pathogenic mutations that can present with phenotypic variability. Further study in larger samples is warranted to understand the potential of this score for identifying the earliest cognitive deficits in f-FTLD, which may vary by underlying pathogenic risk. The findings from this study of the TabCAT-EXAMINER score adds further support that digital cognitive tasks are reliable and may serve as effective outcomes for clinical trials (Inan et al., Reference Inan, Tenaerts, Prindiville, Reynolds, Dizon, Cooper-Arnold, Turakhia, Pletcher, Preston, Krumholz, Marlin, Mandl, Klasnja, Spring, Iturriaga, Campo, Desvigne-Nickens, Rosenberg, Steinhubl and Califf2020; Staffaroni, Bajorek, et al., Reference Staffaroni, Bajorek, Casaletto, Cobigo, Goh, Wolf, Heuer, Elahi, Ljubenkov, Dever, Kornak, Appleby, Bove, Bordelon, Brannelly, Brushaber, Caso, Coppola and Dheel2020; Staffaroni et al., Reference Staffaroni, Clark, Taylor, Heuer, Sanderson-Cimino, Wise, Dhanam, Cobigo, Wolf, Manoochehri, Forsberg, Mester, Rankin, Appleby, Bayram, Bozoki, Clark, Darby and Domoto-Reilly2024; Staffaroni, Tsoy, et al., Reference Staffaroni, Tsoy, Taylor, Boxer and Possin2020).
The code to generate these factor scores was written in an open-source software, R, allowing interested users to calculate factor scores using their own data. Additionally, the TabCAT platform is implementing an automatic calculation of the TabCAT-EXAMINER for users. Details about accessing TabCAT software can be found online at: https://tabcathealth.com/.
Our study has some limitations. Although the sample was diagnostically diverse, all participants completed testing in English, and participants were generally highly educated and predominantly white and non-Hispanic. Future studies with non-English speaking and culturally diverse samples are warranted to determine the generalizability of the current findings to other populations. Toward this end, an ongoing study is examining the TabCAT-EXAMINER score within a Latin American Cohort (ReDLat: Multi-Partner Consortium to Expand Dementia Research in Latin America; Ibanez et al., Reference Ibanez, Yokoyama, Possin, Matallana, Lopera, Nitrini, Takada, Custodio, Sosa Ortiz, Avila-Funes, Behrens, Slachevsky, Myers, Cochran, Brusco, Bruno, Brucki, Pina-Escudero, Okada de Oliveira and Miller2021). Our validation sample primarily comprised FTD participants; additional validation in other neurodegenerative and neurological populations would further demonstrate utility of the TabCAT-EXAMINER. The TabCAT-EXAMINER is not directly comparable to the original NIH-EXAMINER due to adaption changes (i.e., computer to tablet) and differences in specific components (e.g., Running dots, Match). A future cross-walk study will develop a method of converting between the executive composites.
Conclusion
Scores on the TabCAT-EXAMINER, a tablet-based battery of executive functioning tests, can be distilled into a single reliable metric with evidence of construct validity and utility for disease monitoring. Since executive functioning is impacted by a range of neurological and medical conditions, this score has broad relevance. These results add to an emerging body of literature supporting the use of digital health technologies for collecting clinical outcome assessments in AD and related dementia research. Future work is needed to understand the relevance of these tests to clinical care and validate their use in different languages and in underrepresented groups.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355617725000177.
Funding statement
This work was supported by the National Institutes of Health [grants AG063911, AG62677, AG045390, NS092089, AG032306, AG016976, AG058233, AG038791, AG02350, AG019724, AG062422, NS050915, AG032289-11, AG077557, K23AG061253, K24AG045333, and U01NS128913], the Association for Frontotemporal Degeneration, the Bluefield Project to Cure FTD, the Rainwater Charitable Foundation, the Global Brain Health Institute, the Larry L. Hillblom Foundation [2014-A-004-NET], the John Douglas French Alzheimer’s Foundation, and AlzOut. Samples from the National Centralized Repository for Alzheimer’s Disease and Related Dementias, which receives government support under a cooperative agreement grant (U24 AG21886) awarded by the NIA, were used in this study. Dr Saloner. is supported by the Alzheimer’s Association (AARF-23-1145318), New Vision Research (CCAD2024-001-1), American Academy of Neurology, Association for Frontotemporal Degeneration, and American Brain Foundation. Dr Boxer is supported by U19AG063911, R01AG073482, R01AG038791, and R01AG071756. Dr Kramer is supported by NIH (R01(s) AG032289 and AG048234) and Larry L. Hillblom Network Grant (2014-A-004-NET). We would also like to thank participants and collaborators from the ALLFTD (https://www.allftd.org/), which was funded by the National Institute on Aging and the National Institute of Neurological Diseases and Stroke (U19: AG063911).
Competing interests
Dr Staffaroni has provided consultation to Alector, Aviado Bio, CervoMed, Eli Lilly & Prevail Therapeutics, Passage Bio, and Takeda. He serves on the scientific review board for the Alzheimer’s Drug Discovery Foundation. He is a co-inventor of four smartphone cognitive tasks (not included in the present study) and receives licensing fees. UCSF receives licensing fees for TabCAT that are used to maintain the software and its use. Dr Rojas is a site Principal Investigator for clinical trials sponsored by Eli-Lilly, Eisai and Amylyx. Dr Boxer has served as a consultant for Aeovian, AGTC, Alector, Arkuda, Arvinas, Boehringer Ingelheim, Denali, GSK, Life Edit, Humana, Oligomerix, Oscotec, Roche, TrueBinding, Wave, Merck and received research support from Biogen, Eisai and Regeneron.
