Introduction
Script generation is a novel cognitive construct involving the ability to produce and organize complex, sequential action plans derived from mental representations of everyday activities (Grafman, Reference Grafman1995). These scripts are activated when a high-level goal is established and triggers its individual component actions and sub-goals. Scripts are crucial to goal-directed behavior (Shallice, Reference Shallice1982) and provide templates for a variety of daily activities. Deficiencies in script generation can, therefore, disrupt instrumental activities of daily living (IADL; Zalla, Plaissiart, Pillon, Grafman, & Sirigu, Reference Zalla, Plaissiart, Pillon, Grafman and Sirigu2001), and direct assessments of script use are predictive of everyday functioning in some neurological populations (e.g., Giovannetti et al., Reference Giovannetti, Bettcher, Brennan, Libon, Kessler and Duey2008).
Neuroimaging studies generally support the involvement of frontostriatal systems in script generation (e.g., Shallice, Reference Shallice1988), in particular dorsolateral and ventromedial prefrontal cortex, as well as striatum (e.g., Koechlin, Danek, Burnod, & Grafman, Reference Koechlin, Danek, Burnod and Grafman2002). Moreover, script generation is commonly impaired in individuals with prefrontal lesions (e.g., Sirigu et al., Reference Sirigu, Zalla, Pillon, Grafman, Agid and Dubois1995) or Parkinson's disease (e.g., Godbout & Doyon, Reference Godbout and Doyon2000). These populations commonly display adequate performance in generating the relevant actions required for a script, indicating intact access to script knowledge, although the types of errors that occur may differ by condition. For example, individuals with Parkinson's disease and prefrontal lesions both display errors in sequencing script events, but Parkinson's disease patients have difficulty inhibiting irrelevant script elements while individuals with prefrontal lesions demonstrate problems staying within the boundaries of scripts (e.g., premature termination).
Considering the prominent frontostriatal neuropathogenesis of HIV infection (González-Scarano & Martín-García, Reference González-Scarano and Martín-García2005) and resultant executive dysfunction (e.g., Grant et al., Reference Grant, Atkinson, Hesselink, Kennedy, Richman, Spector and McCutchan1987), script generation may be of particular relevance to HIV. Prior research shows that action (verb) generation is disproportionately impaired in HIV (Woods et al., Reference Woods, Carey, Tröster and Grant2005) and sensitive to IADL declines (Woods et al., Reference Woods, Morgan, Dawson, Scott and Grant2006), which may be related to difficulties inhibiting the generation of irrelevant actions. If one presumes that deficient generation of verbs and difficulty avoiding irrelevant actions interfere with effective generation of script sequences, then it is reasonable to hypothesize that HIV infection might lead to errors in script generation, which may ultimately lead to errors in script execution (i.e., IADL dysfunction). The aim of this study was, therefore (1) to assess the effect of HIV-associated neurocognitive disorders on script generation, and (2) to examine the convergent (e.g., executive functions) and divergent (e.g., constructional praxis) validity of script generation.
Methods
Participants
Participants included 60 individuals with HIV-1 infection (HIV+) and 26 seronegative healthy adults (HIV-). Individuals with histories of major neuromedical confounds (e.g., head injuries), severe psychiatric disorders (e.g., schizophrenia), substance-related disorders within 1 year of evaluation, or a positive toxicology test for illicit substance use were excluded. Twenty-nine of the HIV+ participants (48%) met research criteria (see Antinori et al., Reference Antinori, Arendt, Becker, Brew, Byrd, Cherner and Wojna2007) for an HIV-associated neurocognitive disorder (HAND) as determined by a comprehensive neuromedical and neuropsychological evaluation (see Woods et al., Reference Woods, Rippeth, Frol, Levy, Ryan, Soukup and Heaton2004). Twenty of the HAND+ participants were diagnosed with Asymptomatic Neurocognitive Impairment, 7 with Minor Neurocognitive Disorder, and 2 with HIV-associated Dementia. Table 1 shows that the study groups were largely comparable in demographic, psychiatric, and HIV disease characteristics.
Demographic, psychiatric, and HIV characteristics of the study sample
Note. WRAT = Wide Range Achievement Test; BDI-II=Beck Depression Inventory-II; cART = combination antiretroviral therapies; CSF = cerebrospinal fluid.
aChi-square analysis compares all ethnicity groups.
bIndicates a lifetime diagnosis.
cFifteen HIV+ participants received the Beck Depression Inventory.
dData represent medians with interquartile ranges.
eSamples available on 38 HIV+ participants.
fBoth HIV+ groups differ from HIV seronegative group, although the two HIV+ groups are comparable.
gData are intended to be descriptive and are part of the basis for group classification, and thus F values and significance are not shown.
Procedures
The study was approved by the UCSD Human Research Protections Program. After providing written informed consent, each participant was administered a standardized test of script generation and comprehensive neuropsychological, psychiatric, and neuromedical evaluations. The script generation test required participants to verbally generate and organize the necessary steps for completing 6 daily activities. Five scripts were from a normative study of frequency in engaging in daily scripts by Rosen, Caplan, Sheesley, Rodriguez, and Grafman (Reference Rosen, Caplan, Sheesley, Rodriguez and Grafman2003), which were chosen to cover a range of novelty and complexity and for their ecological relevance for clinical populations. These scripts were as follows: (1) going shopping for a meal; (2) attending a dentist's appointment; (3) preparing to leave the house in the morning; (4) getting into a car accident (from the time of the accident); and (5) doing the laundry. An additional script, “getting a new prescription filled,” was added given its relevance to the daily activities of HIV+ individuals. The administration sequence of scripts was randomized for each participant.
Administration was standardized after Godbout and Doyon (Reference Godbout and Doyon2000). The examiner defined the overall purpose/goal of each script and the script's starting and ending points. Instructions were given verbally and displayed on a cue card that remained visible throughout the task. For example, for “Preparing to leave the house in the morning,” the examiner stated: “You need to get up in the morning to go to work or attend an appointment. Tell me, in order, all of the things you need to do, starting when you go to sleep the night before and stopping when you leave the house.” The examiner offered an example response to ensure understanding of the task requirements. After each script instruction, the examiner recorded the participant's responses in the order specified and then confirmed the responses and their sequence (and scored the corrected responses separately). When generated events were similar, such as “take a shower” and “take a bath,” they were classified as belonging to the same action category (Sirigu et al., Reference Sirigu, Zalla, Pillon, Grafman, Agid and Dubois1995). The lead author and a trained research assistant administered and scored the test in accordance with standardized procedures, and each battery was double-scored to ensure accuracy. The examiner was blinded to participants’ HIV status. Inter-rater reliability for script generation was established before study initiation with 10 healthy, HIV seronegative participants not included in the study sample. Two-way, random-effects intraclass correlations for consistency revealed good-to-excellent inter-rater reliability (range = 0.83–0.98).
In line with Godbout and Doyon (Reference Godbout and Doyon2000), performance on script generation was measured by the following: (1) Sequencing Errors (physically impossible or inconsistent); (2) Repetitions; (3) Intrusions (irrelevant to the script); (4) Script Boundary Errors (participant ends a script before or beyond the prescribed endpoint); and (5) Total Errors (sum of 1–4). We also created a variable to index the total Accuracy of responding, which was calculated as [(accurate responses—errors)/accurate responses]. The total number of script elements that participants omitted from among the top five most frequently generated actions for each script (Rosen et al., Reference Rosen, Caplan, Sheesley, Rodriguez and Grafman2003) were summed to give a measure of important script elements that were omitted (excluding the prescription script, which did not have normative information). The total numbers of actions generated and mean generation time (total time/number of actions) were used as potential confounding variables (i.e., to determine whether findings might be due to reduced fluency). In addition, participants were read a list of 10 actions for each sequence—five that were most frequently generated in a normative sample (Rosen et al., Reference Rosen, Caplan, Sheesley, Rodriguez and Grafman2003) and five that did not belong—and asked to respond “Yes” or “No” to whether each action belongs in a sequence, providing a measure of recognition for the semantic content of scripts. Participants were also asked to rate the frequency that they performed the given tasks on a scale from 1 (never) to 5 (very frequently) to assess general script familiarity.
The neuropsychological battery was constructed in accordance with the recommendations of the Frascati NIH working group (Antinori et al., Reference Antinori, Arendt, Becker, Brew, Byrd, Cherner and Wojna2007) and is described in detail elsewhere (Rippeth et al., Reference Rippeth, Heaton, Carey, Marcotte, Moore, Gonzalez and Grant2004). All participants scored above the 90% correct cutoff on the Digit Memory Test (Hiscock & Hiscock, Reference Hiscock and Hiscock1989). Population-based Z scores were created from raw scores of measures within select neurocognitive domains, which were chosen on an a priori conceptual basis to explore the convergent and divergent validity of script generation. These Z scores were adjusted so that high scores represented better performance and were averaged to generate composite cognitive domain scores. For speed of information processing, the measures used were: (a) Wechsler Adult Intelligence Scale-III (WAIS-III) Digit Symbol; (b) WAIS-III Symbol Search; and (c) Trailmaking Test, Part A. For attention/working memory, the measures were: (a) Paced Auditory Inhibition Test (PASAT) number correct (2.4-s inter-stimulus interval); and (b) Wechsler Memory Scale-III (WMS-III) Spatial Span total. For executive functions, the measures were: (a) Wisconsin Card Sorting Test-64 (WCST-64) perseverative responses; (b) Trailmaking Test, Part B; and (c) Stroop Color-Word Test (incongruent trial). Correlations between Action Fluency (Piatt, Fields, Paolo, & Tröster, Reference Piatt, Fields, Paolo and Tröster1999) and script generation were also examined given their putative association. In terms of divergent validity, for learning, the measures used were: (a) Hopkins Verbal Learning Test-Revised (HVLT-R) Learning; and (b) Brief Visuospatial Memory Test-Revised (BVMT-R) Learning. For memory, the measures were: (a) HVLT-R Delayed Recall; and (b) BVMT-R Delayed Recall. For constructional praxis, the measure used was: BVMT-R Copy Trial. Neuropsychological domain Z scores are presented in Table 1 for reference.
Data Analyses
Data were screened for significant outliers (i.e., >3.5 SD from the overall group mean) and evaluated for normality (i.e., Shapiro-Wilk W test, p < .05). The script generation error variables were nonnormally distributed, and Kruskal-Wallis tests (and follow-up Wilcoxon Ranked Sums) were, therefore, used. Spearman's rho correlations were used to examine the association between script generation variables and neuropsychological performance.
Results
As displayed in Table 2, significant omnibus between-group differences were observed for total errors, intrusion errors, script boundary errors, and total accuracy (ps < .05). Pair-wise comparisons revealed that HAND+ individuals evidenced significantly more intrusion errors (Cohen's d = .79), total errors (d = .94), and script boundary errors (d = .75) as well as decreased total accuracy (d = 1.3) as compared to the HIV- sample. HAND+ participants also displayed significantly lower accuracy in script generation (d = 1.3) compared to the HAND- participants. HAND- participants were not significantly different on any script generation variable from HIV- individuals. There were no group differences in inclusion of the most relevant script elements, recognition of script steps, script generation time, or number of elements generated (ps > .10). Using the corrected responses of participants in analyses or removing individuals with dementia from the HAND sample did not alter the study findings.
Script generation performance in the HAND+, HAND-, and HIV- healthy comparison samples
Note. FPs = False Positives; Chi-square values are from nonparametric Kruskal-Wallis omnibus tests.
aAverage number of individual elements (i.e., actions) per script.
bOut of a total 30 possible.
Within the HIV+ sample, action fluency was correlated with script generation boundary errors (rho = −.29; p < .05), total errors (rho = −.28; p < .05), and accuracy (rho = .34; p < .01). The speed of information processing Z score was negatively associated with sequencing errors (rho = −.35; p < .01), total errors (rho = −.27; p < .05), and boundary errors (rho = −.35; p < .01), and positively associated with accuracy (rho = .31; p < .05). The executive functions Z score was associated with sequencing errors (rho = −.28; p < .05), total errors (rho = −.28; p < .05), and boundary errors (rho = −.25; p < .05). The attention/working memory Z score was negatively associated with intrusions (rho = −.32; p < .05), boundary errors (rho = −.27; p < .05), and total errors (rho = −.29; p < .05), and positively associated with accuracy (rho = .37; p < .01). All other correlations between these domains and script generation variables were nonsignificant (ps > .10). Moreover, no significant associations were observed between script generation variables and measures of learning, memory, or constructional praxis (range: rho = .01–.21; all ps > .10).
Discussion
In this study, the first to examine script generation in HIV disease, we found differences of a medium-to-large magnitude in script generation errors and accuracy between individuals with HAND and HIV seronegative participants. Individuals with HAND displayed increased intrusions, script boundary errors, and total errors but, unexpectedly, did not display increased errors in sequencing script elements. These differences persisted even when examining the responses of participants after they were given the opportunity to change their responses. In addition, individuals with HAND did not have difficulties generating the most relevant elements for each script. Taken together, our results indicate that HIV+ individuals with HAND display adequate performance in generating and sequencing the relevant actions required for a script (suggestive of intact access to script knowledge) but have difficulty inhibiting the expression of irrelevant steps (i.e., actions) and staying within the prescribed boundaries of scripts.
Our results extend the findings of Woods and colleagues (Reference Woods, Carey, Tröster and Grant2005, Reference Woods, Morgan, Dawson, Scott and Grant2006), who reported that individuals with HIV demonstrated deficient overall performance and increased intrusion errors on a test of action fluency, which was also predictive of IADL dysfunction. Thus, the processes of generating script action sequences and searching for, accessing, and retrieving mental representations of actions may be related. To this end, correlational analyses showed that action fluency was modestly associated with script boundary errors, total errors, and accuracy, supporting the relatedness of these measures. Interestingly, however, the pattern of errors displayed by individuals with HAND was somewhat different than previous script generation results in Parkinson's disease and prefrontal lesions (e.g., Godbout & Doyon, Reference Godbout and Doyon2000; Sirigu et al., Reference Sirigu, Zalla, Pillon, Grafman, Agid and Dubois1995). Specifically, individuals with HAND did not display sequencing errors, which are characteristic of these disorders, although they evidenced a combination of other errors that distinguish these two disorders (i.e., intrusions similar to Parkinson's disease and boundary errors similar to prefrontal lesion patients).
Although little research has examined the construct validity of script generation, the present study supports its convergent validity by demonstrating correlations between script generation errors and accuracy and clinical measures of executive functions, attention/working memory, and processing speed. Thus, script generation errors and accuracy share a modest proportion of variance with tests measuring these related cognitive constructs. Importantly, evidence of divergent validity was provided by nonsignificant correlations with tests of episodic memory and praxis. Taken together, these findings provide evidence that HIV-associated difficulties with script generation involve slowed processing and executive dysfunction, both of which are common in HIV.
Regarding this study's clinical significance, although raw script generation error rates were relatively low (and accuracy was relatively high), our between-group effect size estimates fell within the medium-to-large range. Moreover, while script generation errors are rare in neurologically healthy individuals, HAND+ individuals made errors on average once out of every 20 script steps generated (using accuracy as a metric), which may be clinically significant. However, these speculative contentions await specific hypothesis testing regarding the clinical significance of script generation, including studies that relate script generation performance to “real world” outcomes, such as informant reports of functional abilities or readily measurable IADLs (e.g., medication adherence).
One might argue that familiarity with the script actions could have impacted these results. Post hoc analyses examining self-reported frequency of engaging in these actions revealed that both HIV+ groups had prescriptions filled more frequently compared to the HIV- group, but this frequency was not correlated with any script generation variable (ps > .10). Moreover, HAND+ participants evidenced a trend for leaving the house less frequently than the HIV- group (p = .06), which was negatively correlated with the number of script generation errors in the HAND+ sample, albeit at a trend level (rho = −.24; p = .06). Thus, the frequency that HAND+ individuals left the house was modestly associated with their script generation errors.
It is unlikely that differences observed between the HAND and HIV- groups were due to demographic factors, as the groups were comparable in this regard. Results were also likely not dependent on affective status, as depressive symptoms were not associated with script generation performance (all ps > .10). However, limitations of the study include restricted generalizability due to the demographic (e.g., largely well-educated males) and HIV disease characteristics (e.g., current immunocompetence) of the study sample. In addition, although some normative data are available for script generation, the sensitivity and specificity of script generation errors to neurocognitive disorders remain largely unexplored.
In summary, these data provide novel insights into the nature of script generation difficulties in persons with HIV. In combination with our previous study examining the multitasking of everyday actions in HIV infection (Scott et al., Reference Scott, Woods, Vigil, Heaton, Schweinsburg, Ellis and Marcotte2011), results highlight the potential clinical relevance of everyday action organization in the expression of HAND. In fact, a growing literature points to the benefits of directly assessing everyday actions in several neurologic and neuropsychiatric disorders (e.g., Giovannetti et al., Reference Giovannetti, Bettcher, Brennan, Libon, Kessler and Duey2008; Kessler, Giovannetti, & MacMullen, Reference Kessler, Giovannetti and MacMullen2007). However, although efforts have begun to standardize and obtain psychometric data for such tasks, adequate justification for adding such measures to a clinical neuropsychological battery requires multiple additional demonstrations of reliability and construct validity.
Acknowledgments
The authors thank Jordan Grafman for valuable discussions regarding script generation. This research was supported by Developmental Award-759 from the Core Support Program for AIDS Research, the 2007 Benton-Meier Scholarship from the American Psychological Foundation, and a dissertation award from the UCSD Department of Psychiatry. The HIV Neurobehavioral Research Center [HNRC] is supported by Center award MH 62512 from NIMH. The San Diego HNRC group is affiliated with the University of California, San Diego, the Naval Hospital, San Diego, and the Veterans Affairs San Diego Healthcare System, and includes: Director: Igor Grant, M.D.; Co-Directors: J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., and J. Allen McCutchan, M.D.; Center Manager: Thomas D. Marcotte, Ph.D.; Jennifer Marquie-Beck, M.P.H.; Melanie Sherman; Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Scott Letendre, M.D., Edmund Capparelli, Pharm.D., Rachel Schrier, Ph.D., Terry Alexander, R.N., Debra Rosario, M.P.H., Shannon LeBlanc; Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), Steven Paul Woods, Psy.D., Mariana Cherner, Ph.D., David J. Moore, Ph.D., Matthew Dawson; Neuroimaging Component: Terry Jernigan, Ph.D. (P.I.), Christine Fennema-Notestine, Ph.D., Sarah L. Archibald, M.A., John Hesselink, M.D., Jacopo Annese, Ph.D., Michael J. Taylor, Ph.D.; Neurobiology Component: Eliezer Masliah, M.D. (P.I.), Cristian Achim, M.D., Ph.D., Ian Everall, FRCPsych., FRCPath., Ph.D. (Consultant); Neurovirology Component: Douglas Richman, M.D., (P.I.), David M. Smith, M.D.; International Component: J. Allen McCutchan, M.D., (P.I.); Developmental Component: Cristian Achim, M.D., Ph.D.; (P.I.), Stuart Lipton, M.D., Ph.D.; Participant Accrual and Retention Unit: J. Hampton Atkinson, M.D. (P.I.), Rodney von Jaeger, M.P.H.; Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman (Data Systems Manager); Statistics Unit: Ian Abramson, Ph.D. (P.I.), Florin Vaida, Ph.D., Reena Deutsch, Ph.D., Anya Umlauf, M.S., Tanya Wolfson, M.A. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government. This manuscript has not previously been published either electronically or in print. Aspects of these data were presented at the 118th Annual Convention of the American Psychological Association in San Diego, California. Authors report no conflicts of interest affecting this manuscript.
