INTRODUCTION
Episodic memory impairment is quite common among HIV+ persons, with most estimates ranging between 40 and 60% (e.g., Heaton et al., Reference Heaton, Grant, Butters, White, Kirson, Atkinson and Ellis1995; Rippeth et al., Reference Rippeth, Heaton, Carey, Marcotte, Moore and Gonzalez2004). The etiology of the episodic memory impairment in HIV is likely multifactorial, likely reflecting the adverse effects of the virus on frontostriatal circuits (Ellis, Langford, & Masliah, Reference Ellis, Langford and Masliah2007; Ernst & Chang, Reference Ernst and Chang2004) as well as psychiatric (e.g., substance use disorders) and medical (e.g., hepatitis C co-infection) (Antinori et al., Reference Antinori, Arendt, Becker, Brew, Byrd, Cherner and Wojna2007) comorbidities. Of clinical relevance, HIV+ persons with episodic memory deficits are at heightened risk for experiencing problems with a variety of different everyday activities, including health behaviors.
For example, Hinkin et al. (Reference Hinkin, Hardy, Mason, Castellon, Durvasula, Lam and Stefaniak2004) reported that impairment in memory was associated with poor adherence to combination anti-retroviral therapy (cART). In that study, approximately two-thirds of HIV+ participants with memory impairment were classified as poor adherers. Moreover memory functioning in HIV+ persons has been identified as an independent predictor of real-world tasks such as return to work (van Gorp et al., Reference van Gorp, Rabkin, Ferrando, Mintz, Ryan, Borkowski and Mcelhiney2006) and household shopping (Heaton et al., Reference Heaton, Marcotte, Mindt, Sadek, Moore, Bentley and Grant2004). Thus, research to date clearly shows that episodic memory deficits are both common and impactful among HIV+ persons. However, we know much less about how to improve or restore memory deficits in this complex clinical population (see Weber, Blackstone, & Woods, Reference Weber, Blackstone and Woods2013 for a review).
One potentially viable approach is to translate promising findings from the rich cognitive psychology literature in healthy older adults into effective memory rehabilitation techniques for neuropsychological populations. This approach has been successfully implemented with people living with HIV. For example, Weber, Woods, Kellogg, Grant, and Basso (Reference Weber, Woods, Kellogg, Grant and Basso2011) tested the self-generation effect (i.e., information generated by an individual is learned more effectively than material that is presented didactically) in HIV+ persons via paired associate learning task. Results revealed that HIV+ persons with clinical memory impairment benefitted more from self-generation as opposed to simply reading the materials. Thus, it may be possible to improve memory functions in persons living with HIV disease using approaches that have shown success in basic cognitive psychology experiments.
The testing effect, also known as retrieval practice effect, is well-established memory strategy that produces particularly robust memory improvements in both healthy (see Roediger & Butler, Reference Roediger and Butler2011, for a review) and clinical populations (see Balota, Duchek, & Logan, Reference Balota, Duchek and Logan2007) and, thus, may be another effective means of improving memory in HIV+ persons. The testing effect is a cognitive phenomenon by which the act of retrieval of newly learned material improves subsequent recall more than other learning strategies such as massed study and spaced restudy (Carpenter & DeLosh, Reference Carpenter and DeLosh2005; Roediger & Karpicke, Reference Roediger and Karpicke2006).
Both practical and theoretical reasons exist for examining the testing effect in adults with HIV. It has been reported that the testing effect has improved memory performance in clinical groups (for reviews, see Dunlosky, Rawson, Marsh, Nathan, & Willingham, Reference Dunlosky, Rawson, Marsh, Nathan and Willingham2013; Middleton & Schwartz, Reference Middleton and Schwartz2012). In fact, the effect has been observed in a variety of neuropsychological populations including multiple sclerosis (e.g., Basso, Lowery, Ghormley, Combs, & Johnson, Reference Basso, Lowery, Ghormley, Combs and Johnson2006; Chiaravalloti & DeLuca, Reference Chiaravalloti and DeLuca2002; Sumowski, Chiaravalloti, & DeLuca, Reference Sumowski, Chiaravalloti and DeLuca2010; Sumowski et al., Reference Sumowski, Leavitt, Cohen, Paxton, Chiaravalloti and DeLuca2013), traumatic brain injury (e.g., O’Brien, Chiaravalloti, Arango-Lasprilla, Lengenfelder, & DeLuca, 2007; Sumowski, Wood, et al., Reference Sumowski, Wood, Chiaravalloti, Wylie, Lengenfelder and Deluca2010), and mild dementia in early Alzheimer’s disease (e.g., Lipinska, Bäckman, Mäntylä, & Viitanen, Reference Lipinska, Bäckman, Mäntylä and Viitanen1994).
For example, Sumowski, Chiaravalloti, et al. (Reference Sumowski, Chiaravalloti and DeLuca2010) in a study with healthy adults and persons with multiple sclerosis (MS) found a large main effect of learning condition on memory (η2=.54) revealing large mnemonic advantages of tests for both groups. In fact, testing was the best learning condition for 90% of the participants (MS: 91% and healthy adults: 88%). Practically, if we are to encourage and expand the use of effective and simple manipulations such as the testing effect, then we need to be confident that the approach works in a variety of clinical groups with various types of memory deficits (e.g., HIV+ persons). Indeed, the cognitive rehabilitation literature shows that the relative magnitude of benefit from memory manipulations can vary across different clinical populations who may have different memory profiles (i.e., strengths and weaknesses). As such, from a scientific perspective, it is important to do the groundwork of demonstrating that the basic testing effect can be observed in HIV+ persons under controlled laboratory conditions before beginning the process of integrating the manipulation into a more comprehensive rehabilitation program.
Theoretical motivation for examining the testing effect in HIV− infected persons is more complex than the practical motivation given the lack of consensus among researchers regarding the factors that best explain this robust cognitive phenomenon. The testing effect has been explained in terms of changes in semantic representations through elaboration or the restriction of the search-set to relevant associations, effortful retrieval processes that become easier with repetition, and the potentiation of subsequent encoding (Carpenter, Reference Carpenter2011; Karpicke, Lehman, & Aue, Reference Karpicke, Lehman and Aue2014; Pyc & Rawson, 2010; Roediger & Butler, Reference Roediger and Butler2011). Thus, several cognitive theories exist explaining the nature of the testing effect and its essential processes.
In this study, we do not directly test any of these theories; rather we seek to apply the testing effect in an effort to bolster memory performance among HIV+ persons. We argue that the testing effect will benefit HIV+ persons due to the nature of the neurocognitive impairment that is commonly observed in this population. It is well established that HIV preferentially (but not exclusively) impacts frontostriatal circuits (Ellis et al., Reference Ellis, Langford and Masliah2007; Ernst & Chang, Reference Ernst and Chang2004) and that HIV+ persons show a corresponding pattern of impairment that includes slowed information processing efficiency, executive dysfunction, and deficient episodic memory encoding and retrieval (e.g., Gongvatana, Woods, Taylor, Vigil, & Grant, Reference Gongvatana, Woods, Taylor, Vigil and Grant2007; Murji et al., Reference Murji, Rourke, Donders, Carter, Shore and Rourke2003; Woods et al., Reference Woods, Morgan, Dawson, Cobb Scott and Grant2006).
For example, episodic memory deficits, specifically diminished free recall but broadly spared retention and recognition, are evident in HIV (Woods et al., Reference Woods, Scott, Dawson, Morgan, Carey and Heaton2005). Hence, given the encoding and retrieval deficits experienced by this population, we would expect that the testing effect could improve memory due to: (1) enhancement of encoding via several cognitive processes (e.g., strengthening semantic representations through elaboration or through effortful retrieval processes, the restriction of the search-set to relevant associations); and/or (2) multiple retrieval routes, which may be particularly beneficial in a population with strategic retrieval impairment.
To our knowledge, no study to date has investigated the testing effect in HIV+ persons. Still, given the importance of intact memory functioning for downstream health behaviors in individuals living with HIV (Hinkin et al., Reference Hinkin, Hardy, Mason, Castellon, Durvasula, Lam and Stefaniak2004) and previous studies suggesting that individuals with HIV can benefit from some memory improvement techniques (Weber et al., Reference Weber, Woods, Kellogg, Grant and Basso2011), the present study aimed to evaluate the effectiveness of retrieval practice for improving memory in HIV+ adults following short (i.e., 30 min) and long retention intervals (i.e., 30 days). We expected that the participants, regardless of HIV serostatus, would perform better in testing condition than in restudy conditions.
METHOD
Participants
The study sample consisted of 73 participants aged 50 years or older recruited from the San Diego community and local HIV clinics as part of an NIH-funded study on aging and memory in HIV disease. HIV serostatus was confirmed using standard Western blot and/or a point-of-care test (MedMira Inc., Nova Scotia, Canada) and yielded 52 HIV-seropositive (HIV+) individuals and 21 HIV-seronegative comparison individuals (HIV−). Exclusion criteria included having a severe psychiatric disorder (e.g., schizophrenia) or neuromedical condition including an active central nervous system opportunistic infection, a seizure disorder, head injury with loss of consciousness for more than 30 min, stroke with persistent neurological sequelae, non–HIV-associated dementia, and an estimated verbal IQ score<70 on the Wechsler Test of Adult Reading (WTAR; Psychological Corporation, 2001). Individuals were also excluded if they tested positive on a breathalyzer or urine toxicology screen for illicit drugs (except marijuana) on the day of testing.
The participants’ demographic, psychiatric, and HIV disease and treatment characteristics are displayed in Table 1. Although the groups were matched on the majority of demographic characteristic, HIV+ group was younger than HIV− group (p<.05) and included significantly higher proportions of men than the seronegative group (p<.05). Additionally memory impairment and depression were more common in the HIV+ group than in the HIV− group (p<.05).
Cohort’s demographic, psychiatric, medical, and HIV disease characteristics
Note. Data represent M (SE) or %. AIDS=acquired immune deficiency syndrome; CD4=cluster of differentiation 4; cART=combination antiretroviral therapy.
MATERIALS AND PROCEDURE
The university’s human subjects committee approved this study. All participants provided written, informed consent before completing comprehensive medical, psychiatric, and neuropsychological research evaluations for which they received financial compensation.
Retrieval Practice Paradigm
This task was developed by Sumowski, Chiaravalloti, et al. (Reference Sumowski, Chiaravalloti and DeLuca2010) and Sumowski, Wood, et al. (Reference Sumowski, Wood, Chiaravalloti, Wylie, Lengenfelder and Deluca2010). In a within-subjects design, participants were presented with 48 weakly associated verbal paired associates (VPAs), which were divided across three learning conditions: Massed-Restudy, Spaced-Restudy, and Spaced-Testing. Each pair was individually presented on a computer screen three times in accordance with condition-specific schedules and VPAs were counterbalanced across learning conditions. Each presentation lasted for 6 seconds. The initial presentation of VPAs in the Massed-Restudy condition was immediately followed by two restudy trials; hence the participants were shown the same word pair in three consecutive trials. In the Spaced-Restudy condition, the initial VPA presentation was followed by three filler trials (i.e., other VPAs), a restudy trial, six filler trials, and a second restudy trial. In the Spaced-Testing condition, the initial presentation was followed by three filler trials (i.e., other VPAs), a test trial, six filler trials, and a second test trial. Each test trial consisted of a 5-s cued-recall test (e.g., party - ____ ) followed by a 1-s feedback screen showing the completed VPA (e.g., party - serve). The paradigm is illustrated in Figure 1.
The retrieval practice paradigm adopted from Sumowski, Chiaravalloti, et al. (Reference Sumowski, Chiaravalloti and DeLuca2010) and Sumowski, Wood, et al. (Reference Sumowski, Wood, Chiaravalloti, Wylie, Lengenfelder and Deluca2010).
The paradigm isolated the effect of retrieval practice relative to restudy on memory via different types of cognitive process required by Spaced-Restudy and Spaced-Testing conditions. The only difference between the Spaced-Restudy and Spaced-Testing conditions was the type of cognitive reprocessing, which isolated the effect of retrieval practice during testing. The task also isolated the effect of spaced learning via different presentation schedules in Massed-Restudy and Spaced-Restudy conditions. The only difference between the Massed-Restudy and Spaced-Restudy conditions was the presentation schedule, which isolated the effect of spaced learning on memory (see Sumowski, Chiaravalloti et al., Reference Sumowski, Chiaravalloti and DeLuca2010 and Sumowski, Wood, et al., Reference Sumowski, Wood, Chiaravalloti, Wylie, Lengenfelder and Deluca2010) for further information on the development of the task).
The dependent variables were the performance on (1) short-delay cued recall (i.e., 30-min) and (2) long-delay cued recall (i.e., 30-day) tests. During the cued-recall task, the first word of each VPA was presented on the computer screen, and participants were asked to orally provide the second word of the pair. No feedback was provided in the delayed cued-recall tests. The order of VPAs during the test was quasi-randomized so that VPAs from each learning condition were equally represented at the beginning, middle, and end of the task.
Medical Evaluation
Participants were administered a brief medical evaluation conducted by a research nurse and included a review of systems, medications, history, urine toxicology, and a blood draw.
Psychiatric Evaluation
Current (i.e., within the last 30 days) and lifetime affective disorders (i.e., Major Depressive Disorder, Generalized Anxiety Disorder) were evaluated via the Composite International Diagnostic Interview (CIDI; version 2.128). The CIDI also provided lifetime diagnoses of Substance Use Disorders. Participants also completed the Profile of Mood States (POMS; McNair, Lorr, & Droppleman, Reference McNair, Lorr and Droppleman1971), which assesses symptoms of anxiety, depression, and other mood states over the past week.
Picture Sequence Memory
The Picture Sequence Memory Test in NIH Toolbox was used for the assessment of episodic memory. It involved recalling increasingly long series of illustrated objects and activities that have been presented in a particular order on the computer screen. The participants were asked to recall the sequence of pictures that were presented over two learning trials (for a detailed description of the test, see Weintraub et al., Reference Weintraub, Dikmen, Heaton, Tulsky, Zelazo, Bauer and Gershon2013). Consistent with the recommendations of the Frascati criteria (Antinori et al., Reference Antinori, Arendt, Becker, Brew, Byrd, Cherner and Wojna2007), we used demographically adjusted normative standards (Casaletto et al., Reference Casaletto, Umlauf, Beaumont, Gershon, Slotkin, Akshoomoff and Heaton2015) to quantify performance and used a 1 SD cut-point for identifying impaired memory function.
Data Analysis
To investigate our hypotheses regarding the performance on Verbal Paired Associate Learning paradigm as a function of learning conditions and HIV serostatus in short (30 min) retention interval, a 2 (HIV+ and HIV−) × 3 (Massed-Restudy, Spaced-Restudy, and Spaced-Testing) mixed-design analysis of variance (ANOVA) was performed. To determine covariates for this model, we considered only variables that related to both independent and dependent variables at a critical alpha of .05. We analyzed comorbid variables (memory impairment, generalized anxiety disorder, major depressive disorder, substance abuse, and hepatitis C) and demographic variables (age, gender, estimated premorbid IQ, diabetes, hyperlidemia). Of the variables (i.e., age, gender, memory impairment, and depression) that were associated with our fixed effect (see Table 1), memory impairment, and depression scores were also associated with the short-delay memory performance outcomes; therefore, these two variables were included in the models as fixed effects factor.
When appropriate, Cohen’s d or partial eta squared (ηp 2) was used to report effect size estimates. Of note, the Shapiro-Wilk test of normality revealed that the data collected in cued-recall tests (i.e., short delay) failed the assumption of normality of parametric tests (ps<.05). Since ANOVA-based tests are considered robust tests against normality assumption and there is no non-parametric corresponding test of ANOVA, the mixed-design ANOVA was determined to be the most appropriate statistical approach to analyze data in short-delay condition.
The analyses were conducted using the JMP 11.2 statistical program and the critical alpha was set at .05.
RESULTS
The model with HIV, memory impairment, and depression as between-subject factors and learning conditions (Massed-Restudy, Spaced-Restudy, and Spaced-Testing conditions) as within-subjects factors yielded a large effect of learning condition (Pillai’s Trace=.17, F(2,68)=32.76; p<.001) where participants recalled 7.91 (SE=.08) of VPAs studied through Spaced-Testing compared with only 3.39 (SE=.58) through Massed-Restudy and 5.26 through Spaced-Restudy (SE=.02) with medium to large effect sizes between Massed-Restudy and Spaced-Testing (d=.98) and between Spaced-Restudy and Spaced-Testing (d=.58). The main effect of HIV status was not significant (p=.22) (See Figure 2a). The main effect of depression was marginally significant (F(1,69)=3.35; p=.07; ηp 2=.05).
Short-delay cued-recall memory performance across all three study conditions. Error bars represent standard errors.
Participants with history of major depression tended to remember fewer words pairs (M=4.77; SE=.81) than participants with no history of major depression (M=6.81; SE=.56). HIV status and depression did not interact with learning conditions (ps>.05). The analysis revealed a significant main effect of memory impairment (F(1,69)=5.36; p=.02; ηp 2=.08) such that individuals with memory impairment were less likely to remember VPAs. The interaction between memory impairment and learning conditions was marginally significant (F(2,68)=2.86; p=.06; ηp 2=.05). Since there was only 1 HIV− participant with memory impairment, we sought to inspect the effect of memory impairment on outcome variables by performing within group comparisons only in the HIV+ group.
We categorized participants in the HIV+ group into those with memory impairment (HIV+Imp) (n=14) and those without memory impairment (HIV+Normal) (n=38) according to their performance on the memory task from NIH Toolbox: As noted above, the cut-point for identifying impaired memory function has been established at 1 SD of the normative mean (Antinori et al., Reference Antinori, Arendt, Becker, Brew, Byrd, Cherner and Wojna2007). The analysis of variance showed that Spaced-Testing generated a superior memory performance than Massed-Restudy (d=1.13 and d=.52, for HIV+Normal and HIV+Imp, respectively) and than Spaced-Restudy (d=.75 and d=.39, HIV+Normal and HIV+Imp, respectively) (ps<.05). The difference between Massed-Restudy and Spaced-Restudy was significant in HIV+Normal (d=.39; p=.004) but not in HIV+Imp group (d=.15; p=.11) (See Figure 2b).
Preliminary analysis of long-delay memory performance (i.e., number of pairs recalled in the cued-recall test) revealed that the participants scored poorly with very little variance (M=.12; SE=.03). Therefore, floor effects prevented further analysis of these data.
DISCUSSION
Although memory impairment is common and functionally impactful in HIV+ persons, there are few established ways to improve it. The present study examined the testing effect via a verbal paired associate learning paradigm in an HIV+ sample as a potential memory improvement strategy. The results revealed that retrieval practice generated a strong and reliable memory gain for both people with and without HIV disease.
Findings showed that the testing condition improved memory in short-delay memory test, which was associated with medium to large effect sizes. Even spaced study, which is a well-established method to help people with dementia or other memory impairments (Balota et al., Reference Balota, Duchek and Logan2007), did not generate comparable learning rates. This point is important to mention because there is a large body of experimental work investigating spaced practice and these studies have generally reported a robust beneficial effect of distributed (over massed) presentation of to-be-remembered material (for meta-analysis, see Cepeda, Pashler, Vul, Wixted, & Rohrer, Reference Cepeda, Pashler, Vul, Wixted and Rohrer2006) as was reported in the present study. Based on the findings of our study, we further suggest that spacing and testing can be combined, that is, spaced retrieval practice, to obtain the benefits of both.
Incidentally, the medium-to-large effect sizes observed in Spaced-Testing over the other two learning conditions (d between .58 and 1.13) deserve particular attention. The magnitude of these effects attests to the potential use of retrieval practice strategy for improving real-life memory performance of people with memory impairment (Sumowski, Chiaravalloti et al., Reference Sumowski, Chiaravalloti and DeLuca2010; Sumowski, Wood, et al., Reference Sumowski, Wood, Chiaravalloti, Wylie, Lengenfelder and Deluca2010) especially considering the scarcity of cognitive rehabilitation studies producing such large effect sizes. For example, methodical review of the literature concerning the effectiveness of cognitive rehabilitation of people with traumatic brain injury or stroke revealed that neurocognitive rehabilitation following brain injury are associated with improvements in attention, memory, and executive function (Cicerone et al., Reference Cicerone, Dahlberg, Kalmar, Langenbahn, Malec, Bergquist and Morse2000, Reference Cicerone, Dahlberg, Malec, Langenbahn, Felicetti, Kneipp and Catanese2005, Reference Cicerone, Langenbahn, Braden, Malec, Kalmar, Fraas and Ashman2011). However, only a small fraction of improvement was directly attributable to cognitive rehabilitation, which was evidenced by small treatment effect size (d=.30) (for meta-analyses see Rohling, Faust, Beverly, & Demakis, Reference Rohling, Faust, Beverly and Demakis2009). The medium-to-large effect sizes in this study and a similar study (Sumowski, Wood, et al., Reference Sumowski, Wood, Chiaravalloti, Wylie, Lengenfelder and Deluca2010) suggest that retrieval practice might be an effective memory improvement tool and should incite further exploration of the strategy in rehabilitation literature.
Another interesting finding was the absence of an interaction between HIV serostatus group and learning conditions. The serostatus groups performed similarly across all three learning conditions: they recalled more word pairs in the Spaced-Testing condition than in the restudy conditions. Moreover, the testing effect was broadly comparable across the groups with medium to high effect sizes (i.e., d between .58 and .98). This finding indicates that the testing effect is robust cognitive phenomenon and can be used for memory improvement in HIV+ persons. It is equally important to note that the magnitude of the memory advantage of the testing effect relative to restudying in the HIV+ group with memory impairment was approximately half of that observed in the HIV+ group without memory impairment.
The difference in effect sizes is noteworthy, especially since a liberal 1 SD cutoff was used to determine memory impairment. This finding might indicate that the magnitude of the testing effect may be slightly dampened, but still robust among individuals with fairly subtle memory impairment. Indeed, while the HIV+ group with memory impairment demonstrated somewhat inferior memory performance (i.e., recalled fewer number of word pairs) they still benefitted from retrieval practice as compared to the Spaced-Restudy and Massed-Restudy (d between .39 and .52) conditions (see Figure 2b). This might indicate that the effect was present in the individuals for whom such an intervention was most needed. Indeed, spaced testing in the memory impaired group was comparable to that of massed study in the normal group, suggesting that the strategy may be of some clinical benefit.
Although it was not the main focus of the present study, the marginal main effect of a lifetime history of major depression on the paired associates deserves brief discussion. Specifically, participants with a history of depression tended to perform poorly on the memory task than their non-depressed counterparts, irrespective of HIV serostatus (p=.07). Follow-up post hoc tests showed that current symptoms of depression as measured by the POMS depression/dejection scale were significantly associated with poor cued recall memory performance (r s =−.29; p=.01). The literature on the associations between depression and verbal memory is inconsistent (for a review, see Castaneda et al., Reference Castaneda, Tuulio-Henriksson, Marttunen, Suvisaari and Lönnqvist2008), particularly in the setting of HIV disease (e.g., Cysique et al., Reference Cysique, Deutsch, Atkinson, Young, Marcotte, Dawson and Heaton2006). Some studies have observed verbal memory impairments among depressed participants compared to healthy adults (Smith, Muir, & Blackwood, Reference Smith, Muir and Blackwood2006), yet other studies have reported no deficits in this domain (Fossati, Amar, Raoux, Ergis, & Allilaire, Reference Fossati, Amar, Raoux, Ergis and Allilaire1999; Wang et al., Reference Wang, Halvorsen, Sundet, Steffensen, Holte and Waterloo2006). Although our results point to verbal memory impairment associated with depression, the trend level effect, cross-sectional design, small effect size, and small sample size prevent us from drawing any firm conclusions.
This study is not without its limitations. The first limitation concerns the paradigm used in the study. Specifically, one may argue that the absence of a Massed-Testing condition is a weakness of the task, since such a condition could allow one to conclude whether testing alone elicited an advantage relative to spaced restudying in adults with HIV, or if this advantage was only achieved when combined with spacing. However, we argue that the paradigm isolated the effect of retrieval practice during testing relative to restudy on memory: the only difference between Spaced-Testing and Spaced-Restudy was the cognitive processes required by the condition.
Hence, the higher learning rates in the spaced testing were generated by the retrieval practice. The task also isolated the effect of spaced learning via different presentation schedules in Massed-Restudy and Spaced-Restudy conditions, since the only difference between Massed-Restudy and Spaced-Restudy conditions was the schedule of presentation, which isolated the effect of spaced learning on memory. Hence, the higher learning rates generated via spaced restudy as compared to massed restudy was due to the spacing effect. Since the effect of retrieval and spacing are captured separately by the task, we argue that higher learning rates in Spaced-Testing are due to the retrieval practice.
One might argue that the 1 SD cutoff for memory impairment is another limitation of the study. Indeed, there is some controversy in the neuroAIDS literature (as well as other conditions) as to what the optimal cutpoints are to maximize sensitivity and specificity. However, to date, there is no persuasive evidence to support the benefits of using a 1.5 or 2 SD cut-point versus a 1 SD cut-point, with the former improving specificity but also reducing sensitivity (and in all cases, the literature is lacking rigorous gold standards against which to determine true classification accuracy). At present, the most widely used diagnostic cut-points are those offered by the Frascati criteria, and thus we anchor our approach to this literature. If we were to follow the more conservative 2 SD cut-points recommended by other groups (e.g., Gisslén, Price, & Nilsson, Reference Gisslén, Price and Nilsson2011), only 2% (n=1) of the sample would meet the criteria, which would prevent us from conducting statistical analyses.
Another source of weakness in this study is the long-term retention task (i.e., cued-recall task). The long-term retention performance of the participant indicated that cued-recall test following a long delay interval (i.e., 30 days) was difficult for our participants, which manifested through floor effects. An easier task, such as recognition, or a shorter delay interval might have been more suitable for the current population.
In addition, the generalizability of these results is subject to certain limitations. First, the small sample size limits the generalization of our findings. The attrition rate of the participants in the second session limits the generalizability of the testing effect even further. Second, further investigation of the effect in other modalities and ecologically relevant materials (e.g., health-related information) is needed.
To sum up, our findings indicate that the testing effect can enhance episodic verbal memory performance among persons living with HIV, including those with clinical memory deficits. Such findings are consistent with research in other neurological conditions that experience mild-to-moderate memory deficits such as multiple sclerosis (e.g., Basso et al., Reference Basso, Lowery, Ghormley, Combs and Johnson2006; Chiaravalloti & DeLuca, Reference Chiaravalloti and DeLuca2002;), traumatic brain injury (e.g., O’Brien et al., Reference O’Brien, Chiaravalloti, Carlos Arango-Lasprilla, Lengenfelder and Deluca2007), and mild dementia in early AD (e.g., Lipinska et al., Reference Lipinska, Bäckman, Mäntylä and Viitanen1994). The present research provides the first known evidence for the mnemonic value of the testing effect relative to different study schedules among individuals with HIV. Considering the notable adverse effects that memory deficits have on everyday functioning, health behaviors, and quality of life in HIV, we argue that it is particularly important to identify effective ways to improve such deficits. Hence, a theoretically driven and empirically supported neurocognitive rehabilitation approach is imperative to this long-term goal to improve quality of life among persons with HIV disease.
ACKNOWLEDGMENTS
This work was supported by the National Institutes of Health under grants R01-MH073419 and P30-MH62512. The authors have no financial conflicts of interest related to this work. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government. The authors thank Donald Franklin, Stephanie Corkran, Jessica Beltran, and Dr. Erin E. Morgan for their help with data processing and study coordination. The San Diego HIV Neurobehavioral Research Program [HNRP] group is affiliated with the University of California, San Diego, the Naval Hospital, San Diego, and the Veterans Affairs San Diego Healthcare System, and includes: Director: Igor Grant, M.D.; Co-Directors: J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., and J. Allen McCutchan, M.D.; Center Manager: Thomas D. Marcotte, Ph.D.; Jennifer Marquie-Beck, M.P.H.; Melanie Sherman; Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Scott Letendre, M.D., Edmund Capparelli, Pharm.D., Rachel Schrier, Ph.D., Debra Rosario, M.P.H., Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), Mariana Cherner, Ph.D., Jennifer E. Iudicello, Ph.D., David J. Moore, Ph.D., Erin E. Morgan, Ph.D., Matthew Dawson; Neuroimaging Component: Terry Jernigan, Ph.D. (P.I.), Christine Fennema-Notestine, Ph.D., Sarah L. Archibald, M.A., John Hesselink, M.D., Jacopo Annese, Ph.D., Michael J. Taylor, Ph.D.; Neurobiology Component: Eliezer Masliah, M.D. (P.I.), Cristian Achim, M.D., Ph.D., Ian Everall, FRCPsych., FRCPath., Ph.D. (Consultant); Neurovirology Component: Douglas Richman, M.D., (P.I.), David M. Smith, M.D.; International Component: J. Allen McCutchan, M.D., (P.I.); Developmental Component: Cristian Achim, M.D., Ph.D.; (P.I.), Stuart Lipton, M.D., Ph.D.; Participant Accrual and Retention Unit: J. Hampton Atkinson, M.D. (P.I.); Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman (Data Systems Manager); Statistics Unit: Ian Abramson, Ph.D. (P.I.), Florin Vaida, Ph.D., Reena Deutsch, Ph.D., Anya Umlauf, M.S.
