Introduction
Patients with medically intractable epilepsy have known central nervous system dysfunction and often perform poorly on tests of intelligence, memory, and attention (for a review, see Lee, Reference Lee2010). Presurgical neuropsychological evaluation often helps to identify epilepsy patients who are at risk for postoperative cognitive decline (Busch & Naugle, Reference Busch and Naugle2008). For example, preoperative cognitive ability (e.g., memory) is one of the best predictors of cognitive outcome, particularly following temporal lobectomy (Busch et al., Reference Busch, Chapin, Umashankar, Diehl, Harvey, Naugle and Najm2008; Chelune, Naugle, Lüders, & Awad, Reference Chelune, Naugle, Lüders and Awad1991; Harvey et al., Reference Harvey, Naugle, Magleby, Chapin, Najm, Bingaman and Busch2008; Naugle, Chelune, Cheek, Lüders, & Awad, Reference Naugle, Chelune, Cheek, Lüders and Awad1993). Specifically, patients with low memory abilities before surgery are less likely to experience postsurgical memory declines after temporal lobectomy than those with intact presurgical memory functioning. Thus, it is important to ensure that preoperative test scores are valid indicators of a patient's actual cognitive ability, as poor effort on neuropsychological tests may lead to inappropriate expectations regarding postsurgical cognitive outcomes.
Accordingly, in both forensic and clinical neuropsychological evaluations, assessment of symptom validity is essential for ascertaining the validity of the neurocognitive results and maximizing the degree of confidence in the diagnoses, prognoses, and recommendations based upon such results (Bush et al., Reference Bush, Ruff, Troster, Barth, Koffler, Pliskin and Silver2005). While the specific methods may vary depending upon the circumstances of an evaluation, symptom validity assessment typically includes the examination of performance on specific tests/indices, as well as the consideration of consistencies (or lack thereof) among self-reported history, documented history, collateral information, behavioral observations, and known patterns of brain functioning.
The Victoria Symptom Validity Test (VSVT; Slick, Hopp, Strauss, & Thompson, Reference Slick, Hopp, Strauss and Thompson1997) is a forced-choice recognition task used to evaluate the validity of a patient's cognitive impairments, particularly self-reported memory deficits. On average, patients in a general clinical setting seeking compensation perform more poorly on the VSVT than their non-compensation-seeking counterparts (Doss, Chelune, & Naugle, Reference Doss, Chelune and Naugle1999; Grote et al., Reference Grote, Kooker, Garron, Nyenhuis, Smith and Mattingly2000). Moreover, some studies have reported that non-compensation-seeking patients with dense anterograde amnesia, severe memory impairment, and/or neurological disorders such as epilepsy are able to obtain perfect or near perfect scores on the VSVT (Grote, et al., Reference Grote, Kooker, Garron, Nyenhuis, Smith and Mattingly2000; Slick et al., Reference Slick, Tan, Strauss, Mateer, Harnadek and Sherman2003).
Loring, Lee, and Meador (Reference Loring, Lee and Meador2005) previously described VSVT performance in a sample of 120 non-litigating epilepsy surgical candidates. Patients with VSVT hard item scores categorized as invalid (<18/24 VSVT hard items) and questionable (18–20/24) demonstrated poorer performance on intelligence indices and multiple memory measures than patients with valid scores (>20/24). In addition to their findings regarding associations with intelligence and memory, Loring and colleagues reported that age was negatively correlated with VSVT hard item performance in their sample. Their study did not examine relationships between VSVT hard item performance and seizure history variables.
In light of observations that 13 of the 20 patients in their sample with FSIQ ≤69 had VSVT hard item scores within the invalid and questionable ranges, Loring and colleagues suggested a potential confound of low FSIQ on VSVT performance. At the same time, however, they acknowledged that low FSIQ scores alone could not explain low VSVT hard item scores since 6 patients with FSIQs between 60 and 69 obtained scores within the valid range. Providing additional modest support for the possibility of a confound of low FSIQ on VSVT hard item performance among epilepsy surgery candidates, subsequent research has observed that performance on a variety of symptom validity measures (which did not include VSVT) is inversely related to FSIQ in a heterogeneous sample of non-compensation-seeking neuropsychology clinic outpatients (Dean, Victor, Boone, & Arnold, Reference Dean, Victor, Boone and Arnold2008).
FSIQ is a global measurement composed of a variety of cognitive domains. It is unknown whether the influence of FSIQ on VSVT hard item performance is related to a particular cognitive domain versus overall intellectual functioning. Anecdotal clinical observations indicate that patients who perform poorly on the VSVT hard items often demonstrate concomitant working memory deficits, and based on the nature of the task, working memory is a likely cognitive domain to focus investigation. Furthermore, there has been little investigation into the effect of demographic and seizure variables on VSVT hard item performance.
The primary goal of the current investigation was to replicate and extend the study by Loring et al. in a larger sample of patients with medically intractable epilepsy (N = 404). Specifically, we further evaluated the relationship between VSVT hard item performance and intelligence by examining VSVT performance across different levels of FSIQ. In addition, given anecdotal clinical observations that patients who perform poorly on the VSVT hard items often demonstrate concomitant working memory deficits, many times in the context of intact episodic memory, we explored the potential mediating role of working memory in the relationship between VSVT hard item performance and FSIQ. Finally, we explored whether VSVT hard item performance varies as a function of demographic characteristics (i.e., age, education) and seizure history variables (i.e., age at seizure onset, duration of epilepsy).
Methods
Participants
This retrospective, IRB-approved study included 404 adult (≥18 years of age) patients (170 male, 42.1%; 368 Caucasian, 91.1%) with medically intractable focal epilepsy who had completed 6 or more years of education. The sample was derived from a consecutive case series of patients who were seen for comprehensive neuropsychological assessment between June 1988 and June 2011 as part of investigations to determine surgical candidacy. All patients had focal epilepsies (273 temporal, 55 frontal, 3 parietal, 2 occipital, 62 multi-lobar, and 9 non-localizable). The mean age at seizure onset was 19.72 years (SD = 14.17) and the mean duration of epilepsy was 18.72 years (SD = 14.11). Data regarding side of seizure onset were available for 92.3% (373/404) of the sample; of those, 149 had seizures arising from the right hemisphere, 212 from the left hemisphere, and 12 had bilateral seizure foci. Side of seizure onset was not systematically related to VSVT total score (F = 1.19; p = .31), VSVT easy item score (F = 1.68; p = .19), or VSVT hard item score (F = .726; p = .49). The patients ranged in age from 18 to 73 years (M = 38.56; SD = 12.09) and in education from 7 to 20 years (M = 13.27; SD = 2.15).
None of the patients in this clinically referred sample were known to be involved in litigation regarding their medical status or seeking financial compensation at the time of their neuropsychological evaluations. While it is possible that some patients may have had applications pending for disability, this scenario is unlikely given the mean seizure duration of 19 years. Furthermore, patients with intractable epilepsy would likely qualify for disability on the basis of their seizure severity, independent of any evidence of cognitive dysfunction. Theoretically, some patients could have had a non-financial motivation to perform poorly on testing (e.g., a desire to continue to be dependent on caregivers); this is unlikely, however, as most patients are highly motivated to undergo surgery.
Measures
As part of a comprehensive preoperative neuropsychological assessment, patients completed the Victoria Symptom Validity Test (VSVT; Slick et al., Reference Slick, Hopp, Strauss and Thompson1997), Wechsler Adult Intelligence Scale-Third Edition (WAIS-III; Wechsler, Reference Wechsler1997a), and Wechsler Memory Scale-Third Edition (WMS-III; Wechsler, Reference Wechsler1997b).
The VSVT manual provides recommended cut-off scores for hard memory items on the basis of binomial probability theory (Slick et al., Reference Slick, Hopp, Strauss and Thompson1997; see also Slick, Strauss, & Spellacy, Reference Slick, Strauss and Spellacy1996); hard item scores that are significantly above chance (i.e., 16–24), at chance (i.e., 8–15), and below chance (i.e., 0–7) have suggested interpretations of valid, questionable, and invalid, respectively. However, the VSVT hard item cut-scores used in the present investigation were those that were previously identified as maximally efficient in discriminating between compensation-seeking patients involved in personal injury lawsuits and non-compensation-seeking patients with medically intractable epilepsy (Grote et al., Reference Grote, Kooker, Garron, Nyenhuis, Smith and Mattingly2000). The use of the Grote et al. (Reference Grote, Kooker, Garron, Nyenhuis, Smith and Mattingly2000) cut-scores in the present investigation also allowed for direct comparison with the Loring et al. (Reference Loring, Lee and Meador2005) study. Thus, for the purposes of the current study, scores >20/24 on the VSVT hard items were classified as valid (n = 351), scores ranging from 18/24 to 20/24 were classified as questionable (n = 31), and scores of <18/24 were classified as invalid (n = 22).
Data Imputation
To avoid possible introduction of sampling bias due to missing data, we used the multiple data imputation procedure developed by Graham and Schafer (Graham & Schafer, Reference Graham and Schafer1999; Schafer, Reference Schafer2002). In this procedure, three sets of values are imputed for each individual case with missing data. In each data set, the missing values are estimated, with slightly different estimations for each missing value in each data set, and complete data remain consistent across data sets. Missing values are estimated using an iterative procedure based upon parameter estimates derived from the expectation maximization (EM) algorithm (Enders, Reference Enders2001). For the three imputed data sets, the standard error of imputations will tend to be only 1.04 times as wide as the standard error of an infinite number of data sets (Graham & Schafer, Reference Graham and Schafer1999). This indicates that the three data sets in the present study were likely to include highly similar imputed values for all of the missing data from the original data matrix.
Analytic Strategy
All statistical analyses were performed using Statistical Package for the Social Sciences (SPSS) software, version 19 (Chicago, IL, 2010). For all analyses, the criterion for statistical significance was set at p < .05.
Replication of Loring et al. (Reference Loring, Lee and Meador2005)
As in the Loring et al. (Reference Loring, Lee and Meador2005) study, a series of one-way ANOVAs were conducted on the WAIS-III and WMS-III Index scores to analyze differences in intellectual and memory performance among the three VSVT hard item score groups (i.e., valid, questionable, and invalid). Then, in cases where significance was found, a series of t tests were conducted to examine the differences in cognitive performance between the invalid and questionable groups and between the questionable and valid groups; differences between the invalid and valid groups were not examined. Finally, a series of correlations were calculated to determine the strength of the relationships between VSVT hard item score and: (a) demographic factors and (b) cognitive test scores, and, to extend findings, we also examined (c) seizure variables.
Extension and Further Exploration
Ordinary least squares (OLS) & quantile regression analyses
To further explore the relationship between FSIQ and VSVT hard item performance, we conducted both OLS and quantile regression analyses to examine the overall relationship between FSIQ and VSVT hard item performance as well as changes in this relationship at various points of the FSIQ distribution. Whereas OLS regression is used to assess the relationship between a continuous level independent variable (IV) and the mean of a continuous dependent variable (DV), quantile regression allows for the evaluation of the relationship of IVs across the full range of the DV (Koenker & Bassett, Reference Koenker and Bassett1978; Koenker & Hallock, Reference Koenker and Hallock2001). Consequently, computation of different regression coefficients via quantile regression across the conditional distribution of FSIQ can provide a more sophisticated understanding of how VSVT hard item performance and FSIQ are related. We first constructed an OLS regression model of VSVT hard item scores, constraining the relationship with FSIQ to be linear. We then constructed quantile regression models for the 10th, 20th, 30th, 40th, 50th, and 60th percentiles of FSIQ. Regression models for higher percentiles were not examined because of the truncated (i.e., negatively skewed) nature of the VSVT hard item score distribution.
Mediation analyses
To examine working memory as a potential mediator in the relationship between FSIQ and VSVT hard item performance, a series of OLS regression analyses were conducted according to the methods outlined by Baron and Kenny (Reference Baron and Kenny1986). Accordingly, three sets of OLS regression analyses were examined. In Model 1, the dependent variable (i.e., VSVT hard item score) was regressed on the independent variable (i.e., WAIS-III FSIQ). In Model 2, the mediator variable [i.e., WMS-III Working Memory Index (WMI)] was regressed on the independent variable (i.e., FSIQ). In Model 3, the dependent variable (i.e., VSVT hard item score) was regressed on both the independent variable (i.e., FSIQ) and the mediator variable (i.e., WMS-III WMI). The third model provided estimates of the relationship between the mediator and dependent variable as well as the relationship between the independent variable and the dependent variable controlling for the mediator.
The Sobel test (Sobel, Reference Sobel1982) was conducted to determine whether there was a statistically significant reduction in the variance explained by the independent variable when the proposed mediator was included in the model. The Sobel test uses unstandardized coefficients and standard errors of the independent variable-mediator and the mediator-dependent variable relationships to determine statistical significance. In the present study, mediation is conceptualized as simply statistical mediation rather than mediation based on the full MacArthur framework, where logical and statistical criteria are required to demonstrate mediation (Kraemer, Kiernan, Essex, & Kupfer, Reference Kraemer, Kiernan, Essex and Kupfer2008).
Results
Missing Data Analyses
Complete data were available for VSVT performance and all of the demographic characteristic variables (i.e., sex, age, and education). Approximately 0.5% (n = 2) of the seizure history data (i.e., age at seizure onset and duration of epilepsy) was missing.
Only 0.65% of all neuropsychological test data was missing, with the majority of missing data attributed to WMS-III WMI (smallest n = 397) and WMS-III Auditory Recognition Delayed Index (smallest n = 400). Over 98% of the sample had neuropsychological data from at least one of the neuropsychological measures besides the VSVT, and 97% (n = 392) of the sample completed all neuropsychological measures. To determine the influence of missing data on observed and imputed values, correlations between a dichotomous measure specifying whether data were complete or imputed was correlated with each variable, separately in each imputed data set. Correlations were then averaged across variables and across imputed data sets. On average, the potential influence of missing data on observed scores was modest in size (absolute value of range of r = 0.00035−0.08127; smallest p = .10; average r across all three imputed data sets = .03). The data were well-suited for multiple imputation procedures given completeness of the available data and the small relationship between missing data and observed imputed values. Analyses of the three imputed data sets and the original data produced a nearly identical pattern of results. For simplicity, the following results presented are those based on the first imputed data set (N = 404).
Replication of Loring et al. (Reference Loring, Lee and Meador2005)
Figure 1 presents the frequency distribution of scores obtained on VSVT hard items. The vast majority of patients (i.e., n = 351; 87%) scored between 21 and 24; 31 scored between 18 and 20 (8%), and 22 scored between 11 and 17 (5%). By far, the most common scores were 24/24 and 23/24 (n = 262). No participant obtained a score significantly below chance level (i.e., less than 8/24); the lowest VSVT hard item score obtained by any participant was 11. The average VSVT hard item score for the overall sample was 22.43 (SD = 2.35; median = 23; inter-quartile range = 2).
Frequency distribution of Victoria Symptom Validity Test (VSVT) hard item score (N = 404).
Table 1 presents the means and standard deviations of the demographic, seizure, and cognitive test performance variables for each of the three VSVT hard item performance groups. Table 1 also presents the results of a series of one-way ANOVAs in which VSVT hard item category was used as the grouping variable. A statistically significant group effect was observed for education, F = 3.30, p = .04. However, this corresponded to a small effect, partial η2 = .016 (Cohen, Reference Cohen1988). Contrary to the results of Loring et al. (Reference Loring, Lee and Meador2005), the current study did not observe a significant group effect for age. Statistically significant group effects were also observed for all of the WAIS-III IQ indices, as well as for all WMS-III memory indices. Medium effects were observed for WAIS-III FSIQ, F = 26.59, p < .001, partial η2 = .117, and WMS-III WMI, F = 30.15, p < .001, partial η2 = .131 (Cohen, Reference Cohen1988).
Demographic factors, seizure variables, and cognitive test performance as a function of VSVT hard item score categorization
*Imputed values are reported where appropriate.
aValid vs. questionable difference p < .001.
bValid vs. questionable difference p < .05.
cQuestionable vs. invalid difference p < .05.
dThere were no significant differences for either the valid and questionable groups or the questionable and invalid group comparisons.
According to a series of t tests, the valid group performed significantly better than the questionable group on the following WAIS-III and WMS-III Indices: Verbal IQ (p < .05), Performance IQ (p < .001), Full Scale IQ (p < .001), Auditory Immediate (p < .05), Immediate Memory (p < .05), Auditory Delayed Recognition Memory (p < .05), and Working Memory (p < .001). In addition, the questionable group performed significantly better than the invalid group on measures of Verbal IQ, Performance IQ, Full Scale IQ, Visual Immediate, Visual Delay, and Working Memory (all p's<.05). There were no significant effects of education, age at seizure onset, or duration of epilepsy when comparing the valid versus questionable groups or the questionable versus invalid groups.
Table 2 presents Pearson's r correlations between VSVT hard item score and the demographic factors, seizure variables, and cognitive test performance variables. VSVT hard item score was significantly correlated with education (r = .169; p = .001). In exploratory analyses, there were no significant group effects of either age at seizure onset or duration of epilepsy on VSVT hard item scores among the valid, questionable, and invalid groups. In addition, there were no significant correlations between VSVT hard item score and either of the seizure history variables considered in the current investigation.
Correlations between VSVT hard item score and demographic characteristics, seizure history variables, and cognitive test performance
Note. According to Cohen (Reference Cohen1992), Pearson's r values of 0.10 and 0.30 are the minimum thresholds for a small* and medium** effect size, respectively.
However, as in the investigation by Loring et al. (Reference Loring, Lee and Meador2005), significant positive correlations were found between VSVT hard items and all indices of intellectual and memory performance. Of note, VSVT hard item scores correlated most highly with FSIQ (Pearson's r = .378) and WMS-III WMI (r = .391); both correlations corresponded to a medium effect size (Cohen, Reference Cohen1992). Other WMS-III Indices, namely General Memory, Auditory Immediate Memory, and Visual Immediate Memory Indices also demonstrated medium-sized correlations.
Extension of Previous Work
OLS & quantile regression analyses
Figure 2 presents a summary of the OLS and quantile regression results for the model; the left and right panels depict the intercept and slope (i.e., unstandardized regression coefficients) estimates, respectively. According to the OLS regression model, higher FSIQ scores were associated with higher VSVT hard item scores (B = .061; SE = .007; β = .378; R 2 = .143; adjusted R 2 = 0.141; p < .001). The magnitudes of the estimated intercept decreased across the quantiles (i.e., from the lowest to highest quantiles; see left panel of Figure 2), simply reflecting that lower FSIQ levels were observed across FSIQ quantiles. Unstandardized quantile regression coefficients decreased from lower to higher quantiles (see the right panel of Figure 2), reflecting a decreasing magnitude of the relationship between VSVT hard scores and FSIQ at higher levels of FSIQ. Of note, the quantile regression coefficients for the first two quantiles (10th and 20th percentiles) fell above the 95% CI for the OLS regression estimates. Thus, in these low levels of FSIQ, the relationship between VSVT hard item scores and FSIQ was considerably stronger than observed for the full sample. Conversely, for the final three quantiles (40th, 50th, 60th) the relationship was substantively weaker than in the full sample.
OLS and quantile regression estimates of intercept (left panel) and β coefficients (right panel) for VSVT hard item scores with FSIQ as predictor. OLS = ordinary least squares; CI = confidence interval; VSVT = Victoria Symptom Validity Test; FSIQ = Full Scale IQ; β = regression coefficient.
Figure 3 presents a summary of the overall OLS versus quantile regression model fits for VSVT hard item scores with FSIQ as a predictor. When compared to the quantile regression results, the OLS regression model underestimated the influence of lower FSIQ values on VSVT hard item scores and over estimated the relationship for higher FSIQ values. This figure highlights the need to consider FSIQ ranges when conceptualizing the relationship between general ability and VSVT hard item performance.
Overall model fit VSVT hard item scores with FSIQ as predictor: ordinary least squares versus quantile regression models. VSVT = Victoria Symptom Validity Test; FSIQ = Full Scale IQ; OLS = ordinary least squares.
Mediation analyses
Three sets of OLS regression analyses were examined to determine whether working memory mediates the relationship between FSIQ and VSVT hard item performance; see Tables 3 and 4. First, as presented in the preceding section, the independent variable (FSIQ) was significantly related to the dependent variable (VSVT hard item score). Higher FSIQ scores were associated with higher VSVT hard item scores; see Table 3, Model 1. Second, the independent variable (FSIQ) was significantly associated with the mediator variable (WMS-III WMI); see Table 4, Model 2. Third, the mediator (WMS-III WMI) was significantly related to the dependent variable (VSVT hard item score); see Table 3, Model 3. Of note, when the mediator was included in Block 2 of Model 3, the standardized coefficient (i.e., β weight) for the FSIQ decreased from 0.378 to 0.199; this represents a statistically significant decrease (Sobel test statistic = 3.327; p < .001).
OLS regression analyses predicting VSVT Hard Item score
Notes. OLS = ordinary least squares; VSVT = Victoria Symptom Validity Test; WAIS-III FSIQ = Wechsler Adult Intelligence Scale (3rd edition) Full Scale IQ; WMS-III WMI = Wechsler Memory Scale (3rd edition) Working Memory Index.
*p < .01, **p < .001.
OLS regression analyses predicting WMS-III Working Memory score
Notes. OLS = ordinary least squares; VSVT = Victoria Symptom Validity Test; WAIS-III FSIQ = Wechsler Adult Intelligence Scale (3rd edition) Full Scale IQ; WMS-III WMI = Wechsler Memory Scale (3rd edition) Working Memory Index.
*p < .01, **p < .001.
Among the WMS-III indices, WMI demonstrated the strongest correlation with VSVT hard item scores, followed by FSIQ. As noted above, other WMS-III Indices also demonstrated medium effect size correlations, namely General Memory, Auditory Immediate Memory, and Visual Immediate Memory Indices. However, exploratory analyses of these WMS-III Indices provided no evidence of mediation in the relationship between FSIQ and VSVT hard item performance.
Discussion
Results from the present study provide further evidence that VSVT hard item performance is related to measures of intelligence and memory among epilepsy surgery candidates with no known external incentive to perform poorly. Thus, the current results largely replicate the findings of Loring et al. (Reference Loring, Lee and Meador2005) in a larger sample of patients. Statistically significant group effects among the valid, questionable, and invalid groups were observed for all of the WAIS-III IQ and WMS-III memory indices. More specific between-group comparisons revealed several significant differences between the valid and questionable groups, as well as between the questionable and invalid groups (see Table 1). Although all WAIS-III and WMS-III Indices demonstrated significant correlations with VSVT hard item performance, FSIQ and WMS-III WMI correlated most highly with VSVT hard item scores. The correlation observed between VSVT hard item score and WMS-III WMI is not surprising given the task demands of the VSVT hard items—that is, asking the patient to hold a 5-digit number within the working memory buffer for up to 15 s before attempting to discriminate between the target 5-digit string and another 5-digit foil in which only the middle digits have been transposed. Thus, the use of other symptom validity measures [e.g., Test of Memory Malingering (Tombaugh, Reference Tombaugh1996), Word Memory Test (Green, Allen, & Astner, Reference Green, Allen and Astner1996)] that place greater demand on declarative memory than working memory (Merten, Bossink, & Schmand, Reference Merten, Bossink and Schmand2007) may be more appropriate in epilepsy patients and other neurologic populations in which working memory deficits are common.
To address the hypothesis of Loring et al. (Reference Loring, Lee and Meador2005) of a potential confound of low FSIQ on VSVT hard item performance among epilepsy surgery candidates, we examined regression analyses of VSVT hard item scores at the mean (via OLS regression) as well as at various points of the FSIQ distribution (via quantile regression). Compared to quantile regression results, the OLS regression model underestimated the influence of lower FSIQ values on VSVT hard item scores. Consequently, consideration of OLS regression results alone would mask the larger relationship at lower FSIQ scores (see Figure 3).
Extending the findings by Loring et al. (Reference Loring, Lee and Meador2005), results of the current investigation indicate that working memory is an important mediator in the relationship between VSVT hard item performance and FSIQ. Although other WMS-III Indices also demonstrated medium-sized correlations with VSVT hard scores, exploratory analyses did not provide similar evidence of partial mediation of the association between FSIQ and VSVT hard item performance. The finding of partial mediation for WMS-III WMI suggests that VSVT hard item performance is likely impacted to some extent by cognitive ability, particularly among those with lower FSIQ. Thus, low VSVT hard item scores may not always be merely an indication of frank malingering or insufficient effort, but may instead reflect significant deficits in working memory among candidates for epilepsy surgery. At the same time, however, it is also possible that poor VSVT hard item performance among some epilepsy patients may be associated with other factors that can affect performance on measures of intelligence and working memory—for example, antiepileptic drug (AED) regimen side effects, as well as the type, location, and frequency of seizures (Loring, Marino, & Meador, Reference Loring, Marino and Meador2007; Meador, Reference Meador2002; Motamedi & Meador, Reference Motamedi and Meador2003; Ortinski & Meador, Reference Ortinski and Meador2004). Also of note, additional research suggests that recent seizure activity may confer greater risk of failure on symptom validity testing (Williamson et al., Reference Williamson, Drane, Stroup, Wilensky, Holmes and Miller2005).
Unlike the findings in the study by Loring et al. (Reference Loring, Lee and Meador2005), the current investigation did not observe a significant association between age and VSVT hard item score. Interestingly, Loring et al. (Reference Loring, Lee and Meador2005) described their finding of negative correlation between age and VSVT hard item performance as “not anticipated” (p. 615). Despite its statistical significance, the magnitude of the age-VSVT correlation in the study by Loring et al. (Reference Loring, Lee and Meador2005) was relatively small (r = −.24) according to Cohen (Reference Cohen1992) criteria. Also of note, there was no significant correlation between age and VSVT hard item performance in another, albeit smaller, sub-sample of 30 non-compensation-seeking epilepsy patients in a separate investigation (Grote et al., Reference Grote, Kooker, Garron, Nyenhuis, Smith and Mattingly2000).
VSVT hard item scores were significantly correlated with education; this corresponded to a small effect (see Table 2). In addition, a small, statistically significant group effect among the valid, questionable, and invalid groups was observed for education (see Table 1). However, specific between-group comparisons revealed no significant differences for education between either the valid and questionable groups or the questionable and invalid groups. Moreover, consideration of the modest differences among the group means argues against the clinical significance of the observed group effect.
Clinical Utility of Current Findings
In summary, the current study suggests that the VSVT hard item scores may be suppressed among patients with medically intractable epilepsy for reasons other than malingering. Thus, when evaluating epilepsy surgery candidates, clinicians are encouraged to interpret VSVT hard item performance in the context of a patient's intellectual and working memory capabilities, as well as their performance on episodic memory tasks. When the VSVT hard item score is within the normal range and there are no other apparent symptom validity concerns/profile inconsistencies, clinicians may generally assume that the cognitive profile is valid. However, when VSVT hard item performance is poor, the cognitive profile should not automatically be judged invalid. Instead, clinicians will need to consider the possibility that lower performance on the VSVT hard items may be due to intellectual or working memory limitations. In these cases, it may be useful to administer symptom validity measures that require less working memory capacity (if not already completed) and look for convergence of findings.
Limitations and Future Directions
A few limitations of the current study warrant brief mention, and suggestions for clinical practice and future research are offered. First, the present investigation considered only one ‘stand alone’ symptom validity instrument, the VSVT. However, we conducted some brief exploratory analyses to examine VSVT hard item scores in relation to performance on so-called “embedded” validity measures. Among those participants who had data available to calculate Mittenberg's index [i.e., Vocabulary Age-Corrected Scaled Score (ACSS) minus Digit Span ACSS; Mittenberg et al. (Reference Mittenberg, Theroux, Aguila-Puentes, Bianchini, Grere and Rayls2001); n = 352], VSVT hard item score was not significantly correlated with Mittenberg's index (r = −0.04; p = .45). Interestingly, only 2 of the 22 (9.1%) participants with VSVT hard item scores categorized as invalid based on the Grote et al. (Reference Grote, Kooker, Garron, Nyenhuis, Smith and Mattingly2000) criterion had a Mittenberg index value above the cut-off of 4. There was a significant correlation between WMS-III WMI and Mittenberg index (r = −.173; p = 001); however, the magnitude of this effect (considered “small” per Cohen, Reference Cohen1992) is much smaller than the relationship between VSVT hard item performance and WMI (medium effect size). Also of note, among those participants who had data available regarding WCST failure-to-maintain-set (FMS; n = 391), VSVT hard item performance was not significantly correlated with WCST FMS (r = .004; p = .93). Future studies should investigate the impact of IQ and working memory on other symptom validity measures. It may be particularly worthwhile to simultaneously compare multiple symptom validity measures in this population.
Second, data regarding participants’ AED regimens (number and type of AED, dosage level, etc.) were not available for analysis in the current study. Certain AEDs (e.g., topiramate) may have a greater impact on cognition, and working memory in particular, than others (Kim, Lee, Jung, Suh, & Park, Reference Kim, Lee, Jung, Suh and Park2006; Kockelmann, Elger, & Helmstaedter, Reference Kockelmann, Elger and Helmstaedter2004; Sommer & Fenn, Reference Sommer and Fenn2010). The risk for cognitive side effects increase with polypharmacy (vs. monotherapy), and some AED-related cognitive deficits tend to emerge in a dose-dependent manner (Loring et al., Reference Loring, Marino and Meador2007; Ortinski & Meador, Reference Ortinski and Meador2004). Thus, future studies should examine the potential role of medication side effects.
Finally, additional studies are needed that more carefully evaluate the relationship between symptom validity test performance and cognitive abilities in diverse neurologically impaired populations. These studies would benefit from adopting methodological approaches that focus on multiple symptom validity measures, using distinct cognitive paradigms. This work would also benefit from adopting statistical approaches that assist in determining the precise nature of these relationships across score distributions (e.g., quantile regression) or empirical approaches that focus on identifying distinct patterns of performance on symptom validity measures/indices (e.g., classification tree, taxometric, latent class, or mixture modeling analyses). In the absence of a flawless gold-standard evaluation of symptom validity, clinical approaches should continue to prioritize the most valid information (standardized symptom validity indices), but also take into account possible clinical factors that may decrease the value of these indices.
Acknowledgments
There are no financial or other relationships that could be interpreted as a conflict of interest affecting this manuscript. In addition, there are no sources of financial support for this manuscript. The authors would like to thank Adrian Johnson for his assistance with preparation of the figures.
