In response to the high rate of traumatic brain injury (TBI) in service members deployed to Iraq and Afghanistan (Hoge et al., Reference Hoge, McGurk, Thomas, Cox, Engel and Castro2008; Terrio et al., Reference Terrio, Brenner, Ivins, Cho, Helmick, Schwab, Scally, Bretthauer and Warden2009; Wilk et al., Reference Wilk, Thomas, McGurk, Riviere, Castro and Hoge2010), the US Veterans Health Administration instituted a system to screen and evaluate brain injury for all Veterans who served in the recent conflicts in Iraq and Afghanistan. Although the majority of individuals with mild TBI experience full recovery of post-concussive symptoms (PCSs) days to weeks following injury, a sizable minority continue to experience PCS months to years past the expected window of recovery (Lew et al., Reference Lew, Otis, Tun, Kerns, Clark and Cifu2009; Vanderploeg, Curtiss, & Belanger, Reference Vanderploeg, Curtiss and Belanger2005). Common comorbid conditions such as posttraumatic stress disorder (PTSD) and depression contribute notably to PCS reporting (Andrews, Fonda, Levin, McGlinchey, & Milberg, Reference Andrews, Fonda, Levin, McGlinchey and Milberg2018; Belanger, Kretzmer, Vanderploeg, & French, Reference Belanger, Kretzmer, Vanderploeg and French2010; Meares et al., Reference Meares, Shores, Taylor, Batchelor, Bryant, Baguley, Chapman, Gurka and Marosszeky2011), further complicating diagnosis and treatment of PCS.
Neuropsychological evaluation is often recommended to patients with persistent PCS to aid in diagnostic clarification and provide treatment recommendations. However, failure to adequately engage in testing, as measured by performance validity tests (PVTs) occurs in 17–68% of clinical evaluations of Veterans (Armistead-Jehle, Reference Armistead-Jehle2010; Jak et al., Reference Jak, Gregory, Orff, Colón, Steele, Schiehser, Delano-Wood, Jurick and Twamley2015; Lippa et al., Reference Lippa, Pastorek, Romesser, Linck, Sim, Wisdom and Miller2014; Russo, Reference Russo2012; Whitney, Shepard, Williams, Davis, & Adams, Reference Whitney, Shepard, Williams, Davis and Adams2009), with the highest PVT failure rates among mTBI and PTSD referrals (Young, Roper, & Arentsen, Reference Young, Roper and Arentsen2016). PVT failure is associated with poor outcomes such as lower community reintegration (Lippa et al., Reference Lippa, Pastorek, Romesser, Linck, Sim, Wisdom and Miller2014), increased healthcare utilization (Horner, VanKirk, Dismuke, Turner, & Muzzy, Reference Horner, VanKirk, Dismuke, Turner and Muzzy2014), and higher financial burden on the healthcare system (Denning & Shura, Reference Denning and Shura2019), as well as artificially reduced cognitive test scores (Grills & Armistead-Jehle, Reference Grills and Armistead-Jehle2016) and elevated reporting of psychiatric symptoms (Jurick et al., Reference Jurick, Twamley, Crocker, Hays, Orff, Golshan and Jak2016; Larrabee, Reference Larrabee2003; Suchy, Chelune, Franchow, & Thorgusen, Reference Suchy, Chelune, Franchow and Thorgusen2012). PVT failure can invalidate neuropsychological test results and lead to inconclusive findings (Bush et al., Reference Bush, Ruff, Tröster, Barth, Koffler, Pliskin, Reynolds and Silver2005; Carone, Iverson, & Bush, Reference Carone, Iverson and Bush2010; Heilbronner, Sweet, Morgan, Larrabee, Millis, Reference Heilbronner, Sweet, Morgan, Larrabee and Millis2009), providing clinicians with limited information to reach diagnostic conclusions and offer treatment recommendations to the patient and providers involved with their care.
Certainly, there are many potential factors contributing to PVT failures. Iraq and Afghanistan Veterans frequently present with high levels of psychiatric distress, sleep disturbances, chronic pain, and other psychosocial stressors (Balba et al., Reference Balba, Elliott, Weymann, Opel, Duke, Oken, Morasco, Heinricher and Lim2018; Hoge et al., Reference Hoge, McGurk, Thomas, Cox, Engel and Castro2008; Seal et al., Reference Seal, Bertenthal, Samuelson, Maguen, Kumar and Vasterling2016; Stojanovic et al., Reference Stojanovic, Fonda, Fortier, Higgins, Rudolph, Milberg and McGlinchey2016) that can greatly impact engagement in the assessment process and contribute to PVT failure (Jak et al., Reference Jak, Gregory, Orff, Colón, Steele, Schiehser, Delano-Wood, Jurick and Twamley2015; Lange, Pancholi, Bhagwat, Anderson-Barnes, & French, Reference Lange, Pancholi, Bhagwat, Anderson-Barnes and French2012). Thus, when neuropsychologists encounter PVT failure in a clinical context, a referral for psychotherapy may be made to address underlying factors such as mental health symptoms that could be contributing to a patient’s distress and/or PVT failure (Carone et al., Reference Carone, Iverson and Bush2010). However, there is at present little empirical evidence regarding the efficacy of psychotherapy following invalid PVT performance. Given that these mental health factors can impact engagement in the assessment process, it is possible that these factors also affect engagement in psychotherapy. Thus, an important question to be addressed is whether those with invalid PVT performance can benefit from psychotherapy interventions to the same degree as those with valid PVT performance, and moreover, whether treatment subsequently leads to better test engagement and improved PVT performance.
Although psychotherapy interventions have not been well examined in the context of invalid neuropsychological test performance, providing education regarding symptom management and trajectory in the early period following brain injury has been shown to be effective in improving outcomes in subgroups of individuals at risk for poor outcomes following mTBI (Snell, Surgenor, Hay-Smith, & Siegert, Reference Snell, Surgenor, Hay-Smith and Siegert2009). Notably, many Veterans with a history of mTBI and their family members hold incorrect beliefs regarding mTBI, and only a small minority ever receive brain injury education while in the military (Block et al., Reference Block, Fabrizio, Bagley, Hannah, Camp, Mindingall, Labbe and Lokken2014). In the post-acute and chronic phase following mTBI, preliminary studies evaluating combined approaches to treatments (e.g., psychoeducation with concurrent mental health treatment and multidisciplinary treatments including cognitive rehabilitation and integrated behavioral health care) have demonstrated improvement in PCS and mental health symptoms (Belanger et al., Reference Belanger, Barwick, Silva, Kretzmer, Kip and Vanderploeg2015; Janak et al., Reference Janak, Cooper, Bowles, Alamgir, Cooper, Gabriel, Pérez and Orman2017). Furthermore, a growing body of research suggests that objective cognitive performance can also improve following cognitive rehabilitation in Veterans with chronic PCS complaints following mTBI (Cooper et al., Reference Cooper, Bowles, Kennedy, Curtiss, French, Tate and Vanderploeg2017; Storzbach et al., Reference Storzbach, Twamley, Roost, Golshan, Williams, O’Neil, Jak, Turner, Kowalski, Pagulayan and Huckans2017). However, the studies described above did not report the rate of PVT performance in their samples, with the exception of Cooper and colleagues who had an exceptionally small subgroup who failed a PVT (n = 5/126; <4%).
Our group recently published a randomized controlled trial comparing SMART-CPT, a novel hybrid treatment combining cognitive processing therapy (CPT) for PTSD with components of compensatory cognitive training from Cognitive Symptom Management and Rehabilitation Therapy (CogSMART; Twamley, Jak, Delis, Bondi, & Lohr, Reference Twamley, Jak, Delis, Bondi and Lohr2014) to standard CPT in Veterans with PTSD and history of mild-to-moderate TBI (Jak et al., Reference Jak, Jurick, Crocker, Sanderson-Cimino, Aupperle, Rodgers, Thomas, Boyd, Norman, Lang, Keller, Schiehser and Twamley2019). Findings demonstrated comparative improvements in mental health symptoms including PCS and quality of life across conditions; however, those in the SMART-CPT condition showed greater improvement in objective cognitive performance than those in the standard CPT condition. Although excluding those with poor PVT performance at baseline did not significantly alter the results, the extent to which clinically meaningful improvements in the symptom measures occurred for those who failed PVTs was not specifically examined in this study. Furthermore, whether rates of PVT failure improved following treatment and whether these outcomes differed based on condition (CPT vs. SMART-CPT) are questions still to be addressed. Thus, the purpose of the current study was to determine whether (1) PVT status at baseline impacted improvement in mental health symptoms including PCS; (2) lower rates of PVT failure were observed following treatment; and (3) SMART-CPT reduced PVT failure rates to a greater degree than CPT.
METHOD
Participants and Procedures
The current study was a secondary analysis from a study described previously (Jak et al., Reference Jak, Jurick, Crocker, Sanderson-Cimino, Aupperle, Rodgers, Thomas, Boyd, Norman, Lang, Keller, Schiehser and Twamley2019). Participants were enrolled if they (1) were Iraq or Afghanistan Veterans with PTSD [confirmed by chart review of a diagnostic clinical interview performed by a mental health professional or the administration of a structured clinical interview, the Clinician Administered PTSD Scale for DSM-IV (CAPS; Blake et al., Reference Blake, Weathers, Nagy, Kaloupek, Charney and Keane1995)], (2) had a history of mild-to-moderate TBI (defined by VA/DOD criteria; Management of Concussion-mTBI Working Group, 2016), (3) reported current cognitive complaints, and (4) were stable on psychiatric medications for at least 6 weeks. Exclusion criteria included history of severe TBI, current substance dependence, suicidal intent or attempt within the past month, psychotic disorder, dementia, current participation in other intervention studies, or more than five prior sessions of CPT or CogSMART. Participants were recruited from various VA San Diego Healthcare System (VASDHS) clinics, Veterans’ centers, and local colleges through informational sessions, advertisements, and clinician referrals (see Jak et al., Reference Jak, Jurick, Crocker, Sanderson-Cimino, Aupperle, Rodgers, Thomas, Boyd, Norman, Lang, Keller, Schiehser and Twamley2019 for full information regarding reasons for exclusion throughout the enrollment process and differences between treatment completers and non-completers). The present study was approved by the VASDHS Human Research Protection Program and University of California, San Diego Institutional Review Board.
Veterans were randomly assigned to one of the two 12-week treatment conditions: (1) standard CPT or (2) CPT with embedded cognitive rehabilitation strategies from CogSMART (SMART-CPT) including psychoeducation about TBI and simplified CPT worksheets. Participants received comprehensive neuropsychological assessment including PVTs and mental health measures at baseline, immediately following treatment completion, and 3 months later (6 months following baseline assessment). Notably, neither condition included any feedback regarding performance on neuropsychological assessment nor specifically addressed the importance of trying one’s hardest during the assessments. In addition to the three assessment visits, participants completed a PTSD symptom questionnaire each week of therapy and a PCS questionnaire 6 weeks into treatment.
Measures
TBI Characteristics
The Warrior Administered Retrospective Casualty Assessment Tool (Terrio et al., Reference Terrio, Brenner, Ivins, Cho, Helmick, Schwab, Scally, Bretthauer and Warden2009) is a structured interview that was used to gather information regarding TBI characteristics. Information collected regarding TBI included number and mechanism of TBIs, and presence and duration of loss of consciousness (LOC), alteration of consciousness, and post-traumatic amnesia (PTA).
Performance validity tests
The following measures were used to assess performance validity: Test of Memory Malingering (TOMM; Tombaugh, Reference Tombaugh1996), California Verbal Learning Test-Second Edition (CVLT-II) Forced Choice (Delis, Kramer, Kaplan, & Ober, Reference Delis, Kramer, Kaplan and Ober2000), and Reliable Digit Span from the Wechsler Adult Intelligence Scale (Wechsler, Reference Wechsler2008). Veterans were included in the performance invalid (PVT-Fail) group if they performed below 41 on TOMM Trial 1 (Denning, Reference Denning2012), 45 on TOMM Trial 2 and Retention Trial (Tombaugh, Reference Tombaugh1996), 15 on CVLT-II Forced Choice Recognition (Moore and Donders, Reference Moore and Donders2004), and/or 7 on Reliable Digit Span (Spencer et al., Reference Spencer, Axelrod, Drag, Waldron-Perrine, Pangilinan and Bieliauskas2013). Veterans were included in the performance valid group (PVT-Pass) if they performed above cutoffs on all PVT indices. These measures have been widely used to detect poor performance validity in Veterans with TBI history (Flaherty, Spencer, Drag, Pangilinan, & Bieliauskas, Reference Flaherty, Spencer, Drag, Pangilinan and Bieliauskas2015; Jak et al., Reference Jak, Gregory, Orff, Colón, Steele, Schiehser, Delano-Wood, Jurick and Twamley2015; Lippa, Reference Lippa2018, Whitney, Davis, Shepard, Bertram, & Adams, Reference Whitney, Davis, Shepard, Bertram and Adams2009; Young et al., Reference Young, Roper and Arentsen2016) and have shown adequate sensitivity, specificity (false positive rates <10%), and positive predictive power (Denning, Reference Denning2012, Reference Denning2014; Fazio, Denning, & Denney, Reference Fazio, Denning and Denney2017; Haber & Fichtenberg, Reference Haber and Fichtenberg2006; Kulas, Axelrod, & Rinaldi, Reference Kulas, Axelrod and Rinaldi2014; Schwartz et al., Reference Schwartz, Erdodi, Rodriguez, Ghosh, Curtain, Flashman and Roth2016; Spencer et al., Reference Spencer, Axelrod, Drag, Waldron-Perrine, Pangilinan and Bieliauskas2013; Tombaugh, Reference Tombaugh1997; Young, Sawyer, Roper, & Baughman, Reference Young, Sawyer, Roper and Baughman2012). Furthermore, it has been recommended that failure of even one PVT should warrant consideration of performance invalidity, particularly for tests with adequate positive predictive power used in populations with relatively high base rates including individuals with remote history of mTBI (Denning, Reference Denning2019; Inman & Berry, Reference Inman and Berry2002; Iverson & Franzen, Reference Iverson and Franzen1996; Lippa, Reference Lippa2018; Proto et al., Reference Proto, Pastorek, Miller, Romesser, Sim and Linck2014; Vickery et al., Reference Vickery, Berry, Dearth, Vagnini, Baser, Cragar and Orey2004).
Mental health symptoms
The 17-item PTSD Symptom Checklist – Specific Trauma (PCL-S) was used to assess PTSD symptoms consistent with DSM-IV criteria for PTSD over the past week (Weathers, Litz, Herman, Huska, & Keane, Reference Weathers, Litz, Herman, Huska and Keane1993). There were up to 15 data points for the PCL-S (baseline, 12 weeks of treatment, post-treatment, and follow-up). The 22-item Neurobehavioral Symptom Inventory (NSI) was used to assess self-reported cognitive, emotional, affective, and somatosensory PCS over the past 4 weeks (Cicerone & Kalmar, Reference Cicerone and Kalmar1995). There were four data points for the NSI (baseline, 6 weeks into treatment, post-treatment, and follow-up). The Beck Depression Inventory-II (BDI-II) is a 21-item measure designed to assess depressive symptomatology over the past 2 weeks (Beck, Steer, & Brown, Reference Beck, Steer and Brown1996). The BDI-II was collected at three time points (baseline, post-treatment, and follow-up). For each measure, a total score was derived by summing all items, in which higher scores represented greater symptoms.
Estimate of premorbid intellectual functioning
The Wide Range Achievement Test—Fourth Edition (WRAT-4; Wilkinson & Robertson, Reference Wilkinson and Robertson2006) reading subtest was used to assess premorbid intellectual functioning. Given associations between low intellectual functioning and poor performance on PVTs (for review, see Lippa, Reference Lippa2018), all analyses were replicated controlling for WRAT-4 performance to ensure that low premorbid intellectual functioning was not contributing to results.
Statistical Analyses
Consistent with prior work (Jak et al., Reference Jak, Jurick, Crocker, Sanderson-Cimino, Aupperle, Rodgers, Thomas, Boyd, Norman, Lang, Keller, Schiehser and Twamley2019; Crocker et al., Reference Crocker, Jurick, Thomas, Keller, Sanderson-Cimino, Boyd, Rodgers, Twamley and Jak2018), multilevel modeling (MLM) was used to address the first aim of determining whether the PVT-Fail group demonstrated similar improvement in mental health symptoms (including PCS) compared to the PVT-Pass group. MLM has several advantages over repeated measures ANOVA including the ability to take into account that repeated measurements across time are not independent but nested within individuals and utilize cases with missing data without using biased procedures (Schafer & Graham, Reference Schafer and Graham2002; Singer & Willett, Reference Singer and Willett2003; Woodard, Reference Woodard2017). Thus, all available data were used and no interpolation was conducted. Separate MLMs using a full information maximum likelihood method with the intent-to-treat sample were tested for the PCL-S, NSI, and BDI-II, with all possible time points evaluated (15, 4, and 3, respectively). Each MLM model included a random intercept and fixed effects of time, treatment condition, treatment condition × time interaction, PVT group, and PVT group × time (interaction of interest). Follow-up analyses explored whether treatment condition (standard CPT vs. SMART-CPT) moderated the interaction between PVT group and time on symptom change for all three symptom measures. Therefore, the three-way interaction (PVT group × time × treatment condition) as well as the PVT group × treatment condition interaction (to include all relevant two-way interactions) were added to the models described above.
To address the second aim of determining whether PVT failure rates reduced following treatment, generalized estimating equation (GEE) was employed. GEE is an extension of the general linear model that allows for repeated categorical outcome measures in which measurements on the same individual are assumed to influence the estimation of model parameters (Heck, Thomas, & Tabata, Reference Heck, Thomas and Tabata2013; Liang & Zeger, Reference Liang and Zeger1986). An exchangeable working correlation structure assumes that correlations are the same over each time interval and was selected based on the Quasi-Information Criterion which measures the relative goodness-of-fit of the data for GEE (Burnham & Anderson, Reference Burnham and Anderson2002; Pan, Reference Pan2001). In the binary logistic model, time, treatment condition (CPT vs. SMART-CPT), and time × treatment condition interaction were entered as predictor variables and PVT performance was entered as the dependent variable. Time was modeled as a continuous variable in all analyses. All analyses were conducted in SPSS version 25.
RESULTS
Table 1 provides descriptive information regarding demographics, mental health symptoms, and PVT performance assessed at baseline for the entire sample (N = 100) and each PVT group. The PVT-Pass group had higher premorbid intellectual functioning (t(96) = −2.14, p = .035) and lower PTSD (t(96) = 2.36, p = .020), PCS (t(96) = 2.71, p = .008) and depressive symptom severity (t(95) = 2.47, p = .015) compared to the PVT-Fail group. The PVT groups did not differ with regard to age, education, gender, presence of LOC or PTA, number of TBIs, severity of TBI injury (mild vs. moderate), or rates of treatment completion (all p’s > .05). As shown in Table 1, 57.9% of the PVT-Pass group and 46.5% of the PVT-Fail group completed treatment. Rates of treatment completion did not differ as a function of treatment condition (48.9% of the CPT and 56.9% of the SMART-CPT group completed treatment;
$\({\chi ^2} = .623(1)\)$
, p = .430). See Table 2 for detailed information about PVTs and number of participants in each treatment condition at each assessment.
Participant characteristics by group
Note: *p < .05; PVT = Performance Validity Test; WRAT4 = Wide Range Achievement Test 4; TBI = traumatic brain injury; PCL-S = PTSD Checklist – Specific Trauma; NSI = Neurobehavioral Symptom Inventory; BDI-II = Beck Depression Inventory – Second Edition; TOMM = Test of Memory Malingering; CVLT-II = California Verbal Learning Test – Second Edition; WAIS-IV = Wechsler Adult Intelligence Scale – Fourth Edition.
Symptom severity and performance validity measures at each assessment by treatment group
CPT = Cognitive Processing Therapy; SMART-CPT = Cognitive Symptom Management and Rehabilitation Therapy combined with CPT; PCL-S = PTSD Checklist – Specific Trauma; NSI = Neurobehavioral Symptom Inventory; BDI-II = Beck Depression Inventory – Second Edition; TOMM = Test of Memory Malingering; CVLT-II = California Verbal Learning Test – Second Edition.
With regard to the first aim of evaluating whether the PVT-Fail and PVT-Pass groups equally benefited from treatment in terms of symptom change, Table 3 presents the relevant MLM parameter estimates, p values, and effect sizes (reported as r values, small = 0.10; medium = 0.30; large = 0.50) for each MLM conducted. In terms of PTSD symptom severity (PCL-S), there was a main effect of time (b = −1.64, SE = .11, p < .001, r = .57) as well as a PVT group × time interaction (b = .69, SE = .13, p < .001, r = .24). Figure 1a illustrates this model-predicted interaction, which appears to show that individuals in both groups displayed a clinically significant decrease in PTSD symptoms (at least 10 points; Monson et al., Reference Monson, Gradus, Young-Xu, Schnurr, Price and Schumm2008); however, those in the PVT-Pass group exhibited a greater reduction in symptoms.
Parameter Estimates of Multilevel Models Predicting Change in Symptom Scores
Note: PCL-S = Posttraumatic Stress Disorder Checklist – Specific Trauma; NSI = Neurobehavioral Symptom Inventory; BDI-II = Beck Depression Inventory – Second Edition; PVT = performance validity test.
* p < .05.
Graphical depictions of the MLM results predicting symptom change. Gray shading represents 95% confidence intervals. (a) PCL-S = Posttraumatic Stress Disorder Checklist – Specific Trauma, (b) NSI = Neurobehavioral Symptom Inventory, (c) BDI-II = Beck Depression Inventory – Second Edition.
In terms of the NSI, there was a main effect of PVT group (b = 8.32, SE = 2.92, p = .005, r = .28) in which the PVT-Fail group had higher scores compared to the PVT-Pass group, as well as a main effect of time (b = −.48, SE = .13, p = .001, r = .42), consistent with what appears to be a clinically significant decrease in PCS (at least 8 points; Belanger et al., Reference Belanger, Lange, Bailie, Iverson, Arrieux, Ivins and Cole2016; see Figure 1b); however, there was no PVT group × time interaction (b = .02, SE = .16, p = .900, r = .02). Similarly, depressive symptom severity decreased over time, as demonstrated by a main effect of time on BDI-II (b = −.39, SE = .09, p < .001, r = .52). Figure 1c indicates the decrease in depression symptoms also appears to be clinically significant (at least 10 points; Titov et al., Reference Titov, Dear, McMillan, Anderson, Zou and Sunderland2011; Westbrook & Kirk, Reference Westbrook and Kirk2005). In addition, there was a main effect of PVT group (b = 5.59, SE = 2.10, p = .009, r = .27) in which the PVT-Fail group had higher scores compared to the PVT-Pass group, but the PVT group × time interaction was not significant for BDI-II (b = −.11, SE = .11, p = .351, r = .13). Because validity groups differed in years of education, WRAT-4 standard scores, and baseline mental health/PCS symptoms, MLMs were repeated with these variables entered as covariates (although the symptom measure was not entered into the MLM if it was part of the dependent variable). Results were largely consistent, with significant main effects of time for all three measures (all p’s ≤ .001), and a significant PVT group × time interaction for the PCL-S (p < .001). However, the PVT group effect observed for the NSI and BDI-II became non-significant (NSI: p = .121, BDI-II: p = .350).
Follow-up analyses testing whether treatment condition moderated the relationship between PVT group and time on symptom change for the three measures indicated that none of the three-way interactions were significant (all p’s > .201; see Table 3). These results remained consistent when including covariates in the MLMs (all p’s > .212).
Regarding the aim of evaluating reduction in PVT failure rates, there was a significant main effect of time [χ2 (1, N = 184) = 4.06, p = .044, OR = .969 (95% CI: .941, .999)] and treatment condition × time interaction [χ2 (1, N = 184) = 7.27, p = .007, OR = .915 (95% CI: .858, .976)], but no effect of treatment condition [χ2 (1, N = 184) = 1.03, p = .311, OR = .668 (95% CI: .306, 1.458)]. Although Veterans in both treatment conditions demonstrated a reduction in PVT failure rates, Veterans in the SMART-CPT condition demonstrated a greater reduction in PVT failure rates compared to the CPT condition (see Figure 2). Additionally, the same analysis was run controlling for years of education, premorbid intellectual functioning, PCS, and depression and PTSD symptoms in five separate models. The interaction term remained significant in all analyses (all p’s < .02), whereas the main effect of time was significant for most covariates (p’s ≤ .05) with the exception of education and NSI in which the main effect of time dropped to trend level (p’s ≤ .086). All results reported were replicated when excluding those with a history of moderate TBI (n = 6) from analyses.
Percentage of Veterans with one or more performance validity tests in the invalid range at each assessment. PVT = performance validity test; CPT = Cognitive Processing Therapy; SMART-CPT = Hybrid treatment combining cognitive processing therapy (CPT) for PTSD with components of compensatory cognitive training from Cognitive Symptom Management and Rehabilitation Therapy (CogSMART).
DISCUSSION
The present study sought to determine (1) whether Veterans who scored within the invalid versus valid range on PVTs at baseline equally benefited from psychotherapy, (2) if lower rates of PVT failure were observed following treatment, and (3) whether SMART-CPT reduced PVT failure rates to a greater degree than CPT. In sum, Veterans who scored in the valid range on PVTs at baseline demonstrated a greater reduction in PTSD symptoms compared to those with invalid PVTs, however, both PVT groups exhibited a clinically meaningful reduction in PTSD symptoms. Veterans who scored in the invalid range on one or more PVTs benefited from treatment to a similar degree as those who scored in the valid range on all PVTs at baseline with regard to depressive symptoms and PCS. Additionally, reduction in rates of PVT failure was observed following treatment, with those in the SMART-CPT condition demonstrating greater reduction in PVT failure rates compared to those in the CPT condition.
The present study demonstrated that those who failed PVTs exhibited clinically meaningful improvements in mental health symptoms including PTSD, depression, and PCS. Although psychotherapy is often recommended in feedback following an invalid neuropsychological assessment (Carone et al., Reference Carone, Iverson and Bush2010), this is the first study to compare pre- and post-treatment outcomes for those who pass versus fail PVTs. Thus, the findings lend empirical support to the clinical practice of recommending psychotherapy in the context of invalid neuropsychological test performance in those with persistent PCS presentations. Although both groups demonstrated clinically significant improvement in symptoms, the PVT-Pass group demonstrated a greater reduction of PTSD symptoms than the PVT-Fail group, suggesting that Veterans in the PVT-Pass group were better able to engage in and benefit from treatment compared to the PVT-Fail group. Iraq and Afghanistan Veterans with history of mTBI and a service-connected disability rating may be more likely to exhibit invalid PVT performance (Armistead-Jehle, Reference Armistead-Jehle2010), and higher service-connected disability rating is a factor that is related to poorer response to evidence-based treatment for PTSD (Crawford et al., Reference Crawford, Wolf, Kretzmer, Dillon, Thors and Vanderploeg2017). Service-connected disability rating did not differ between PVT groups in the present study, so it is not clear what factor(s) among those in the PVT-Pass group contributed to their improved symptom reduction. An alternate hypothesis regarding the differential PTSD symptom improvements between the PVT groups is that the PVT-Fail group was more likely to over-report symptoms at the follow-up assessments and/or was more reticent to acknowledge symptom resolution compared to the PVT-Pass group. However, the relationship between performance and symptom validity is tenuous (Jurick et al., Reference Jurick, Crocker, Keller, Hoffman, Bomyea, Jacobson and Jak2019; Van Dyke, Millis, Axelrod, & Hanks, Reference Van Dyke, Millis, Axelrod and Hanks2013) and no measure of symptom validity was included in the present study to directly test this hypothesis. Future research should focus on moderators of psychotherapy treatment response in those with invalid PVT performance.
PVT failure rates reduced following trauma-focused treatment, both post-treatment and at the 3-month follow-up. Both treatment conditions resulted in lower rates of PVT failure following treatment. It can be hypothesized that the reduction of mental health symptoms increased Veterans’ ability to fully engage in the neuropsychological evaluation; however, this is speculative. To our knowledge, the only published studies demonstrating improvement in rates of invalid PVT performance have directly provided guidance or feedback regarding PVT performance and have yielded mixed results (Roor et al., Reference Roor, Knoop, Dandachi-FitzGerald, Peters, Bleijenberg and Ponds2018; Suchy et al., Reference Suchy, Chelune, Franchow and Thorgusen2012). A recent study demonstrated that having Veterans read and sign an informational document emphasizing the importance of trying one’s hardest and the consequences of invalid responding resulted in a lower frequency of invalid responding only for Veterans seeking disability benefits at the time of the evaluation (Horner, Turner, VanKirk, & Denning, Reference Horner, Turner, VanKirk and Denning2017). However, Niesten, van Impelen and Merckelbach (Reference Niesten, van Impelen and Merckelbach2018) raised concerns about Horner and colleagues’ findings, and several authors have raised concerns about this technique more broadly such as test security and the potential for patients to use the information provided to engage in less blatant exaggeration rendering poor test engagement harder to detect (Lippa, Reference Lippa2018; Niesten et al. (Reference Niesten, van Impelen and Merckelbach2018); Suhr & Gunstad, Reference Suhr and Gunstad2000; Youngjohn, Lees-Haley, & Binder, Reference Youngjohn, Lees-Haley and Binder1999). Thus, the present study adds to the extant literature by providing initial evidence that in Veterans with comorbid PTSD and PCS, PVT performance can improve following evidence-based psychotherapy without giving direct feedback or information regarding PVT invalidity.
As for treatment condition, PVT pass rates increased over time for Veterans in the SMART-CPT condition compared to the CPT condition. Although the study was not designed to definitively identify what specific treatment factors positively influenced PVT performance, a working hypothesis is that components of CogSMART unique to the SMART-CPT condition such as psychoeducation regarding TBI may have played a positive role beyond the impact of distress reduction. Although speculative, psychoeducation about the typical course of recovery following TBI and contributions of comorbid psychiatric symptoms provided in the SMART-CPT condition may have influenced perceptions regarding mTBI recovery allowing for possible alternative attributions of nonspecific PCS (e.g., feeling fatigue may be related to poor sleep rather than related to their mTBI per se). Additionally, improvements in mental health symptoms and PVT failure rates were observed to an even greater degree at 3-month post-treatment follow-up, supporting that long-lasting positive effects of treatment also seem to contribute to Veterans’ ongoing ability to demonstrate adequate test engagement during a neuropsychological evaluation.
An alternative explanation for the reduction in PVT failure rates is that perceived expectations of performance (e.g., demand characteristics, stereotype threat), particularly in the SMART-CPT condition, may have influenced performance on PVTs. Although no data were included in the present study to directly test this hypothesis, future research could attempt to further probe these possibilities by including questionnaires regarding perceived expectancies of performance.
Although this study has a number of strengths, including randomization of conditions, multiple measures of performance validity, and a well-characterized sample of Veterans with PTSD and history of mild-to-moderate TBI, there are some limitations that should be noted. First, the sample consisted of predominately male Iraq and Afghanistan Veterans with history of mild-to-moderate TBI, and therefore the findings may not generalize to non-Veterans, Veterans from other eras, those with history of severe TBI, or female Veterans. Second, the limited exclusion criteria in the present study created a sample with multiple comorbidities, including high rates of depressive symptoms. However, it also allows for generalizability of the results to those receiving standard clinical care, given that depression is a common comorbidity in this population. Additionally, it is possible that results may differ depending on the type of PVTs, criteria, and cutoffs used to define PVT failure, and future studies should explore alternate decision rules and measures. Finally, nearly half of the Veterans enrolled in the present study did not complete treatment. Although future research should focus on strategies to improve treatment completion rates, this rate of dropout is not uncommon in clinical care (Garcia, Kelley, Rentz, & Lee, Reference Garcia, Kelley, Rentz and Lee2011; Kehle-Forbes, Meis, Spoont, & Polusny, Reference Kehle-Forbes, Meis, Spoont and Polusny2016) and other treatment studies (Goetter et al., Reference Goetter, Bui, Ojserkis, Zakarian, Brendel and Simon2015; Schottenbauer, Glass, Arnkoff, Tendick, & Gray, Reference Schottenbauer, Glass, Arnkoff, Tendick and Gray2008). Furthermore, PVT failure groups did not differ with regard to dropout rate, suggesting that this was not a confounding factor in the results.
To our knowledge, this is the first study to demonstrate benefit in mental health symptoms in Veterans with invalid neuropsychological test performance at baseline and reduction in PVT failure rates following psychotherapy. Thus, we believe the present study holds clinical relevance in the process of triaging, assessing, and treating Iraq and Afghanistan Veterans with PTSD and history of TBI. Consistent with VA/DOD guidelines (Management of Posttraumatic Stress Disorder Work Group, 2017), the present findings add empirical support that Veterans presenting with PTSD and history of mild-to-moderate TBI should be enrolled in trauma-focused treatment and may benefit from comprehensive neuropsychological assessment after, rather than before, trauma-focused treatment. Additionally, SMART-CPT includes adaptations to CPT that may be particularly helpful for Veterans with invalid neuropsychological performance. Future research should focus on understanding moderators of treatment dropout and treatment response in Veterans with invalid PVT performance to improve mental health outcomes in this population.
Acknowledgements
This work was supported by the Department of Defense (award W81XWH-11-1-0641). This material is further supported with resources of the Veterans Affairs Office of Academic Affiliations Fellowship Program and the VA San Diego Center of Excellence for Stress and Mental Health, and Laura Crocker received salary support during this work from Career Development Award Number IK2 RX002459 from the VA Rehabilitation R&D Service. Special thanks to Briana Boyd and Mark Sanderson-Cimino.
CONFLICTS OF INTEREST
The authors have nothing to disclose.
