Introduction
Anxiety is the most prevalent mental health problem in the older adult population, being more frequently reported than either depressive disorders or substance use disorders (Kessler et al., Reference Kessler, Chiu, Demler and Walters2005). Social anxiety (also called social phobia) is one of the most common anxiety disorder in older adults, with a 12-month prevalence estimated between 1.3% and 6.5% within the older adult population (Cairney et al., Reference Cairney, McCabe, Veldhuizen, Corna, Streiner and Herrmann2007; Chou, Reference Chou2009; Byers et al., Reference Byers, Yaffe, Covinsky, Friedman and Bruce2010; Reynolds et al., Reference Reynolds, Pietrzak, El-Gabalawy, Mackenzie and Sareen2015; Canuto et al., Reference Canuto2017). Social anxiety disorder (SAD) is characterized by anxiety and distress related to social situations, including fear of embarrassing oneself, or excessive concern over negative evaluations from others (American Psychiatric Association, 2013). Despite its prevalence, research on social anxiety in older adults is relatively limited (for exceptions see Gretarsdottir et al., Reference Gretarsdottir, Woodruff-Borden, Meeks and Depp2004; Ciliberti et al., Reference Ciliberti, Gould, Smith, Chorney and Edelstein2011).
As individuals age, the nature of their social interactions adapts and evolves (Carstensen, Reference Carstensen2006). Changes in responsibilities (e.g. leaving the workforce), social roles (e.g. becoming a widow/widower), and living situation (e.g. entering an assisted living community) can alter how often and with whom older adults interact. Studies have shown that although older adults tend to report strong familial ties, the number of non-family relationships (i.e. friends) appears to decline in older age (Shaw et al., Reference Shaw, Krause, Liang and Bennett2007; Cornwell et al., Reference Cornwell, Laumann and Schumm2008). Additionally, the strength and vulnerability integration (SAVI) model posits that while older adults may have adequate coping skills for familiar stressors (e.g. interpersonal conflict, work/job stress), the novelty of stressors encountered in old age (e.g. social isolation, cognitive decline) may reduce the effectiveness of these coping skills, and result in greater psychological distress (Charles and Luong, Reference Charles and Luong2013). Given these qualitative and quantitative changes to socialization in later-life, it is reasonable to assume that the nature of anxiety and worry over social situations would evolve in tandem.
A research has indicated that anxiety disorders in general are present differently in older adults than they do in younger adults (Kogan et al., Reference Kogan, Edelstein and McKee2000; Ciliberti et al., Reference Ciliberti, Gould, Smith, Chorney and Edelstein2011). For instance, older adults may report anxiety symptoms as physical or somatic complaints, rather than psychological in nature (Wolitzky-Taylor et al., Reference Wolitzky‐Taylor, Castriotta, Lenze, Stanley and Craske2010). Additionally, many anxiety symptoms can overlap with cognitive deficits and medical comorbidities, making the detection of anxiety in older adults challenging (Krasucki et al., Reference Krasucki, Howard and Mann1998). There is also a difference in the type and frequency of anxiety provoking situations encountered by older adults. Socioemotional selectivity theory (Carstensen, Reference Carstensen2006) posits that older adults may be more prone to self-select activities with a lower-risk of emotional distress (cf. more risk-averse than younger counterparts). Furthermore, some situations that may provoke anxiety in older adults are relatively infrequent in younger samples. For example, Hedberg-Kristensson and colleagues (Reference Hedberg-Kristensson, Ivanoff and Iwarsson2007) found that older adults reported that using mobility assistive devices is a source of anxiety and shame. Similarly, older adults may be sensitive about appearing forgetful in front of others, for fear of being perceived as cognitively impaired or demented (Martin et al., Reference Martin, Leary and Rejeski2000). Many common measures of anxiety fail to take these unique presentations into account, calling into question their content validity in the older adult population.
To address the limitations of extant social anxiety measures, the Older Adult Social Evaluative Scale (OASES; Gould et al., Reference Gould, Gerolimatos, Ciliberti, Edelstein and Smith2012) was developed to be a content-valid assessment of social anxiety in older adults. Although not a diagnostic tool, the OASES can be used to identify potentially anxiety-provoking situations, as well as rate and monitor symptom intensity. Initial findings suggested that the OASES has strong psychometric properties, demonstrating good reliability as well as preliminary construct validity. Despite this, the evidence in support of the OASES is lacking in that (1) the OASES has never been used with a clinical/treatment-seeking population, and (2) it lacks validation against a gold-standard measure (cf. criterion-validity). The present study was designed to address these limitations by examining the psychometric performance of the OASES against the Structured Clinical Interview for DSM-5 (SCID-5; First et al., Reference First, Williams, Karg and Spitzer2015) in a sample of anxious older adults. The SCID was chosen to allow for comprehensive assessment of mental health disorders, in order to detect co-morbid conditions. Using a gold-standard diagnostic interview to establish criterion-validity of the measure, we first aimed to examine the criterion-validity and clinical utility of the OASES. Second, we aimed to identify clinically relevant OASES cut-offs using receiver operating characteristic (ROC) analysis. Third, the concurrent and discriminant validity of the OASES was evaluated by comparing its performance to other measures of anxiety and a variety of clinical measures, to explore the relationship between social anxiety and other factors.
Methods
Participants
Participants were community-dwelling individuals who were recruited as part of a study investigating treatment of late-life anxiety (NIH Clinical Trial No.: NCT02429778; Gould et al., Reference Gould, Ma, Kok, Wetherell, O'Hara and Beaudreau2017). Inclusion criteria for the study included being over the age of 60, and having an anxiety disorder as defined by the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5; American Psychiatric Association, 2013), to include: generalized anxiety disorder, SAD (social phobia), panic disorder, agoraphobia, specific phobia, and unspecified anxiety disorder. Exclusion criteria included: a history of bipolar disorder, schizophrenia, or other psychotic disorder; being engaged in any current psychotherapy (individual or group); daily use of benzodiazepines for conditions other than sleep (i.e. daytime-anxiety reduction), and cognitive impairment. All participants underwent an initial telephone screen that assessed eligibility criteria, which included the Short Blessed Test (SBT; Katzman et al., Reference Katzman, Brown, Fuld, Peck, Schechter and Shimmel1983) to screen for cognitive impairment; participants who scored ≥6 on the SBT indicating significant impairment (Katzman et al., Reference Katzman, Brown, Fuld, Peck, Schechter and Shimmel1983) were considered ineligible for the study. The present study includes all participants who provided consent and met the baseline eligibility assessment for the larger treatment study.
Measures
Older Adult Social Evaluative Scale
The OASES (Gould et al., Reference Gould, Gerolimatos, Ciliberti, Edelstein and Smith2012) is a 37-item measure of social anxiety specifically designed for use with older adults. The measure lists 37 social situations, including general scenarios (e.g. “Talking with people you do not know very well”) and scenarios tailored to older adults (e.g. “Forgetting information in front of others;” “Using medical treatment in public”). For each item, the respondent is asked to make two ratings: (1) How uncomfortable they feel in the situation (anxiety subscale), and (2) How often they avoid these situations (avoidance subscale). Both scales use 4-point Likert-type response options (0 “Not at all”/“Never” to 3 “Severely”/”Usually”). A total score is calculated by summing the responses to all 37 items on both scales (range of possible scores: 0–222). Although the OASES has demonstrated good psychometric properties in a non-clinical sample of community-residing older adults, the clinical utility of the OASES has not been established.
Structured Clinical Interview for DSM-5 (SCID-5)
The SCID-5 (First et al., Reference First, Williams, Karg and Spitzer2015) is a semi-structured interview that assesses mental health disorders as conceptualized in the DSM-5.The SCID is one of the most widely used and validated diagnostic instruments, and SCID-derived diagnoses are typically the standard to which other assessments are compared. To date, no studies on the psychometrics of the SCID-5 in older adults have been published; however, previous versions of the SCID have shown good interrater reliability in older adults (Segal et al., Reference Segal, Hersen, Van Hasselt, Kabacoff and Roth1993). Administration of the SCID-5 typically takes between 1 h and 2 h, and requires a trained interviewer to administer. The research version of the SCID was used in the present study (SCID-5-RV), and were conducted by PhD or graduate-level clinicians who were trained in the administration and scoring of the measure.
Geriatric Anxiety Scale (GAS)
The Geriatric Anxiety Scale (Segal et al., Reference Segal, June, Payne, Coolidge and Yochim2010) is a self-report measure that assesses the frequency of 25 anxiety-related symptoms. In addition to the 25 scored items, five additional items provide insight into the nature/type of worry. Responders rate how often each symptom occurred over the past week using a 4-point Likert-type scale ranging from 0 (Not at all) to 3 (All the time) on which higher scores indicate greater symptom frequency. The measure provides subscale scores based on three anxiety domains: somatic symptoms, cognitive symptoms, and affective symptoms. The GAS has excellent internal consistency, good convergent validity, and demonstrated discriminant validity within the community-residing older adults (Segal et al., Reference Segal, June, Payne, Coolidge and Yochim2010; Yochim et al., Reference Yochim, Mueller, June and Segal2010). The internal consistency of the GAS in the present study was excellent (α = 0.91).
PHQ-9
The PHQ-9 is a 9-item self-report screen for depressive symptoms that is based on the nine depressive symptoms of the DSM-IV (Kroenke et al., Reference Kroenke, Spitzer and Williams2001). Reponses are rated on a 4-point Likert-type scale with scores ranging from 0 to 27; one additional item assesses functional impairment caused by the nine symptoms. The PHQ-9 is commonly used in mental health and primary care settings, and its psychometric properties have been well-studied (Kroenke et al., Reference Kroenke, Spitzer and Williams2001). The internal consistency of the PHQ-9 in the present study was good (α = 0.89).
Somatic Symptom Scale-8 (SSS-8)
The Somatic Symptom Scale-8 (Gierk et al., Reference Gierk2014) is an 8-item measure that assesses somatic symptom burden over the past week. The eight items are scored on 5-point Likert-type scale, and summed to provide a total score (range 0–32). The scale is an abbreviated version of the PHQ-15 scale (Kroenke et al., Reference Kroenke, Spitzer and Williams2002), and has demonstrated good reliability and construct validity in mixed-age samples (Gierk et al., Reference Gierk2014). The internal consistency of the SSS-8 in the present study was acceptable (α = 0.76).
Procedures
Participants (N = 71) were initially screened over the phone to ascertain age and screen for potential exclusion criteria. Fifty-six participants met inclusion criteria, and were invited to complete the initial in-person assessments; however, eight participants did not complete an in-person assessment (e.g. due to no show, no longer interested). After explaining the study and obtaining informed consent, 48 participants were administered an assessment battery that included the SCID-5, OASES, GAS, SSS-8, and PHQ-9, as well as a demographic questionnaire. One participant was excluded during the intake assessment due to likely psychotic disorder, leaving the final sample at 47. All data were collected between August 2015 and February 2017.
Analytic plan
Inferential statistics were conducted to test for between-group differences in individuals with and without a SAD diagnosis with respect to demographic variables and OASES scores. Point-biserial correlations were calculated to examine associations between OASES and SAD-diagnosis relative to other measures of general anxiety (i.e. GAS) and depression (i.e. PHQ-9). The internal consistency of the OASES was determined by calculating Cronbach's α. A receiver operating characteristic (ROC) analysis was run to evaluate the performance of the OASES against the SCID. Youden's index (Youden, Reference Youden1950) was calculated as an additional metric to assist in identifying an optimal cut-point. Positive predictive value (PPV) and negative predictive values (NPV) were calculated using an Excel macro; all other analyses were done using SPSS (v 21.0).
Receiver Operating Characteristic
ROC analysis is a product of signal detection theory that allows the comparison of multiple decision points (i.e. cut-offs) in determining classification into a binary category/group (Metz, Reference Metz1978). This is done by calculating sensitivity (number of predicted positives/number true positives) and specificity (number predicted negatives/number true negatives) for all possible cut-offs of the assessment score. Each pair of sensitivity and specificity values is plotted along a curve, which can be used to determine diagnostic accuracy. By looking at area under the curve (AUC), the analysis can determine the probability that the predicted model can accurately determine caseness; because the group is binary (yes/no), AUC values range from 0.5 (random chance) to 1.0 (perfect classification). In psychological research, this typically involves comparing scores on a given assessment to gold-standard diagnostic classifications; by comparing the ratio of sensitivity and specificity across multiple decision points, one can determine an optimal cut-off that provides greatest accuracy in diagnosis (Metz, Reference Metz1978; Zweig and Campbell, Reference Zweig and Campbell1993).
Handling Missing Data
Because of the specificity of certain items on the OASES (e.g. using a medical treatment or device in public), some participants marked items that do not apply to them as “not applicable” or “N/A.” Responses in which a participant clearly indicated the activity did not apply were treated as non-missing data, and scored as zero (no anxiety/no avoidance) in the analyses; we comment on this response pattern in the discussion section. Other missing items that we could not determine whether they were missing because the item did not apply or for another reason were also not imputed and were scored as zero and omitted from the total score. Participants who responded to fewer than 80% of OASES items were excluded from analysis (N = 0).
Results
Forty-seven participants were included in the final analyses. Slightly over half the participants were female (57.4%), and participant age ranged from 60 to 88 years old (M = 69.2, SD = 7.0). SCID administration times ranged from 41 to 156 minutes, with an average duration of 91 minutes (SD = 23.4). Forty-one participants (87%) met criteria for one or more current anxiety disorders as determined by SCID-5. Eleven participants (23.4%) met criteria for current SAD. Ages of onset for SAD symptoms were: 8, 11, 15, 19, 30, and 55; three participants reported lifelong/always-present symptoms, two had unknown onset. Between-group analyses revealed only one demographic significantly differed by SAD-status; individuals meeting criteria for SAD were significantly younger than those without a SAD diagnosis, t (45) = 2.48, p = 0.02. See Table 1 for full sample demographics and diagnostic information. Thirty-nine of 47 participants (83%) responded to all 74 OASES items, and no participant omitted more than two responses. The common anxiety-inducing situations in this sample were Confronting/Disagreeing with Others (M = 1.74, SD = 1.03), Giving a Presentation (M = 1.68, SD = 1.04), and Forgetting Information in Front of Others (M = 1.49, SD = 0.91). The most frequently avoided situations were Confronting/Disagreeing with Others (M = 1.51, SD = 0.98), Giving a Presentation (M = 1.26, SD = 1.13, and Wearing Revealing Clothing in Public (M = 1.21, SD = 1.02). Item-level means for all 74 items are available as supplementary material online attached to the electronic version of this paper (Table S1 available at http://journals.cambridge.org/).Three OASES situations were marked “N/A” by ≥10% of the sample and scored as zero: Using Medical Treatment in Public, Using Assistive Devices in Public, and Interacting with Son/Daughter In-Law. Sensitivity analyses were conducted to determine whether these three items had an impact on any of the following analyses (data not shown). The findings of the sensitivity analyses suggested that including and excluding the items yielded the same findings and thus the items were retained.
Study demographics and descriptive statistics for main study variables. Values are provided for the total sample, as well as by subgroup based on SAD-status
Note: GAS = geriatric anxiety scale; PHQ-9 = patient health questionnaire-9; SAD = social anxiety disorder; SSS-8 = somatic symptom scale-8.
a Independent samples t-test for equal means between SAD+ and SAD− groups (df = 45 for all t values).
*p ≤ 0.05.
**p ≤ 0.01.
Psychometrics of the OASES
Internal Reliability
Total scores on the OASES ranged from 1 to 148 (M = 55.0; SD = 37.0) in this sample. Scores on the anxiety subscale ranged from 1 to 80 (M = 29.8; SD = 19.3). Scores on the avoidance subscale ranged from 0 to 77 (M = 25.1; SD = 18.3). The intercorrelation between the two subscales were expectedly high (r = 0.94, p < 0.001), and both subscales-scores were highly correlated with the OASES total score (r = 0.99 for anxiety, r = 0.98 for avoidance; both p < 0.001). The internal consistency of the OASES in this sample was found to be high (Cronbach's α = 0.97). Internal consistency was similarly high within the anxiety subscale (Cronbach's α = 0.95) and avoidance subscale (Cronbach's α =0.95). Corrected item-total correlations for the OASES ranged from r = 0.26 to 0.84 (see Supplementary Table S1).
Concurrent Validity
Mean OASES total scores were significantly higher in participants with SAD (as determined by SCID-5; M = 90.4, SD = 36.1) than in those without SAD (M = 44.1, SD = 30.3), t (45) = −4.2, p < 0.001.
Convergent and Discriminant Validity
Pearson correlations (bivariate and point biserial) of OASES total score and subscale-scores study measures examined relative relationships (Table 2). OASES total scores were significantly correlated with all other study measures (all p < 0.05). The strongest correlations were found between GAS scores and OASES total scores, whereas weaker correlations were found between PHQ-9 or SSS-9 and OASES total scores. Binary correlations ranged in strength from r = 0.35 (PHQ-9) to r = 0.58 (GAS Affective Subscale). SCID-5 diagnoses of SAD were most strongly correlated with GAS total score (r = 0.61).
Correlation matrix for primary study measures
Note: GAS = geriatric anxiety scale; OASES = older adult social evaluation scale; PHQ = patient health questionnaire; SAD = social anxiety disorder; SCID = structured clinical interview for DSM-5; SSS = somatic symptom scale.
*p ≤ 0.05.
**p ≤ 0.01.
ROC Analysis
The ROC analysis indicated that the OASES had an AUC of 0.86 (95% CI: 0.75, 0.97) indicating diagnostic performance well above chance (Figure 1). The diagnostic performance of specific cut-off values is presented in Table 3. A total score cut-off of ≥76 was found to have the highest Youden's index score (J = 0.61), demonstrated good diagnostic performance (sensitivity = 0.73, specificity = 0.89) as well as good, positive- and negative-predictive validity (0.67 and 0.91, respectively). Comparison of this cut-point against SCID diagnoses revealed that of the twelve participants who scored over 76 on the OASES, eight were true-positives, and four were false-positives. Qualitative examination of the four false-positives showed that they met criteria for lifetime SAD but not current (N = 1), or met criteria for sub-threshold lifetime SAD but not current (N = 3).
ROC curve representing performance of the OASES against SCID-5 derived diagnosis of social anxiety disorder. Note: Dashed line represents chance performance.
Sensitivity and specificity of various OASES total score cut-offs in classifying SCID-5 social anxiety disorder
Note: Bolded value indicates optimal cut-off. NPV = negative predictive value; PPV = positive predictive value.
Discussion
Despite demonstrating adequate diagnostic performance against the SCID-5, the OASES is not intended to be used as a diagnostic tool. The items on the OASES measure an individual's subjective reaction across a variety of context-specific situations (cf. state-anxiety). The cut-offs reported in this paper are intended as a way to quantify and monitor clinically significant SAD distress and impairment as a part of on-going research or clinical treatment. Thus, the OASES can be used to inform clinical care by means of identifying situations, and quantifying the severity of SAD. Identifying a clinical cut-off is valuable in establishing clinical benchmarks (cf. symptom thresholds), and although non-diagnostic, can be used to indicate the need for additional follow-up. The OASES cut-off of ≥76 demonstrated optimal balance between sensitivity and specificity, correctly identifying eight individuals with SAD, with four false-positives. Qualitative examination of four false-positives (i.e. participants who scored above 76 on the OASES, but had no SCID-derived SAD diagnosis) revealed that each had either clinical or sub-clinical levels of SAD throughout their lifetime, providing support for the OASES’ potential to identify elevated anxiety in and avoidance of social symptoms. This finding highlights the importance of further assessment of impairment associated with social anxiety symptoms in order to diagnose SAD. Although the present study identified ≥76 as the optimal cut-point, clinicians may wish to err on the side of caution by selecting a cut-point with greater sensitivity (e.g. ≥54 or ≥47) to maximize detection of SAD symptoms, at the cost of potentially having more false-positive screens. Because the cut-point is primarily descriptive, rather than diagnostic in nature, there is flexibility in its application. Although the OASES has two subscales (anxiety and avoidance), only the total score was used as the in this study for the ROC analysis. Although the two subscales can provide qualitative information about the nature and types of situations a person fears, each taps into a different facet of social anxiety. The total score captures both cognitive and behavioral aspects of SAD, providing a more comprehensive assessment of social anxiety symptoms.
The significant correlation between OASES scores and both the SCID-derived diagnosis of SAD and GAS scores provides evidence of concurrent validity for the OASES. As mentioned, the OASES does not directly assess anxiety symptoms, but rather perceptions and behaviors related to social situations. Because the GAS asks about specific symptoms (e.g. heart racing) the measure may better reflect an individual's level of distress (social or otherwise), and higher distress would be associated with a clinical-level diagnosis. In addition to these correlational findings, bivariate correlations between OASES scores and the PHQ-9 and SSS-8 were notably lower, providing preliminary evidence for discriminant validity for the OASES. Lower correlations were also observed between the OASES and the somatic subscale of the GAS, reflecting the relative importance of cognitive and emotional symptoms on the OASES.
The significant effect of age was an unexpected finding. Post-hoc analyses examining the relationship between age and other study variables (i.e. OASES, GAS, SSS-8, and PHQ-9) revealed that only SAD diagnosis was significantly correlated with age (r = −0.35, p = 0.01; results not shown). This finding is consistent with other studies that have reported lower social anxiety rates in old-old versus young–old populations (Cairney et al., Reference Cairney, McCabe, Veldhuizen, Corna, Streiner and Herrmann2007; Canuto et al., Reference Canuto2017). One possible explanation for this finding could be that as individuals age, their social interactions diminish (Shaw et al., Reference Shaw, Krause, Liang and Bennett2007), and fewer opportunities for socialization may correspond to less social anxiety. It should be noted, however, the lack of social support has been identified as a risk factor for poorer psychological outcomes (Petkus et al., Reference Petkus2017); additional research into the role of how quality and quantity of social interactions influences mental health outcomes in older adults.
Clinical utility of the OASES
The OASES can also be administered as part of a clinical intake, and used as a way to identify what social situations are most distressing to an individual. By identifying the most anxiety-provoking situations, clinicians can better tailor treatments (e.g. create exposure hierarchies, process specific thoughts associated with each situation). Because older adults may be reluctant to spontaneously identify anxiety-provoking situations, having a comprehensive list of situations provides a thorough and efficient assessment. In addition to operating as a “checklist” of older adult-specific situations, the OASES provides additional information on situational/behavioral avoidance. Responses on the avoidance subscale indicate how often a person tries to avoid a given situation or activity; changes in avoidance ratings could be used as a proxy for treatment progress, with lower scores indicating fewer avoidant behaviors. Thus, the avoidance subscale (in conjunction with the anxiety subscale) can be used as a qualitative tool for clinical-outcome monitoring, or in the development of in vivo exposure hierarchies.
Limitations
The most prominent limitation in this study is the small sample size. Because the data were collected as part of a pilot study, recruitment was limited. Despite being small, the sample was representative in terms of gender, age, and race/ethnicity, and to-date is the only study to examine the validity of the OASES. There was also considerable comorbidity between anxiety disorders in this study. Each of the 11 individuals who met criteria for SAD also met criteria for at least one other anxiety disorder, most notably generalized anxiety disorder (N = 8) and specific phobia (N = 5). It is, therefore, possible that some of the self-report measures represent findings from multiple sources of anxiety. From a generalizability standpoint, however, mixed-anxiety populations are common (e.g. Fresco et al., Reference Fresco, Mennin, Heimberg and Turk2003; Grant et al., Reference Grant2005), and this community-based sample is likely representative of anxious older adults in the general population. Future research on the OASES would also benefit from the inclusion of comparative samples (i.e. younger adults), and the inclusion other measures of social anxiety, to examine the performance of the OASES compared to non-older-adult-specific assessments.
Analysis of response patterns revealed that three OASES situations were relatively uncommon in our sample: Using Medical Treatment in Public, Using Assistive Devices in Public, and Interacting with Son/Daughter In-Law. As described in the results section, sensitivity analyses revealed that omitting these items did not impact the overall performance of the OASES. The low frequency of endorsement of these items may justify exclusion in future studies; however, at the present time, we recommend that these items be retained in the measure. Item-diversity is one of the strengths of the OASES, and practitioners should consider their population and clinical goals prior to modifying the measure.
Conclusion
Social anxiety remains a prominent issue in the older adult population, and having valid measures is essential to the identification and treatment of the condition. Despite the aforementioned limitations, this study is the first to validate the performance of the OASES against a structured interview-derived diagnosis, and one of the few studies to use the SCID-5 with an older adult sample. The OASES demonstrated adequate performance, supporting its use in the assessment of social anxiety in older adults.
Conflict of interest
None.
Description of authors’ roles
B.C. Kok helped to collect data, carried out statistical analyses, and wrote the paper. V.K. Ma helped to collect data and assisted with writing the paper. C.E. Gould designed the study, supervised data collection, and assisted with writing the paper. All authors reviewed the manuscript and approved it for submission.
Acknowledgments
This work was supported by Ellen Schapiro and Gerald Axelbaum through a 2014 NARSAD Young Investigator Grant from the Brain & Behavior Research Foundation and by Career Development Award (IK2 RX001478; PI: Gould) from the United States (U.S.) Department of Veterans Affairs Rehabilitation Research and Development Service. The authors have no conflicts of interest to disclose related to the manuscript. Views expressed in this paper are those of the authors and not necessarily those of the Department of Veterans Affairs or the Federal Government.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S104161021700285X
