INTRODUCTION
Valid assessment of traumatic brain injury (TBI) symptoms is critical to clinical care and research (Broglio et al., Reference Broglio, Cantu, Goia, Guskiewcz, Kutcher, Palm and Valovich McLeod2014). The primary mode of symptom assessment in TBI research is self-report inventory via instruments like the Rivermead Post Concussion Symptoms Questionnaire (RPQ; King et al., Reference King, Crawford, Wenden, Moss and Wade1995), the Sport Concussion Assessment Tool (SCAT; Guskiewicz et al., Reference Guskiewicz, Register-Mihalik, McCrory, McCrea, Johnston, Makdissi and Meeuwisse2013) symptom checklist, and the Neurobehavioral Symptom Inventory (NSI; Cicerone & Kalmar, Reference Cicerone and Kalmar1995). Yet questionnaire-based assessment introduces potential sources of bias and may yield information that is less valid than information obtained by other approaches such as clinical interviews (Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010). Given evidence that questionnaires and interviews yield different information about TBI symptoms (Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010), systematic comparisons of differing symptom assessment methods are needed to inform empirically supported research and practice guidelines.
Previous studies indicate that mild TBI (mTBI) patient and non-patient samples freely volunteer fewer mTBI symptoms than they report when prompted with specific symptoms (via questionnaire or structured interview; Gerber & Schraa, Reference Gerber and Schraa1995; Edmed & Sullivan, Reference Edmed and Sullivan2012; Edmed & Sullivan, Reference Edmed and Sullivan2014; Edmed et al., Reference Edmed, Sullivan, Allan and Smith2015; Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010; Nolin et al., Reference Nolin, Villemure and Heroux2006; Villemure et al., Reference Villemure, Nolin and Le Sage2011). Similarly, patients report less severe mTBI symptoms when assessed via open-ended interview than questionnaires (Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010; Edmed et al., Reference Edmed, Sullivan, Allan and Smith2015). These data align with studies outside the TBI field finding that direct questioning elicits more symptom reporting than open-ended questioning (Stapleton & Mills, Reference Stapleton and Mills2008). However, limitations in the prior work make it difficult to discern the main cause of discrepant reporting and the degree to which findings apply to patients. For example, studies comparing open-ended interview to questionnaire responses confounded mode of assessment (questionnaire, interview) with the level of structure (open- vs. closed-ended) of the questions (Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010; Edmed et al., Reference Edmed, Sullivan, Allan and Smith2015). Additionally, other studies included participants without TBI (university students; Edmed & Sullivan, Reference Edmed and Sullivan2012; Edmed & Sullivan, Reference Edmed and Sullivan2014; Krol et al., Reference Krol, Mrazik, Naidu, Brooks and Iverson2011). However, one cannot assume that mTBI-related symptoms present the same in mTBI patients vs. non-TBI samples. Relatedly, results from studies that use mock or simulated interviews may not generalize to genuine clinical assessments (Edmed & Sullivan, Reference Edmed and Sullivan2012; Edmed & Sullivan, Reference Edmed and Sullivan2014).
One concern when patients report more mTBI symptoms to questionnaires or direct questions (as compared to interview or open-ended questioning) is that excess symptoms are invalid. This concern might be supported by the finding that freely-reported symptoms, but not checklist-reported symptoms, differentiate mTBI from orthopedically injured and uninjured control groups (Gerber & Schraa, Reference Gerber and Schraa1995). Perhaps direct questions about symptoms prompt patients to attribute ambiguous experiences to TBI (Gerber & Schraa, Reference Gerber and Schraa1995). Such phenomena might be explained or enhanced by sources of response bias such as acquiescence bias (Baumgartner & Steenkamp, Reference Baumgartner, Steenkamp, Grover and Vriens’2006; Nolin et al., Reference Nolin, Villemure and Heroux2006; Wood, Reference Wood2004), the “good old days” bias (Gunstad & Suhr, Reference Gunstad and Suhr2001), or the “expectation as etiology” effect (Mittenberg et al., Reference Mittenberg, DiGuilio, Perrin and Bass1992). Others have proposed that checklists put pressure on a patient to give a response and endorse a symptom that they do not experience (Stapleton & Mills, Reference Stapleton and Mills2008). In contrast, direct questions provide linguistic and contextual cues about a topic, which may prompt individuals who were unsure what to report to open-ended questions to acknowledge legitimate symptoms.
As compared to questionnaires, which are commonly viewed as screening tools (Decaluwé, & Braet, Reference Decaluwé and Braet2004), structured or semi-structured interviews are widely regarded as the gold standard for assessing psychiatric disorders (e.g., Davison et al., Reference Davison, McCabe and Mellor2009; Nordgaard et al., Reference Nordgaard, Revsbech, Sæbye and Parnas2012; Zimmerman, Reference Zimmerman2003). Given the vast majority of mTBI studies have relied on questionnaire-assessed symptoms, alongside evidence that this could lead to biased reporting, there may be value in assessing mTBI symptoms via interviews. As mTBI-related symptoms are not specific to mTBI and often occur in the general population (Chan, Reference Chan2001; Gouvier et al., Reference Gouvier, Uddo-Crane and Brown1988; Iverson & Lange, Reference Iverson and Lange2003), interviews may provide a particularly strong opportunity to ascertain whether symptoms endorsed by patients were really worsened since injury or caused by the injury as opposed to reflecting preinjury or situational factors. Interviews also provide more opportunity to detect confusion about questions or contextual cues, such as the time frame in which to report symptoms.
The present study examined relationships between a novel TBI symptom interview (the Structured Interview of TBI Symptoms; SITS) with two commonly used measures of mTBI symptom reporting—the RPQ and SCAT-3 symptom checklist. To overcome some limitations of past assessment approaches, the SITS includes both open-ended and closed-ended questions as well as queries to discern preinjury from injury-related symptoms and clarify the time frame symptoms were experienced. Our primary aim was to compare the frequency and pattern of symptom endorsement across the SITS, RPQ, and SCAT-3 symptom checklist. We hypothesized that (a) symptoms would be more prevalent when assessed via structured (closed-ended) interview than open-ended interview on the SITS and (b) that a strong, positive association between structured interview (SITS closed-ended questions) and questionnaire-assessed (RPQ, SCAT-3) symptoms would be present given both of these methods elicit symptoms through direct questioning. To the degree that the instruments were not similarly associated, we expected somewhat stronger positive SITS-RPQ than SITS-SCAT-3 associations given that both the SITS and RPQ explicitly inquire about injury-related (vs. any current) symptoms. Additionally, we examined the relationship between administration order and acquiescence response bias with total symptom burden reported on the three instruments. Finally, an exploratory analysis tested whether the propensity to report fewer mTBI symptoms to open- than closed-ended interview questions was predicted by the degree to which patients felt that non-mTBI injury symptoms were more bothersome than their mTBI symptoms.
METHODS
Participants
This study was approved by the Institutional Review Board at the Medical College of Wisconsin and conducted in compliance with the Helsinki Declaration. For a thorough description of this study’s recruitment procedures and participants, see Harfmann et al. (Reference Harfmann, deRoon-Cassini, McCrea, Nader and Nelson2020). In brief, 162 civilian patients with TBI were recruited from a level 1 trauma center in Milwaukee, WI in person (n = 124) or by phone (n = 38) within 2 weeks of injury. Clinical assessments were completed in-person at 3-months post-injury. Inclusion criteria were age 18 years or older, English speaking, meets the study definition of TBI, and able to provide informed consent. Exclusion criteria included prisoner population and individuals unable to provide consent (e.g., activated power of attorney). The study definition of TBI was consistent with the definition used by the American Congress of Rehabilitation Medicine (Menon-et al., Reference Menon, Schwab, Wright and Maas2010): an alteration in brain function, or other evidence of brain pathology, caused by an external force. Altered brain function was classified as any evidence of altered mental status—i.e., unconsciousness, peritraumatic amnesia, or witnessed or subjective report of AMS (e.g., confusion, disorientation). TBI severity was based on admission Glasgow Coma Scale score (GCS 13-15 = mild, GCS 9-12 = moderate, GCS 3-8 = severe). Information about acute injury characteristics was obtained from a combination of chart review and semi-structured interview at the time of enrollment.
Of the 162 TBI patients enrolled, 93.8% were mTBI (17.9% with positive acute imaging findings), 1.2% were moderate TBI, and 4.9% were severe TBI. Seventy eight returned for 3-month follow-up and on average were slightly older (M = 45.12 years, SD = 15.70 years) than those who did not return (M = 39.88, SD = 15.79), t(160) = 2.11, p = .036. There were no significant differences between groups on race (X 2[4] = 4.30, p = .367), gender (Fischer’s exact test p = .531), admission GCS (X 2[3] = 4.63, p = .201), or TBI symptom severity at enrollment (3-item RPQ total score; t[160] = −.12, p = .903). One participant who completed follow-up was removed from analyses due to the research staff observing that the participant was not adequately engaged in testing to provide valid data. Another participant was removed due to being over the recommended threshold for suspecting clinically significant acquiescence (MMPI-2 RF TRIN-r T = 88T)Footnote 1. Of the remaining 76 participants, 96.0% sustained mTBIs, 0 sustained moderate TBI, and three sustained severe TBIs. The severe TBI cases were excluded given the current study’s focus on mTBI-related symptoms and the sample size of severe TBI being insufficient to permit direct analysis of TBI severity as a moderating variable. Table 1 presents demographic and injury characteristics for the 73 participants included in analyses.
Sample characteristics
N = 73 mTBI participants (Glasgow Coma Scale scores 13–15) for all measures except for the MMPI-2 RF TRIN-r (N = 71) and SITS open-ended questions (N = 71). WRAT-4 = Wide Range Achievement Test, 4th edition; MMPI-2-RF TRIN-r = Minnesota Multiphasic Personality Inventory-2 Restructured Form True Response Inconsistency; RPQ = Rivermead Post Concussion Symptoms Questionnaire; SCAT-3 = Sport Concussion Assessment Tool-3 symptom checklist; SITS = Structured Interview of TBI Symptoms; IQR = interquartile range.
1 Witnessed and suspected categories collapsed.
Procedures
At enrollment (< 2 weeks post-injury), patients completed a demographic questionnaire and the 3-item RPQ estimate of acute TBI symptom severity. The 3-month follow-up (M = 91.7 days, SD = 4.9) assessment was completed in-person and comprised the Wide Range Achievement Test-4th edition (WRAT-4) Word Reading test, followed by questionnaire and interview measures of TBI symptoms (described in the next section), quality of life (QoL), and return to preinjury functioning relevant to address the aims of the parent study. Each participant was assigned to one of four administration orders to counterbalance the order of administration within and across outcome domains (symptoms, function, QoL). The orders were created to ensure some separation in time between instruments of the same domain, as well as a different order of presentation of each instrument. Thus, TBI symptom measures were counterbalanced in order and intermingled with other measures (e.g., QoL). Although we intended to assign orders sequentially, because subject files were produced in advance and some subjects were lost to follow-up, we later strayed from this approach by filling in small order groups to ensure similar order group sizes.
The WRAT-4 (Wilkinson & Robertson, Reference Wilkinson and Robertson2006) Word Reading test was administered to estimate reading level. When performance indicated less than a 6th grade reading level, research staff read the questionnaires to participants whenever feasible. However, this protocol was not consistently followed, in part because some participants requested that questionnaires be read to them, and examiners were instructed to flex procedures within their judgment to prioritize getting through the examination whenever possible. Overall, nine (12.3%) participants had a WRAT-4 Word Reading grade estimate below the 6.0 grade level, but eight (11.0%) participants had questionnaires read to them and six (8.2%) had a combination of independently reading questionnaires and having the questionnaire read to them. As reported in the Results, sensitivity analyses confirmed that primary findings were not affected by the subset of individuals who did not independently read the questionnaires.
Clinical Assessment Measures
The RPQ (King et al., Reference King, Crawford, Wenden, Moss and Wade1995) is a 16-item questionnaire that assess symptom severity (relative to preinjury symptoms) over the past 24 hours on a 5-point scale (0 = not experienced at all to 4 = a severe problem). Ratings of 1 (no more of a problem than preinjury) are recoded to 0, and item ratings are summed to produce a total symptom severity score (0–64). Items were considered endorsed if rated 2–4. A 3-item RPQ (composed of the first three symptoms from the questionnaire: headache, dizziness, and nausea) was administered at enrollment to estimate baseline symptom severity. The full RPQ was administered at follow-up.
The SCAT-3 (Guskiewicz et al., Reference Guskiewicz, Register-Mihalik, McCrory, McCrea, Johnston, Makdissi and Meeuwisse2013) is a 22-item questionnaire that assesses current symptom severity on a 7-point scale (0 = none to 6 = severe). Ratings are summed to produce a symptom severity score (range 0–132).
The Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, Reference Ben-Porath and Tellegen2008) is a questionnaire of personality and psychopathology administered as true/false items. Specific MMPI-2-RF scales were administered (TRIN, RC1, FBS; 87 items total) in the parent study, as it was infeasible to collect the entire instrument. The TRIN-r scale was relevant to the present study and was included to assess acquiescence response bias.
The SITS was developed for this study to assess TBI symptoms using both open- and closed-ended question formats. An initial draft of the interview was reviewed by four experts in the field who provided feedback, and revisions were made before the SITS was piloted on the study sample. The SITS version administered comprised four parts; however, only data from parts one and two were used in the current study and will be described in detail here. Part one consisted of open-ended questions about any symptoms experienced since injury. Participants were asked (1) “Tell me about the injuries you had as a result of your [fall/accident, etc.],” then, (2) “Tell me about the symptoms you had after your (concussion/traumatic brain injury/injury).”
Part two was a structured, closed-ended interview of 31 symptoms. The examiner instructed participants that they would be asked specific questions about symptoms they may have experienced as a result of their concussion/TBI. Participants were read the list of TBI symptoms using various question prompts (A–D). The items were selected to cover the content assessed by the RPQ, SCAT-3, and NSI because these are the main instruments used in the civilian, athlete, and military TBI populations, respectively. A final item (question 32) asked the participant to report on any “other” TBI symptoms experienced that were not included on the list. Prompt A asked: “Has [symptom] ever been a problem after your injury?” If the symptom was endorsed on prompt A, prompt B was administered next: “Was this problem something you dealt with before your injury?”. If prompt B was endorsed, prompt C was administered: “Was this problem worse or more frequent after your injury?”. If prompt B was answered “no”, prompt D1 was administered: “Is this still a problem (i.e., in the past week)?” If prompt C was endorsed, prompt D2 was administered: “Is this problem still worse than it was before your injury (i.e., in the past week?).” Two SITS closed-ended question scores were calculated, reflecting the number of any symptoms experienced since injury (Prompt A responses) and any current, injury-related symptoms (Prompt D; range 0–32).
Interviewers recorded participant responses onto paper forms, which were double entered by two individuals and reconciled by a third rater in cases of entry inconsistencies. Coding of open-ended question responses used an approach similar to other studies (e.g., Edmed & Sullivan, Reference Edmed and Sullivan2012; Edmed & Sullivan Reference Edmed and Sullivan2014; Edmed et al., Reference Edmed, Sullivan, Allan and Smith2015; Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010). Responses were coded (by two independent raters) as being either consistent (score = 1) or inconsistent (score = 0) with the 31 items asked about during closed-ended questioning from part two. Responses deemed synonyms of items from closed-ended questions were coded as the closest closed-ended question item (e.g., “worried” reported during open-ended questioning was recorded under “feeling anxious or nervous”). Responses deemed similar but too vague to be declared any particular TBI symptom were coded as an “other” TBI symptom (e.g., “vision change,” “feels slow”). Responses that were inconsistent with TBI symptoms were not scored (e.g., “disbelief,” “feeling dependent”). Symptoms reported to open-ended questioning were summed to produce an index that paralleled the total number of symptoms score to closed-ended prompt A responses (i.e., reflecting any TBI symptoms experienced since injury).
Finally, participants were asked about whether TBI or non-TBI injury symptoms were more bothersome and, while not included in the present study, were questioned about overall severity and duration of TBI and peripheral/non-TBI symptoms.
Statistical Analyses
Statistical analyses were performed with SPSS version 24.0 with the exception of multiple comparison correction (performed in R version 3.5.2). Alpha was set to .05. Data were complete for all clinical assessment outcome measures except the SITS open-ended interview question, for which one interviewer neglected to record the participant’s response (n = 71) and the MMPI-2-RF TRIN-r (n = 71). Because of skew in the RPQ, SCAT-3, and the SITS closed-ended question scores, nonparametric statistics were used for the main analyses.
Spearman’s ρ correlation coefficients examined associations among symptom severity and TRIN-r raw score variables, with 95% confidence intervals computed using syntax from Bonett and Wright (Reference Bonett and Wright2000). Correlation strength was interpreted using guidelines from Dancey and Reidy (Reference Dancey and Reidy2007): .10 to .39 = weak, .40 to .69 = moderate, and .70 to .99 = strong. Cochran’s Q (three-group comparisons) and McNemar’s (two-group comparisons) analyses examined differences in the prevalence of symptom endorsement across measures. The false discovery rate (FDR) control method (Benjamini & Hochberg, Reference Benjamini and Hochberg1995) was used to correct for multiple comparisons. Only items that were the same across all three symptom assessment instruments (12 items) were included when comparing the questionnaires to the closed-ended interview questions. Wilcoxon signed-rank and Friedman’s analysis of variance tests were used to compare the total number of post-injury symptoms reported across assessment instruments. Kruskal–Wallis H tests for independent samples examined potential effects of administration order on TBI total symptom scores, and Mann–Whitney tests were used for follow-up pairwise comparisons with r as a measure of effect size (Field, Reference Field2013). Mann–Whitney tests also compared predominant symptom-type groups (non-TBI vs. TBI more bothersome) on the number of symptoms reported to open-ended interview questions and on the difference in symptoms reported to closed- vs. open-ended interview questions.
RESULTS
TBI Symptom Reporting: Open- vs. Closed-Ended Interview Questions
Mean and median number of symptoms reported across TBI measures are displayed in Table 1. At 3-month follow-up, participants retrospectively reported experiencing significantly more symptoms since injury on the SITS closed-ended questions (i.e., prompt A items) compared to the SITS open-ended questions (z = 7.33, p < .001, r = .62). A difference score was calculated (closed-ended minus open-ended total symptoms; Mdn = 14 symptoms, range 2–27). Table 2 displays the percentages of individual symptoms experienced since injury that were retrospectively reported at 3 months during the SITS open- and closed-ended questioning. Across all 31 TBI symptoms, the prevalence of endorsement was significantly higher via closed- than open-ended interview questioning. Results remained significant after FDR correction was applied (p’s < .001). There was no significant difference in number of “other” TBI symptoms reported to open- and closed-ended questions (p = .250).
Prevalence of TBI symptoms retrospectively recalled as occurring between injury and 3-month assessment for open-ended versus closed-ended interview methods
N = 71. SITS = Structured Interview of TBI Symptoms. Results of McNemar’s analyses comparing percentages of symptoms on open-ended questioning and closed-ended questioning of any symptoms experienced after injury (asked at 3 months post-injury). Bolded p values are significant after FDR correction for multiple comparisons. Other TBI-like symptoms reported included symptoms that did not clearly match the 31 primary symptoms, such as visual problems, hearing problems, anxiety or mood symptoms (e.g., frightened, fear, jumpy, worried, panicky, mood swings, and angry/aggressive), and cognitive symptoms (e.g., slowness in brain functions, repeating self, and feeling dazed).
TBI Symptom Reporting: Closed-Ended Interview Questions and Questionnaires
Table 3 displays the percentage of individual symptoms experienced currently at 3-month follow-up and deemed injury related (new or worsened since preinjury) on the SCAT-3, RPQ, and SITS-closed-ended questions (i.e., responses to prompt D) for the 12 items that overlapped across measures. No comparisons between the three instruments were significant after FDR correction. Results remained the same when analyses were computed including only the subset of participants who independently read their own questionnaires (N = 59). Total number of current, injury-related symptoms reported (among the 12 overlapping symptoms) were not significantly different across the measures, χ F 2 (2) = 4.82, p = .090.
Percentage of TBI symptoms experienced currently (at 3-month follow-up) by assessment instrument
N = 73. RPQ = Rivermead Post Concussion Symptoms Questionnaire; SCAT-3 = Sport Concussion Assessment Tool-3 symptom checklist; SITS = Structured Interview of TBI Symptoms. Closed-ended responses represent current injury-related symptoms experienced at 3-month follow-up. Cochran’s Q analyses restricted to symptoms represented across all TBI instruments. No statistical tests were significant after correction for multiple comparisons.
Correlations among symptom measures are provided in Table 4. Results of correlations showed that at 3-month follow-up, higher total number of current injury-related symptoms on the SITS closed-ended questions (i.e., prompt D responses) was strongly and positively associated with greater total severity of current symptoms measured by the RPQ and the SCAT-3 (p’s < .001). The SITS-RPQ association was significantly stronger than the SITS-SCAT-3 association (z = −2.59, p = .009).
Spearman’s correlations across TBI symptom measures and acquiescence response bias
SITS = Structured Interview of TBI Symptoms; SCAT-3 = Sport Concussion Assessment Tool-3 symptom checklist symptom severity score; RPQ = Rivermead Post Concussion Symptoms Questionnaire symptom severity score; MMPI-2-RF TRIN-r = Minnesota Multiphasic Personality Inventory-2 Restructured Form True Response Inconsistency raw score.
* 2-tailed p < .05, ** p < 0.01
1 Number of current (3-month) injury-related symptoms reported to closed-ended questions, selected to maximize comparability to SCAT-3 and RPQ scores.
2 Number of symptoms reported to open-ended questions (any since injury).
3 Closed-ended vs. open-ended symptom difference score (any since injury).
Relationships Between TBI Symptoms and Acquiescence Bias
Correlations between TBI symptom measures and the MMPI-2 RF TRIN-r are reported in Table 4. Higher levels of acquiescence bias (TRIN-r raw score) were weakly and positively associated with higher current symptom burden on SCAT-3 and SITS closed-ended questions (p’s < .05) and greater difference between number of symptoms (any since injury) reported to closed- vs. open-ended SITS questions (p = .008). The association between current symptom burden on the RPQ and acquiescence failed to reach significance, though trended in that direction (p = .06). The association between acquiescence and total number of symptoms (any since injury) reported to SITS open-ended questions was not significant (p = .326).
Effects of Test Administration Order on TBI Symptom Reporting
Effects of test administration order on symptom reporting are summarized in Table 5. There were no significant effects of order on symptom burden reported to the SITS closed-ended questions (current injury-related symptoms), the RPQ or the SCAT-3 symptom checklist (p’s > .05). However, order was related to number of symptoms reported to the SITS open-ended question (any symptoms since injury; p = .008, η2 = .083). Follow-up pairwise comparisons with adjusted p-values demonstrated that more symptoms were reported to open-ended interview when the SITS was administered third compared to first (z = −2.99, p = .008, r = .41). There was a trend for more symptoms reported when the open-ended question was administered second (vs. first; z = −2.36, p = .054, r = .38) and no difference between administration order 2 and 3 (z = −.33, p > .999, r = .04).
Descriptive statistics and statistical comparison of effect of administration order on symptom endorsement
SITS = Structured Interview of TBI Symptoms; SCAT-3 = Sport Concussion Assessment Tool-3; RPQ = Rivermead Post Concussion Questionnaire. Results of Kruskal–Wallis H test.
1 Number of symptoms reported to open-ended questions (any since injury).
2 Number of current (3-month) injury-related symptoms reported to closed-ended questions, selected to maximize comparability to SCAT-3 and RPQ scores.
3 More symptoms reported to open-ended questions when the SITS was administered third compared to first.
Association Between Predominant Symptom Type and SITS TBI Symptom Responses
Most of the sample experienced a combination of TBI and other injuries, leading us to question whether symptom reporting to different question types might be predicted by the predominant source of injury symptoms experienced by the patients. An exploratory analysis found that 50.7% of the sample endorsed their TBI symptoms as more bothersome and 49.3% endorsed their non-TBI injury symptoms as more bothersome. Total number of symptoms experienced since injury reported to SITS open-ended questions did not differ between those who were more bothered by TBI symptoms (Mdn = 3.5) than those who were more bothered by other injury symptoms (Mdn = 4.0), U = 623.00, z = .34, p = .73, r = .04. However, the difference between number of symptoms reported to closed- vs. open-ended interview questions was significantly greater for the group who endorsed their TBI symptoms as more bothersome (Mdn = 16.0) compared to the group that endorsed their non-TBI symptoms as more bothersome (Mdn = 11.0), U = 367.50, z = −2.74, p = .006, r = .33.
DISCUSSION
The current study investigated relationships between mTBI symptoms assessed via a novel TBI interview (the SITS) and well-established self-report inventories. We compared the frequency and pattern of symptom endorsement across these measures and explored the association between symptom reporting and acquiescence response bias. The key findings of this study include (1) mTBI symptom endorsement is highly concordant between closed-ended interview questions and questionnaires; (2) symptom reporting varies by level of question structure (open- vs. closed-ended), when comparing instruments equivalent in mode of assessment (i.e., within an interview); and (3) symptom endorsement to closed-ended questions, as well as the propensity to report more symptoms to closed- than open-ended questions, is significantly but weakly related to acquiescence.
Consistent with hypotheses, we found strong, positive correlations between the SITS closed-ended questions with the RPQ and SCAT-3 symptom checklist, as well as similar rates of symptom endorsement across these three measures, suggesting these measures are highly concordant. The stronger association between the SITS and RPQ than between the SITS and the SCAT-3 could be related to the SITS and RPQ both assess symptoms relative to preinjury, whereas the SCAT-3 checklist merely solicits current symptom burden. The strong concordance of ratings across the three measures may result from the highly structured questioning across the instruments. In contrast, consistent with hypotheses and prior work (Gerber & Schraa, Reference Gerber and Schraa1995; Edmed et al., Reference Edmed, Sullivan, Allan and Smith2015; Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010; Nolin et al., Reference Nolin, Villemure and Heroux2006; Villemure et al., Reference Villemure, Nolin and Le Sage2011), patients reported substantially more symptoms to closed-ended (vs. open-ended) interview questions, implying a stronger effect of question structure than mode of assessment (interview, questionnaire) on mTBI symptom reporting.
Our finding that TRIN-r was positively correlated with the closed- vs. open-ended difference score suggests that acquiescence response bias may play a small role in examinees’ tendencies to endorse more symptoms to closed-ended/direct questions. However, the weak size of this relationship implies that other mechanisms likely contribute to increased symptom endorsement to closed-ended questioning more so than acquiescence. Secondary analyses ruled out that patients who endorsed relatively few symptoms to open- than closed-ended questions were simply more preoccupied with non-TBI injury symptoms. In contrast, patients who were more relatively bothered by their mTBI (vs. other injury) symptoms tended to report even more new symptoms to closed- than open-ended questioning.
It is also possible that subtle differences in how open- vs. closed-ended interview questions were framed contributed to the symptoms reported in these subsections of the interview. Following open-ended questioning, the most commonly reported symptoms were headache, dizziness, nausea/vomiting, and forgetfulness. These symptoms closely align with those identified by others to be typical acute symptoms (i.e., headache, dizziness, and nausea per Eyres et al., Reference Eyres, Carey, Gilworth, Neumann and Tennent2005; Ryan & Warden, Reference Ryan and Warden2003), which may suggest that the open-ended question prompted participants to reflect on their acute experience of symptoms. In contrast, the most common symptoms endorsed since injury to closed-ended questioning were headache, fatigue, dizziness, and taking longer to think. The closed-ended question prompt called for any symptoms recognized between injury and the 3-month interview, which may have prompted subjects to report symptoms that were not immediately noticeable or not attributed by participants to TBI given their expectancies and knowledge about traumatic injuries.
We also found evidence of a dose–response relationship between number of mTBI symptom questionnaires administered before the interview and number of symptoms freely reported to open-ended interview questions, with more symptoms freely reported when other mTBI checklists were administered first. Despite these order effects for the open-ended questions, participants still freely volunteered fewer symptoms overall compared with the number of symptoms reported to the closed-ended questions (Mdn = 18 to closed-ended questioning vs. 3–4 to open-ended questioning depending on administration order). This may provide support for Iverson et al.’s (Reference Iverson, Brooks, Ashton and Lange2010) hypothesis that checklists remind patients of symptoms they forgot they had. Alternatively, perhaps exposure to questionnaires clarified what examiners meant by “TBI symptoms.”
While not assessed in the current study, researchers have suggested that some patients may report fewer symptoms during an interview because they feel rushed or uncomfortable discussing symptoms (Iverson et al., Reference Iverson, Brooks, Ashton and Lange2010). Anecdotally, our participants have occasionally reported that they perceived the symptom prompts of purportedly different symptoms to reflect the same symptom to them, implying repetition of content that inflates perceived differences between responses to open- and closed-ended questions.
The study findings have several clinical and research implications. First, clinicians should recognize that how they frame and structure questioning about symptoms can significantly influence the number and types of symptoms reported by their patients (Edmed & Sullivan, Reference Edmed and Sullivan2014). This is particularly important because it can be difficult for clinicians to plan treatments when they get conflicting information about patients’ symptoms across different assessment instruments (Kondiles et al., Reference Kondiles, Starr, Larson and Zollman2015) or when using different types of questioning. The findings are also important to consider in research settings, where direct assessment of symptoms via questionnaires has predominated. In particular, the high concordance between different instruments that employed direct questioning implies that research findings that used one instrument may generalize to other similar instruments. On the other hand, given uncertainty around why symptom reporting differs markedly between open- and closed-ended question formats, research findings cannot be readily compared across studies that employ different levels of question structure. Thus, researchers should always clearly report how they assessed symptoms (Krol et al., Reference Krol, Mrazik, Naidu, Brooks and Iverson2011). Investigating alternative hypotheses regarding the cause of discrepant information obtained across approaches would be valuable to continue to advance evidence-based assessment strategies.
The strong concordance across the SITS closed-ended questions, the RPQ, and the SCAT-3 symptom checklist supports the validity of symptoms assessed by the SITS. However, these measures should not be used interchangeably, as the SITS and other symptom interviews are likely to add additional clinically relevant information not assessable with a brief, structured symptom checklist. For example, open-ended questions help identify symptoms that are most problematic for the patient, which may help identify symptom-specific treatments that the patient may benefit from (Edmed et al., Reference Edmed, Sullivan, Allan and Smith2015). Additionally, the SITS explicitly queries patients about preinjury symptoms and participants’ peripheral/non-TBI injury symptoms, which were more bothersome than TBI symptoms for half of the sample. Such additional information gained is critical to provide patient-centered care, particularly in civilian polytrauma populations where comorbidities are prevalent.
Limitations
This study had several novel strengths including using multiple methods of TBI assessment and levels of question structure (open-ended questions, closed-ended questions, and questionnaires), consideration of examinee reading level, individually proctored assessments to provide participants with the questionnaire instructions, and counterbalanced administration of symptom measures. On the other hand, limitations of the study included the relatively small sample size and attrition rate between enrollment and 3-month follow-up. However, other than being slightly older than those who were lost to follow-up, there were no other differences on demographic or injury characteristics between those who did vs. did not return, so we do not believe this substantially skewed the findings. While reassuring that no other measured variables predicted loss to follow-up, unfortunately the reasons for attrition are unclear. Because the most robust predictor of 3-month symptoms (acute symptom severity) was not predictive of follow-up, and the mean 3-month RPQ score is consistent with the literature in this population (e.g., Sigurdardottir et al., Reference Sigurdardottir, Andelic, Roe, Jerstad and Schanke2009), it seems unlikely that those who returned for follow-up had more or less prolonged symptoms than those lost to follow-up.
The lack of symptom validity tests may be considered a limitation. However, we were primarily interested in comparing instruments and had no reason to suspect that potential overreporting/underreporting in some participants would differentially affect the relationship between instruments.Footnote 2 Administration of an abbreviated MMPI-2 RF may have altered the meaning of the TRIN-r scale. However, administration of the entire MMPI-2 RF was impractical and not relevant to the primary aims of our study. The order effects observed on open-ended symptom reporting may be considered a limitation. Using only those who were administered the SITS open-ended question first (n = 20), correlations among instruments continued to demonstrate high concordance between the SITS closed-ended questions and TBI symptom checklists, yet this subset of the data are too small to draw firm conclusions about this or other analyses.
Conclusions
This study demonstrated that self-report of mTBI symptoms varies by form of question structure but is highly concordant between widely used questionnaires and a novel structured interview. Acquiescence bias appears to play a small role in the additional symptoms reported to closed- vs. open-ended questions. The findings help inform selection of clinical outcome measures for mTBI studies. Findings also support the continued use of the RPQ and SCAT-3 symptom checklists to assess mTBI-related symptoms (at least under the carefully proctored conditions of this study), even if those symptoms might warrant follow-up queries to discern if symptoms were related to the mTBI or not (e.g., if symptoms existed premorbidly or related to interim circumstances). Additional study of when and why participants endorse more symptoms to closed-ended questions is warranted. Structured interviewing (e.g., with the SITS) may provide additional valuable information about patients’ symptom context, trajectories, and experiences that, when practical to administer, could better inform clinical management decisions and spark more informed research to mitigate the burden of mTBI symptoms on patients.
ACKNOWLEDGMENTS
We thank Amy Nader, Hannah Bartels, and Alexa Wild for their contributions to coordinating and coding data for the study. We thank our colleagues, Heather Belanger, PhD, James Hoelzle, PhD, Michael Kirkwood, PhD, and Nathaniel Nelson, PhD for providing feedback on an initial draft of the SITS. The University of Minnesota Press provided permission to administer a portion of the MMPI-2-RF in this study.
FINANCIAL SUPPORT
The study was funded by the Medical College of Wisconsin Center for Advancing Population Science (CAPS) and National Institutes of Health (NIH) grant R01NS110856. The REDCap database used for the study was supported by the MCW Clinical and Translational Science Institute (National Institutes of Health grant UL1TR001436). The manuscript’s contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
CONFLICT OF INTEREST
The authors have no conflicts of interest to report.
