Introduction
Persistent impairment in functional outcome following moderate-to-severe traumatic brain injury (TBI) is consistently reported in several domains, including motor, behaviour, emotion and cognition (Andrewes, Reference Andrewes2001; Engberg & Teasdale, Reference Engberg and Teasdale2004; Hammond et al., Reference Hammond, Grattan, Sasser, Corrigan, Bushnik and Zafonte2001; Schretlen & Shapiro, Reference Schretlen and Shapiro2003). In the cognitive domain, although an initial global cognitive impairment is apparent in the 1- to 3-month period after moderate-to-severe TBI (Axelrod, Fichtenberg, Liethen, Czarnota, & Stucky, Reference Axelrod, Fichtenberg, Liethen, Czarnota and Stucky2001; Brooks & Aughton, Reference *Brooks and Aughton1979; Haut, Petros, & Frank, Reference Haut, Petros and Frank1991; Rawlings & Crewe, Reference *Rawlings and Crewe1992; Shelton, Reference Shelton2004), the patterns of cognitive recovery during the subsequent months (specifically 4–24 months) are not so clear. Overall, most studies show a trend for continuing, often significant, cognitive impairment, typically in the attention, executive and memory domains, but deficits also are reported in language, psychomotor, motor and visuo-constructional functions. These results are evident at 1, 5 and even 30 years post-injury (Chu et al., Reference Chu, Millis, Arango-Lasprilla, Hanks, Novack and Hart2007; Dikmen, Machamer, Winn, & Temkin, Reference *Dikmen, Machamer, Winn and Temkin1995; Draper & Ponsford, Reference Draper and Ponsford2008; Novack, Alderson, Bush, Meythaler, & Canupp, Reference Novack, Alderson, Bush, Meythaler and Canupp2000; Schmitter-Edgecombe & Nissley, Reference Schmitter-Edgecombe and Nissley2000; Skandsen et al., Reference Skandsen, FInnanger, Andersson, Lydersen, Brunner and Vik2010; Tate, Fenelon, Manning, & Hunter, Reference Tate, Fenelon, Manning and Hunter1991; Trexler & Zappala, Reference Trexler and Zappala1988). However, the magnitude of reported impairment levels varies widely in this literature, depending upon injury severity, measures used and time assessed. In addition, the focus has been predominately on cross-sectional data, which provides a snap-shot of cognitive functioning at specified time-points but does not inform the natural progression of recovery. The primary purpose of the current review is to integrate the available evidence from longitudinal studies to form a clearer understanding of the natural course of cognitive recovery in adults with moderate-to-severe TBI. The secondary aim is to examine methodological issues arising from the literature to formulate recommendations for future longitudinal research.
A small number of meta-analyses are published which have attempted to synthesise the varying residual deficits reported in a range of cognitive domains following moderate-to-severe TBI (Mathias & Wheaton, Reference Mathias and Wheaton2007; Ruttan, Martin, Liu, Colella, & Green, Reference Ruttan, Martin, Liu, Colella and Green2008; Schretlen & Shapiro, Reference Schretlen and Shapiro2003). In terms of recovery, the results from these reviews describe different recovery patterns but are largely reliant on cross-sectional studies. Schretlen and Shapiro (Reference Schretlen and Shapiro2003), who investigated global cognitive outcome following TBI at various time-points (15.6 days to 6 years) and severity levels, report improvement until the 2-year period and then decline in cognitive function. By contrast, Ruttan et al. (Reference Ruttan, Martin, Liu, Colella and Green2008), who analysed performance on timed and untimed cognitive tasks, report minimal differences in effect sizes between 6–18 months and 4.5 years post-injury. Mathias and Wheaton (Reference Mathias and Wheaton2007) found a broad range of impairment levels on a variety of tests targeting attention and processing speed, but collapsed the assessment time-points together (mean = 3.23 years post-injury), hence recovery per se was not examined. The above disparate results may have arisen because of the different method of grouping assessments used by the reviews. Grouping data from various time-points may have masked the changes that occur over time during the recovery process.
The methodological problems of cross-sectional data analysis can be overcome with the use of prospective longitudinal data. Singer and Willet (Reference Singer and Willett2003) state that there are three essential methodological requisites for study of change: (1) multiple waves of data – more than two data-points are required to examine an individual's unique change trajectory and to differentiate between measurement error and true change; (2) a sensible metric for time – there must be adequate data sampling to provide a description of the individual's recovery pattern; and (3) an outcome which changes systematically. Change over time can only be measured truly with longitudinal data; by contrast cross-sectional data can only demonstrate differences in cognitive function between different groups at varying time-points – it cannot describe change over time within a single group (Singer & Willet, Reference Singer and Willett2003). The Schretlen and Shapiro (Reference Schretlen and Shapiro2003) review demonstrates the inherent problems that occur when data are collapsed over time, because it is not clear whether changes over time were due to recovery or differences between the different samples used at the different time-points. Therefore, the current review will focus solely on longitudinal data.
In summary, there is still limited information regarding cognitive recovery in the first 2 years post-trauma, when most of the natural recovery is purported to occur (Katz, Reference Katz1992; Stuss & Buckle, Reference Stuss and Buckle1992). Therefore, in order to gain a better understanding of the process of cognitive recovery, it is necessary to analyse longitudinal data with more frequent assessment time-points dispersed over the initial 2 years post-injury.
This paper is organised in two parts. Part I of the paper provides a systematic review of the longitudinal research in this area from the past 50 years. Part II examines the methodological issues identified in the literature, to develop recommendations for future research. This systematic review was prepared in accordance with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement of reporting guidelines for systematic reviews (Liberati et al., Reference Liberati, Altman, Tetzlaff, Mulrow, Gøtzsche, Ioannidis and Moher2009).
PART I – SYSTEMATIC REVIEW
Method
Literature Search
Three electronic databases – Medline (1950 to October 2012), PsycINFO (1806 to October 2012) and CINAHL (1982 to January 2009) – were searched using combinations of the terms ‘head injury, brain injury, traumatic brain injury, cerebral ischemia, closed head injury and craniocerebral trauma’ and ‘recovery, recovery of function, spontaneous recovery and recovery (disorders)’. Potential search terms were refined, with the above terms providing the most sensitive and specific results (for specific terms see Appendix 1). Possible articles were further identified via manual searching of the reference lists of relevant articles and textbooks, and the authors’ own reference catalogues.
Inclusion Criteria
The following inclusion criteria were used: (1) human population; (2) published in English, in a peer-reviewed journal after 1960; (3) adult moderate-to-severe TBI population (defined as a period of post-traumatic amnesia (PTA) >1 day (Jennett & Teasdale, Reference Jennett and Teasdale1981)); (4) examined cognitive recovery with objective, performance based neuropsychological tests; (5) testing conducted after the emergence from PTA and not exclusively during a vegetative or minimally conscious period; (6) minimum of three assessment time-points; and (7) initial three assessments conducted in the first 2 years post-injury. Papers were excluded if they involved a treatment study, a review article, a single-subject design, case study or used only self-report measures or rating scales.
Two reviewers (the authors) independently applied these criteria to a sample (approximately 30%) of the yielded results, by reading the titles and abstracts of the papers. The level of agreement was 95.1%; disagreements were then discussed and a selection process was confirmed. The subsequent 70% of potential papers were then reviewed for inclusion by the first author. Hard copies of selected papers were retrieved and read by both reviewers to confirm eligibility, and disagreements were resolved by consensus.
Data Extraction
The data were extracted for two purposes: (1) to provide a summary description of the current longitudinal data; and (2) to calculate effect sizes from the available data. Two reviewers (the authors) used a data extraction form to independently extract the data; disagreements were resolved by consensus. The methodological quality of the included papers was not assessed because currently no appropriate rating tool is available in this area (see Sanderson, Tatt, & Higgins, Reference Sanderson, Tatt and Higgins2007 for a review of rating scales in observational studies).
(1) Summary description
The following aspects of each study were identified: study design, participant characteristics, cognitive functions assessed, neuropsychological measures used, and the assessment time-points. The classification of individual neuropsychological tests into exclusive cognitive domains is a widely controversial topic. Typically, approaches are based on theoretical models, factor analytic studies or clinical convention (Al-Khindi, Zakzanis, & van Gorp, Reference Al-Khindi, Zakzanis and van Gorp2011). These approaches all have advantages and disadvantages, and no single method can circumvent the likelihood that each individual test contains components from more than one cognitive domain (for further discussion see Al-Khindi et al., Reference Al-Khindi, Zakzanis and van Gorp2011). For the current review, the individual neuropsychological measures were categorised into seven cognitive domains (attention, executive function, general cognition, language, memory, motor function and visual-perception) based on the classification of neuropsychological abilities and tests outlined in the chapter titles from Strauss, Sherman and Spreen (Reference Strauss, Sherman and Spreen2006, Chapters 6–12 and 14; when further clarification was required Lezak, Howieson, & Loring (Reference Lezak, Howieson and Loring2004) was used). The full list of measures by cognitive domain is provided in Table 1.
Measures Identified in Systematic Review and Corresponding Classification by Cognitive Domain (indicates the number of studies that included each measure)
WAIS/WAIS-R/WAIS-III = Wechsler Adult Intelligence Scale/Revised/III; WMS/WMS-III = Wechsler Memory Scale/III; FSIQ = Full Scale Intelligence Quotient; PIQ = Performance Intelligence Quotient; VIQ = Verbal Intelligence Quotient.
The assessment time-points were categorised into six epochs based on time since injury: (1) 0–3 months, (2) 4–6 months, (3) 7–9 months, (4) 10–12 months, (5) 13–18 months and (6) 19–24 months. For the first 12 months, 3-month periods between epochs were used because this was frequent enough to observe improvement but sufficiently large to capture the variable assessment time-points in the literature.
(2) Effect sizes
Effect sizes were calculated in order to provide an estimation of the difference in test performance between the TBI sample and non-brain-injured controls, that is, to provide the magnitude of impairment at each assessment time-point. Cohen's d was selected because of its appropriateness for neuropsychological research (Zakzanis, Reference Zakzanis2001). The descriptors provided by Cohen (Reference Cohen1988) were used to describe the magnitude of effect sizes: 0.2 small, 0.5 moderate and ≥0.8 a large effect. Effect sizes were computed for the studies which included an appropriate comparison group and provided the statistical data necessary to calculate effect sizes (e.g., means, standard deviations, t-tests, F values from an ANOVA; Olejnik & Algina, Reference Olejnik and Algina2000; Zakzanis, Reference Zakzanis2001). When means and standard deviations were reported the following equations were used:
\begin{equation*}
{\rm Cohen'}s\,d\, = \,\frac{{M_1 - M_2}}{{SD\,{\rm Pooled}}}
\end{equation*}
\begin{equation*}
SD\,{\rm Pooled} = \sqrt {\frac{{(N_1 - 1)SD_1^2 + (N_2 - 1)SD_2^2}}{{N_1 + N_2 - 2}}}
\end{equation*}
\begin{equation*}
{\rm Cohen's}\,d = \frac{{2t}}{{\sqrt {df}}}
\end{equation*}
To ensure independence of the data included in the analyses, the following provisions were implemented: (1) when it was apparent that multiple papers reported on the same data set, the more complete data set was used or the data were entered as a single study for the analysis (n = 3); and (2) when there was more than one assessment in a single epoch, the last time-point was used.
Results
Combining the searches from the three databases and excluding duplicates produced 5718 potential papers (search finalised on 31 October 2012). After applying the eligibility criteria, 5698 papers were excluded; all 20 papers eligible for inclusion (as well as an additional 8 studies with related data sets) were included in the review (see Figure 1 for the PRISMA flow diagram).
PRISMA flow diagram of the different stages of the systematic review.
(1) Summary Description
Table 2 provides a summary of the 20 papers included in the review; reporting the study design, participant characteristics, cognitive functions assessed, neuropsychological measures used and the assessment time-points. Eleven papers were classified as observational cohort studies with a comparison group, and nine were observational cohort studies without a comparison group. As a set, the 20 articles with 2095 participants (1658 TBI and 437 comparison participants) measured all seven cognitive domains. In general, the TBI participants were male (mean = 80.9%) and the average age at injury was 30.94 years (all but one study included the average age). Only 11 studies reported the educational level of the TBI participants; six of these provided average years of educational attainment (mean = 11.56 years). Few studies reported the previous vocational status or pre-morbid estimates of cognitive functioning. Great variability was evident in reporting the participant characteristics and the composition of the comparison groups. Some samples comprised non-brain-injured trauma patients; some, healthy volunteers; and others were family and friends of the TBI sample. From the information provided, the mean age of the comparison group was 29.14 years, the majority were male (mean = 69.5%) and the average years of education was 11.67 years.
Assessment Time-points and Measures Used in Each Paper
FSIQ = Full Scale Intelligence Quotient; GCS = Glasgow Coma Scale; GSW = gunshot wound; LOC = loss of consciousness; LOF = level of functioning; LOS = length of stay; N/A = non-applicable; NR = not reported; PIQ = Performance Intelligence Quotient; PTA = post-traumatic amnesia; PTD = post-traumatic disorientation; RLS = Reaction Level Scale; RTC = road traffic crash; TBI = traumatic brain injury; TFC = time to follow command; VIQ = Verbal Intelligence Quotient; WAIS/WAIS-R/WAIS-III = Wechsler Adult Intelligence Scale/Revised/III; WMS/WMS-III = Wechsler Memory Scale/III.
*All assessment times are post-trauma unless otherwise stated.
Injury severity was difficult to compare among the studies because a range of severity measures was used; these included length of PTA (n = 10), Glasgow Coma Scales scores (GCS; n = 9), loss of consciousness or coma duration (n = 7) and time to follow commands (n = 4). In addition, several papers (n = 7) did not provide relevant injury severity descriptive data. Forty-five per cent (n = 9) reported trauma type; of these a road traffic crash (RTC) was the most common trauma.
Most studies (n = 14) examined cognitive recovery in several domains, ranging from three to six functions assessed in a single paper. The remaining six articles were more focused, investigating only one or two domains. Attention, executive function and memory were examined by the majority of papers, with 15 papers assessing at least two of these domains. Specifically, 16 papers contained measures of attention, 14 tested memory, 13 examined executive function and 11 assessed general cognition. Seven investigated visual-perception, five examined motor function and four assessed language. Sixty-seven separate measures were used in the 20 studies but there was little commonality in the measures used among the investigations. Fifty-two per cent of the measures (35/67) were used by more than one study; the remaining 32 (48%) measures were used in a single study. Table 1 indicates the number of studies that included each measure and Table 2 reports the measures used in each paper.
The timing of the assessments was reviewed to examine the comparable data points. Overall, there was a wide fluctuation in the timing of assessments. Most assessments occurred in the first 12 months after injury; only three studies completed assessments in the interval between the first and second years post-trauma (Eker, Hagstadius, Linden, Schalen, & Nordstrom, Reference *Eker, Hagstadius, Linden, Schalen and Nordstrom2001; Parker & Serrats, Reference *Parker and Serrats1976; Van Zomeren & Deelman, Reference *Van Zomeren and Deelman1978). The largest number of assessments in a single study was seven in the first 2 years (Parker & Serrats, Reference *Parker and Serrats1976); most other studies (n = 11) used at least four assessment time-points. Despite the overall variation in the timing of assessments, a common testing sequence was identified, with eight papers reporting assessments staged at 3, 6 and 12 months post-injury. These methodological differences limited a detailed examination of cognitive recovery over time.
(2) Effect Sizes
A series of effect sizes was computed to ascertain the pattern of cognitive recovery. Seventeen of the 20 papers were excluded because they did not report appropriate data to calculate effect sizes (n = 5), did not include a suitable comparison group (n = 4) or reported neither appropriate data nor had a suitable comparison group (n = 8). Appropriate data were requested from the authors of the studies that did not provide sufficient data but did have a suitable comparison group. Data from a single study (Dikmen, Machamer, Temkin, & McLean, Reference Dikmen, Machamer, Temkin and McLean1990) were forwarded to the current authors. Thus, altogether effect sizes could be computed for four studies (Dikmen et al., Reference Dikmen, Machamer, Temkin and McLean1990; Mandleberg & Brooks, Reference *Mandleberg and Brooks1975; Spikman, Timmerman, van Zomeren, & Deelman, Reference *Spikman, Timmerman, van Zomeren and Deelman1999; Spitz, Ponsford, Rudzki, & Maller, Reference *Spitz, Ponsford, Rudzki and Maller2012), with 24 measures from five cognitive domains (attention, executive function, general cognition, memory and visual-perception).
In total 523 participants were included in the effect size calculations (242 TBI and 281 comparison participants). The mean age at injury of the TBI group was 27.1 years and 28.2 years for the comparison group, mean length of education was 11.1 years for the TBI group and 11.5 years for the comparison group; 73.3% and 69.8% were males, respectively. There were no statistically significant differences between the TBI and comparison groups for age at injury, years of education or sex (see Table 3 for descriptive data, t-test results and p values). Injury severity was again difficult to compare among the four studies due to the variable severity measurements used, and only two studies (Spikman et al., Reference *Spikman, Timmerman, van Zomeren and Deelman1999; Spitz et al., Reference *Spitz, Ponsford, Rudzki and Maller2012) provided descriptive data. However, participants from the four studies recorded a PTA duration of at least 4 days post-trauma, indicating that these participants were in the severe range of impairment.
Descriptive Data, t and p Values for Analyses of Demographic Characteristics of the TBI and Comparison Groups Included in the Effect Sizes
Effect sizes were obtainable for epochs 1, 2, 4 and 6 (see Table 4 for Cohen's d). It was not appropriate to perform meta-analyses or to aggregate the effect sizes because of the small number of studies available. Figures 2–5 display the effect sizes over time for the measures in each cognitive domain.
Effect Sizes for Each Measure Classified by Cognitive Domain
Arith = Arithmetic subtest from the WAIS batteries; BD = Block Design subtest from the WAIS batteries; Category = Category test; Comp = Comprehension subtest from the WAIS batteries; Distraction = Reaction time distraction task; Dspan = Digit Span subtest from the WAIS batteries; Dsymbol = Digit Symbol subtest from the WAIS batteries; Dual = Reaction time dual task; FSIQ = Full Scale Intelligence Quotient from the WAIS batteries; Info = Information subtest from the WAIS batteries; MCST (RPES) = relative perseveration error score from the Modified Card Sorting Test; MQ = Memory Quotient; OA = Object Assembly subtest from the WAIS batteries; PA = Picture Arrangement subtest from the WAIS batteries; PASAT = Paced Auditory Serial Attention Test; PC = Picture Completion subtest from the WAIS batteries; PIQ = Performance Intelligence Quotient from the WAIS batteries; Sims = Similarities subtest from the WAIS batteries; Stroop I = Stroop Test Part 1 – Word Naming; Stroop II – Stroop Test Part 2 – Colour Word Naming; Trails A = Trails A subtest from the Trail Making Test; Trails B = Trails B subtest from the Trail Making Test; VIQ = Verbal Intelligence Quotient from the WAIS batteries; Vocab = Vocabulary subtest from the WAIS batteries; WAIS/WAIS-R/WAIS-III = Wechsler Adult Intelligence Scale/Revised/III; WMS/WMS-III = Wechsler Memory Scale/III
Effect sizes from the measures in the attention domain.
Effect sizes from the measures in the general cognition domain.
Effect sizes from the measures in the executive function domain.
Effect sizes from the measures in the memory and visual-perception domains.
Epoch 1 (0–3 months)
The results from the individual measures demonstrated widespread cognitive impairment, for the five domains examined at Epoch 1. Only one measure, the relative perseveration error score from the Modified Card Sorting Test (MCST), classified in the executive function domain, had a small effect size (d = 0.2), the remainder had at least a moderate effect size, with many demonstrating a large magnitude (range: d = 0.5–2.8).
Epoch 2 (4–6 months)
At this epoch, residual impairment was still apparent on most of the individual measures in the five domains. Seven measures were now small to moderate in size (range: d = 0.2–0.4) but the majority (n = 21) remained in the moderate to large range (d = 0.5–1.0). The executive function tests showed the least residual effect sizes (range: d = 0.2–0.6), attention, general cognition and memory (ranges: d = 0.3–1.0; d = 0.3–1.0 and d = 0.6–0.9, respectively) demonstrated moderate to large remaining effect sizes, and visual-perception displayed mostly large effect sizes (range: d = 0.6–0.9). In terms of recovery from Epoch 1 to 2, nearly all of the measures demonstrated a decrease in effect size, with the average effect size change over time of 0.33 (range of change: d = 0–0.7).
Epoch 4 (10–12 months)
In the following 6 months, considerable deficits persisted on most tests but improved performance was evident on some measures. The effect sizes in the attention, executive function and general cognition domains were quite variable. In general, the attention measures displayed moderate effect sizes (range: d = 0.3–1.4). For executive function, the more verbal reasoning tests and planning tasks (Zoo Map and the Similarities and Comprehension subtests from the WAIS (Wechsler Adult Intelligence Scale) batteries) demonstrated nil to small effect sizes (range: d = 0–0.3), while tasks requiring selective attention, generativity, inhibition of habitual responses and visuo-spatial problem solving (the COWAT, Stroop and Category tests) showed moderate to large effect sizes (range: d = 0.5–1.5). The general cognition measures also provided variable magnitudes of effect sizes (range: d = 0.1–1.7). The Verbal Information and Perceptual Information Quotients (VIQ and PIQ) from the WAIS batteries showed varying effect sizes from the two different studies, but, overall, there was at least a small to moderate effect size for these two variables at 1 year post-injury. The memory effect size remained mostly in the moderate to large range (d = 0.7–1.3), apart from the WMS-III Immediate Memory Index which displayed a small effect size (d = 0.1). The visual-perception measures had a small to moderate effect size (range: d = 0.3–0.5). In summary, recovery was still apparent from Epoch 2 to 4, but to a lesser degree than from Epoch 1 to 2; with the average change over time in effect size of 0.23 (range: d = 0.0–0.6).
Epoch 6 (19–24 months)
At the end of the second year post-trauma, cognitive performance of the TBI sample compared to the control group was more inconsistent than at the previous observations. More than half of the measures showed either no or minimal differences between the TBI and control groups (n = 6), or small to moderate effect sizes (n = 7), suggesting resolution of the previous impairments on these tests. In general, the remaining measures demonstrated a large effect size (n = 8; range: d = 1.0–1.3). The memory domain demonstrated little recovery, with the effect size remaining large (d = 1.1). The executive function and general cognition effect sizes (range: d = 0.0–1.1 and d = 0.1–1.1, respectively) again provided varying effect sizes, but these were mostly consistent with the results seen at Epoch 4. The attention measures were similarly variable, most demonstrated a moderate to large effect size (range: d = 0.5–1.2), with one measure displaying only a small effect size (d = 0.2). The visual-perception effect sizes showed near to complete resolution of the previous deficits on all tests. In summary, recovery was evident in each of the domains between Epoch 4 and 6, and was similar in magnitude to the change between Epoch 2 and 4; the average change in effect size was 0.18 (range: d = 0.1– .5). These preliminary results suggest that recovery continues on a number of neuropsychological measures into the second year post-injury.
Discussion
The current systematic review provides new information about the process of cognitive recovery during the early post-acute phase. Understanding recovery during this stage is important, because it is suggested that the majority of recovery occurs in the first 6 months following trauma, and although recovery continues, this is at a minimal rate, plateauing around the 12-month mark (Katz, Reference Katz1992; Stuss & Buckle, Reference Stuss and Buckle1992). The current findings partially support this position. A general pattern of recovery was evident, with cognitive recovery continuous throughout the initial 12 months and with more limited data suggesting that recovery persists during the second year post-trauma (albeit at a reduced rate).
In addition, the current findings suggest that the degree of impairment varied among the specific cognitive domains and that the domains (and more notably the individual measures of them) recovered at different rates. Deficits in the visual-perception domain were initially large but showed consistent recovery over time, with complete resolution by 2 years post-injury. In contrast, the memory domain demonstrated some recovery but significant impairments persisted at 2 years post-trauma. Results also suggest that those cognitive processes which recover quickly (e.g., visual-spatial domain) are more likely to show minimal deficits by 2 years, while impairments will persist on those that recover more slowly (e.g., memory domain). There was a large degree of variability between and within the other three domains (attention, executive function and general cognition) in terms of their initial degree of impairment and the rate of recovery over the 2 years post-injury. The heterogeneity observed in the domains suggests that the individual measures are probably assessing different neuropsychological processes. This highlights the difficulty in classifying neuropsychological tools into discrete cognitive domains.
It is acknowledged that this review was subject to limitations. There was a dearth of studies specifically examining cognitive recovery using longitudinal designs. The searches of the current paper yielded a large volume of studies, yet only a small subset was methodologically robust enough to be included the review and even fewer provided the necessary data to calculate effect sizes.The effect sizes were thus calculated with data from only four studies, and the pooled sample size of 523 participants is relatively small. Most notably, several high-quality studies (those with a large sample size, variety of measures and frequent assessment time-points) had to be excluded from the effect size analyses because comparison groups were not included and/or the appropriate data were not provided. However, the current results clearly indicate that it is important to view cognitive recovery as a heterogeneous process and not as a homogeneous entity. In relation to the aims of this systematic review, a number of the 20 studies included had methodological limitations. Yet, it needs to be said that investigating cognitive recovery was not the primary aim of many of these studies, and therefore it is not appropriate to critique the methodology of these studies based on the objectives of the present review.
PART II – METHODOLOGICAL ISSUES AND RECOMMENDATIONS
The available literature from the systematic review highlights a number of methodological issues and provides a basis for suggestions to enable the development of a more standardised data set for future research.
Longitudinal Data
In the existing literature there was a wide variation in the timing of assessments, making it difficult to compare recovery rates between the samples. In accordance with Singer and Willett (Reference Singer and Willett2003), studies need to employ a minimum of three assessment time-points in the first 2 years following injury; the epochs used in the current review (every 3 months in the first year and then 6 monthly in the second) are at a sufficient cadence to detect change. Ideally, cohorts would complete regular follow-up assessments (i.e., every 3–5 years) to examine cognitive recovery over the longer term.
Core Battery of Cognitive Measures
The present review found relatively little commonality in the measures employed to investigate cognitive recovery in the existing literature. Due to the heterogeneous cognitive recovery process, an agreed core battery of measures that assesses the full range of cognitive domains needs to be used consistently by different research groups. Recently, several sets of measures have been recommended to assess TBI outcome (see, for example, Bagiella et al., Reference Bagiella, Novack, Ansel, Diaz-Arrastia, Dikmen, Hart and Temkin2010; the NIH Toolbox (Gershon et al., Reference Gershon, Cella, Fox, Havlik, Hendrie and Wagster2010); Wilde et al., Reference Wilde, Whiteneck, Bogner, Bushnik, Cifu, Dikmen and von Steinbuechel2010). Typically, global outcome is examined by assessing several domains (e.g., emotion, motor, social participation). Cognition is a core outcome variable, but generally a limited set of cognitive measures is included (range: n = 3–7). A notable advantage to the set of measures suggested by Wilde et al. (Reference Wilde, Whiteneck, Bogner, Bushnik, Cifu, Dikmen and von Steinbuechel2010) is its hierarchical structure, which delineates between core, supplemental and emerging outcome measures. This hierarchy is important because it may circumvent the ongoing debate over the use of ‘fixed’ versus ‘flexible’ batteries (Bigler, Reference Bigler2007, Reference Bigler2008; Hom, Reference Hom2008; Larrabee, Reference Larrabee2008; Russell, Reference Russell2007). Certainly, given the relative paucity of longitudinal data, obtaining reliable data from a core set of consistent measures would provide valuable groundwork to build a standardised data set. The risk is, however, that the specific details of cognitive recovery may be missed by using too few measures, which do not adequately sample those cognitive processes sensitive to disruption after TBI.
There is some commonality in the measures advocated and they are largely consistent with those frequently used by the studies included in the present paper. For example, the inclusion of list-learning measures (Rey Auditory Verbal Learning and California Verbal Learning Tests), tests of auditory working memory and processing speed (Digit Span and Processing Speed Index from the WAIS), and verbal fluency (Controlled Oral Word Association Test) are recommended. Also included are tests of attention and mental flexibility (Trail Making Test) and measures of inhibition control (Stroop test). However, other important neuro-psychological processes were not included. For example, the current review found significant impairments on visual-perception tests at 3 and 6 months post-trauma. Although, visual-perception deficits mostly resolved over the subsequent 18 months, there still needs to be adequate assessment of these cognitive processes to describe accurately the recovery process. Adequate assessment of cognitive domains that are more robust to TBI impairment, such as basic language and motor skills, should also be incorporated to ensure that any potential deficits in these areas are not influencing other domains and to observe the differential recovery (or lack thereof) in these domains.
Although practice effects were generally not addressed by the studies included in the review, Rawlings and Crewe (Reference *Rawlings and Crewe1992) focused specifically on this topic using the index and subtest scores from the WAIS-R. They argued that practice effects have a differential influence on healthy control participants compared to TBI samples. Therefore, Rawlings and Crewe (Reference *Rawlings and Crewe1992) compared performances of a TBI group who were assessed four times (2, 4, 8 and 12 months post-injury) with a TBI group assessed only twice (2 and 12 months post-injury). Their results suggested that recovery accounted for the majority of improvement in performance over time but there was evidence of practice effects on some tasks. However, the clinical significance of these practice effects is questionable, given that they translate to, at most, a PIQ score increase of 3 index points. Nonetheless, including measures with several parallel forms will ameliorate the effects of repeated exposure to the assessment stimuli.
There is a limited number of established batteries which have been developed for general neuro-psychological assessment purposes, such as the Halstead–Reitan Battery (HRB; Reitan & Wolfson, Reference Reitan and Wolfson1993), and the Luria–Nebraska Neuro-psychological Battery (LNNB; Golden, Purisch, & Hammeke, Reference Golden, Purisch and Hammeke1991). Although these batteries have been used with the TBI population, they were not specifically developed for the TBI group and have been criticised as not reliably capturing the typical pattern of impairments following a moderate-to-severe TBI and hence their appropriateness (Bigler, Reference Bigler2007, Reference Bigler2008; Hom, Reference Hom2008; Larrabee, Reference Larrabee2008; Lezak et al., Reference Lezak, Howieson and Loring2004; Russell, Reference Russell2007). Volbrecht, Meyers and Kaster-Bundgaard (Reference Volbrecht, Meyers and Kaster-Bundgaard2000) developed a semi-flexible battery to be used specifically in the TBI population; however, this battery contains many of the same measures as found in the HRB and LNNB. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, Reference Randoph1998) and the Neuropsychological Assessment Battery (NAB; Stern & White, Reference Stern and White2003) may be preferable because they contain at least one parallel form and are co-normed. Yet, these batteries are not without their limitations (Strauss et al., Reference Strauss, Sherman and Spreen2006); for example, the low internal reliability of the NAB items and limited number of measures within the RBANS, making it more useful as a screening tool, rather than as a comprehensive measure of cognitive functioning. Critically, this highlights the current limited pool of measures from which to select, and those that are available are not necessarily ideal. This represents a clear gap in neuro-psychological literature and is a major obstacle to methodologically robust research in this area.
Consistent Measurement of Injury Severity
In the current review, it was difficult to examine the effect of injury severity on the pattern of cognitive recovery because of the inconsistency in the severity measures used, as well as many studies not reporting appropriate descriptive data for these measures. The TBI population is not a homogeneous group; generally, the magnitude of impairment differs by injury severity, that is, more severe injuries are typically associated with greater levels and combinations of impairments (Dikmen et al., Reference *Dikmen, Machamer, Winn and Temkin1995). Therefore, injury severity is an important factor to consider when examining cognitive recovery. The results from the present review suggest that PTA duration, GCS scores and length of coma were the most common measures used. While these are important (and historical) indicators of injury severity, there is currently not an agreed-upon ‘gold standard’ measure of injury severity. Until there is an agreed recommendation for a single injury severity measure, it is advisable for studies to report several measures to enable comparative analysis in the literature.
Reporting of Appropriate Descriptive Statistics
Several significant papers could not be included in the effect size analysis because these papers did not report the appropriate descriptive statistics required for the calculation of effect sizes. This limited the data available (only four studies) to examine the cognitive recovery process. There is debate in the extant literature regarding the most appropriate statistical tests to examine cognitive recovery. Statistical analyses advocated in the literature include data transformation, multi-level modelling, and non-linear and linear mixed effects modelling (Christensen et al., Reference *Christensen, Colella, Inness, Hebert, Monette, Bayley and Green2008; Dikmen, Reitan, & Temkin, Reference Dikmen, Reitan and Temkin1983). Although the consistent use of one statistical approach would improve comparative analysis of the emerging data set, it is arguably just as important for the descriptive data to be consistently reported (e.g., mean scores and standard deviation). Providing these would allow for the calculation of effect sizes, as required for the current review and for any future meta-analyses, and it increases the clinical application of the data set. Consequently, clinicians could more readily use such a data set to track the progress of an individual patient or to examine whether a clinical sample is similar to the general TBI population. Therefore, data need to be reported in a metric which is easily understood and usable by both researchers and clinicians.
Inclusion of an Appropriate Comparison Group
The current review found only 11 studies that included an appropriate comparison group. To adequately examine cognitive recovery in the TBI population, the performance of the TBI sample needs to be compared to an appropriate comparative sample. An appropriate comparative sample is one that has data on all measures from a single sample and comprises non-brain-injured participants who are closely matched to the TBI sample on demographic characteristics. Concurrently collecting data from a matched local reference group controls for the effect of other extraneous variables on the pattern of recovery. Without such data, the alternative is to use the normative data from the specific measures to compare with the TBI samples’ performance. However, this method is problematic because it involves comparing the TBI group to several different normative samples (depending on the number of measures used). Consequently, it would be difficult to determine if any differences in impairment among the measures were due to actual differences in performance or different normative samples. Nearly half of the studies in the present paper did not include an appropriate comparison sample and this subsequently limited the data available to describe the natural process of cognitive recovery. The use of co-normed measures could potentially circumvent this difficulty, especially once a comparative data set with consistent measures was established.
Inclusion of Other Non-Cognitive Measures
Many of the studies in the current review failed to provide basic demographic variables, therefore it was difficult to make clear comparisons among the TBI samples. Cognitive impairment and recovery are influenced by a range of contextual issues that cannot be evaluated solely with neuropsychological measures. The finding highlights the need to improve the reporting of demographic information in future research; core descriptive data to be reported include sex, age, years of education, and pre-morbid vocational level and status. In addition, other important key prognostic variables to assess are estimates of pre-morbid intellectual functioning, current and pre-morbid psychological symptomatology, and measures of motor–sensory function. Systematically incorporating these other variables into cognitive recovery research will help to contextualize recovery and provide detailed information about those variables that may confound or predict later cognitive functioning.
Conclusion
In conclusion, based on the available systematically reviewed evidence, a discernable pattern of cognitive recovery emerged; improvement in cognitive functioning continued throughout the entire first year following moderate-to-severe TBI and into the second year. Moreover, the current results demonstrate that the rate of recovery was different for various cognitive domains. Given these findings, it is misleading to discuss cognitive recovery as a homogeneous entity. The present paper has also highlighted the relative scarcity of methodologically robust studies in the literature. Consequently, the current results are limited by the modest data sets available. To improve the evidence base, the paper made recommendations for future research based on the apparent methodological issues within the literature. The evidence from the systematic review revealed the need for a shift in the methodology used to examine cognitive recovery after moderate-to-severe TBI, from largely cross-sectional research paradigms to prospective longitudinal designs.
Acknowledgements
We wish to thank Noeline Monaghan for her statistical advice. There are no conflicts of interest to report. This manuscript has not been published elsewhere and it has not been submitted simultaneously for publication in another source. This study was supported by the University of Sydney, and funding was provided by the Rehabilitation Studies Unit, University of Sydney; the Northern Medical Research Foundation and the Northern Clinical School, University of Sydney; and the Australian Government through an Australian Postgraduate Award to the first author.
Appendix 1 - Database search terms
Medline (Search finalised 31 October 2012):
1. head injury.mp. or Craniocerebral Trauma/
2. traumatic brain injury.mp. or Brain Injuries/
3. closed head injury.mp. or Head Injuries, Closed/
4. 1 or 2 or 3
5. recovery.mp. or “Recovery of Function”/
6. 4 and 5
PsycINFO (Search finalised 31 October 2012):
1. head injury.mp. or exp Head Injuries/
2. traumatic brain injury.mp. or exp Traumatic Brain Injury/
3. closed head injury.mp. or exp Head Injuries/
4. 1 or 2 or 3
5. exp “SPONTANEOUS RECOVERY (LEARNING)”/ or exp “RECOVERY (DISORDERS)”/ or recovery.mp.
6. 4 and 5
CINAHL (Search finalised 30 January 2009):
1. head injury.mp. or exp Head Injuries
2. exp Brain Injuries/ or traumatic brain injury.mp. or exp Cerebral Ischemia
3. exp Head Injuries/ or closed head injury.mp. or exp Brain Injuries
4. 1 or 2 or 3
5. exp RECOVERY/ or recovery.mp.
6. 4 and 5
