DISCUSSION

The primary aim in this study was to establish whether DEX ratings, particularly FM-DEX ratings, provide an accurate source of information about the presence of ED. The DEX was found to be a statistically reliable scale. In addition, DEX ratings by clinical neuropsychologists and occupational therapists, each familiar with the participant and each qualified to assess the cognitive and functional consequences of TBI, were found to be strongly associated. NP- and OT-DEX ratings were moderately associated with impaired processing speed and length of PTA, both good predictors of outcome following TBI. They were not, however, associated with estimated premorbid intelligence. Scores from the DEX were strongly associated with corresponding scores from the eDEX, a comprehensive 65-item scale developed on the basis of a functional model of ED formulated by Banich (1997).

While mean family member rating scores for both the DEX and eDEX were statistically indistinguishable from those provided by neuropsychologists and occupational therapists, FM ratings on both scales were only moderately associated with ratings provided by professionals. In addition, FM ratings were moderately negatively associated with estimated premorbid intelligence, and were associated only weakly with impaired processing speed and not at all with PTA. This indicates that the family members in this study did not provide an accurate assessment of the participant's postinjury level of functioning.

This finding is potentially problematic, as family members and/or carers have been used in previous studies to complete the DEX (Burgess et al., 1998; Chan, 2001; Channon & Crawford, 1999; Evans et al., 1997; Norris & Tate, 2000; Wilson et al., 1996). It is not surprising, however, that family members in this study were less sensitive to ED than were professional persons trained to identify this disorder. The population was relatively acute, with many participants remaining hospitalized at the time of assessment. Even a family member who visits the TBI patient on a regular basis is unlikely to observe subtle signs of ED while the patient remains in a structured setting, since this environment obviates the need for “executive” skills. In addition, the instructions for completing the DEX do not specify a time frame, so it is possible that at least some family members adopted a long-term perspective, assessing the patient's premorbid behavior rather than their behavior following TBI. This is consistent with the finding that family member ratings were moderately negatively associated with estimated premorbid intelligence and could easily be avoided by providing more specific instructions with the DEX. The results, however, serve to caution against using family member ratings without careful consideration of the population being studied, and may partially account for conflicting results obtained previously.

In most previous studies using the DEX, the brain-injured participants came from a chronic population. Family members were therefore probably better able to assess their daily functional capacity, and it would be more appropriate to adopt a long-term perspective when doing so. This might explain why FM-DEX ratings have previously been reported to be strongly associated with other putative measures of ED (Burgess et al., 1998), including the BADS total score (Wilson et al., 1996). In one study, however, a poor correlation between FM-DEX ratings and the total BADS score was obtained for one of two patient groups (Evans et al., 1997). This may indicate that FM-DEX ratings failed to provide an appropriate reference point in this population, or that the BADS was inaccurate as a measure of ED. Another relevant study by Norris and Tate (2000) tested patients with either TBI or multiple sclerosis. This study found that neither the overall BADS score nor five of the subtest scores were significantly associated with FM-DEX ratings. Even more problematic, the significant association between the final subtest and the FM-DEX was opposite to that expected. Again, therefore, this may indicate that the FM-DEX is not always the best measure against which to evaluate test performances or, alternatively, that the tests used were not sensitive to ED, as measured by the DEX in these samples.

If it is not appropriate to assume that FM-DEX ratings are always an accurate indicator of ED, then an alternative source of information is required in order to evaluate the usefulness of formal testing instruments. Family members are generally acknowledged to be more accurate raters than are participants (Evans et al., 1997; Prigatano, 1996; Sbordone et al., 1998) and this was confirmed in this study, with participant ratings not being associated with any other indicator of ED. In the present study, however, a far more accurate assessment was found to be provided by the DEX (and eDEX) ratings provided by the treating clinical neuropsychologist and occupational therapist, an approach previously suggested by Channon and Crawford (1999) and employed by Knight et al. (2002).

Unlike family members, the neuropsychologists in our study had the advantage of having conducted at least one formal assessment of each patient, using standard cognitive tests. This has the potential to confound the use of neuropsychologists ratings as a “gold standard” against which other ED measures can be compared, since their ratings may at least partially reflect the same test results being evaluated as putative indicators of ED. To guard against this possibility, the neuropsychologists in this study were required to complete the rating scales prior to accessing the test data collected as part of the study. While they did potentially have access to test data obtained during previous assessments for some participants, the entire BADS had never been administered previously to any participant. At most, a small number of participants may have previously been administered one or two individual subtests from the BADS (Zoo Map Test and Key Search Test). This could not be ascertained but no BADS subtest had been administered within the most recent assessment.

In addition, while the occupational therapists had access to the neuropsychologist's report for some participants, they had no direct access to neuropsychological test data. The fact that rating scores from the two professional groups were so strongly associated, and that many of the same measures were found to be associated with both the NP- and OT-DEX, therefore strengthens the argument that the professional rating scores provided unbiased and accurate indicators against which test performances could be assessed. ED-related behavior observed by occupational therapists is clearly reasonably consistent with that identified by neuropsychologists on the basis of a formal assessment, although those few differences observed may indicate subtle differences in the way that particular behaviors are perceived and this should be followed up in subsequent research.

A secondary aim in this study was to determine the sensitivity of the BADS to ED, as measured using the NP- and OT-DEX scores. Moderate negative associations between the BADS total score and the NP- and OT-DEX were found, supporting the hypothesis that the BADS may be reasonably sensitive to ED. The degree of association between several subtests and NP- or OT-DEX ratings was equally if not more robust, however, indicating that some of the subtests may be effective in isolation. Of more concern, three BADS subtests were not significantly associated with either NP- or OT-DEX ratings. Moreover, stepwise regression analyses revealed that only one subtest, the Modified Six Elements Test, made a significant unique contribution to the prediction of NP-DEX scores. Similarly, only the Action Program Test made a significant unique contribution to the prediction of OT-DEX ratings. These results indicate that the BADS battery may be of only limited utility to those seeking to identify ED in a clinical population; one or two subtests from the BADS may be just as sensitive to ED as is the entire battery.

If ED represents a multidimensional construct, then it is likely and indeed desirable, that individuals with different types of ED should fail only selected tests within a comprehensive test battery. However, the population in this study was particularly selected because of the known heterogeneity of ED associated with TBI. Thus, it is difficult to explain why this study should fail to find significant correlations between each subtest and the DEX, when these were reported by Wilson et al. (1996) in a similar population. Differences on the Temporal Judgement Test can probably be attributed to the use of norms inappropriate to an Australian population, as reported by Norris and Tate (2000). This cannot account, however, for the failure of the Zoo Map Test and Key Search Test to be associated with NP- or OT-DEX ratings.

Several previous studies that have directly compared BADS scores obtained by brain-injured patients and intact controls have reported similarly disturbing results. For example, while Knight et al. (2002) found that most BADS subtests correlated with the DEX, with the Zoo Map test being most sensitive, Evans et al. (1997) found that only the Action Program Test, Temporal Judgement Test, and Modified Six Elements Test discriminated between their sample of persons with schizophrenia and normal controls. Norris and Tate (2000) also found that only three BADS subtests (Action Program test, Modified Six Elements Test, and Zoo Map Test) discriminated between a sample comprised of people with either TBI or multiple sclerosis and uninjured controls, even though the groups were distinguished by several other putative tests of ED. A study by Chan (2001) found no association between the Modified Six Elements Test and DEX ratings in a normal population. Differences between available studies may be partially attributable to the exact type of ED associated with specific clinical populations, although these are not sufficiently characterized as yet to permit this hypothesis to be tested. At present, it would clearly be remiss to conclude that measurement difficulties traditionally associated with ED have been satisfactorily resolved.

The results of this study have significant implications. Importantly, while neuropsychologists often reported only limited contact with the participants in this study, sometimes as little as three one-hour assessment sessions, neuropsychologists' ratings were strongly associated with objective measures of injury severity and with ratings provided by occupational therapists, who often worked with the participants on a daily basis. This suggests that even standard neuropsychological instruments, while not providing quantitative evidence of ED, may provide a context within which subtle cognitive deficits can be noted by a trained observer. In their original description of the Six Elements Test and Multiple Errands Test, Shallice and Burgess (1991) emphasized the importance of observing qualitative difficulties on these tasks, a view echoed by Knight et al. (2002). Perhaps, as previously suggested by Rosenthal and Millis (1992), therefore, future research should concentrate not so much on criticizing the utility of existing measures of ED, but on examining how these measures might best be scored and interpreted so as to convey the information they elicit. Assessment of subtle functional deficits should take into account not only whether a person is able to perform a test or activity, for example, but also the manner by which they reach the solution, and also perhaps how well they may have been expected to perform given premorbid skills and personal characteristics.

The present study was limited by the use of a relatively acute sample, this being necessary to obtain participants familiar to both a neuropsychologist and occupational therapist. It was also limited by the use of a pure TBI sample since, if ED is as multifactorial as claimed (Burgess et al., 1998; Stuss & Alexander, 2000), it might be expected that other aetiologies may lead to systematic differences in the way that ED is expressed behaviorally. It would be useful to collect sufficient data so that participant groups could be analyzed according to the location of brain injury sustained. One might well find that ED is manifest differently according to the exact location of damage (see Stuss & Alexander, 2000, and Burgess et al., 2000, for discussions relevant to this issue) but this could not be addressed in this study. Recent attempts to factor analyze multiple measures from individual neuropsychological tests sensitive to ED suggest that a more comprehensive analysis of available indicators, requiring far more participants that were available in the present study, may prove useful in identifying the range of skills which contribute to ED (Greve et al., 1997).

In summary, family members may not provide an accurate source of information concerning the presence of ED in a population of TBI patients presenting for rehabilitation, many still in an acute stage of recovery. This is particularly problematic when an instrument is used that fails to specify the period during which the patient's behavior should be assessed. With the exception of this oversight in the instructions for the scale, the DEX seems to provide a useful instrument. When administered to an appropriate respondent it can be used as an indicator of ED, against which other putative indicators can be evaluated. Using this approach, it seems that the BADS contains several useful subtests. Other components of the BADS may be less useful and, in a TBI sample at least, may not add to the sensitivity of the overall battery. When used judiciously a combination of neuropsychological instruments may facilitate detection of ED. It seems, however, that qualitative information may be equally useful, perhaps because the concept of ED is simply not yet sufficiently specified to permit a rigorous, quantitative approach. Executive difficulties in the initiation and regulation of complex behaviors are likely to reflect a number of syndromes, not yet clearly defined or able to be distinguished empirically. If so, then it is not surprising that different studies, using different participant populations and different combinations of tests, continue to report conflicting results. Future research, using larger sample numbers and either qualitative data analyses or multivariate statistical techniques, is needed to address a number of outstanding issues.