Design

We employed a single-case experimental design with multiple baselines across settings in the present study (Domholdtz, 2005). The four-week baseline, eight-week treatment, and four-week maintenance and generalization periods for one setting were conducted in conjunction with a 12-week baseline and four-week treatment period in the second setting. This design enabled evaluation of any changes in the participant's performance in response to the intervention in one setting, whilst examining any changes in performance in a different setting in which a behavioral observation procedure was used but no intervention was provided until the 13th week.

Participant background information

At the time of his injury in 2000, JM was single and working as an assistant driller on an offshore oilrig. Since leaving high school after Year 10, with “average” grades, JM had worked consistently in a range of skilled laboring positions. At age 32 JM sustained a very severe open head injury in a car accident. A Glasgow Coma Scale score of 3/15 was recorded at hospital admission. Neurosurgery was performed to remove a large piece of fence post from the right frontal region of JM's skull. A CT head scan revealed a large compound comminuted open-skull fracture with linear extension down the frontal bone to the right midline and along the floor of the anterior cranial fossa. There was an area of extensive haemorrhage beneath the fracture with moderate midline shift. The mass of the haemorrhage was greater in the right frontal region but also extended into the left frontal lobe. The hospital discharge report documented that JM had not emerged from posttraumatic amnesia (PTA) at six months post-injury. Consistent with right hemisphere damage, JM displayed left-sided neglect and his left upper limb was nonfunctional except to stabilize objects with prompting. JM attended outpatient occupational and physical therapy for 18 months. At two years post-injury, a prevocational assessment provided the opinion that JM was not commercially employable but might have the capacity to perform volunteer work, which he commenced in 2003. JM consented to participate in the present study, which was conducted in accordance with institutional research standards for human research.

Neuropsychological and psychological assessment

JM received a neuropsychological assessment at two and a half years post-injury (2002) and review at four years post-injury (2004), which included the following measures.

Measures of self-awareness. The Self-Awareness of Deficits Interview (SADI; Fleming et al., 1996) and the Awareness Questionnaire (AQ; Sherer et al., 1998a) are two standardized measures of self-awareness. The SADI is a semistructured, clinician-rated interview that assesses awareness of deficits across three subscales and yields a total SADI score of 0–9 (0 = accurate self-awareness to 9 = very poor awareness). JM's parents completed a checklist concerning JM's current level of functioning, which assisted scoring. The AQ measures self-awareness using a discrepancy score based upon individuals' and significant others' ratings on 17 items which compare current abilities to pre-injury cognitive, behavioral, affective, and motor/sensory functioning. Items are rated on a five-point scale (1 = much worse to 5 = much better), which yields a total score of 17–85 for each version. A high positive discrepancy score between the self- and significant other versions indicates that the individual has overestimated his or her level of functioning.

Measures of psychological functioning. Based on an established approach (see Ownsworth et al., 2002) for assessing the contribution of psychological factors to awareness deficits, JM completed the Marlowe-Crowne Social Desirability Scale (M-CSDS; Crowne & Marlowe, 1960) and the Symptom Expectancy Checklist (SEC; Mittenberg et al., 1992). The M-CSDS is a 33-item questionnaire that measures the degree to which individuals display defensiveness. Based upon norms (M = 15.0, SD = 5.9) (Reynolds, 1982), a score equal to or greater than one SD above the mean (i.e., ≥ 21/33) reflects a high level of defensiveness (Ownsworth et al., 2002). The SEC is a 30-item checklist of relatively benign head-injury symptoms; it measures the degree to which individuals report common symptoms compared with controls (M = 4.9, SD = 4.8) and individuals with head injury (typical score range = 11/30–18/30) (Mittenberg et al., 1992). Scores less than or equal to 10 suggest that individuals are minimising their symptoms (Ownsworth et al., 2002).

JM's emotional status was assessed using the Hospital Anxiety and Depression Scale (Snaith & Zigmond, 1994), which has 14 items rated on a four-point scale to reflect how an individual has been feeling in the past week on a range of symptoms related to depression and anxiety. The anxiety and depression subscales are each scored from 0 to 21 (0–7 = normal, 8–10 = mild, 11–14 = moderate, and 15–21 = severe).

Measures of executive functioning. Standardized measures of executive functioning were selected to identify the influence of neurocognitive factors underlying deficits in self-awareness (Ownsworth & Fleming, 2005). These included the Health and Safety subtest of the Independent Living Scales (Loeb, 1996), Tinker Toy Test (Lezak, 1993), Five-Point Test, FAS test (see Spreen & Strauss, 1998), and the Key Search Test and Zoo Map Test from the Behavioural Assessment of Dysexecutive Syndrome (Wilson et al., 1996).

Assessment findings, opinion, and rationale for the intervention

JM's performance on the assessment measures at two and a half years and four years post-injury is summarized in Table 1. JM displayed a global cognitive decline with severe deficits in visuo-spatial functioning, visual memory, and processing speed. His performance on measures of executive functioning varied between the “low average” and “severe impairment” range. Some improvement was observed over time in relation to aspects of verbal memory, auditory attention, and visual neglect. Overall, JM's neuropsychological profile was consistent with right hemisphere and frontal lobe damage. JM demonstrated a severe impairment in self-awareness at two and a half years post-injury and at four years post-injury. JM's parents reported extensive physical, cognitive, and behavioral changes. JM reported only minor difficulties with physical functioning (e.g., reduced strength on his left side). His explanation of difficulties was related to his premorbid self. For example, he attributed his left-sided weakness to being right handed and his poor fine-motor manipulation to his pre-injury work duties, which involved mainly gross manual handling. JM's responses on self-report measures indicated that he was not displaying defensive denial.

The Results of Assessments Conducted at 2.5 years and 4 Years Post-Injury

An interview with JM's parents and observation of JM in social settings suggested that people in his home and volunteer work environments had adjusted their expectations of his performance on functional tasks. For example, JM was not expected to cook, remember appointments, or assist with household tasks. His parents and co-workers made corrections on his behalf and provided reassuring comments. When JM was sweeping at work he did not move out of the way of the forklift driver, who accommodated for this by driving in wide arc around JM, or waiting for him to finish sweeping (note: JM explained that as a higher-level employee on the oilrig before his injury, forklift drivers used to drive around him).

Overall, neurocognitive factors were considered to be the main issue underlying JM's awareness deficits due to his very severe TBI, extensive area of haemorrhage in the right hemisphere and frontal region, and evidence of global cognitive and executive deficits, including poor error self-regulation. However, his tendency to attribute difficulties to his pre-injury functioning and the lack of feedback opportunities indicated that psychological and socioenvironmental factors had maintained his awareness deficits. Thus, an intervention based upon guidelines for neurocognitive and socioenvironmental interventions was developed.

The need for an intervention and potential for emotional distress were initially discussed with JM and his family. JM identified two main goals, which included becoming more independent at home and returning to paid employment. JM had not cooked meals since his injury due to his parents' safety concerns and had not attempted to return to paid work. He had volunteered in a second-hand clothing and furniture factory for approximately twelve months. It was decided that a systematic feedback approach be used to target error behavior (i.e., self-monitoring and correction) on functional tasks to achieve his cooking and paid employment goals. This approach was chosen in preference to individual or group psychoeducation approaches that focus primarily on facilitating global self-knowledge of deficits (Ownsworth et al., 2000). However, JM received feedback about his post-injury changes as part of the rationale for the intervention. The intervention incorporated education for JM's social supports and demonstrated the use of effective feedback and prompting during his performance on cooking and work tasks.

Initial observations and baseline assessments

JM chose four meals to prepare, including spaghetti bolognaise, rissoles, chicken stir-fry, and a casserole. Prior to the intervention, behavioral observations of JM's performance were conducted in the cooking and work settings. Behavioral measures of on-line awareness were developed based upon the procedure by Hart et al. (1998). These involved recording error frequency and error behaviors during performance in 1-hour observations of JM's performance in the cooking and work settings. The following guidelines were developed for recording these aspects of performance.

Error frequency. An error was recorded when JM's actions were considered to compromise: (a) his own or others' safety, (b) the outcome of the meal or work task, or (c) time efficiency. Independent ratings of error frequency were conducted during an initial observation by two therapists in each setting. Exact percentage inter-rater agreement was 83% in the cooking setting and 91% in the work setting. To maximize accuracy using this approach and provide training to JM's social supports throughout the 16-week intervention, the therapist and either JM's mother or his work supervisor recorded errors in respective settings and compared observations to reach agreement.

Error behavior. The classification of error behavior and systematic prompting procedure was derived from principles of the “Pause, Prompt, and Praise” technique (Glynn et al., 1979; McNaughton et al., 1987), and tailored to meet JM's needs in the present study. This included delayed response to errors detected by the observer, which provided JM with the opportunity to self-correct, or to attempt to self-correct—an action that is unlikely to occur when an observer immediately intervenes to correct an individual's error (McNaughton et al., 1987). When the observer identified that JM was making an error, he or she would pause to provide an opportunity for JM to identify and self-correct his errors (This was not possible on two occasions when JM's safety was considered at risk). When self-correction did not occur following the pause, standard nonspecific prompts were used which included: “can I get you to stop and tell me what you are up to right now?” If JM was unable to correct his error with a nonspecific prompt, the observer provided a specific prompt, such as “can you check the recipe and see what goes in the mixing bowl first?” These error behaviors were classified as “self-correction,” “nonspecific prompts,” and “specific prompts.” Based on these guidelines, two therapists' independent ratings of error behavior yielded an exact agreement of 88% for cooking and 94% for volunteer work.

Treatment, maintenance, and generalization

Prior to the baseline assessment, JM practiced cooking each meal with his mother's support. A subsequent four-week baseline assessment identified a trend for a gradual decline in the frequency of errors (Hersen, 1990). The treatment intervention commenced in the fifth week with structured opportunities for JM to spontaneously identify and correct his errors, or receive systematic external prompting for correction.

During the first of the eight treatment sessions, a role-reversal technique (Toglia, 1998) was used in which JM observed his mother cooking and making the same type of errors he made at baseline. JM was encouraged with prompting from the therapist to stop his mother and describe the errors she was making and corrective actions. JM made the same meal immediately after this demonstration. An electronic timer was introduced as an alerting device to remind JM to check the recipe every 3 minutes. In response to the alarm, JM was to stop what he was doing (if cooking, he turned the frying pan down), read out aloud the last step, the current step, and the future step of the recipe, then reset the alarm and continue cooking. In a post-cooking discussion, JM was encouraged to describe the errors he observed and corrective actions, and received additional feedback about errors from the therapist.

During the second treatment session, JM initially observed a videotape of his cooking performance during the baseline period. With prompting, he was encouraged to stop the videotape, identify his errors, and describe corrective actions throughout the meal. Following a discussion of the errors and corrective actions, JM proceeded to cook the same meal according to the procedure described for the first treatment session. The remaining six treatment sessions involved the use of the electronic timer, a precooking discussion (5–10 minutes) with feedback on his baseline performance, and a post-cooking feedback discussion.

During the four-week maintenance period, the electronic timer was withdrawn and JM's mother assumed the supportive role of providing feedback and prompting, to approximate a more “real life” situation. In conjunction with the maintenance period in the cooking setting, the therapist introduced similar treatment techniques to provide feedback on JM's performance on volunteer work activities (i.e., sweeping, sorting items, and cleaning equipment) over four sessions. Additionally, his work supervisor received education and training regarding strategies for providing effective feedback. The Self-Awareness of Deficits Interview, Awareness Questionnaire, and Hospital Anxiety and Depression Scale were re-administered at the end of the program.

Error frequency

During the four-week baseline period on the cooking task (see “a” in Figure 2) JM made an average of 21 errors with the error frequency gradually declining over the baseline period. However, in the first week of treatment (week 5), a marked decline in the error frequency was observed. During the 8-week treatment period in the cooking setting, a 44% reduction in the average error frequency (11.8) was observed relative to the baseline period. The average error frequency (11.0) in the maintenance period indicated that the treatment effect was maintained. The error frequency observed in the volunteer work setting during the 12-week baseline period (see “b” in Figure 2) varied from 9 to 16 (M = 12.3) with no particular trend evident. However, when treatment was introduced in the work setting in week 13, a 39% reduction in errors (M = 7.5) was observed and appeared stable over the four-week period.

The frequency of errors observed in performance during the baseline (A), treatment (B) and maintenance (C) periods for the cooking setting and the baseline (A) and treatment (B) periods for the volunteer work setting.

Error behaviors

Figure 3 presents the proportion of error behaviors observed on the cooking task over the 16-week period. During the baseline period, JM mainly required specific prompting for correction and self-corrected only 4–15% of errors. During the treatment period, the proportion of self-corrected errors increased to 9–27% and JM mainly required nonspecific prompting to correct errors. During the maintenance period, JM self-corrected 25–46% of errors, required nonspecific prompts for 27–42% of errors, and specific prompts for 27–33% of errors. Thus, over time JM displayed greater self-correction and correction of errors with nonspecific prompts.

The proportion of error behaviors observed in performance on the cooking task over the 16-week period (A = baseline, B = treatment and C = maintenance).

General awareness of deficits

Compared with JM's pre-intervention performance on the Self-Awareness of Deficits Interview (2,2,2 = 6/9), no appreciable improvement (2,2,1 = 5/9) was observed in his level of awareness after the 16-week program. On the Awareness Questionnaire (AQ), a comparison of pre-intervention scores (self = 45; significant other = 27) with post-intervention scores (self = 43; significant other = 33) indicated a decrease in the level of discrepancy between ratings. However, the 8-point decrease occurred mainly as a result of JM's parents rating his skill level as higher on the AQ. Overall, there was no clinically meaningful change observed in JM's self-awareness on these measures. JM's self-reported level of emotional well-being at post-intervention (anxiety score = 6, depression score = 4) was similar to pre-intervention.

Programming for generalization

Some general observations provided evidence of spontaneous generalization, particularly on the cooking task. For example, during the treatment period JM began checking the recipe as needed between the 3-minute alarm intervals. Additionally, during the maintenance period, without prompting, JM turned down the frying pan when the telephone and doorbell rang, attended to these interruptions and then returned to the cooking task. However, it was apparent that JM would always require a degree of supervision and prompting during cooking. Additional strategies were discussed with JM and his parents to promote his independence.

Due to the positive outcome of the cooking and volunteer work interventions, JM was encouraged to seek paid employment with assistance from a disability employment service. Three weeks after the intervention he gained paid part-time work (16 hours per week) in a plant nursery with initial one-to-one support from a job coach. The therapist conducted a behavioral observation and provided training to JM's job coach and supervisor regarding the use of effective feedback for errors. After four weeks, JM no longer required a job coach and his supervisor reported a decrease in errors and an associated reduction in the level of supervision required.

Mechanisms underlying change

There are a number of proposed mechanisms underlying the decline in errors in JM's performance. One possible explanation is that the systematic error prompting procedure contributed to the initial decrease in errors observed during the four-week cooking baseline period. However, the concurrent use of this procedure in the work setting during the 12-week baseline was not associated with a decline in errors. Alternatively, it is possible that the reduction in errors in the cooking baseline period reflected behavioral change associated with task-specific learning and habit formation (Sohlberg et al., 1998). Although JM had been volunteering for approximately 12 months, he started cooking just prior to the baseline and, thus, the gains made during this period may have represented practice effects. Therefore, it may also be argued that the further reduction in errors and associated functional gains observed during treatment were due to practice. However, the lack of improvement in self-correction of errors during the cooking baseline was not consistent with this explanation.

Although a stable error behavior baseline was not established, JM's ability to self-correct errors without prompting or with nonspecific prompts improved following an introduction of the systematic feedback techniques. These techniques were designed to promote JM's self-evaluation of performance, anticipation of errors, use of corrective strategies and “on-line” self-correction during performance (Toglia, 1998). The greater proportion of self-corrected errors and corrections to nonspecific prompts following the baseline period suggested that an increase in self-regulatory behavior contributed to JM's functional gains (McNaughton et al., 1987). It is possible to speculate about underlying mechanisms for functional gains using the Cognitive Awareness Model (Morris & Hannesdottir, 2004). Specifically, the provision of systematic feedback may have served to update JM's personal database (PDB) concerning other people's expectations of his performance. In terms of self-appraisal of his performance, it is unclear whether errors were only perceived in the immediate sense, thus influencing behavioral change (i.e., self-correction) through implicit knowledge, or if errors were maintained in the PDB either with or without input received by the conscious awareness system (Morris & Hannesdottir, 2004). Other researchers have indicated that behavioral gains can occur without conscious awareness through implicit processes (e.g., Bieman-Copland & Dywan, 2000; Sohlberg et al., 1998). Overall, it appeared that JM's global self-knowledge concerning his impairments was unchanged and that the increase in self-monitoring was context-specific, although the extent to which JM was consciously aware of his errors on-line cannot be determined.

Such findings suggest that a metacognitive contextual intervention might have more clinical efficacy for facilitating on-line awareness than updating self-knowledge. As previously discussed, an a priori decision was made on the basis of JM's neurocognitive deficits and personal goals to target his error behavior in specific real-life contexts rather than primarily aiming to increase his self-knowledge. The latter therapeutic objective would appear to require an alternative rehabilitation approach, such as intensive psychoeducation or group programs (Ownsworth et al., 2000), which might not be specifically tailored to achieving functional goals. The key implication of this study is that rehabilitation planning requires due consideration of the nature of the clients' awareness deficits and the type of clinical outcomes that are perceived to be most beneficial or valued by the client. Whilst it may be important for some clients to clearly understand that their difficulties are due to a brain injury rather than pre-injury lifestyle or circumstances, other individuals, such as JM, may be taught to monitor their performance and use corrective strategies without a deeper understanding of their deficits. Further, some individuals might be at risk of heightened emotional distress if confronted by their deficits and by developing a deeper understanding of these (Ownsworth & Clare, 2005).

Methodological considerations

A number of limitations need to be acknowledged in this study. In particular, different feedback techniques were introduced in combination and it is not possible to isolate the therapeutic effects of specific treatment components. Additionally, the assessment of error behavior involved an individualised approach relevant for JM and although inter-rater agreement was examined, this approach has not been standardized. Various limitations of single-case designs have been discussed in relation to internal validity and generalisability (see Domholdtz, 2005). In the present study, the multiple-baseline design enabled examination of extraneous factors contributing to the changes observed throughout the program, which have been discussed. Detailed neuroimaging data were not available to permit a more precise description of the extent of JM's injury. For instance, more specific localization of JM's injury in terms of the relative damage to the ventral medial and lateral frontal regions would potentially have yielded greater understanding of neurocognitive mechanisms of self-awareness and guided recovery (Stuss & Anderson, 2004). However, the provision of comprehensive baseline data concerning JM's neuropsychological and psychological functioning may help readers ascertain the relevance of the findings to other individuals with TBI.

Additionally, it was not possible to determine in this exploratory study the extent to which self-correction on the cooking task increased as a function of practice, particularly given the variability observed in the four-week cooking baseline. An extended baseline would have enabled a more detailed examination of trends in error behavior and potential practice effects prior to introducing the feedback techniques. Thus, whilst it may be assumed that JM learnt to self-monitor and remain on task to enable him to self-correct or adjust his behavior according to non-specific cues, the extent to which metacognitive changes occurred at a more intrinsic level, as a function of the intervention, remains unclear.

Finally, it may have been enlightening to examine JM's theory of mind, or ability to make inferences about others' mental states, using perspective-taking, false beliefs, faux pas or related tasks (Stone et al., 1998; Stuss & Anderson, 2004). Despite the growing body of literature on self-awareness and theory of mind, there is a relative absence of empirical research examining the association between self-awareness deficits and theory of mind following TBI.

In conclusion, the present study provides some initial empirical support for the efficacy of a metacognitive contextual intervention for improving self-correction and functional performance. The importance of considering the relative contribution of factors underlying awareness deficits was emphasized. Further research is needed to develop standardized and ecologically valid procedures for assessing error awareness and to evaluate metacognitive contextual interventions. The efficacy of the present approach needs to be evaluated in a group study of individuals with awareness deficits related to neurocognitive and socioenvironmental factors.