INTRODUCTION
One of the greatest challenges faced in the treatment of aphasia is how to deliver intervention that yields real language improvements and facilitates generalisation to everyday communication. Group treatment, by allowing participants to practice communication with peers, has been proposed to be better suited to improving real-life communication than the typical clinician–patient dyad of individual treatment (Elman, Reference Elman2007; Kearns & Elman, Reference Kearns, Elman and Chapey2001).
Group treatment for aphasia is not uncommon. In a survey of service delivery by Australian speech pathologists, Verna, Davidson and Rose (Reference Verna, Davidson and Rose2009) found that it was used across rehabilitation settings, including 58.5% of hospital outpatient services and 100% of university clinics. However, while there is evidence that group treatment can lead to increased participation in everyday communication (e.g., Aten, Caligiuri, & Holland, Reference Aten, Caligiuri and Holland1982; Bollinger, Musson, & Holland, Reference Bollinger, Musson and Holland1993) and psychosocial gains (Elman & Bernstein-Ellis, Reference Elman and Bernstein-Ellis1999), questions remain regarding the specific language improvements that may result. In this study, we aimed to investigate this underexplored area by considering the impact of group treatment on language production with particular focus on word retrieval.
Evaluation of the group-treatment literature is complicated by the considerable variability in goals and treatment methods (Lanyon, Rose, & Worrall, Reference Lanyon, Rose and Worrall2013). While some studies have focused on compensatory techniques (e.g., Aten et al., Reference Aten, Caligiuri and Holland1982; Bollinger et al., Reference Bollinger, Musson and Holland1993), others have aimed to improve specific language deficits (e.g., Eales & Pring, Reference Eales and Pring1998; Maher et al., Reference Maher, Kendall, Swearengin, Rodriguez, Leon, Pingel, Holland and Rothi2006; Meinzer, Djundja, Barthel, Elbert, & Rockstroh, Reference Meinzer, Djundja, Barthel, Elbert and Rockstroh2005; Pulvermüller et al., Reference Pulvermüller, Neininger, Elbert, Mohr, Rockstroh, Koebbel and Taub2001). Consequently, group-treatment studies have used a range of techniques including group discussion (e.g., Eales & Pring, Reference Eales and Pring1998; Elman & Bernstein-Ellis, Reference Elman and Bernstein-Ellis1999; Wertz et al., Reference Wertz, Collins, Weiss, Kurtzke, Friden, Brookshire and West1981), semantic feature analysis (e.g., Antonucci, Reference Antonucci2009; Falconer & Antonucci, Reference Falconer and Antonucci2012), and role play (e.g., Aten et al., Reference Aten, Caligiuri and Holland1982; Bollinger et al., Reference Bollinger, Musson and Holland1993; Elman & Bernstein-Ellis, Reference Elman and Bernstein-Ellis1999).
Prompted by the limited evidence for transfer from word-retrieval therapy delivered individually (e.g., Carragher, Conroy, Sage, & Wilkinson, Reference Carragher, Conroy, Sage and Wilkinson2012; Webster et al., Reference Webster, Whitworth and Morris2015), we were particularly interested in whether group treatment could facilitate word retrieval in connected speech. Eales and Pring (Reference Eales and Pring1998) examined the effects of combined individual and group treatment for word retrieval. Participants were first trained on a specific word set in individual treatment and then received discussion-based group treatment focusing on the same topic area as the treated words. Eales and Pring argued that participants showed improved retrieval of treated words in isolation following individual treatment, which was maintained following group treatment (although lack of a stable baseline period makes the data hard to interpret). Two of the four participants were also argued to improve on untreated items. However, the design of this study (with a fixed treatment order) means that we cannot directly compare the outcomes of the two treatment approaches, and in particular, it is not possible to determine if the generalisation to untreated items experienced by half the participants was due to the inclusion of group treatment.
In a more recent study, Fama, Baron, Hatfield and Turkeltaub (Reference Fama, Baron, Hatfield and Turkeltaub2016) compared individual and group treatment for people with severe nonfluent aphasia in an acute hospital setting, using impairment-based activities such as auditory comprehension tasks and a structured language activity (e.g., a card game) as well as conversation. Fama et al. found that participants initiated communication more frequently in group treatment and produced a greater variety of unique words in individual sessions. Unfortunately, this study was constrained to a single session of each treatment approach and only considered communication within those sessions. It is therefore impossible to determine how these approaches impacted on communication outside of treatment.
Possibly, the clearest evidence for group treatment as an intervention for word-retrieval difficulties comes from studies using semantic feature analysis. Antonucci (Reference Antonucci2009) investigated semantic feature analysis therapy conducted in a small-group setting. She reported that two of their three participants showed increased rated informativeness of connected speech (using Nicholas and Brookshire’s Reference Nicholas and Brookshire1993 methods). These participants also showed some improvements in correct information unit production. In a similar study, Falconer and Antonucci (Reference Falconer and Antonucci2012) included an additional home programme in which strategies developed in the group therapy were practiced at home and all four participants showed increases in rated informativeness in discourse.
While these results are positive, the nature of treatment in these studies differed from group treatment as it is typically practiced. Lanyon et al. (Reference Lanyon, Rose and Worrall2013) suggested that the majority of aphasia group treatments in Australia are discussion-focused and conducted in community settings (e.g., community centres) or in outpatient services. In addition, Verna et al. (Reference Verna, Davidson and Rose2009) indicated that aphasia treatment is most commonly conducted once per week (with the exception of inpatient care). Hence, it remains to be determined whether more typical approaches to group treatment can lead to specific language improvements in connected speech.
Given the prevalence of discussion-focused treatment groups, in Nickels, McDonald and Mason (Reference Nickels, McDonald and Mason2016), we evaluated the effects of a typical discussion-focused group-treatment programme for three individuals with aphasia, when conducted once a week in an outpatient clinic. The group focused on the discussion of topics of general interest (e.g., health, education, pollution) aiming to facilitate the participants’ ability to converse on these topics. To examine whether additional word-retrieval support would further ‘add-value’ to the group outcomes, participants also received supplementary word-retrieval treatment with a home programme (using repetition in the presence of the picture, RIPP) for half of the treated topics. We examined whether this form of treatment improved word retrieval at the single-word level (measured by picture naming). We found that although there was no difference in naming gains made during treatment periods compared with no treatment periods, there was evidence of significantly improved naming for the topic-related treated words over the course of the study (but not for control words). This improvement was significantly greater for topics that had also received the home programme. However, preliminary examination of semistructured interviews with the participants about the treated topics showed that there was little indication of change in retrieval of treated target words.
This finding is consistent with the literature showing relatively little evidence that improvement in word retrieval as a result of individual (or home programme) therapy leads to improvement in word retrieval in connected speech (e.g., Best et al., Reference Best, Grassly, Greenwood, Herbert, Hickin and Howard2011; Croot et al., Reference Croot, Taylor, Abel, Jones, Krein, Hameister, Ruggero and Nickels2015; Mason et al., Reference Mason, Nickels, McDonald, Moses, Makin and Taylor2011; see Carragher et al., Reference Carragher, Conroy, Sage and Wilkinson2012, and Webster et al., Reference Webster, Whitworth and Morris2015, for review). For example, in Mason et al. (Reference Mason, Nickels, McDonald, Moses, Makin and Taylor2011), we found that only one of the three participants treated with a home programme for word retrieval (using RIPP) showed any indication of transfer to connected speech.
This lack of generalisation seems surprising: Given that word retrieval at the single-word level correlates with word retrieval in connected speech (Herbert, Hickin, Howard, Osborne, & Best, Reference Herbert, Hickin, Howard, Osborne and Best2008), then improvements in word retrieval would be predicted to improve retrieval of the same words in connected speech. One possible explanation is that the treatment tasks did not facilitate the wider use of the treated lexical items. An advantage of discussion-focused group treatment is that it allows language skills to be practiced in conversation and may facilitate the transfer of therapy effects to connected speech. This paper examines this prediction. We present detailed analyses of previously unanalysed connected speech data (semistructured interviews about each topic) collected as part of Nickels et al.’s (Reference Nickels, McDonald and Mason2016) group-treatment study.
Determining whether word-retrieval treatments generalise to connected speech is a challenge for aphasia research, and there is ongoing debate regarding the most appropriate method of measurement (e.g., Carragher et al., Reference Carragher, Conroy, Sage and Wilkinson2012; Croot et al., Reference Croot, Taylor, Abel, Jones, Krein, Hameister, Ruggero and Nickels2015; Hickin, Best, Herbert, Howard, & Osborne, Reference Hickin, Best, Herbert, Howard and Osborne2001; Webster et al., Reference Webster, Whitworth and Morris2015). In this study, we selected four outcome measures that had been demonstrated in previous studies to be sensitive to improvements in word retrieval as a result of treatment (and change in communication more generally): use of treated words, the Profile of Word Retrieval and Errors in Speech (POWERS; Herbert, Best, Hickin, Howard, & Osborne, Reference Herbert, Best, Hickin, Howard and Osborne2012), propositional idea density, and perceptual discourse ratings
In each treatment session, treatment focused on a set of core vocabulary that was relevant to the focus topic. Hence, our first measure considered the use of treated vocabulary in semistructured interviews. This is the measure that is most clearly predicted to improve given that the trained words were chosen to be directly relevant to these discussion topics (see Appendix D for a list of treated words). However, in a review of discourse treatments for word retrieval, Boyle (Reference Boyle2011) noted that treatment did not always result in measurable improvements to treated vocabulary in connected speech. Indeed, as noted above, our preliminary (visual) inspection of the data suggested that there was no improvement.
As gains to word retrieval more generally (rather than specifically for treated stimuli) have been more often evident in the literature (Boyle, Reference Boyle2011), we used two further analyses predicted to reflect such change: POWERS (Herbert et al., Reference Herbert, Best, Hickin, Howard and Osborne2012) and the Computerized Propositional Idea Density Rater-3 (CPIDR-3: Brown, Snodgrass, & Covington, Reference Brown, Snodgrass and Covington2007). The POWERS quantifies rates of accurately produced words and evidence of word-retrieval difficulties (such as paraphasias) in connected speech and has been shown to be sensitive to changes in word-retrieval processes in conversation (Herbert et al., Reference Herbert, Best, Hickin, Howard and Osborne2012). CPIDR-3 is an automated measure of propositional idea density, which provides an indicator of speech fluency and lexical diversity (Kemper & Sumner, Reference Kemper and Sumner2001). As word-retrieval difficulties in aphasia can result in overt breakdown in word retrieval as well as reduced lexical diversity (e.g., Fergadiotis & Wright Reference Fergadiotis and Wright2011) and disfluencies (e.g., Brown & Cullinan, Reference Brown and Cullinan1981), we would predict that group treatment with a focus on word retrieval would result in improved POWERS indices and higher propositional idea density.
Finally, we conducted perceptual ratings of discourse (Best, Howard, Bruce, & Gatehouse, Reference Best, Howard, Bruce and Gatehouse1997; Le Dorze, Boulay, Gaudreau, & Brassard, Reference Le Dorze, Boulay, Gaudreau and Brassard1994). Given the potentially subtle and multifaceted changes as a result of improved lexical accessibility, the ratings may capture changes to communication that might not be revealed through specific counts of language units. Improved word retrieval in connected speech would be predicted to lead to improvement on all three of our ratings: ability to transmit message, ability to find adequate words, and quantity of information.
To our knowledge, our study is the first to look at the impact of group treatment on connected speech using a wide range of quantitative measures focusing specifically on word retrieval. By doing so, we aimed not only to identify the extent to which discussion-focused group treatment can result in improved connected speech in areas related to lexical retrieval but also to promote the discussion of the utility of these measures.
METHOD
Participants
Three participants (two females, one male, aged 32–73 years) took part in the study (RAA, RYT and JDC). An additional fourth participant was initially recruited, however was absent for four of the six group-treatment sessions, and has therefore been excluded from analysis. The remaining participants attended all group-treatment sessions. The participants were drawn from the outpatients department of a rehabilitation hospital in Sydney. Inclusion criteria were diagnosis of aphasia (with anomia, without severe apraxia of speech or dysarthria) as a result of stroke, native English speaker, normal or corrected-to-normal vision and hearing, at least 12 months post onset and not currently receiving other speech pathology interventions. Participants provided informed consent prior to taking part, provided in aphasia-friendly written format supplemented with a verbal explanation, as approved by Macquarie University and St. Vincent’s Health Australia.
Two participants were diagnosed with mild nonfluent aphasia and one with moderate conduction aphasia. Participants were assessed prior to beginning the study using the Comprehensive Aphasia Test (Swinburn, Porter, & Howard, Reference Swinburn, Porter and Howard2005) and, critically, all displayed impaired naming. See Table 1 for biographical details and summary of assessments (Nickels et al., Reference Nickels, McDonald and Mason2016, provides further detail).
Participant biographical details and Comprehensive Aphasia Test assessment summary
Note. Memory, auditory comprehension, visual comprehension, repetition, naming, reading, and writing scores are totals from component subtests. Full subtest outcomes can be found in Nickels et al. (Reference Nickels, McDonald and Mason2016). Asterisks indicate scores below control cut-off.
Treatment
Weekly group treatment
Group-treatment sessions took place at a Sydney rehabilitation facility once a week for 6 weeks. Each session was facilitated by the third author, BM, an experienced speech pathologist. Treatment was designed to replicate as closely as possible group treatment as it is conducted in the clinical setting, in our experience in Australia. We aimed to provide multiple exposure to, and opportunities for, retrieval of, topic-related vocabulary. Each session was 1.5 hr and comprised three parts: (1) 30-min passive viewing of a topic-related video; (2) 30 min of facilitated group discussion about the video topic; and (3) 30 min of structured treatment tasks facilitated by the speech pathologist. These aimed to maximise opportunities for productions of a target word set, focused around a set of 30 core topic-related nouns, adjectives, and verbs (see Nickels et al. Reference Nickels, McDonald and Mason2016 for details regarding the word selection process). The tasks used techniques typical of word-retrieval treatment such as naming to a definition, with cueing and/or repetition as required (Nickels, Reference Nickels2002; Nickels & Best, Reference Nickels and Best1996). Although the tasks were presented to the group, the facilitator would direct questions to specific participants if required to ensure equal response opportunity.
The videos were episodes of the Australian Broadcasting Corporation documentary series ‘Australian Story’ (ABC TV, 2004–2012) inspired by Cartwright and Elliott (Reference Cartwright and Elliott2009). Each session focused on a different topic, based on the video viewed at the beginning of the session. These were education, health, refugees, pollution, indigenous issues, and war. Two videos comprising control topics (dementia, domestic violence) were viewed: one before and one following the treatment programme. In the control condition, participants watched the documentary video in a quiet room in their home with one of the authors present, with no discussion following the viewing.
Home programme
In addition to the group-treatment sessions, participants completed a word-retrieval therapy home programme for three of the six treated topics (health, pollution, war). This aimed to determine whether a treatment of proven efficacy for word retrieval at a single-word level (e.g., Mason et al., Reference Mason, Nickels, McDonald, Moses, Makin and Taylor2011) would increase the efficacy of the group treatment. As noted above, Nickels et al. (Reference Nickels, McDonald and Mason2016) found that this was the case at the single-word level.
Each home programme was completed over 2 weeks: 1 week preceding the group session for that topic and 1 week following (see Appendix A for a timeline). The home programme targeted the same 30 topic-related target words as the group session using RIPP (as described in Mason et al. Reference Mason, Nickels, McDonald, Moses, Makin and Taylor2011), presented with a PowerPoint programme using colour photographs accompanied by an audio file of the word. The participant was instructed to repeat the word after the recording. Participants were required to independently complete RIPP once a day on any 5 days within a week. Participants received aphasia-friendly written instructions and a log sheet where they noted the completion of each home programme session. The participants reported that they completed the home programme as instructed, except that RAA missed 2 days in the first week of the pollution topic due to ill health and RYT missed 1 day of the health topic due to computer problems.
Data Collection
The outcomes of treatment were assessed for each topic using discourse samples from semistructured interviews conducted at four testing points: two pretests at 3 and 2 weeks prior to the treatment session for that topic; a post-test within 24 hr of the topic group-treatment session and a final testing point 1 week following the topic group-treatment session. The home programme was conducted over 2 weeks either side of the group session, and thus (for topics that received it), the third testing point was midway through home programme treatment. Hence, this testing point is referred to as ‘mid’ for the topics that also received the home programme (as, while it follows the group treatment, it is in the middle of the home programme) and ‘post1’ for the topics that did not receive the home programme (see timeline, Appendix A). For control topic 1, the final testing point was conducted more than 1 week after the preceding testing session. Therefore, only the first three testing points are included in analysis of the control topics.
The interviews were conducted by BM and CM. Each participant had the same interviewer for all interviews. The interviews were based around set questions (with pictorial and written aphasia-friendly supports) on the topic of the video. The questions served as starters; however, the interviews were meant to be natural and free flowing and could lead to other topics. The interviewer followed the lead of the participant in terms of the direction and flow of the interview with emphasis being placed on the thoughts, ideas, and opinions of the participant and input from the interviewer kept to a minimum. There was no set duration and the participants could talk for as long as they wanted. The interviews were audio- and video-recorded. The audio recordings were transcribed by a professional transcription service with accuracy checked by the authors prior to analysis.
Analyses
For each analysis, we examined average performance across the topics treated within each condition (three topics treated with group alone, three with group and home programme, and two control topics).
For analysis of data with categorical outcomes (e.g., correct/incorrect; target word naming and discourse ratings), we used Weighted Statistics (WEST; Howard, Best, & Nickels, Reference Howard, Best and Nickels2015). These statistics offer a means of analysing single-case study data using one-sampled t-tests to evaluate the pattern of change over multiple testing points (pretests and post-tests) compared to the null hypothesis of no change with the use of appropriately determined weighting coefficients. The weights used in this study for the testing schedule of pretest 1, pretest 2, post-test 1, and post-test 2 were −3, −1, 1, and 3 to evaluate the trend across the whole study (WEST-Trend) and 1, −3, 3, and −1 to compare the rates of change (ROCs) across treatment and no treatment phases (WEST-ROC). For the topics that received the home programme, the testing schedule was pretest 1, pretest 2, mid, and post-test, and therefore, WEST-ROC weights were 3, −4, −1, and 2. Control topic analysis was based on three sessions (two pretests and one post-test) and had weights of −2, 0, and 2 for the WEST-Trend and 1, −2, and 1 for WEST-ROC.
A treatment effect was indicated by a significant improvement across the study in WEST-Trend and significantly greater improvement in the treated phase compared with the untreated phase with WEST-ROC (Howard et al., Reference Howard, Best and Nickels2015). Raw scores were used for target word counts. For perceptual discourse ratings, we converted raw scores to Z-scores for each rater to provide a distribution from pretests to post-treatment tests. All p-values are one-tailed, given the prediction that the treatment should benefit participants.
Analyses considering proportional scores (CPIDR-3, POWERS) used Poisson Trend Test, and the ROC was measured using Poisson Test with Arbitrary Weights. Weights were the same as those described in the previous section.
Results
Analysis 1: Use of target words in semistructured interviews
We analysed target word production in the semistructured interviews, considering the total number of unique target words. Correct productions or productions with minor phonological errors (one incorrect phoneme with an unambiguous target) were included, as were productions with the same stem but morphological variants (e.g., plurals, past tense).
As can be seen from Figure 1, very few target words were produced in the semistructured interviews (see Table 2). While some WEST-ROC analyses were significant (RAA control topics; JDC topics that received group treatment with home programme; JDC control topics), no individual showed significance in both WEST-Trend and WEST-ROC for any condition, indicating that there was no evidence of improved performance as a result of treatment (also see Table 2 for details of the statistical analysis).
Average number of target words produced for each topic within a treatment condition across four testing points. *Mid/Post1 is post-test 1 for group treatment and control sets and midway testing point for group with home programme set (Group + HP) (cross-hatched). Control topics have three testing points.
Semistructured interview target words, words and length, and statistical analyses for production of target words
Note. Number of target words, number of total words, and length show the average across the interviews for the treatment/control sets with standard deviations provided in parenthesis. Statistical results indicate one-tailed p-values. ‘Treatment effect’ requires significant effect in both WEST-Trend and WEST-ROC analyses.*p < .05, **p < .01.
Analysis 2: POWERS
The POWERS analysis was completed by an independent assessor, blind to the phase of the sample, with order of samples randomised. The middle 5 min of the interviews was analysed (as is the standard procedure; Herbert et al., Reference Herbert, Best, Hickin, Howard and Osborne2012), with interviews that were less than 5 min analysed in their entirety. Because of the difference in length across samples (see Table 2), our analysis focused on proportional results rather than raw counts.
The POWERS focuses on indicators of successful word retrieval such as production of nouns, production of content words, and number of substantive turns (conversational turns containing at least one content word) and indicators of problematic word retrieval (termed ‘trouble-indicating behaviours’; TIBs) such as circumlocutions, filled pauses, and phonological paraphasias. As we included target words from several word classes, we expected that any change would be evident in overall content word productions rather than specifically nouns. Furthermore, due to the interviewer restricting their interaction in the interview, we did not expect changes to substantive turns. Consequently, our analysis focused on the two areas in which we most expected change to occur: production of content words and TIBs [both measured as a proportion of the total number of speech units (all tokens including words and filled pauses)].
Neither proportion of content words nor proportion of TIBs showed significant change following the treatment for any participant (Poisson Trend Test and Poisson Test with Arbitrary Weights all ps > .05, see Appendix B and Figure 2).
POWERS analysis results. Left hand column: content words as a proportion of total speech units, Right hand column: POWERS trouble-indicating behaviours (TIB) as a proportion of total speech units; *Mid/Post1 is post-test 1 for group treatment and control sets and midway testing point for group with home programme set (Group + HP) (cross-hatched). Control topics have three testing points.
Analysis 3: Propositional Idea Density (CPIDR-3)
Semistructured interview responses were analysed using CPIDR 3.2 (Brown et al., Reference Brown, Snodgrass and Covington2007). We analysed the written transcriptions of the interviews using the CPIDR-3 ‘speech mode’ option, which excludes repetitions and filled hesitations (e.g., ‘um’) from the total word count. The participants showed little change in the average propositional idea density of the speech samples (see Figure 3), and no significant changes were found for any participant across any of the conditions (Poisson Trend Test and Poisson Test with Arbitrary Weights all p > .05; see Appendix C). Notably, the pretest propositional idea density scores for our participants were at the upper range of the aphasic speakers (range 0.009–0.582) found by Bryant et al. (Reference Bryant, Spencer, Ferguson, Craig, Colyvas and Worrall2013), and in some cases within the range of unimpaired speakers (0.505–0.573). This issue is explored further in the Discussion section.
Propositional density of participants’ output in semistructured interviews (mean propositional density for the topics within each condition). *Mid/Post1 is post-test 1 for group treatment and control sets and midway testing point for group + home programme set (Group + HP) (cross-hatched). Control topics have three testing points.
Analysis 4: Discourse Ratings
The aim of the perceptual rating was to determine whether listeners identified that communication had improved following the treatment in areas predicted to change with a focus on word retrieval. Following Le Dorze et al. (Reference Le Dorze, Boulay, Gaudreau and Brassard1994), each interview was rated on: the participant’s ability to transmit their message, their ability to find adequate words, and the quantity of information.
The ratings were completed by 12 final year Speech Pathology Masters students who were blind to the purpose of the study. The raters were familiar with aphasia and had previous exposure to aphasic speech. Prior to commencing ratings, the raters were required to familiarise themselves with the rating system and received detailed written instructions. The raters measured the participants’ communication using a 0–5 scale with 0.5 intervals with ‘0’ indicating a complete absence of ability to transmit message, find adequate words, or provide content and ‘5’ indicating unimpaired performance. For rating of ‘ability to transmit message’, the raters were asked to consider how well the speaker had communicated their ideas, while ‘ability to find adequate words’ was described as a rating of whether the speaker used accurate words and/or displayed any behaviours consistent with word-finding difficulties, such as paraphasias. Finally, for ‘quantity of information’, the raters were asked to consider whether the speaker provided either too little or too much information to effectively communicate their message.
The ratings were based on a sample of the first 5 min of the interview. For interviews that did not exceed 5 min in length, the entire sample was used for the ratings. The recordings were presented in randomised order but blocked by participant so as to facilitate consistency of ratings within participants. Z-scores were calculated for each participant for each rater and then averaged across raters to provide a distribution from pretests to post-tests for each participant.
There was substantial variability in the ratings across testing sessions for each participant (see Table 3 and Figure 4). There were no significant treatment effects for RYT or JDC. However, RAA showed significantly improved ratings for ‘ability to retrieve adequate words’ for topics treated with combined group treatment and home programme (WEST-Trend: t = 0.691, p = .009; WEST-ROC: t = 0.909, p = .042 one-tailed). Analysis of RAA’s rated ability to retrieve adequate words for the topics treated with the combined group and home programme treatment compared with topics treated without a home programme was significant for the WEST-Trend test only (Two sample t-test: WEST-Trend: t(70) = −2.969, p = .004 two-tailed; WEST-ROC: t(70) = 0.871, p = 387).
Discourse ratings across the three parameters: ability to transmit message, ability to find adequate words, and quantity of information
West-Trend and WEST-ROC show one-tailed p-values. Treatment effect required a significant result on both statistical measures. ** p < .05.
Distribution of Z-scores, taking the average across the three topic areas for each set: group (education, refugees, and indigenous issues), group with home programme (health, pollution, and war), control (domestic violence and dementia). *Mid/Post1 is post-test 1 for group treatment and control sets and midway testing point for group + home programme set (Group + HP). Control Topics have three testing points. Asterisk indicates significant result (p < 0.05 one-tailed on both Poisson Trend and Poisson Arbitrary Weights).
Discussion
This study examined whether group-based aphasia treatment (with a focus on word retrieval) had an effect on later connected speech. However, we also sought add to the discussion within aphasia research regarding how changes in connected speech that may result from group treatment can be measured given the acknowledged difficulty in determining a suitable assessment approach. In Nickels et al. (Reference Nickels, McDonald and Mason2016), we presented the outcomes of this treatment programme for word retrieval measured by picture naming, in which our participants showed some evidence for improvement. However, one of the claimed benefits of group treatment is that it facilitates improved participation in everyday conversation (Elman, Reference Elman2007; Kearns & Elman, Reference Kearns, Elman and Chapey2001). Therefore, here, we considered the impact of our group-treatment protocol on word retrieval in connected speech.
It has been established that word retrieval in connected speech is difficult to measure (e.g., Croot et al., Reference Croot, Taylor, Abel, Jones, Krein, Hameister, Ruggero and Nickels2015; Hickin et al., Reference Hickin, Best, Herbert, Howard and Osborne2001). For this reason, we took a comprehensive approach to analysis using four different measures that have been shown to be sensitive communication measures for people with aphasia: target word count, the POWERS, propositional idea density (measured using the CPIDR-3), and a perceptual rating of discourse. Importantly, as far as we are aware, this study was the first attempt to quantitatively measure the effects of group-based treatment on word retrieval and its impact in connected speech.
Only a single gain was evident: RAA showed an increase in her rated ability to retrieve words in topics that received group treatment supplemented with a word-retrieval treatment home programme. Given the lack of improvement for topics treated with the group alone, both for RAA and the other participants, it is clear that we have no evidence that group treatment alone improves connected speech. However, as we did not include a ‘home programme alone’ condition, we are unable to determine whether the group treatment (in combination with the home programme) did play a role in improving RAA’s rated ability to find words.
Nevertheless, in general, our findings cannot be said to support a claim that our ‘typical’ group-based treatment led to substantially improved word retrieval at the discourse level for these participants. This is despite an increase in word retrieval measured by picture naming over the course of the study (see Nickels et al., Reference Nickels, McDonald and Mason2016). There are two key possible accounts for our findings: that treatment was ineffective at the discourse level, or that the measures used were unable to capture change.
The first possibility is that this treatment was ineffective in achieving the discourse level improvements in word retrieval for the participants in our study. This may be because lexical retrieval itself was not improved sufficiently. Indeed, our previous findings showed that improvements in lexical retrieval at the single-word level were small for the topics treated with the group alone (see Nickels et al., Reference Nickels, McDonald and Mason2016). However, much larger improvements occurred for those topics also treated with a supplementary home programme, yet we found no evidence of improvement to word retrieval in discourse for these either set of topics for RYT and JDC. The one participant, RAA, who did show significant improvement in her rated ability to find words, was also the participant who showed the largest treatment gains at the single-word level in the same condition (group plus home programme). Consequently, it is possible that more substantial gains in word retrieval at the single-word level are required to observe carry over to word-retrieval improvements in discourse. This implies that in order for measureable changes to occur at the level of connected speech, group treatment needs to be optimised to lead to greater improvements to word retrieval. Possible areas for consideration include improvements to the selection of treatment words, an increased duration of intervention, and/or further supplementation with specific word-retrieval treatment through home programmes or individual therapy.
Choosing treatment words that will have the greatest impact on an individual’s communication is difficult (see Renvall, Nickels, & Davidson, Reference Renvall, Nickels and Davidson2013), and it is possible that the words used in this study were not ideal. In this study, we selected words by asking unimpaired subjects to list words associated with a topic (see Nickels et al., Reference Nickels, McDonald and Mason2016 for details). However, other methods, such as conducting the same semistructured interviews with unimpaired subjects and determining a core vocabulary set, may have led to the selection of more appropriate words for use in the interviews. It was also noted that some topics (such as refugees) led to more robust discussions in the group session than others (such as pollution), indicating that careful consideration is needed for topics that will be engaging for a specific group of participants.
It is also possible that the relatively short duration of treatment (for each topic) had an impact on our findings. Each topic only received one group session and, for some topics, 2 weeks of the home programme. Possibly, a longer duration of treatment for each topic was needed to yield measurable change in discourse. We chose a design where we evaluated at the level of a single topic, as this gave the greatest experimental control and most precise ability to monitor retrieval of treated lexical items. However, this design decision meant that we could not observe cumulative effects of the 6 weeks of group treatment on discourse. It is possible that improvements related to general lexical retrieval (such as rated word-finding ability) may have become evident in such an analysis.
The alternative account of our findings is that our participants did experience improvements; however, our measures were ineffective in capturing this change. As discussed, measuring specific changes in connected speech presents an ongoing difficulty for researchers and clinicians alike. While the measures we used had been shown to be sensitive to aphasic language, there are factors that may have limited their effectiveness in our study.
First, we were faced with the difficult issue of how to collect connected speech samples that would be stable over time. Test–retest reliability has been identified as a key difficulty when assessing discourse-level language in aphasia research (e.g., Boyle, Reference Boyle2011, Reference Boyle2014; Dietz & Boyle, Reference Dietz and Boyle2018). Researchers in this field are presented with the difficulty of maximising comparability between samples (in terms of topic difficulty, properties of relevant lexical items, interactional styles), while also maintaining the ‘naturalness’ of the discourse. We approached this problem by using a semistructured interview in which the participants were asked the same questions about each topic by the same researcher across testing points. However, we noted that this created a problem, as while the participants would provide a full response to a direction of inquiry in the initial session, there was a tendency to summarise in the subsequent sessions, typically evidenced by a reduction in the number of words the participants produced following the first pretest (see Table 2). Thus, improvements could be made to the sampling methodology.
Berarducci (Reference Berarducci2008) compared the stability of several measures (e.g., type-token ration, number of clauses) across three discourse samples with people with aphasia. She identified that the biggest differences were between the first and second samples, even when there was a large gap of 4 weeks between sessions. Consequently, one possibility is that when collecting discourse samples, multiple (at least three) pretreatment baselines are collected and the first is excluded from analysis. Another possibility is that a different interviewer is engaged at each sampling point, while minimising that interviewer’s personal impact through restriction of interaction, such as by employing a more rigid structure.
In addition, our approaches to analysis were selected because they had been shown to be effective at distinguishing aphasic speech from controls and/or at detecting change. However, some techniques may not have been optimal for our specific participants. As noted in the Results section, pretreatment propositional idea density for our participants was similar to the unimpaired controls described in Bryant et al. (Reference Bryant, Spencer, Ferguson, Craig, Colyvas and Worrall2013). This may indicate that this measure is not sensitive for individuals with mild–moderate anomia such as our participants. We also measured the use of specific treated words as retrieval of these words would be predicted to improve following the treatment. However, as we found, opportunities to use specific words in connected speech can be few in number and difficult to predict.
The question of how discourse and changes in discourse can be effectively measured is increasingly gaining attention in aphasia research, with calls for the development of a ‘Core Outcome Set’ of measurements (Dietz & Boyle, Reference Dietz and Boyle2018; Wallace, Worrall, Rose, & Le Dorze, Reference Wallace, Worrall, Rose and Le Dorze2014; Wallace, Worrall, Rose, & Le Dorze, Reference Wallace, Worrall, Rose and Le Dorze2016). We agree that this discussion is important for aphasia research but suggest that consideration of whether the measures that are appropriate may differ depending on aphasia severity and type, and the type of discourse changes that are predicted is vital. Clearly, if we believe that group treatment offers promise as a means of improving (word retrieval) in connected speech, there is a need for sensitive and reliable measures for examining outcomes over time.
Conclusions
Group treatment offers potential benefits as a treatment delivery method by allowing several individuals to receive treatment simultaneously, while also providing opportunity for conversation practice with peers. However, our study provides little evidence that ‘typical’ discussion-focused aphasia groups result in substantial gains in connected speech, albeit with the caveat that we only have data from a small group of participants. Consequently, it remains to be determined which language impairments are most suited to group-treatment approaches and how such treatments may be best designed to optimise outcomes. Given the prevalence of word-finding difficulties in aphasia (e.g., Goodglass & Wingfield, Reference Goodglass and Wingfield1997), continued research into the effectiveness of group-treatment methods for this impairment is required. Importantly, there is a need to better understand what are the essential components in order that group treatment can facilitate generalisation of gains in word retrieval to everyday speech.
This study also demonstrates once again that the issue of how we can best measure word-retrieval outcomes (and language outcomes generally) in discourse remains a barrier in aphasia research (e.g., Carragher et al., Reference Carragher, Conroy, Sage and Wilkinson2012; Dietz & Boyle, Reference Dietz and Boyle2018; Wallace et al., Reference Wallace, Worrall, Rose and Le Dorze2014). We hope that this paper will contribute to this discussion by addressing the difficulty identifying discourse measures to examine group-treatment outcomes. This question is not only pressing for researchers but also for clinicians seeking an effective means of measuring their clients’ improvements from group treatment.
ACKNOWLEDGMENTS
We would like to express our gratitude to our participants; RYT, RAA, and JDC and their families for sparing their time and energy to take part in our study. Our thanks also go to Solène Hameau for her valuable help in checking transcriptions and assistance with POWERS scoring. During the preparation of this manuscript, Lyndsey Nickels was funded by an Australian Research Council (ARC) Future Fellowship (FT120100102). Catherine Mason was funded by the ARC Centre of Excellence for Cognition and its Disorders (CCD, CE110001021). We know of no conflict of interest, financial or otherwise, relating to the content or purpose of the study.
CONFLICT OF INTEREST
The authors have nothing to disclose.
APPENDIX A Timeline of assessments and intervention for a topic. The third testing point (post-test 1/mid-test) occurred at the end of the group treatment but midway through the home programme
APPENDIX B POWERS results and statistics. Statistical measures were the Poisson Trend Test and Poisson Arbitrary Weights
APPENDIX C Propositional idea density results and statistics. Statistical measures were the Poisson Trend Test and Poisson Arbitrary Weights
APPENDIX D Target words organised by topic. ‘T’ indicates treatment topic, ‘C’ indicates control topic
