Pragmatic language impairment (PLI) is characterized as an impairment in the use of language in social contexts (Bishop, Reference Bishop, Leonard and Bishop2000), and has originally been classified as a language disorder. The impairment emerged around 20 years ago, although at the time it was referred to as semantic pragmatic syndrome. This term was reserved for children suffering from problems such as stereotyped conversational responses, literal interpretations, and impairments in the ability to take turns and to maintain a topic in discourse (Rapin & Allen, Reference Rapin, Allen and Kirk1983). In subsequent years, the diagnosis has been modified in both its name and its symptoms. Because the symptoms of PLI seem to resemble those of autism spectrum disorders (ASDs), some researchers argue that PLI is merely a description of the language problems experienced by children with ASD. However, although pragmatic impairments often co-occur with autistic symptoms, not all children with PLI demonstrate marked deficits in the areas of social interaction or restricted interests and stereotypical behaviors (Conti-Ramsden, Crutchley, & Botting, Reference Conti-Ramsden, Crutchley and Botting1997), although they clearly do experience at least some problems in the area of social interaction. To date, no consensus has been achieved regarding the question whether PLI should be considered a language disorder, an ASD, a separate disorder altogether (Botting, Reference Botting2002; Brook & Bowler, Reference Brook and Bowler1992). As a consequence, there is no clarity regarding its symptoms. The present paper reports on the narrative abilities in children with PLI in the context of other abilities such as theory of mind and executive functioning (EF). Findings may lead to improvements in the clinical assessment of PLI and may provide valuable clues for therapy.
The ability to understand and generate narratives is a skill that has recently received increased attention in the communication disorders research area (Botting, Reference Botting2002; Diehl, Bennetto, & Young, Reference Diehl, Bennetto and Young2006; Losh & Capps, Reference Losh and Capps2003; Norbury & Bishop, Reference Norbury and Bishop2003; Wetherell, Botting, & Conti-Ramsden, Reference Wetherell, Botting and Conti-Ramsden2007). Mar (Reference Mar2004) defines narratives as “a series of actions and events that unfold over time, according to causal principles” (p. 1415). They are widely considered to be an ecologically valid measure to assess pragmatic performance (Botting, Reference Botting2002; Paul & Smith, Reference Paul and Smith1993). In addition, as opposed to syntax or vocabulary measures, narrative competence serves as a strong predictor of later academic achievement (Feagans & Applebaum, Reference Feagans and Applebaum1986) and peer acceptance (Paul, Reference Paul1995).
Narrative competence requires semantic and syntactic skills, the skill to organize information, and the ability to adapt to the listener's level of background information (Losh & Capps, Reference Losh and Capps2003). As such, it is a complex task involving “higher level language skills,” which requires the integration of information beyond the word level. The advantage of using narratives as an assessment tool is that they can be used to gain information on several levels; it is possible to analyze specific aspects of narrative complexity both within and between sentences, but also to assess the organization of content (Liles, Duffy, Merritt, & Purcell, Reference Liles, Duffy, Merritt and Purcell1995). Alternatively, Coelho (Reference Coelho2002) describes narratives in terms of sentence production (narrative productivity, e.g., number of words per utterance), cohesion (e.g., percentage complete ties out of total ties), and story grammar (organization of story content, e.g., number of episodes mentioned) when assessing narrative competence. These different aspects of narrative competence are subject to developmental growth throughout the school years. Berman (Reference Berman1988), for instance, found significant increases in story length during the preschool and school-age years. By Age 6, most of the narratives produced consist of complete episodes with initiating events, motivating states, attempts, and consequences (Peterson & McCabe, Reference Peterson and McCabe1983).
Fictional narrative assessment can be performed using either a story generation design or a story retelling design. In story generation, children are generally asked to look at pictures and provide a story. In contrast, in a story retelling design, children first listen to a model story and are then asked to retell the story in their own way, possibly also assisted by pictures. Both story generation and story retelling are useful measures of narrative competence. Merritt and Liles (Reference Merritt and Liles1989) compared the results of both story generation and story retelling by language disordered children and typically developing (TD) children. They found that both groups told longer stories with more grammar components in the story retelling condition. In addition, stories in the retelling condition were more reliably scored.
Narrative deficits have been well documented in children with communication disorders (Botting, Reference Botting2002; Merritt & Liles, Reference Merritt and Liles1989; Pankratz, Plante, Vance, & Insalaco, Reference Pankratz, Plante, Vance and Insalaco2007). For example, using a story retelling design, Pankratz et al. (Reference Pankratz, Plante, Vance and Insalaco2007) found 4-year-old children with a language impairment to produce shorter stories with an overall poorer story content compared to TD children. Evidence for impaired narrative competence in children with ASD comes from Diehl et al. (Reference Diehl, Bennetto and Young2006), who also applied a retelling design. They report on problems (re)producing important plot structure components in a story in combination with higher rates of expressing nonrelated events for a group of 9-year-old children. Although they did not find specific deficits related to morphosyntactic errors, Capps, Losh, and Thurber (Reference Capps, Losh and Thurber2000) did find that children with autism and children with developmental delays told shorter stories with a lower proportion of complex syntax during narratives using a story generation task. It is interesting that a study by Dodwell and Bavin (Reference Dodwell and Bavin2007) showed that 6-year-old children with specific language impairment (SLI) performed more poorly on a narrative task that required retelling compared to tasks that required generation. Whereas the SLI group did show marked delays in a retelling task, they performed at a normal level on the generation task, a result that is countered by several studies (i.e., Reilly, Losh, Bellugi, & Wulfeck, Reference Reilly, Losh, Bellugi and Wulfeck2004; Reuterskiold Wagner, Sahlen, & Nettelbladt, Reference Reuterskiold Wagner, Sahlen and Nettelbladt1999).
Based on the different findings for different populations, some researchers have suggested that narrative profiles can be used to identify different disorders, such as language disorders and ASD (Botting, Reference Botting2002; Miniscalco, Hagberg, Kadesjö, Westerlund, & Gillberg, Reference Miniscalco, Hagberg, Kadesjö, Westerlund and Gillberg2007). This insight might apply to PLI as well, but there have been few studies that investigate the narrative competence of children with PLI. Conti-Ramsden, Crutchley, and Botting (Reference Conti-Ramsden, Crutchley and Botting1997) studied story content organization and found that 7-year-old children with PLI contributed less information compared to children with SLI in a story generation task. Norbury and Bishop (Reference Norbury and Bishop2003) also applied a story generation design but failed to find many significant differences between TD children and children with SLI, PLI, or autism aged 9, although some of the differences bordered significance. In one of the few studies that investigates the narrative competence of children with PLI in more detail, Botting (Reference Botting2002) found problems in the organization of content similar to those reported by Conti-Ramsden et al. (Reference Conti-Ramsden, Crutchley and Botting1997) in both a story retelling and a story generation task. Detailed examination showed that children with PLI expressed lower rates of utterances related to the setting and ending, compared to rates found in other studies for groups with autism or SLI. However, whereas Botting did find a lower sentence length and a reduced number of subordinate clauses in the story retelling design, there were no differences in the story generation task. Although these results give a first indication of impaired narrative competence of children with PLI, the limited sample size (n = 5) in addition to the absence of a TD control group makes it difficult to draw any definite conclusions. In addition, questions have been raised regarding the structural language skills of children with PLI, which, although frequently believed to be intact, have been found to be impaired by some researchers (Norbury & Bishop, Reference Norbury and Bishop2003). It remains to be seen whether the impaired narrative competence can be explained by linguistic impairments rather than impaired pragmatic competence.
UNDERLYING MECHANISMS OF NARRATIVE COMPETENCE
The use of narratives offers a host of information on the pragmatic skills as well as linguistic skills of children. Research has shown that narrative competence might be affected by several underlying cognitive mechanisms.
Theory of mind
As Mar (Reference Mar2004) points out, narrators have to assume the perspective of the character in order to understand the actions of this character and subsequently explain these to others. Furthermore, a narrator has to be able to take into account the perspective of the listener, a skill that requires theory of mind abilities. However, studies pertaining to the relationship between theory of mind skills and narrative competence use varying populations and thus yield varying results. For example, in the case of autism, Capps et al. (Reference Capps, Losh and Thurber2000) found correlations between false belief understanding (which according to many researchers is considered the litmus test of theory of mind because it indicates an appreciation of a distinction between matters of mind and the real world; Wellman, Cross, & Watson, Reference Wellman, Cross and Watson2001), and several measures of narrative competence for a population of 12-year-olds using a story generation task. The correlations included measures of syntactic diversity, evaluative devices, and the use of mental states. They did note, however, that some correlations could be (partly) explained by the connection with language ability. Using a similar population and design, Tager-Flusberg and Sullivan (Reference Tager-Flusberg and Sullivan1995) found a relationship between measures of theory of mind skills and mental state referencing in narratives. This finding has been replicated for a group of deaf children by Peterson and Slaughter (Reference Peterson and Slaughter2006). Losh and Capps (Reference Losh and Capps2003), however, failed to find evidence for a relationship between advanced theory of mind skills and narrative competence in their population of 11-year-olds with autism using a story generation task and personal narratives. They did find evidence for a relationship between emotion understanding and skills related to story length and diversity of complex syntax. In the only study known to the authors on theory of mind abilities in narratives for children with PLI, it was found that 7- and 8-year-old children with PLI used more references to mental states compared to children with autism or children with SLI in a story retelling and story generating task (Botting, Reference Botting2002). However, because no norm group was investigated, it is not clear whether the PLI group showed normal levels of mental state referencing in narratives. Moreover, in the case of PLI, no systematic approach has been taken to relate different aspects of narrative competence to theory of mind abilities.
EF
In addition to theory of mind skills, a second underlying mechanism that might affect narrative competence is EF. EF is an umbrella term that refers to control processes dealing with generating, planning, and evaluating behavior. Brain studies suggest the involvement of executive functions as necessary skills for narrative competence (for a comprehensive review, see Mar, Reference Mar2004). Narrative competence requires organization of information regarding the content, but it also requires organization by securing a connection between sentences (Tannock, Purvis, & Schachar, Reference Tannock, Purvis and Schachar1993). The main components of EF are working memory, attentional processes, and cognitive flexibility, the ability to inhibit immediate behavior (inhibitory control), and planning abilities (Goldberg et al., Reference Goldberg, Mostofsky, Cutting, Mahone, Astor and Denckla2005; Sinzig, Bruning, Schmidt, & Lehmkuhl, Reference Sinzig, Bruning, Schmidt and Lehmkuhl2008). The relation between working memory and narrative competence is based on the fact that important aspects of the narrative need to be held in mind. Attentional processes, cognitive flexibility, inhibitory control, and planning abilities may play a pivotal role in the ability to maintain the topic, the ability to name key plot structure elements, and the ability not to be distracted by less important information. Certainly in the case of a retelling design, one could expect the involvement of EF, due to the involvement of memory processes. As suggested by Dodwell and Bavin (Reference Dodwell and Bavin2007), narrative assessment could even be used to assess EF.
In recent work on the relationship between working memory and narrative competence as measured using the number of events in the narrated story, Dodwell and Bavin (Reference Dodwell and Bavin2007) made a distinction in narrative tasks that are based on retelling and narrative tasks that require generation. They found a significant correlation between working memory and narrative retelling skills in 6-year-old children with SLI. Unfortunately, they did not report on the relationship between working memory and narrative generation. In addition, Montgomery, Polunenko, and Marinellie (Reference Montgomery, Polunenko and Marinellie2009) found evidence for a relation between aspects related to working memory (resource capacity and processing speed) on the one hand and understanding narratives on the other hand. Other evidence for a relationship between narrative competence and EF comes from studies on abilities of individuals with known executive dysfunction, such as individuals with brain damage (for an overview, see Mar, Reference Mar2004). For example, Coelho (Reference Coelho2002) found modest correlations between a measure of cognitive flexibility and narrative skills relating to narrative productivity and story content. Some of the findings were negative, however, with better executive functions relating to lower narrative competence. No significant correlations between EF and cohesion were found.
PRESENT RESEARCH
Although there has been much research into narrative competence of children with communicative disorders, little research exists on the narrative competence profile of children with PLI specifically. It remains to be seen whether children with PLI show deficits limited to the story content organization component as described by the model of Coelho (Reference Coelho2002), or whether their problems extend to narrative productivity and cohesion. Moreover, although many researchers have suggested the involvement of theory of mind abilities and executive functions, the underlying cognitive mechanisms of narrative competence are still largely unknown, especially for the PLI population. The possibility should not be excluded that linguistic abilities play an important role in narrative competence, more so than the aforementioned cognitive functions. Regarding cognitive profiles of children with PLI, there is some evidence of theory of mind deficits (Shields, Varley, Broks, & Simpson, Reference Shields, Varley, Broks and Simpson1996), and only recently research has started to unravel possible executive dysfunction (Bishop & Norbury, Reference Bishop and Norbury2005a, Reference Bishop and Norbury2005b). However, no attempt has been made to relate narrative competence to theory of mind and EF in children with PLI. The present study uses story retelling as a measure of narrative competence. If children with PLI indeed show deficits in narrative competence, and if these deficits are related to problems in theory of mind and/or EF, this finding could provide valuable clues for clinical assessment and intervention.
We will attempt to answer the following research questions:
1. To what extent do children with PLI show a reduced narrative competence as measured using a retelling design, in the areas of narrative productivity, organization of content, and cohesion, compared to TD children?
2. To what extent are problems in the areas of EF and theory of mind related to narrative competence as measured using a retelling design?
3. To what extent are the group differences and relationships mediated by language skills?
METHOD
Participants
Participants were recruited from primary schools in the Netherlands. A total of 84 children with pragmatic language problems were selected ranging in age between 4 years, 11 months (4;11) and 6;1 (M = 5;6 years, SD = 3.5 months). The group consisted of 59 boys and 25 girls, a ratio that is to be expected based on earlier reports on gender ratios of language disorders (Silva, Reference Silva1980), although more recently a lower gender ratio has been found (Tomblin, Reference Tomblin1997). The children with PLI were selected based on the pragmatic composite score of the Children's Communication Checklist (CCC; Bishop, Reference Bishop1998) as rated by their teacher. The CCC is a teacher/therapist questionnaire that can be used to identify children with pragmatic language difficulties. Children with a pragmatic composite below the cutoff score of 132 were identified as children with PLI. This cutoff has been identified as a marker for discriminating children with pragmatic language impairment from children with more typical SLI and has been used extensively in research (e.g., Bishop, Reference Bishop1998; Bishop & Baird, Reference Bishop and Baird2001; Geurts et al., Reference Geurts, Verte, Oosterlaan, Roeyers, Hartman and Mulder2004). Recently, the potential of the CCC has been established for use in the general population as well (Ketelaars, Cuperus, Van Daal, Jansonius, & Verhoeven, Reference Ketelaars, Cuperus, Van Daal, Jansonius and Verhoeven2009). A matching group of 81 TD children (TD group) was selected ranging in age between 4;11 and 6;1 (M = 5;6, SD = 3.5 months). Matching was based on classroom, gender, and age (within 6 months). As confirmation of their normal development, the children had to show a pragmatic composite above 140 (lowest normal score of a TD sample of Bishop & Baird, Reference Bishop and Baird2001). For each child, the teachers were asked to indicate if the children experienced any problems and if so, whether they received any kind of treatment. Children who were judged to have impaired functioning in any area were excluded from the control group. Measures of narrative performance were missing for 7 children in the PLI group and 5 children in the TD group, due to either excessive shyness of the child or failure of the audio equipment. The final groups consisted of 77 children with PLI, and 77 TD children.
Background measures
Nonverbal reasoning. All children were assessed using the Raven Coloured Progressive Matrices (CPM; Raven, Reference Raven1956). In this task children are presented with unfinished patterns. They are asked to point to the correct picture out of six pictures that would complete the presented figure. Standardized scores were based on a Dutch norm group (Van Bon, Reference Van Bon1986) (M = 5, SD = 2).
Receptive vocabulary. The receptive vocabulary subtest of the Dutch Language Test for Children (Verhoeven & Vermeer, Reference Verhoeven and Vermeer2001) was administered. This is a standardized test for 4- to 10-year-old children. In the receptive vocabulary subtest the child is presented with a word and is asked to select the picture illustrating that word out of four pictures. The maximum number of items is 96, and the task is discontinued after five consecutive errors. Raw scores were used as an indication of receptive vocabulary.
Expressive vocabulary. To assess children's expressive vocabulary, the Dutch adaptation of the Renfrew Word Finding Vocabulary Test (Jansonius-Schultheiss, Borgers, DeBruin, & Stumpel, Reference Jansonius-Schultheiss, Borgers, DeBruin and Stumpel2006) was administered. The task consists of 50 pictures that have to be named. Raw scores were used as an indication of expressive vocabulary.
Sentence comprehension. The sentence comprehension subtest of the Dutch Language Test for Children (Verhoeven & Vermeer, Reference Verhoeven and Vermeer2001) was administered. In this task the child is presented with 32 sentences in which syntactic patterns play a key role. For each sentence, the child has to choose the correct picture out of three. Raw scores were used as an indication of sentence comprehension.
Narrative competence
Narrative competence was assessed using a Dutch adaptation of the Renfrew Bus Story Test (Jansonius-Schultheiss et al., Reference Jansonius-Schultheiss, Borgers, DeBruin and Stumpel2006), which uses a story about a naughty bus. The Dutch adaptation differs from the original in that some story components were added to attain a better episodic plot structure. With the additional story elements (a more elaborate description of the setting, an additional emotional response at the end of the story, and an utterance on the morality of the story), plot structure component analysis was deemed possible, whereas the original Renfrew Bus Story Test only provides an opportunity to score the number of relevant pieces of information given by the child (the Renfrew information score) as measured using only the exact wording. The course of the story as well as the basic story components were retained. Children were told the story while being shown a picture book which depicts some, but not all, key components of the story. After having heard the story, the children were asked to retell the story with the aid of the picture book. The narratives were recorded on tape and transcribed. Using Hunt's method (Hunt, Reference Hunt1970), each utterance was segmented into T-units. In line with Coelho (Reference Coelho2002), we included measures for three narrative components: narrative productivity, organization of content, and cohesion. As measures of narrative productivity we analyzed three measures: total number of T-units, mean length of five longest T-units (ML5LU), and number of subordinate clauses. Subordinate clauses are sentences dependent of a head sentence or grammatical constituents, introduced by subordinate conjuncts. Specifically, we counted as subordinate clauses adverbial clauses, adjective clauses (relative clause), and complement clauses. The narrative as told to the child contained 11 subordinate clauses (3 adverbial clauses, 3 adjective clauses, and 5 complement clauses).
Organization of content was measured by the proportion of plot structure components (number of plot structure components divided by the total number of T-units), and the proportion of irrelevant T-units (T-units containing no information that was part of the model story).
Plot structure components. The following subdivision in plot structure components (Jansonius-Schultheiss et al., Reference Jansonius-Schultheiss, Borgers, DeBruin and Stumpel2006) was derived from Stein and Glenn (Reference Stein and Glenn1979). The setting is information on the main characters and background information on the place and time. The initiating event is considered to be the main event, which triggers a response of the main character. The subsequent events belonging to the episode of the story consist of following events triggering plans and actions. The outcome is information given on the end result. Morality is considered information on the learning experience of the story character as provided by the child (i.e., never to run away again). Emotional responses were singled out as information regarding affective and emotional states throughout the story. The plot structure components are somewhat similar to the Renfrew information score, but allow for more own wording compared to the Renfrew information score. Moreover, a division into separate plot structure components was considered useful because research performed by Botting (Reference Botting2002) indicated that 7- to 8-year-old children with PLI scored within the normal range on the Renfrew information score, although detailed analysis of specific aspects of content information did reveal specific impairments. The proportion of plot structure components was computed relative to the total number of T-units because some children supplied much information that did not belong to the original story. Although this results in a longer story, the information pertaining to the story itself would be low. Alternatively, telling a longer story often results in the inclusion of more plot structure components. Appendix A shows the subdivision of the plot structure components of the Bus Story.
Irrelevant T-units. Information given by the child that did not correspond to the story as told by the examiner were considered irrelevant T-units (Jansonius-Schultheiss et al., Reference Jansonius-Schultheiss, Borgers, DeBruin and Stumpel2006). Irrelevant T-units were counted, and the proportion of irrelevant T-units was computed as a proportion of the total number of T-units, because the amount of irrelevant T-units should be considered in the context of the length of the story. T-units were considered irrelevant when the semantic content of the T-unit was unclear (incomprehensible), when the T-unit consisted of information that did not belong in the original story (made up), and when the T-unit consisted of correct information that was provided in the wrong chronological order.
As a measure of cohesion we assessed the number of implicit references that occurred in those T-units that were considered relevant in the light of the model story and expressed them as a proportion of the total number of relevant T-units. Implicit references were considered the use of anaphoric references to objects, persons or places, in situations that required the use of nouns. For example, the utterance “And the bus ran away. Then they made funny faces at each other” was judged an implicit reference because [they] does not refer correctly to persons mentioned before. The utterance “There, he saw a cow” was also judged an implicit reference when the pasture the cow was residing in was not mentioned earlier. Irrelevant T-units, that is, T-units not part of the model story, were not coded for implicit references, because some of these T-units could not be judged on the appropriateness of the references.
The coding of the measures was judged separately on interrater reliability. Interrater reliability for coding was assessed for 13% of all children (20 transcripts) and was performed by an experienced clinical linguist who was blind to group status. Agreement exceeded 80% for all measures.
Theory of mind
Theory of mind was measured using three change of location false belief stories; for instance, a girl is playing with her teddy bear. When she needs to leave to room for a minute, she puts the bear in a cabinet. After she has left the room, a second child takes the teddy bear out of the cabinet and throws it into the trash can. Subsequently, the girl comes back into the room and wants to play with her bear. The stories were examined by a linguist to ensure low levels of narrative complexity. After hearing the story, children were asked about the false belief of one of the story characters: “Where would [story character] look for [the object]?” Additional support was provided through the use of pictures during the verbalisation of the story. To obtain a score of 1 for a story, a memory question pertaining to the actual location of the object had to be answered correctly in addition to the false belief question. The maximum score for this task was 3.
EF
In order to determine the general EF level of the participants, we applied a broad battery of EF tests that are designed to measure different aspects of EF. In the analyses the results of these tests were processed by a data reduction technique in order to obtain a general EF score.
Planning. Planning was measured using the tower task of the NEPSY (Korkman, Kirk, & Kemp, Reference Korkman, Kirk and Kemp1998). The task consists of 20 items, in which the child has to rearrange three colored balls to a configuration presented on a picture. There is a set of rules pertaining to the number of moves, the number of balls that can be moved at a time (one), and the amount of time. After four consecutive incorrect responses the task is terminated. The tower task measures higher order executive functions such as planning ability, cognitive flexibility, and inhibitory control.
Selective attention and cognitive flexibility. Selective attention and cognitive flexibility were measured using the Auditory Attention and Response Set of the NEPSY (Korkman et al., Reference Korkman, Kirk and Kemp1998). The Auditory Attention and Response Set task consists of two parts, in which children are asked to respond to a set of auditory stimuli. Children are presented with auditory stimuli and coloured foam squares of different colours. In the first part, they are asked to put a red square into a box every time the auditory stimulus red is heard. This part measures simple, selective auditory attention. In the second part, the child learns a new set of rules that consist of both contrasting and matching stimuli (putting a yellow square into the box after hearing the stimulus word red, putting a red square into the box after hearing the stimulus word yellow, and putting a blue square into the box after hearing the stimulus word blue). The child gets a score of 2 for every correct response within 1 s of the auditory stimulus, and a score of 1 for every correct response in the next 2 s. The total score can be computed by summing the scores and subtracting the number of commission errors (using a wrong color or responding to a nontarget word).
Working memory. Working memory was measured using Number Recall, a subtest of the Kaufman Assessment Battery for Children (Kaufman & Kaufman, Reference Kaufman and Kaufman2004). The child has to repeat strings of numbers, which are of increasing lengths. After three consecutive errors the task is discontinued.
Procedure
Children were tested at their schools in two sessions of approximately 50 min each. Upon entering the room the children were first familiarized with the situation and with the experimenter. Tasks were presented in a semirandom order, alternating verbal and performance tasks. The selective attention and cognitive flexibility task was presented as a first task in one of the two sessions, to ensure maximum attention.
Data were analyzed in several steps. A multivariate analysis of variance (MANOVA) was conducted to examine group differences in narrative competence between TD children and children with PLI. This analysis and all following analyses were carried out with and without controlling for nonverbal intelligence. To further investigate possible differences in organization of story content a MANOVA was conducted on the specific plot structure components and specific types of irrelevant T-units. An overall language factor was computed by means of principal components analysis (PCA) on the different language tasks, and the preceding analyses were then repeated using this language factor as a covariate. Subsequently, a PCA was conducted to find whether the proposed subdivision into three factors (narrative productivity, organization of story content, and cohesion) was attainable. Next, a MANOVA was performed to examine group differences in false belief understanding and EF, followed by a similar analysis controlling for language skills using the same language factor that was used earlier. Finally, we computed correlations between the relevant narrative factor scores, an EF factor, which was created by means of PCA, and a measure of theory of mind. This was followed by a hierarchical regression analysis to assess the unique contribution of theory of mind and EF independent of language skills.
RESULTS
Background data
Table 1 shows data on age, nonverbal reasoning abilities (standardized scores on the Raven CPM), receptive language skills (raw scores on two subtests of the Dutch Language Test for Children), and expressive language skills (raw scores on the Renfrew Word Finding Vocabulary Test). As the groups were matched on age, no significant age differences were found between both groups. However, a significant difference was visible between both groups on the measure of nonverbal reasoning (as measured with the Raven CPM), with the TD group outperforming the PLI group. Comparing the standardized scores of the PLI group to the original sample showed a normal mean score, whereas the TD group showed a slightly higher than expected score. This might be caused by the fact that our TD children were screened for developmental problems. Because the groups differed in nonverbal reasoning skills, the Raven CPM score was used as a covariate in further analyses. We also found significantly lower scores for the PLI group on measures of receptive language skills and expressive language skills.
Background data on the participants
Note: TD, typically developing; PLI, pragmatic language impairment; CPM, Coloured Progressive Matrices.
Group differences in narrative competence. Boxplots were employed to visually examine possible outliers. Five children (two in the PLI group, three in the TD group) were eliminated from subsequent analyses due to extreme performance on at least two of the narrative and/or language measures. Although this ensured minimal effects of outliers, it did not result in the elimination of many subjects.
The mean scores for the narrative measures can be found in Table 2, along with the results of the MANOVA. The MANOVA proved to be significant (Wilks Λ = 0.75), F (6, 141) = 7.77, p < .001, ηp2 = 0.25. Concerning the narrative length, children with PLI showed a lower number of T-units, combined with a lower ML5LU. Moreover, the T-units of the PLI group were characterized by less complex syntax, as is evident by their lower number of subordinate clauses. Differences were also visible at the level of story content, with the PLI group being able to reproduce roughly 35% of the plot structure components, and the TD children reproducing 50%. The PLI group also produced a higher proportion of irrelevant T-units: 21% for the PLI group compared to 13% for the TD children. Finally, the PLI group also produced a higher proportion of implicit references. All the differences remained significant when nonverbal reasoning was added as a covariate (Wilks Λ = 0.77), F (6, 140) = 6.91, p < .001, ηp2 = 0.23.
Means and standard deviations on narrative measures across groups
Plot structure components. As stated, children with PLI expressed lower thematic content as revealed by a lower score on the total plot structure components. Separate one-way analyses of variance (ANOVAs) were used to examine possible group differences in specific plot structure components. The mean scores for the subdivision into different plot structure components are shown in Table 3, along with the results of the ANOVAs. The children with PLI expressed significantly less information on the setting, the initiating events, the outcome, and the emotional responses. With regard to the events, the PLI group did not show a lower score on the first event, although they did show a lower score on the subsequent events. The groups did not differ in the amount of information regarding the morality of the story. Controlling for nonverbal reasoning skills did not change the outcome for any of the ANOVAs.
Means and standard deviations on specific aspects of organization of story content across groups
Proportion of irrelevant T-units. To find whether specific types of irrelevant T-units accounted for an overall higher rate of irrelevant T-units, we performed additional ANOVAs subdividing irrelevant T-units into incomprehensible T-units, made up T-units, and wrong episode T-units. Significant differences were found in the first two categories, with the PLI group showing higher rates of incomprehensible T-units, F (1, 146) = 6.53, p < .05, ηp2 = 0.04, and made up T-units, F (1, 146) = 8.08, p < .01, ηp2 = 0.05. Children with PLI did not, however, show higher rates of T-units belonging to events expressed in an incorrect order. Results did not change after controlling for nonverbal reasoning skills.
Group differences controlling for language level
Because the groups differed in language skills, all analyses were carried out again using language skills as a covariate. Data reduction on the language measures was performed using PCA with preset eigenvalues of >1.0. The one-factor solution of the PCA, in combination with the high item loadings (all > 0.79) provided evidence of one concept underlying our language measures. Individual regression scores on this factor were therefore used as an indicator of the language skills of the children. The results of the MANOVA remained significant after the language composite was used as a covariate (Wilks Λ = 0.91), F (6, 136) = 2.26, p < 0.05, ηp2 = 0.09. Significant differences were still visible in the total number of T-units, the ML5LU, and the number of subordinate clauses. The remaining measures (proportion of plot structure components, proportion of noise and the proportion of implicit references) did not reveal significant differences once the language composite was added as a covariate.
Concerning the deeper investigation of specific division in plot structure components, covarying for language ability resulted in significant differences in the amount of initiating events and outcome, although the other differences were no longer significant. Differences in the proportion of irrelevant T-units were no longer visible after controlling for language ability.
Factor analysis
To verify that our measures of narrative competence could indeed be subdivided into the different sets of abilities as proposed by Coelho (Reference Coelho2002), we performed a PCA with varimax rotation specifying three factors. As Table 4 shows, three factors were found to explain 85% of the variance. The first factor corresponded to the measurements pertaining to narrative productivity and explained 47% of the variance. The second factor corresponded to the measurements of organization of story content and explained an additional 23% of the variance. The last factor corresponded to the measurement of cohesion and explained 15% of the variance. Because the factor analysis supported the theoretical division as proposed by Coelho, regression coefficients of the three factors were used for subsequent analyses.
Varimax rotated three factor principal components solution on narrative measures on both groups
Note: Values > .30 are reported.
Group differences in false belief understanding and EF
Descriptive information of children's performance on the false belief task and EF tasks is shown in Table 5. The result of the MANOVA proved to be significant (Wilks Λ = 0.66), F (5, 140) = 14.19, p < .001, ηp2 = 0.34. The PLI group performed significantly lower on the false belief task as well as all of the EF tasks. These differences remained significant after adding nonverbal reasoning as a covariate (Wilks Λ = 0.73), F (5, 139) = 10.26, p < .001, ηp2 = 0.27. Although the outcome of the MANOVA remained significant (Wilks Λ = 0.81), F (5, 136) = 6.18, p < .001, ηp2 = 0.19, adding the language composite as a covariate resulted in some changes. The performance of the PLI group remained impaired on the auditory attention and response tasks, and on the number recall task. However, their performance on the false belief task and the tower task was no longer impaired.
Means and standard deviations on false belief understanding and executive function measures across groups
Relationship among narrative competence, EF, and false belief understanding
In order to reduce our data and create a stable EF factor, we performed a PCA on the four measures that were thought to reflect executive functions. The factor analysis yielded one factor with an eigenvalue over 1.0, which accounted for 50% of the variance of the tasks. Factor loadings on this factor fell between 0.56 and 0.85, with a mean factor loading of 0.70. Subsequently, for all children individual regression scores were computed based on the factor loadings of this single factor. Then, Pearson correlations were computed between the narrative factors on the one hand and the false belief understanding score and EF factor on the other hand. Among the TD children, the narrative productivity factor was positively correlated with both the false belief understanding task, r (72) = .39, p < .01, and the EF factor, r (71) = .30, p < .05. The story content organization factor did not show any significant correlations, as was the case with the cohesion factor. Among the PLI group, narrative productivity was significantly correlated with the EF factor, r (69) = .37, p < .01. There was no significant relation with the false belief task as found with the TD children. The story content organization showed a marginally significant correlation with the EF factor, r (69) = .22, p = .07. As with the TD children, no significant correlations were found for the cohesion factor.
To find whether the correlations between our narrative productivity factor and the cognitive measures could potentially be mediated by language skills, and that subsequently group differences in these correlations could be attributed to differences in these language skills, we carried out a hierarchical regression analysis (stepwise method) for each group with the scores on the narrative productivity factor as a dependent variable. In Step 1, we included the language composite score, in Step 2 the EF factor and the false belief understanding score were added. Step 1 resulted in significant models for the TD group, F (1, 68) = 24.55, p < .000, explaining 27% of the variance, as well as for the PLI group, F (1, 65) = 4.48, p < .05, explaining 6% of the variance. Step 2 also resulted in a significant model for both groups, although the predictors differed according to group. For the TD group false belief understanding predicted an additional 7% of the variance of the narrative productivity factor, F (2, 67) = 16.60, p < .000, whereas the predictive power of the language composite score was kept. The EF factor did not account for any variance in the model. Contrary to the regression results of the TD group, the significant predictor in the PLI group was the EF factor, F (2, 64) = 5.86, p < .01, explaining an additional 9% of variance of the narrative productivity factor. In addition, the predictive value of the language composite lost all of its value when the EF factor was added to the model (p = .33). Table 6 shows the results of the final model predicting narrative productivity.
Summary of the hierarchical regression analysis for variables predicting narrative productivity
*p < .05. **p < .01.
DISCUSSION AND CONCLUSION
The aims of the present study were to identify possible difficulties in narrative competence in children with PLI compared with TD children, and to assess to what extent these difficulties could be related to EF and theory of mind. Analyses revealed several findings that can shed more light on the narrative difficulties of children with PLI and the underpinnings of these difficulties.
Narrative competence in children with PLI
As expected, our PLI group showed problems in expressing narratives. Difficulties were visible at the level of narrative productivity, at the level of organization of story content, and at the level of cohesion. With regard to narrative productivity, the narratives of children with PLI were characterized by a lower total number of T-units, a lower ML5LU, and a lower number of subordinate clauses. Regarding story content, children with PLI showed lower rates of expressing plot structure components, combined with higher rates of irrelevant T-units in their narratives. We also found evidence of difficulties relating to cohesion, although we used only one measure as an indicator of cohesion. The results of narrative deficits that are found in this study are partly consistent with results found by Botting (Reference Botting2002).
The role of language skills
Because narrative tasks are considered a complex task measuring higher level language skills, the impaired language skills of our PLI group might have negatively affected their performance. This was actually the case in the area of story content organization. Our measures of story content organization are largely a measure of information conveyed per T-unit, and according to our measurements the PLI group requires more words to convey the same amount of information. It was shown that this impairment is entirely due to linguistic difficulties. The same effect was found in the area of cohesion. In the area of narrative productivity however, the group differences could not be explained entirely by language skills: after controlling for language skill, the PLI group still exhibited lower productivity. Because of the fact that the groups use the same number of T-units to convey a plot structure component (after controlling for language), this implies that the PLI group conveyed less plot structure components, in absolute terms. This is also borne out by, and can be explained by our detailed analysis of specific plot structure components. After controlling for language, it turns out that the PLI group often omits the initiating events and information regarding the story outcome. This is clearly a pragmatic deficit, because it is a failure to adhere to the needs of the listener. It is interesting that these results extend findings by Botting (Reference Botting2002), who found the same results for a PLI group relative to children with autism or SLI.
The role of theory of mind and EF
In addition to the role of language abilities, we were interested in the role of theory of mind and EF in narrative competence. One striking finding that emerged from our data was the fact that the groups exhibited different correlations. Among the TD children, skills related to the first narrative factor (narrative productivity) were positively correlated to our measure of theory of mind as well as to our EF composite score. Deeper investigation using regression analysis revealed that theory of mind accounted for unique variance, whereas EF did not. Theory of mind is said to be important for narrative competence in at least two respects (Mar, Reference Mar2004). First, to understand narratives (and consequently, to be able to tell the narrative correctly) narrators have to be able to grasp the motives that underlie the actions of the characters. Second, theory of mind may also play a pivotal role with regard to taking perspective, when it comes to pragmatic aspects of telling a story, for example, they require an understanding of the listeners’ needs. A possible alternative explanation for the specific relationship between our theory of mind measure and skills related to narrative productivity hinges on the ability to understand complex syntactic structures with an embedded clause (de Villiers & Pyers, Reference de Villiers and Pyers2002; Hale & Tager-Flusberg, Reference Hale and Tager-Flusberg2003). Children who are capable of producing complex syntactic structures are probably more able to understand false belief questions that are posed using an embedded clause. As such, children with higher linguistic skills will also show a higher performance on false belief questions. However, our regression analyses showed that false belief understanding still accounted for variance in narrative productivity once language ability was controlled for.
It is interesting that we did not find a similar relationship between theory of mind and narrative productivity in our PLI group. This is consistent with results from Tager-Flusberg and Sullivan (Reference Tager-Flusberg and Sullivan1995), who studied children with developmental delays, and Losh and Capps (Reference Losh and Capps2003) who studied children with autism. As an explanation, Tager-Flusberg and Sullivan (Reference Tager-Flusberg and Sullivan1995) suggested that EF abilities could be more important for narrative competence than sociocognitive abilities. This seems to be the case for our PLI group as evidenced by the outcome of the correlational analyses and the regression analysis. Although language abilities seem to account for some variance in the relationship between narrative productivity and EF, our results showed that language abilities no longer predicted variance in narrative productivity after EF was added to the model.
Concerning the second factor, skills relating to organization of story content, we did not find significant correlations for our TD group. Among the PLI group we did find a slight positive correlation with skills related to EF, albeit marginally. Finally, concerning the third factor, skills related to cohesion, we did not find any evidence for a relationship with theory of mind and/or EF for either the TD children or the PLI group.
Clearly the PLI group shows a different pattern of correlations compared to the TD children. It may be hypothesized that the deficits in theory of mind and EF prevent the PLI group from using the same strategies for retelling that are used by the TD children. For example, although the visual approach of our narrative task is thought to reduce memory demands (Flory et al., Reference Flory, Milich, Lorch, Hayden, Strange and Welsh2006), it might be hypothesized that children with PLI use an alternative compensatory strategy, which is more reliant on rote reproduction and therefore working memory. Alternatively, it may be hypothesized that some executive functions, for instance planning and attention, play a large part in the ability to adhere to the needs and interests of the listener, for example, to include important information such as initiating events and the story outcome.
Limitations and future directions
Although the present study sheds more light on the narrative competence of children with PLI as well as the underlying mechanisms of narrative competence, some limitations are worth mentioning. First, our PLI group, based on a community sample, was selected using the CCC. It should be mentioned that the CCC is a questionnaire designed to screen for PLI rather than a diagnostic tool, and positive screening should result in in-depth investigation. As such, one should be careful not to base a clinical diagnosis of a child solely on this questionnaire. Furthermore, we used teachers as informants only, when research has shown that the CCC is most effective when completed by more than one respondent. However, it should be noted that in earlier research into the psychometric properties of the CCC, we have found it to be reliable and valid as a screening instrument in the general population (Ketelaars et al., Reference Ketelaars, Cuperus, Van Daal, Jansonius and Verhoeven2009). We argue that given the present lack of a clear diagnostic classification system for PLI, the use of a valid and reliable screening instrument is the only way to achieve a consistent, uniform selection criterion.
Second, the use of a sample from the general population may be a limiting factor in the present use of the results of this study. This is due to the fact that previous studies mainly used samples from clinical groups, which makes it difficult to compare the results. However, we believe that our approach does have its merits. Due to the lack of a clear diagnostic classification system for PLI, the use of samples from clinical groups may be overly restrictive and may not yield results that are representative for the entire PLI population. By using a sample from the general population we have avoided this pitfall.
Third, there are some limitations related to the tasks used in this study. The present study applied a story retelling design rather than a story generating design. We outlined our arguments for story retelling in the Introduction. However, there is still ongoing debate whether story retelling is a good reflection of narrative competence. It might be considered easier than story generation, resulting in an overestimation of the narrative abilities of children with language impairments. In defense of our study, it can be argued that because both groups were subjected to the same condition, and because significant differences were still being observed, retelling as a tool for measuring narrative competence is informative regarding the skills of the subjects.
As with the diversity of ways to assess narrative competence, there is a great variety in the narrative measures one can assess. The measures that have been used in this study are not all independent. For example, the ML5LU will be affected by the number of subordinate clauses that have been used. As a consequence, children with a lower number of subordinate clauses will often also have a lower ML5LU. This weakness has been mitigated by the use of factor scores rather than individual scores.
Similarly, there are some limitations pertaining to the choice of our EF tasks. Although we used tasks that are widely used, a variety of executive functions are involved in each of the tasks. Because our main goal was to establish a relationship between narrative competence and EF abilities in general, we used factor scores instead of single test scores. This reduces errors made by the use of single tests. In addition, because our EF tasks each measure a variety of executive functions, it is difficult to make claims as to the specific EF that results in a relationship. This prevents us from drawing conclusions regarding the exact nature of the relationship with EF. In further studies it would be interesting to differentiate between different executive functions, using a large battery with multiple tasks that are thought to reflect a single executive function.
CONCLUSIONS
In conclusion, the results of the present study suggest that young children with PLI exhibit extensive narrative difficulties relating to narrative productivity and organization of story content. This finding can be used to identify the specific difficulties of children with PLI in relation to children with other disorders. In order to come to more definite answers regarding the nature of the PLI, it would be interesting to investigate narrative competence using a longitudinal design. This could clarify the extent to which the narrative impairments as found in this study should be characterized as a delay, or whether the narrative impairments are qualitatively different and do not improve over time, which would be consistent with the view of PLI as a disorder. In addition, it would be worthwhile to compare the narrative abilities of children with PLI and children with ASD, to gain more insight into the degree of overlap between both disorders, as well as to find markers that might be unique to one of the disorders.
A second important finding is that the narrative difficulties are related to some extent to executive dysfunctioning. This raises the question whether improving specific executive functions could prove useful in mediating linguistic and communicative difficulties as experienced by children with PLI. The possible mediating role of EF has been suggested in children with language-learning disorders (Singer & Bashir, Reference Singer and Bashir1999). In addition, working memory training has been suggested in children with SLI (Montgomery, Reference Montgomery2003), because many children with SLI suffer from deficiencies in working memory, and improvements in this area may result in improved language skills as well. The possibility of training executive functions in order to improve narrative competence would be a worthwhile research subject, because narrative competence is related to social competence as well as later academic skills (Bishop & Edmundson, Reference Bishop and Edmundson1987; Roth, Speece, Cooper, & De La Paz, Reference Roth, Speece, Cooper and De La Paz1996).
APPENDIX A
Plot structure components in the Bus Story Test
