In a world where bilingualism is increasing, numerous challenges arise for researchers and clinicians in finding ways to effectively test a bilingual child's language in cases where standardized tests do not exist or have only been normed on monolingual speakers (Kohnert, Reference Kohnert2010). The development of bilingual and monolingual children's language is broadly similar (e.g., Bialystok & Craik, Reference Bialystok and Craik2010), but there are two crucial differences that will inevitably influence language progress: (a) the organization of language on multiple linguistic levels is necessarily more complex when additional languages are present; and (b) input will vary in its quantity, quality, and timing between a bilingual child's first and second languages (Gathercole, Reference Gathercole and Gathercole2013). A particular challenge in this context is distinguishing between delayed language due to insufficient input, which is likely to improve over time, and actual language impairment requiring early intervention (Chiat et al., Reference Chiat, Armon-Lotem, Marinis, Polišenská, Roy, Seeff-Gabriel and Gathercole2013).
Narrative assessments have shown promise in addressing the challenges involved in assessing bilingual children's language (Squires et al., Reference Squires, Lugo-Neris, Peña, Bedore, Bohman and Gillam2014), but previous results have been inconsistent and focused on production rather than comprehension (Gagarina et al., Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012). Studies in this area have primarily been conducted in the United States (English–Spanish bilinguals) but bilingualism has received little attention of any kind in Eastern Europe. This paper builds on previous studies by testing narrative assessments developed specifically for bilingual children in the context of Eastern Europe with a language combination that has not yet been studied: Slovak–English. This study explores the effect of the children's first language (L1; Slovak) versus second language (L2; English) on story macrostructure and microstructure in this cultural context, and the possible transfer of these structures between languages of successive bilingual children.
There are additional broader challenges involved in assessing bilingual children that this study aims to explore. Researchers and clinicians such as speech and language therapists (SLTs) only often know one of a bilingual child's languages. This has led to suggestions in the literature that nonword repetition (NWR) can contribute to the assessment of bilingual children in situations where specific language knowledge is lacking (e.g., Thordardottir & Brandeker, Reference Thordardottir and Brandeker2013). In this study, we examine NWR and its relationship to narrative performance. Before NWR and narrative tasks are jointly utilized in language assessments for bilingual children, more information is needed to understand how they relate to each other, as addressed in this study.
NARRATIVE ASSESSMENT OF BILINGUAL CHILDREN’S LANGUAGE SKILLS
The use of narrative assessment with bilingual children has a number of motivations. First, narrative assessments are informative and ecologically valid in that they reflect the importance of stories in most children's lives (Botting, Reference Botting2002; Kit-Sum To, Stokes, Cheung, & T’sou, Reference Kit-Sum To, Stokes, Cheung and T'sou2010; Skarakis-Doyle & Dempsey, Reference Skarakis-Doyle and Dempsey2008). Second, narrative assessments provide researchers with the opportunity to examine multiple linguistic levels in one task: the microstructural level (e.g., morphosyntax, referential devices, and lexicon) as well as the more global macrostructural level (i.e., the organization of the story telling above the sentence level; Iluz-Cohen & Walters, Reference Iluz-Cohen and Walters2012; Pearson, Reference Pearson2002). Third, narrative skills have been shown to reflect cognitive development and predict academic success in typically developing (TD) children (Kit-Sum To et al., Reference Kit-Sum To, Stokes, Cheung and T'sou2010), while also revealing language impairment (Botting, Reference Botting2002; Norbury & Bishop, Reference Norbury and Bishop2003).
There have been a number of studies that have investigated the transfer of narrative ability (or of language ability in a narrative context) between TD bilingual children's L1 and L2. Pearson (Reference Pearson2002) found that scores for macrostructure (story elements, sequencing, referencing, internal states, and engagement) were similar for both Spanish (L1) and English (L2). Gutiérrez-Clellen (Reference Gutiérrez-Clellen2002) also found that TD bilingual second graders produced grammatical narratives in both Spanish (L1) and English (L2), but performance on story recall and associated comprehension tasks was actually better in English than in Spanish. Squires et al. (Reference Squires, Lugo-Neris, Peña, Bedore, Bohman and Gillam2014) claimed that macrostructure ability in Spanish (L1) transferred to English (L2) based on their findings for Spanish–English bilinguals. Iluz-Cohen and Walters (Reference Iluz-Cohen and Walters2012) studied sequential English–Hebrew bilingual children. They also reported transfer of story structure (macrostructure) across the languages of a bilingual child, but lexical and morphosyntactic abilities (microstructure) remained more language specific and did not transfer to the same extent. Overall, these findings point to increasing evidence that narrative macrostructure is less reliant on language ability compared to microstructure.
The use of narrative assessments in distinguishing monolingual children with language impairment from their TD peers (see Boudreau, Reference Boudreau2008; Cleave, Girolametto, Chen, & Johnson, Reference Cleave, Girolametto, Chen and Johnson2010) has been extended to bilingual children, with the aim of enabling researchers and clinicians to more effectively differentiate between disordered language and difficulties that may only arise from insufficient input in an L2. TD children are expected to obtain higher scores than children with language impairment regardless of whether they are tested in their L1 or L2 because they will not be compromised by impairment in either language. A notable recent example of assessment aimed at eliciting data from bilingual children is the Multilingual Assessment Instrument for Narratives (MAIN; Gagarina et al., Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012), developed to evaluate narrative abilities across 26 European languages and to aid in distinguishing specific language impairment from TD in bilingual children. The MAIN includes measures of narrative production and comprehension.
COMPREHENSION SKILLS IN MONOLINGUAL AND BILINGUAL CHILDREN
It is well established that comprehension (receptive) difficulties in monolingual children highlight a risk of language impairment before verbal expression emerges (Bishop et al., Reference Bishop, Holt, Line, McDonald, McDonald and Watt2012; Eadie et al., Reference Eadie, Nguyen, Carlin, Bavin, Bretherton and Reilly2014; Polišenská & Kapalková, Reference Polišenská and Kapalková2014a). Thal, Tobias, and Morrison (Reference Thal, Tobias and Morrison1991) suggested that comprehension is an excellent predictor of recovery from expressive language delays, and Bates (Reference Bates1993) argued that children with delayed language who can build on their receptive knowledge of language have a much better chance of catching up with their peers. Comprehension difficulties may also directly affect narrative skills. Boudreau (Reference Boudreau2007, in Boudreau, Reference Boudreau2008) compared the narrative performance of a group of children who had receptive and expressive language impairments with a group of children exclusively with expressive impairment. The study found that children who also had receptive language impairments performed more poorly across a range of micro- and macrostructural measures than children who only had expressive difficulties. Narrative comprehension has also been related to later literacy. For example, Paris and Paris (Reference Paris and Paris2003) assessed monolingual children of a similar age range (61–98 months) to our participants, and found a link between narrative comprehension and reading skills.
There has been limited research on narrative comprehension in bilingual children. A notable exception is a study by Gutiérrez-Clellen (Reference Gutiérrez-Clellen2002) in which English–Spanish bilingual children showed significantly better comprehension in English than in Spanish, although with more variable results for Spanish. Of importance, the author pointed out that it is possible that the stories used in the study did not have the same level of difficulty across their two languages. In the present study, parallel stories across our two languages (see Procedure) ensure that any observed differences between languages cannot be attributed to task difficulty.
We also expect to find the same asymmetry between comprehension and production skills in bilingual children as found in monolingual children. We know that assessment of production skills can reveal delay in bilingual children's language relative to monolingual children. As Leonard (Reference Leonard2009) notes, expressive language problems are unlikely to occur in isolation. Knowing that a child is scoring within the normal range for comprehension in their L2 could help separate those whose language is developing normally from children with mixed receptive–expressive language impairments. The first step in providing better assessments for bilingual children who may be facing language disorder is understanding variation in TD bilingual populations, as investigated in this study.
NWR IN MONOLINGUAL AND BILINGUAL CHILDREN
Like narrative assessments, NWR tasks have also been proposed to identify language impairment in monolingual children (e.g., Botting, Reference Botting2002; Conti-Ramsden, Botting, & Faragher, Reference Conti-Ramsden, Botting and Faragher2001). A number of studies in different languages have shown that TD children consistently achieve higher scores on NWR assessments than children with language difficulties (Bortolini et al., Reference Bortolini, Arfé, Caselli, Degasperi, Deevy and Leonard2006; Coady & Evans, Reference Coady and Evans2008; Girbau & Schwartz, Reference Girbau and Schwartz2007; Kapalková, Polišenská, & Vicenová, Reference Kapalková, Polišenská and Vicenová2013; Thordardottir, Reference Thordardottir2008). Calderón (Reference Calderón2003) and Girbau and Schwartz (Reference Girbau and Schwartz2008) proposed that NWR tasks have the potential to identify language impairment even when only one language of a bilingual child is assessed. Thordardottir and Brandeker (Reference Thordardottir and Brandeker2013) showed that French–English bilingual children did not differ from TD monolingual children on a NWR task but did differ from children with specific language impairment, thus confirming the marker of NWR in bilingual children.
A number of studies have also investigated the relationship between NWR and vocabulary development, and links have been shown between the skills underlying NWR and the skills involved in vocabulary acquisition (e.g., Gathercole & Baddeley, Reference Gathercole and Baddeley1989) link is already present in children under 2 years of age (Hoff, Core, & Bridges, Reference Hoff, Core and Bridges2008; Polišenská & Kapalková, Reference Polišenská and Kapalková2014b). In addition, NWR ability has been shown to be significantly related to vocabulary skills in bilingual children (e.g., Messer, Leseman, Boom, & Mayo, Reference Messer, Leseman, Boom and Mayo2010; Service, Reference Service1992), and Adams and Gathercole (Reference Adams and Gathercole1995) confirmed that the link found between NWR and vocabulary extends to other linguistic domains. Adams and Gathercole's study demonstrated that children who scored better on short-term memory measures including a NWR task were able to produce a wider range of grammatical structures in their spontaneous speech, and the authors proposed that the skills revealed by NWR may be vital for learning new syntactic constructions. Similarly, Montgomery (Reference Montgomery1995) found a positive correlation between NWR performance and performance on a sentence comprehension task.
Together, these studies provide evidence that connections between NWR performance and language exist beyond the word level, and that phonological short-term memory (tapped by NWR) can be seen as a prerequisite for learning new syntactic constructions in addition to being a feature of vocabulary acquisition. In sum, NWR provides another method for distinguishing between language difficulties caused by insufficient input versus actual language impairment.
THE CURRENT STUDY
Using a within-subject design, the current study assessed successive Slovak–English bilingual children on narrative production, narrative comprehension, and NWR performance in order to investigate how these skills relate to each other across and within bilingual children's languages. The study was conducted in Slovakia, a cultural and linguistic context that has rarely been explored in studies on bilingual children. The Slovak language, a member of the West Slavic language group, has a rich morphology, is heavily reliant on inflections, and has a relatively free word order. These features are all in contrast to English with its sparse morphology and reliance on function words within a stricter word order. Slovak is a pro-drop language, and pronominal subjects are usually omitted unless special emphasis is required (Kesselová & Slančová, Reference Kesselová and Slančová2010).
The MAIN, which was developed by Gagarina et al. (Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012), was used for the narrative assessment (see Method). The MAIN's theoretical framework for macrostructure was provided by the “story grammar” model, seen as universal knowledge about story telling and defined by story components, characters, and sequence of events (Stein & Glenn, Reference Stein, Glenn and Freedle1979). It has been suggested that macrostructure performance is partly dependent on cognitive schemas and is thus relatively less dependent on language skills than is microstructure performance (Iluz-Cohen & Walters, Reference Iluz-Cohen and Walters2012; Pearson, Reference Pearson2002). Thus, one might expect children with intensive exposure to an L2 to show roughly similar performance in their L1 and L2. This expectation is reflected in Hypothesis 1: production of macrostructural components in L1 and L2 will not significantly differ in children with a minimum of 12 months intensive exposure to English as an L2.
Narrative production and comprehension can be seen as distinct skills (Boudreau, Reference Boudreau2008), and it is well documented that comprehension skills precede production skills in language development (English: Bates, Dale, & Thal, Reference Bates, Dale, Thal, Fletcher and MacWhinney1995; Slovak: Kapalková, Slančová, Bónová, Kesselová, & Mikulajová, Reference Kapalková, Slančová, Bónová, Kesselová and Mikulajová2010). Previous research has also shown that children with language impairment have difficulties in the comprehension of connected discourse (e.g., Norbury & Bishop, Reference Norbury and Bishop2003). If comprehension scores of TD bilingual children with sustained exposure to their L2 do not differ across languages, this finding could have implications for future use of the MAIN (i.e., gaps in comprehension between the L1 and L2 under similar conditions of exposure might be interpreted as atypical and as needing further investigation to rule out language impairments). In the present study, we explore narrative comprehension of Slovak–English bilinguals, leading to the second hypothesis: comprehension of macrostructural components in participants’ L1 and L2 will not be significantly different in children with a minimum of 12 months intensive exposure to English as an L2.
Microstructure scores directly depend on knowledge of a particular language (L1 or L2) and its lexical items (word level), grammatical rules (sentence level), and links between these elements (discourse level). Our microstructure scoring focused on the word level, which has previously been proposed to effectively reflect language knowledge and used as a measure in bilingual research (Ebert, Pham, & Kohnert, Reference Ebert, Pham and Kohnert2014). Previous research has shown a link between NWR skills and vocabulary measures in L1 and L2 vocabularies (e.g., Gathercole & Baddeley, Reference Gathercole and Baddeley1989). We aimed to establish if this relationship remains the case in successively bilingual children and if this holds across both of their languages, leading to our third hypothesis: NWR skills will be significantly related to lexical diversity both in L1 and in L2.
The last research question investigates whether there are significant relations between bilingual children's performance on a NWR task and macrostructure skills. While macrostructure tasks assess language skills above the word level, NWR is mainly linked with phonological/sublexical processing and knowledge, so it is unclear if performance on these tasks will be linked. However, if children's language skills are weak, they might be weak across a whole range of tasks, and if so, combining two psycholinguistic markers of language difficulties (e.g., narrative and NWR) would strengthen the diagnostic potential of these measures.
METHOD
Participants
Forty TD Slovak–English sequential bilingual children participated in the present study. The 21 girls and 19 boys were all between 5 and 6 years of age, with a mean age of 70.55 months (SD = 7.30). The children had all been exposed to Slovak (L1) from birth, and the mean length of exposure to English (L2) was 38.22 months (SD = 18.98). The children in our sample all met the additional following criteria at the time of the experiment: (a) they had been exposed to English in exclusively English-speaking nurseries or schools in Slovakia (three in Bratislava and one in Trnava), (b) their mothers were all native speakers of Slovak, (c) the children lived in the Slovak Republic, (d) the children were attending a nursery (n = 34) or school (n = 6) where the language of instruction was English, (e) the children attended their nursery/school daily for 30–40 hr per week, (f) the minimum length of exposure to English was 12 months, and (g) the children had not been diagnosed with any emotional or neurological disorders and had no known visual or auditory impairment. In addition, four children from our sample (10%) had parents who used English to communicate with each other.
All of the children attended international schools in Slovakia that followed the UK school curriculum with all schooling and interaction in the nursery/school conducted exclusively in English. A clear majority (85%) of the teachers in the schools were native speakers of English from the United Kingdom or Ireland. Four of the children were born outside of Slovakia (in the Czech Republic, Greece, Luxembourg, and the United States), but no child stayed past the age of 18 months in an English-speaking country. All of the mothers of the children were native speakers of Slovak and 8 were Slovak monolinguals. In addition, among the mothers, 25 could also speak English, and 7 others spoke another language: German (3), Russian (3), or French (1). Most fathers were native, monolingual speakers of Slovak, while 5 of the fathers were native speakers of other languages (2 English, 2 Hungarian, and 1 Dutch). Based on the questionaire, the distribution of L1 (Slovak)/L2 (English) spoken at home for 12 of the children was L1 75% and L2 25%, while for the remainder (n = 28) it was L1 50% and L2 50%.
Parents did not express any concerns about the children's development on the parental questionnaire, nor did teachers report any concerns. No standardized language assessments are available for 5- to 7-year-old Slovak-speaking children (either monolingual or bilingual), so an experimental version of a Slovak sentence repetition task was used to check that all of the participants were TD. Sentence repetition has previously been shown to be a good clinical marker of language impairment across a number of languages (e.g., Conti-Ramsden et al., Reference Conti-Ramsden, Botting and Faragher2001; Kapalková et al., Reference Kapalková, Polišenská and Vicenová2013). No norms were available, but percentiles based on 20 TD children of the same age (and from the same area in Slovakia) were calculated, and all of the children in our sample scored above the 10th percentile.
Materials
Short version of the Beirut–Tours Questionnaire
The questionnaire was adapted to Slovak from the Alberta Language Environment Questionnaire and the Alberta Language and Development Questionnaire (Paradis, Reference Paradis2010, Reference Paradis2011) and included questions about the children's L2 background such as length of exposure, age of initial exposure, and family history of language impairment.
MAIN
The MAIN (Gagarina et al., Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012) was developed within the framework of the COST Action IS0804 Language Impairment in a Multilingual Society: Linguistic Patterns and the Road to Assessment (a pan-European research network funded by COST), in order to assess narrative production and comprehension skills in children from 3 to 10 years. The tool consists of parallel picture sequences to elicit narratives in the two languages of bilingual children. The current study used the version developed for English and Slovak that is divided into production and comprehension tasks. For the production tasks, narratives are elicited in two different modes: story generation (i.e., story telling) and story retelling, each later analyzed in terms of macrostructure. The comprehension task focuses on macrostructure components and internal state terms. The children's narrative productions and their responses to questions are later scored according to the MAIN guidelines (see sections on Scoring and Reliability below).
NWR task
The NWR measure by Polišenská and Kapalková (Reference Polišenská and Kapalková2014b) was also developed as part of the COST Action described in the preceding section. This task consisted of 26 novel items varying in phonological complexity and syllable length. The stimuli were recorded and administered individually over a laptop computer following the protocol described in Polišenská and Kapalková (Reference Polišenská and Kapalková2014b). The administration of the task took approximately 5 min. Whole-item scoring applied and was carried out from recordings: items were scored as correct if all phonemes were produced in the correct order; any omissions, substitutions, or additions resulted in an incorrect answer, earning zero points. The maximum score was 26.
Procedure
The participants were tested individually in a quiet area of their nursery/school by a qualified SLT who was a native speaker of Slovak and an advanced second language learner of English. In accordance with the MAIN manual, data from each language were collected in separate sessions with a testing interval of 5–7 days between each language in order to minimize cross-language influence and also training and carryover effects (Gagarina et al., Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012). Both languages were tested by the same tester, as would usually be the case in a clinical setting.
The experimenter sat opposite the child and started the session with six simple wh-questions (e.g., “What's your name?”; “Who is your best friend?”), with the dual purpose of putting the child at ease and ensuring their ability to understand simple wh-questions (Gagarina et al., Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012). All of the children in our sample were able to provide answers to these questions. Testing of the narrative skills immediately followed the warm-up questions. The four MAIN stories (Dog, Cat, Baby Birds, and Baby Goats) served as prompts for L1 story telling and retelling; L2 telling and retelling. During the first session, L2 telling and retelling was assessed, both for production and for comprehension. The second session, which took place 5–7 days later, assessed L1 telling and retelling skills. The order of story presentation was counterbalanced (for details, see Gagarina et al., Reference Gagarina, Klop, Kunnari, Tantele, Välimaa and Balčiūnienė2012).
The four MAIN stories (six images each) were all hidden in separate envelopes. Three envelopes containing the same picture sequence were placed on the table, and the child was asked to choose one envelope (the child was given to believe that the experimenter did not know which story had been chosen to control for the effect of shared knowledge). The experimenter took out the selected picture story and gave it to the child with instructions to look at the whole story but not to show any of it to the experimenter. When the child was ready to tell the story, the pictures were folded so that only the first two images could be seen. When the child was finished describing Pictures 1–2, the next pair of images were shown so that four pictures were unfolded. After the child had finished their description of Pictures 2–4, the last two pictures were unfolded so the whole story could be seen. The comprehension part of the task was assessed by questions asked after the production part of the narrative task. In the case of story retelling, the experimenter read the story to the child, and then the child was asked to retell the story. The whole session was audio-recorded, and the responses were later transcribed and scored. The overview of the testing sessions is summarized in Table 1.
Overview of testing sessions and assessments administered
Note: There was a 1-week break between sessions.
aAll children sentence repetition.
bAll children nonword repetition.
Scoring
Macrostructure production
A maximum of 17 points was possible from producing the story components in the following ways:
1. setting (information about the time and place that events took place): 2 points for time and place; 1 point for either time or place; and 0 points if neither mentioned
2. initiating event (an event or internal state that sets the events of the story in motion): 1 point
3. goal (an expression of the protagonist's intention of how to deal with the initiating event): 1 point.
4. attempt (an indication of an action to obtain the goal): 1 point
5. outcome (a statement describing if the goal was reached following the attempt, whether successful or not): 1 point
6. mental state as reaction (a statement defining how the protagonist(s) feels, for example, “happy,” “disappointed,” or “angry” or thinks about the outcome): 1 point
Each story contained three episodes, and each episode contained one main character, making it possible within each episode to assess five of the six components above (mental state as initiating event, goal, attempt, outcome, and mental state as reaction). The only exception, setting, was assessed once at the beginning of the story. A child could score a maximum of 17 points per story. Macrostructural scores were derived by combining scores from the story telling and story retelling in each language (maximum 34 points in Slovak and 34 points in English).
Macrostructure comprehension
Nine comprehension questions that focused on macrostructure components and internal state terms were presented as part of the assessment procedure, with 1 point available for each question up to a maximum of 9 points per story. Comprehension scores (maximum 18 points in Slovak and 18 points in English) were derived by combining scores from the story telling and story retelling tasks in each language.
Microstructure production (word lemmas)
The children's narratives were transcribed in the Codes for Human Analysis of Transcripts format of the Child Language Data Exchange System (MacWhinney, Reference MacWhinney2000), and the number of different word lemmas per story was calculated. Microstructure scores were derived by combining the scores from story telling and retelling within each language.
Reliability
The scorer for the Slovak measures was a native speaker of Slovak, while a native speaker of English scored the English tasks. The narratives and comprehension question responses from four children (10% of the sample) were scored by an independent second rater. Following Streiner and Norman (Reference Streiner and Norman1995), the intraclass correlation coefficient (ICC) was used to assess interrater reliability. The reliability coefficients for macrostructure were Slovak α = 0.78 for telling and α = 0.94 for retelling; and English α = 0.78 for telling and α = 0.81 for retelling. The reliability coefficients for microstructure were Slovak α = 0.91 for telling and α = 0.97 for retelling; and English α = 1 for telling and α = 0.96 for retelling. Cicchetti (Reference Cicchetti1994) suggested cutoff points for agreement based on ICC values; values between 0.60 and 0.74 are considered good, and values between 0.75 and 1.0 are considered excellent. Using these cutoffs, the interrater agreement was excellent for all production measures.
The reliability coefficient for Slovak comprehension was α = 0.85 for telling and α = 0.62 for retelling; for English comprehension, α = 1 for telling and α = 0.88 for retelling. All comprehension measures achieved good or excellent levels of interrater agreement. The NWR test has previously been shown to have excellent interrater agreement (ICC = 0.96) and test–retest reliability (ICC = 0.87); for details, see Polišenská and Kapalková (Reference Polišenská and Kapalková2014b).
RESULTS
Compliance rate and outline of analyses
The compliance rate of the participants was high, and only 1 child refused to tell a story in English. Due to an experimenter error, 4 of the children did not receive the comprehension task. All of the children completed the NWR task. Thus, the data of 39 children (97.5 %) were analyzed to address Hypotheses 1 and 3, and the data of 36 children (90%) were analyzed to address Hypothesis 2.
The data generated in the production tasks (macrostructure: story structure and microstructure: number of different word lemmas) were normally distributed and analyzed with a set of analyses of variance (ANOVAs). The macrostructural data of comprehension did not meet the assumption for parametric analysis (data were not normally distributed as many children scored at or close to ceiling) and were therefore analyzed with nonparametric tests.
Preliminary analyses
A series of one-way ANOVAs with story as the independent variable (four levels: Dog, Cat, Baby Birds, and Baby Goats) confirmed that narrative performance did not significantly differ by story (i.e., no set of pictures elicited fewer or more responses, either for macrostructure or for microstructure; all ANOVAs: p > .05). Thus, the results could not be explained by the choice of story alone because all of the stories produced the same degree of difficulty.
Results according to hypotheses
The effect of language on production of macrostructure
The first hypothesis was addressed by a comparison of macrostructure production scores across two languages (L1 vs. L2). A paired-samples t test with production components of macrostructure as the dependent variable revealed a significant effect of language, t (38) = 4.34, p < .001, mean difference 95% CI = 1.12, 3.08, with the children achieving higher scores in their L1 (M = 15.49, SD = 2.69) compared to their L2 (M = 13.39, SD = 3.38).
Data were further analyzed for each narrative component and each language by summing the points per component for all the children in each language, and then dividing this sum by the total possible points to yield a percentage score. For example, for the narrative component goal, 67 points were obtained across children in Slovak, and divided by the maximal score of 234 (3 goals per story × 39 children × 2 stories per child in each language), yielding a score of 28.63%. The percentage scores for each narrative component and language are presented in Table 2. Their order of frequency was similar for L1 and L2: reactions were expressed the least often (20% and 13% for Slovak and English, respectively), followed by setting and goal, which were expressed at a similar rate (28% and 29% in Slovak, 28% and 32% in English). The next component was initiating event, which was expressed for 48% of the Slovak stories and 36% of the English stories. The most often expressed components were outcome (64% and 47% for Slovak and English, respectively) and attempt (71% and 69% for Slovak and English, respectively).
Percentage scores (total points/total possible points across participants) for macrostructural components produced in Slovak and English
Note: L1, First language; L2, second language.
The effect of language on comprehension of macrostructure
The second hypothesis investigated if language had a significant effect on comprehension of the macrostructural components. A Wilcoxon signed ranks test with comprehension components of macrostructure as the dependent variable showed no significant effect of language (z = –1.25, p = .211). The children did not achieve significantly higher scores in their L1 (median = 15) compared to their L2 (median = 15).
The qualitative analysis is summarized in Table 3. There were 9 possible points awarded in the comprehension part of MAIN, fewer than in the production part. Some question responses could be double scored (i.e., a child could receive credit for initiating event and goal for the same answer). Therefore, only questions that addressed a single component were chosen for the qualitative analyses: two questions in each story for initiating event, goal, and reaction. The order of difficulty was similar for L1 and L2. Goal was the least well-comprehended component (70% in Slovak, 65% in English), followed by initiating event (83% in Slovak, 81% in English) and reaction (86% in both Slovak and English).
Percentage scores (total points/total possible points across participants) for comprehension of macrostructural components in Slovak and English
Note: L1, First language; L2, second language.
The relationship between lexical diversity and NWR skills
The third hypothesis was investigated through a correlational analysis that examined if ability to repeat nonwords and lexical diversity were related across the two languages, as suggested by previous research. NWR performance was significantly related to lexical diversity in the children's L1 (r = .37, p < .05) but not in their L2 (r = .26, p = .113).
The relationship between macrostructural narrative abilities and NWR skills
A correlational analysis was also used to assess the relationship of NWR and narrative macrostructure (two measures identified by previous research as having diagnostic potential) in each language. NWR performance was not significantly related to macrostructure scores in Slovak, the L1 (r = .06, p = .729), or in English, the L2 (r = .27, p = .093). A summary of results is provided in Table 4.
Summary of results according to the hypotheses
Note: L1, First language; L2, second language; NWR, nonword repetition.
DISCUSSION
The current study presents data from sequential bilingual Slovak–English children who were tested using the MAIN. Kapalková et al. (Reference Kapalková, Slančová, Bónová, Kesselová and Mikulajová2010) asked parents of 1,750 children living in Slovakia about their children's language background, and 18% of the respondents said that their children had been exposed to a language other than Slovak. This relatively high number of bilingual children provided motivation for employing a task such as the MAIN in the cultural environment of Eastern Europe and for establishing certain psychometric properties of the instrument in this context, while also providing an opportunity to compare results on the MAIN to a NWR task currently used in Slovakia.
The MAIN task elicited high levels of compliance (39 out of 40 children completed the assessment) and provided excellent levels of interrater reliability across all its components (story telling, story retelling, comprehension, and lexical diversity). In addition, the results showed that the four MAIN stimulus stories produced equivalent results, suggesting that the stories are appropriate for assessment of both Ll and L2 and also for test–retest purposes. The lack of significant differences between stories also suggests that the stories are equally appropriate for the cultural environment of Eastern Europe. Overall, the MAIN assessment was easy to administer, reliable, and appeared to be culturally appropriate.
Reviewing the hypotheses in light of the results
Our first hypothesis that there would be no difference in macrostructural story components between L1 and L2 due to cross-linguistic transfer was not supported by the quantitative analysis. Instead, we found that macrostructure scores in the children's L1 were significantly higher than in their L2. These findings are at odds with previous studies such as Fiestas and Pena (Reference Fiestas and Pena2004), Pearson (Reference Pearson2002), and Iluz-Cohen and Walters (Reference Iluz-Cohen and Walters2012), who found that the use of story grammar categories was similar across children's languages. It should be noted that this finding does not exclude the possibility of cross-linguistic transfer, but the extent of any transfer was not sufficient to override the significant differences between L1 and L2. The means for L1 (M = 15.49) and L2 (M = 13.39) suggest that the difference was not large.
The more detailed analysis of the macrostructural components (see Table 2) showed that some components (e.g., goal and attempt) were expressed at similar rates across both languages. In contrast, children expressed initiating event, outcome, and reaction more often in Slovak (L1) than in English (L2), even though the same order of frequency was found across both languages. The same order could be explained by transfer or might be related to more general cognitive development. Children might find it easier to express specific macrostructural components earlier, while the remaining components could be expected to follow as their language/cognition develops (Marková, Reference Marková2011). Further research may reveal if this is the case and whether this development is manifested across both of their languages.
Although L1 macrostructure performance was stronger, the similar L2 score suggests that transfer from L1 to L2 did occur to some extent (though it is also possible that the children were developing similar skills independently in the L1 and L2, or that the L2 was influencing the L1). Some children only had 12 months of exposure to English, and a lower level of proficiency could explain the limited transfer. Using the MAIN task, future studies could investigate language transfer in simultaneously bilingual children to establish if performance is comparable after more evenly distributed exposure across languages.
The second hypothesis regarding the performance on the comprehension part of MAIN was supported by our results, with no significant differences found between children's L1 and L2 comprehension. Although the children in our sample showed differences in their production, comprehension of the macrostuctural components was equivalent for both languages. The qualitative analysis confirmed that the children showed the same order of difficulty for both languages in comprehending specific macrostructural components. Goal was followed by initiating event and the best comprehended component was reaction. In addition, the percentage of successfully comprehended items was very similar across languages, even in the components that were not close to ceiling (e.g., goal). This suggests that the lack of significant differences in comprehension should not be attributed to ceiling effects. The findings are in line with our suggestion that delayed comprehension in bilingual children could be indicative of language impairment. The comprehension results can also be contrasted to the production results. Of particular interest are the results for the component reaction (86% for both Slovak and English), which far exceeded production in both languages (20% and 13% for Slovak and English, respectively), with 100% representing the total number of possible points per component across children in each language. It is possible that the children were too young to productively express this component but more easily understood it when they were asked about it, and this trend was seen in both languages.
The maximum score for the comprehension part in our study was 18 points. The median score for our children (5- and 6-year-olds) was 15, which suggests that they were approaching ceiling and that the comprehension part is unlikely to show enough variation in older children. In clinical practice, the comprehension part could be used as a criterion-referenced task: if we expect 6-year-old children to achieve 15 points for each tested language, a lower score by an older child might suggest that the child is at risk of language impairment.
It appears that the minimum exposure of 12 months in our study was sufficient for comprehending the stories equally in the L1 and L2, but not for producing stories. These findings confirm previous results from typical language development studies that suggest comprehension skills precede production skills (Bates et al., Reference Bates, Dale, Thal, Fletcher and MacWhinney1995; Paris & Paris, Reference Paris and Paris2003), indicating that bilingual children will make gains with their comprehension before gains are made with production. This has important clinical implications. Early recognition of delayed comprehension allows for the important opportunity to identify language impairment (Bishop et al., Reference Bishop, Holt, Line, McDonald, McDonald and Watt2012; Paul & Roth, Reference Paul and Roth2010). The TD bilingual children in our study did not show worse performance in their L2 comprehension after a minimum exposure of 12 months to an L2. This raises the possibility that TD bilingual children with a similar length of exposure will show equivalent comprehension performance in both their L1 and their L2.
Bilingual children with language impairment, in contrast, might be expected to show a weaker profile in both comprehension and production than TD bilingual children, and this would be the case for both their L1 and their L2. Future studies that include bilingual children with language impairment could confirm if such predictions can be substantiated. As Boudreau (Reference Boudreau2008) pointed out, analyses of narrative production alone may not provide a complete picture of narrative skills. As a result, researchers and SLTs should consider separately evaluating comprehension and production when using narrative assessments.
The third analysis assessed the relation between NWR skills and lexical diversity. We found that children's ability to repeat nonwords in their L1 and their ability to learn new words as measured by the lexical diversity score was significantly related. This was in line with findings from previous studies with monolingual children (e.g., Gathercole & Baddeley, Reference Gathercole and Baddeley1990; Polišenská & Kapalková, Reference Polišenská and Kapalková2014b; Stokes & Klee, Reference Stokes and Klee2009). However, we failed to find a significant link to vocabulary acquisition in children's L2. Similarly, Summers, Bohman, Gillam, Peña, and Bedore (Reference Summers, Bohman, Gillam, Peña and Bedore2010) did not find a significant link between NWR and semantics in bilingual children, but did find a link with morphosyntax. Inclusion of a measure of morphosyntax in future studies could lead to finding a significant relation with NWR.
Other possible explanations for the lack of a significant link between NWR and L2 lexical diversity have been suggested in previous research. A study by Gutiérrez-Clellen and Simon-Cereijido (Reference Gutiérrez-Clellen and Simon-Cereijido2010) suggested that children's NWR performance in bilinguals is related to individual differences in language exposure and usage. Chiat, Polišenská, and Szewczyk (Reference Chiat, Polišenská and Szewczyk2012) and Chiat (Reference Chiat, Armon-Lotem, de Jong and Meir2015) have proposed an alternative approach to assessing NWR that involves a novel set of quasiuniversal set of nonwords based on the phonemes and phonological combinations that are most commonly found in European languages. These nonwords decreased the effect of exposure, and a study by Chiat and Polišenská (Reference Chiat and Polišenská2015) suggests that TD bilingual children (English L2, various L1s) who differed on a measure of receptive vocabulary did not differ on the NWR quasi-universal test.
The last analysis was exploratory, and did not show a significant link between NWR skills and macrostructure scores in either the L1 or the L2 of the children in our study. The relationship between the macrostructure scores and phonological processing appeared to be weak, likely due to the fact that NWR is mainly linked to phonological processing while macrostructural scores depend more on story grammar and also cognitive development (e.g., Halamová, Reference Halamová2013; Kim-Sum To et al., Reference Kit-Sum To, Stokes, Cheung and T'sou2010). Because the correlation between macrostructure and NWR tasks was not significant, this suggests that the NWR task and the MAIN task reveal different information about children's language performance.
Limitations and future directions
The current study did not assess proficiency of L2, or examine whether length of exposure to English predicted performance. Future studies could obtain more precise measures of L2 proficiency and establish how this relates to language transfer and NWR performance. It may be that benefits from transfer are likely to increase in simultaneously bilingual children or in children with more extensive exposure to both L1 and L2.
Another possible limitation relates to the languages of the tasks; each child started with tasks in the children's L2 (English). Given that we were testing the hypothesis that there is transfer of macrostructure from L1 to L2, we wanted to test performance in the L2 (less dominant language) first without the influence of L1. Although a 5–7 day interval between sessions and the use of parallel stories would have mitigated practice effects, such effects might still have occurred and could have contributed to the higher L1 performance. Counterbalancing the language of testing, followed by an analysis of order effects, could have improved the design.
The present study did not include a group of bilingual children with language impairment, and therefore it is not possible to establish sensitivity and specificity of the narrative task, the NWR task, or a combination of the two. In order to further evaluate the usefulness of the tasks in identifying language impairment, future research should evaluate performance on the MAIN task by both bilingual TD children and monolingual and bilingual children with language impairment, all in combination with NWR tasks in children's L1 and, if applicable, their L2. If bilingual children with language impairment show a weak narrative comprehension profile in both languages, this could indicate that assessing the child's narrative comprehension in one language (either L1 or L2) may be sufficient in clinical practice. Being able to assess only one language would be particularly helpful in the many cases where the SLT does not speak both of the child's languages and so has no way of assessing the child in both their L1 and L2. In addition, NWR assessment could add valuable information to the child's profile, particularly when low wordlike nonwords are used, for example the quasiuniversal NWR task developed by Chiat et al. (Reference Chiat, Polišenská and Szewczyk2012) and Chiat (Reference Chiat, Armon-Lotem, de Jong and Meir2015).
ACKNOWLEDGMENTS
This work was supported by the Slovak Research and Development Agency under Contract APVV-0410–11. The writing of this paper was supported by COST Action IS0804 Language Impairment in a Multilingual Society: Linguistic Patterns and the Road to Assessment (http://www.bi-sli.org).
