1. Introduction
Late elementary school is an important time related to long-term academic success; assessments during this period predict academic outcomes in 10th grade and graduation rates (Goldhaber et al., Reference Goldhaber, Wolff and Daly2020; Hernandez, Reference Hernandez2011). There is a shift in late elementary school such that children are expected to learn more independently (e.g., reading to learn), and there is an increase in the use of academic language, the more formal and often complex language system used in the classroom (Snow & Uccelli, Reference Snow, Uccelli, Olson and Torrance2009). Therefore, it makes sense that children who are less proficient with language around this time would experience increased long-term challenges in academic attainment. This does seem to be the case; children who are not as proficient in academic language struggle with the increasingly complex tasks of secondary school in the United States (Catts et al., Reference Catts, Adlof and Ellis-Weismer2006; Giguere et al., Reference Giguere, Tulloch, Core and Hoff2024). Some of these children are diagnosed with language or learning disability and receive special education support or services, but there are other populations of children who do not carry a clinical diagnosis but still have lower academic language proficiency: children with subclinical language weaknesses and some bilingual children. Because these children do not carry diagnoses of language impairment, they often go unnoticed, to the detriment of their educational outcomes. While most language screening measures are designed to identify children who are at risk for language impairment, there is a paucity of measures identifying non-impaired children who have challenges with academic language and may ultimately experience poorer academic outcomes (Tong et al., Reference Tong, Deacon and Cain2014).
Identification of clinical and subclinical language weakness continues to be an area of challenge for speech–language pathologists serving in educational settings (McGregor, Reference McGregor2020; Zhang & Tomblin, Reference Zhang and Tomblin2000). Identifying language deficits in children utilise clinical diagnosis practices, which are typically done in the first 9 years (Georgan et al., Reference Georgan, Archibald and Hogan2023). In the United States, qualifying for speech and language services in schools generally requires that children are in the lowest 7th percentile (standard score 1.5 standard deviations below the mean) based on performance on two or more standardised language assessments and that there is educational impact of the impairment. Research typically uses a cutoff of the 16th percentile (standard score 1 standard deviation below the mean) on standardised language assessments to diagnose developmental language disorder (Nitido & Plante, Reference Nitido and Plante2020). Contrasting the two metrics used to diagnose developmental language disorder, approximately 9% of children will score in the low end of normal limits yet fall above the cutoff for school service provision. Here, we refer to these children as having subclinical language weakness. For these children, listening and reading comprehension challenges often result in oral language and reading skills that are below same-age peers (Adlof, Reference Adlof2020; Domsch et al., Reference Domsch, Richels, Saldana, Coleman, Wimberly and Maxwell2012). Subclinical language weakness has also been tied to lower post-secondary educational attainment and lower occupational satisfaction (Durkin et al., Reference Durkin, Toseeb, Botting, Pickles and Conti-Ramsden2017; Winstanley et al., Reference Winstanley, Durkin, Webb and Conti-Ramsden2018). Because its full consequences often only become apparent after secondary school, subclinical language weakness has been referred to as a “hidden” deficit that goes undetected and untreated during critical periods of children’s language and academic development (Adlof, Reference Adlof2020; Nation et al., Reference Nation, Clarke, Marshall and Durand2004). As such, efforts are needed to develop means of identifying older children with subclinical language weaknesses to provide them with the supports they need for academic success.
As mentioned, some bilingual children also struggle with academic language in the classroom. Han (Reference Han2012) reported that children identified as non-English-dominant bilingual and non-English monolingual in kindergarten had lower reading and math scores through fifth grade compared to white English monolingual peers, even when children were receiving English language instruction provided by the schools. Diminished academic outcomes may be due, in part, to difficulty accessing and using academic language (Snow & Uccelli, Reference Snow, Uccelli, Olson and Torrance2009). Bilingual students who struggle with academic language and do not receive supplemental language support experience poorer secondary school outcomes than bilingual students who do receive supplemental language support (Callahan, Reference Callahan2006; Kim, Reference Kim2019). Therefore, like monolingual children with subclinical language weaknesses, it is critical to identify bilingual children who have challenges with academic language to provide them with the support needed to ensure their academic success.
By identifying and tracking the language proficiency of children with subclinical language weaknesses and bilingual children with difficulty accessing academic language, educators, and services providers may strategically deploy supplementary language enrichment opportunities (e.g., targeted reading intervention program; McLeskey et al., Reference McLeskey, Spooner, Algozzine and Waldron2022; Schiller et al., Reference Schiller, Wei, Thayer, Blackorby, Javitz and Williamson2012). However, as noted, most clinical language impairments are identified by age 9, and referrals for the comprehensive language assessment of children decrease throughout the school years (Ireland & Conrad, Reference Ireland and Conrad2016). This results in many children who may be struggling with language, for various reasons, being overlooked for language facilitation support that is crucial to their long-term success.
One way to identify these children is through short, easy-to-administer, and score language screening measures that a variety of professionals can use to identify subclinical language weaknesses in children from broad backgrounds. Although a number of language screening measures exist (see Bao et al., Reference Bao, Komesidou and Hogan2024 for a review of extant language screeners), these measures tend to focus on younger children and identify children at risk for developmental language disorder, not subclinical language weakness. For example, the Preschool Language Scales-5 Screening Test (PLS-5; Zimmerman et al., Reference Zimmerman, Steiner and Pond2011) provides a brief screener for language concerns in children from birth to 7;11. The tasks included on the PLS screener include global receptive and expressive language tasks that approximate the language demands of children during the first 8 years of life. Like language screeners focused on monolingual English speakers, many of the current measures developed for younger bilingual children’s language proficiency do not extend into the later school-age years (Bao et al., Reference Bao, Komesidou and Hogan2024). The Diagnostic Evaluation of Language Variation–Screening Test (Seymour et al., Reference Seymour, Roeper, De Villiers and Pearson2003), for instance, provides screening of the non-contrastive features of multiple dialects of spoken English, but the measure is only intended to extend into the ninth year of life.
As noted, the available screeners for older school-age children have primarily been developed to identify risk of language disorder rather than subclinical language weakness (Bao et al., Reference Bao, Komesidou and Hogan2024). Although normed for a broader age-range of school-age children, screeners like the Clinical Evaluation of Language Fundamentals–Fifth Edition Screening Test are primarily focused on the clinical markers of language impairment associated with younger children with language impairment but occurring at a higher rate (Wiig et al., Reference Wiig, Semel and Secord2013). Additionally, the reported specificity and sensitivity for such measures reflect later identification of language disorder rather than subclinical language weakness (Bao et al., Reference Bao, Komesidou and Hogan2024). To our knowledge, no screener for older school-age children has been developed with the purpose of identifying older school-age children with subclinical language weakness. A first step for better serving older school-age subclinical monolingual and bilingual children is the identification of the predictors of English language proficiency for these children.
Several language tasks have been advanced as possible predictors of language weakness in older school-age children (Dockrell et al., Reference Dockrell, Forrest, Law, Mathers and Charlton2022). Many of these tasks, such as non-word repetition tasks, sentence repetition (SR) tasks, and grammaticality judgement (GJ) tasks, have been incorporated into screeners that are used to detect risk of language impairment in younger children (Bao et al., Reference Bao, Komesidou and Hogan2024). However, as Marchman et al. (Reference Marchman, Wulfeck and Weismer1999) points out, it is possible that tasks sensitive to lower language proficiency in younger children may not prove sensitive for older children. Thus, higher-level language tasks, such as passage comprehension (PC) tasks, have been proposed to be sensitive measure of older school-age children’s developing command of English academic language skills (Ebert, Reference Ebert2020; Phillips Galloway & Lesaux, Reference Phillips Galloway, Lesaux, Santi and Reed2015). In this study, we focus on four tasks as potential predictors of English language proficiency in older monolingual and bilingual children-non-word repetition (NWR), SR, two GJ tasks, and a PC task.
1.1. Four tasks as predictors of English language proficiency
As children with language weakness move beyond the early school-age years, many clinical markers used in the screening of language impairment become less effective. For example, when considering monolingual English-speaking children with developmental language disorder, omission of finiteness markers in spoken simple syntactic structures becomes infrequent in the later elementary grades (Rice et al., Reference Rice, Wexler and Hershberger1998; Werfel et al., Reference Werfel2017). Therefore, it is important to identify predictors of language weakness that pertain to older school-age children.
NWR has been used to measure monolingual English and Spanish-English bilingual children’s ability to process word-like linguistic information (Pawłowska, Reference Pawłowska2014). During NWR tasks, children repeat word-like linguistic stimuli, and the accuracy of their repetition is measured. NWR stimuli adhere to the phonotactic constraints of the test language but are not actual words. Contemporary accounts of NWR posit that a host of underlying mechanisms are involved in successful NWR, including phonological processing, speech perception, and phonological short-term memory (Gathercole Reference Gathercole2006; Roy & Chiat, Reference Roy and Chiat2004; Summers et al., Reference Summers, Bohman, Gillam, Peña and Bedore2010). According to Summers et al. (Reference Summers, Bohman, Gillam, Peña and Bedore2010), NWR requires access to long-term lexical knowledge as well as phonological working memory to allow for a speaker to repeat back word-like stimuli. Thus, NWR taps both the strength of children’s long-term lexical representations as well as the working memory capacity that supports word learning (Roy & Chiat, Reference Roy and Chiat2004).
The utility of NWR tasks in identifying language weakness has been debated in the literature. For monolingual English-speaking children, NWR has emerged as a good predictor of English language proficiency during the early school-age years (Ellis Weismer et al., Reference Ellis Weismer, Tomblin, Zhang, Buckwalter, Chynoweth and Jones2000; Nation et al., Reference Nation, Snowling and Clarke2007; Thordardottir, Reference Thordardottir2008). However, Ebbels et al. (Reference Ebbels, Dockrell and van der Lely2012) found that NWR was not a strong predictor of language weakness beyond early childhood. For bilingual Spanish English-speaking children, the literature is more equivocal on whether NWR tasks are good predictors of English language proficiency. Summers and colleagues (2011) found that NWR performance positively correlated with cumulative English and Spanish language experience in a group of 5- to 6-year-old bilinguals. However, sequence of language exposure likely plays a central role in older bilingual children’s performance on NWR tasks. In a study of elementary school-age sequential Spanish-English bilinguals who learned Spanish first, English NWR was found to be a poor predictor of English language proficiency (Windsor et al., Reference Windsor, Kohnert, Lobitz and Pham2010). Additionally, in a study of early school-age bilingual Spanish-English-speaking children with and without language impairment, Gutiérrez-Clellen and Simon-Cereijido (Reference Gutiérrez–Clellen and Simon–Cereijido2010) found that NWR measures using English words does not predict language proficiency in either English or Spanish.
Like NWR, SR has been used to measure monolingual and bilingual children’s ability to process linguistic information as predictors for their English language proficiency (Ebert, Reference Ebert2014; Pratt et al., Reference Pratt, Peña and Bedore2020). SR tasks require accessing phonological, morphosyntactic, and semantic representations, along with the ability to store and recall linguistic material from memory (Pawłowska, Reference Pawłowska2014). In these tasks, children are asked to repeat sentences exactly as provided to them with. Sentences can vary in length and linguistic complexity following the goal of the task. Accuracy in SR tasks reflects not only children’s familiarity with underlying morphosyntactic representations but also children’s ability to store and manipulate linguistic information (Riches, Reference Riches2012). Participants’ performance on SR tasks reflects involvement of linguistic knowledge in long-term memory (LTM) to assist in the maintenance of verbal information in phonological short-term memory (STM; Jefferies et al., Reference Jefferies, Ralph and Baddeley2004). Linkage to linguistic knowledge in LTM expands informational capacity in phonological STM, allowing for immediate SR that greatly exceeds immediate word span recall (Baddeley et al., Reference Baddeley, Hitch and Allen2009). Like NWR, SR is posited to tap both long-term language knowledge as well as working memory (Pawłowska, Reference Pawłowska2014).
For monolingual English-speaking children, SR has emerged as a strong predictor of language proficiency, including syntax and morphology, during the early school-age years (Polisenska et al., Reference Polišenská, Chiat and Roy2015; Kidd et al., Reference Kidd, Lieven and Tomasello2006; Frizelle & Fletcher, Reference Frizelle and Fletcher2014; Leclercq et al., Reference Leclercq, Quémart, Magis and Maillart2014; Redmond et al., Reference Redmond, Ash, Christopulos and Pfaff2019). Christopulos and Redmond (Reference Christopulos and Redmond2023) found that SR tasks possess good diagnostic value in identification of language disorder in of SR in 5-year olds. SR task performance has also been advanced as a potential predictor of English language proficiency in multilingual children with and without language impairment (Meir et al., Reference Meir, Walters and Armon-Lotem2016; Pratt et al., Reference Pratt, Peña and Bedore2021; Wood & Hoge, Reference Wood and Hoge2019). According to Pawłowska (Reference Pawłowska2014), SR has proven an especially sensitive to language weakness because it taps the fragile linguistic representations and mild processing weaknesses of individuals with lower language proficiency.
GJ tasks are thought to involve not only children’s knowledge of spoken grammatical structures but the meta-linguistic awareness necessary for detecting grammatical or ungrammatical structures (Gottardo et al., Reference Gottardo, Stanovich and Siegel1996; McDonald, Reference McDonald2008). GJ tasks require the child to decide whether they believe a spoken sentence, presented grammatically or ungrammatically, is grammatically correct (Cairns et al., Reference Cairns, Schlisselberg, Waltzman and McDaniel2006; Calder et al., Reference Calder, Visentin, Claessen, Hollingsworth, Ebbels, Smith-Lock and Leitão2023). Calder et al. (Reference Calder, Visentin, Claessen, Hollingsworth, Ebbels, Smith-Lock and Leitão2023) argue that GJ tasks measure both linguistic competence and meta-linguistic awareness skill, requiring a listener’s knowledge of the grammatical components of a well-formed spoken utterance as well as the ability to consider the well-formedness of the spoken utterance. Additionally, GJ tasks require that listeners hold overheard utterances in working memory as they consider components of the spoken utterance (Gottardo et al., Reference Gottardo, Stanovich and Siegel1996; McDonald, Reference McDonald2008).
In a series of studies with children ages 5–18 years, Rice et al. (Reference Rice, Hoffman and Wexler2009, Reference Rice, Earnest and Hoffman2023) found that monolingual English-speaking children with typical language development consistently scored above their age peers with disordered language. Similarly, Dale et al. (Reference Dale, Rice, Rimfeld and Hayiou-Thomas2018) found that GJ tasks have shown some utility in identifying monolingual English-speaking adolescents with language weakness. However, little is known about the utility of GJ tasks for predicting the language proficiency of subclinical bilingual children. Balboa-Castells et al. (Reference Balboa-Castells, Ahufinger, Sanz-Torrent and Andreu2024) found that Spanish-Catalan bilingual children with language impairment performed worse than Spanish-Catalan bilingual children with typical language development on a grammatical correction task that required accurate GJ. It is possible, therefore, that GJ could be a significant predictor of language proficiency in bilingual children as well.
Successful PC tasks require higher-level language skills including listening comprehension and inferencing. In these tasks, children listen to passages then answer questions, literal and/or inferential, about the passage. These tasks align with curricular demands; as children advance into the later elementary grades, they must parse and comprehend spoken and written texts containing inferences and nonliteral language. Oakhill and Cain (Reference Oakhill and Cain2012) assert that PC tasks necessarily involve microstructural and macrostructural language knowledge, posing a challenge for children with weaker language proficiency.
Listening comprehension of passages containing inferential language has been investigated as a possible measure of English language proficiency in older school-age children (Lesaux & Kieffer, Reference Lesaux and Kieffer2010). In studies with monolingual English-speaking children aged 7–12 years, inferential PC tasks were more challenging for children with subclinical language weaknesses (Gough Kenyon et al., Reference Gough Kenyon, Palikara and Lucas2018; Lucas & Norbury, Reference Lucas and Norbury2015). For bilingual speakers, PC shows promise as a predictor of proficiency in the language of academic instruction. According to the task-dependent bidirectional transfer hypothesis (Prevoo et al., Reference Prevoo, Malda, Mesman and van IJzendoorn2016; Proctor et al., Reference Proctor, Harring and Silverman2017), the strength of cross language transfer depends on the type of language proficiency task presented to the child. Language tasks such as PC may highlight knowledge of the language of instruction because students may predominantly complete such tasks in classroom settings.
1.2. Language proficiency and the composite nature of comprehensive language measures
Comprehensive language measures are developed to evaluate different aspects of children’s developing language ability inclusive of multiple domains of language, such as semantics and syntax. One such measure often used in research and clinical settings is the Clinical Evaluation of Language Fundamentals – Fourth Edition (CELF-4; Semel et al., Reference Semel, Wiig and Secord2003) (see Nitido & Plante, Reference Nitido and Plante2020). The CELF-4 includes multiple subtests that can be considered individually or combined in different ways for more comprehensive assessment. For 9–12-year-old children, four subtests comprise the CELF-4 Core Language Scale: word classes, formulated sentences, recalling sentences, and concepts and following directions. Below we will briefly describe each subtest and what it is purported to measure.
In the CELF-4 Word Classes subtest, children see four words and are asked to identify the two words that are most similar and then explain why the words go together. Performance on this subtest is considered to measure children’s expressive and receptive understanding of the relationship between words related by semantic features (Semel et al., Reference Semel, Wiig and Secord2003). In the formulated sentences subtest, children are provided either one word or a short phrase and asked to use the word(s) in a sentence related to a picture. This subtest provides a broad assessment of the ability to integrate morphological, semantic, syntactic, and pragmatic rules expressively (Coret & McCrimmon, Reference Coret and McCrimmon2015; Semel et al., Reference Semel, Wiig and Secord2003). In the recalling sentences subtest, children are asked to repeat sentences of increasing length and complexity. This subtest is considered to assess semantic, morphological, and syntactic competence as well as short-term memory (Semel et al., Reference Semel, Wiig and Secord2003). Finally, in the concepts and following directions subtest, children are presented with a series of instructions (e.g., “point to the middle fish and the black house”) and asked to point to pictures following the instructions. From this subtest, we assess children’s ability to follow directions of increasing length and complexity presented orally (Semel et al., Reference Semel, Wiig and Secord2003).
Taken together, the CELF-4 Core Language Scale subtests provide measures of receptive (word classes, concepts and following directions) and expressive (word classes, recalling sentences, formulated sentences) language and cover the domains of lexical representation and semantics (word classes, formulated sentences, recalling sentences), syntax (formulated sentences, recalling sentences, concepts and following directions), and morphology (formulated sentences, recalling sentences, concepts and following directions), thus making it a broad measure of general language proficiency. Measures of language proficiency used to identify children as clinical or subclinical generally collapse across modalities; clinicians use results on these measures paired with other data to further identify specific areas of strength and weakness, as appropriate. Importantly for this study, the domains of language measured by the experimental tasks that included here overlap with those evaluated by the CELF-4 Core Language Scale subtests. Thus, our aim is not to identify a singular specific language domain that may be particularly effective at screening for subclinical language weakness but a measure that is sensitive to these weaknesses.
Here, as an initial step towards developing a screening measure to identify language weaknesses in older monolingual and bilingual school-aged children, we investigate the utility of using NWR, SR, GJ, and PC tasks in predicting the English language proficiency of older monolingual English and bilingual Spanish-English school-age children (Lam et al., Reference Lam, Wang, Wang, Anaya, Bedore and Peña2025).
1.3. This study
It remains unknown which tasks – NWR, SR, GJ, and/or PC – predict language proficiency in older monolingual and bilingual children. By examining the sensitivity of these linguistic tasks to English language proficiency, we take a first step towards the development of screening measures for school-age children and adolescents with subclinical English language weaknesses. Towards this end, we asked:
-
1) Are NWR, SR, GJ, or PC measures better predictors of English language proficiency in subclinical English monolingual school-aged children and early adolescents?
-
2) Are NWR, SR, GJ, or PC measures better predictors of English language proficiency in Spanish-English bilingual children?
We anticipate that the PC measure will be more sensitive to language proficiency in both subclinical English monolingual school-aged children and Spanish-English bilingual children. Our predictions are informed by both external evidence supportive of the utility of these measures as well as the nature of the comprehensive language measure used to establish language proficiency in our study.
A body of evidence suggests that listening comprehension and inferencing skills, like those required for our PC task, present a significant challenge for children with language weakness (see Lucas & Norbury, Reference Lucas and Norbury2015). Additionally, children require the foundational semantic, morphological, and syntactic language skills addressed across CELF-4 Core Language Scale subtests to complete higher-order language tasks. Therefore, we hypothesise that children with lower language proficiency will differ from children with greater language proficiency on our PC tasks.
Finally, we hypothesise that the other language tasks requiring knowledge of sentence and word-like stimuli will not be adequately taxing for older school-age children and fail to differentiate older school-age children of various language proficiency levels (Werfel et al., Reference Werfel2017). Although NWR, GJ, and SR may address the language skills needed for broad measures of general language proficiency, we hypothesise that these tasks alone will not sufficiently tap general language proficiency.
2. Method
2.1. Participants
Participants were recruited from community sources starting in the fall of 2022. The community sample was recruited through flyers, advertisements, and emails to families who indicated their interest in participation based on previous recruitment efforts. This resulted in a sample of 39 children, aged 9.0–12.9 (years; months; M = 10.9 and SD = 1.1), who completed all measures described below. Study procedures were approved by the San Diego State University Institutional Review Board for research on human subjects. Written consent was obtained from the children’s families, and verbal assent was provided by the children.
Children with varying levels of English-language proficiency were recruited for this study. Exclusionary criteria for both groups included history of developmental disorders (i.e., Down syndrome, autism spectrum disorder), significant neurological history (i.e., head injury, epilepsy), parent report of nonverbal intelligence outside of normal limits, and/or hearing loss. English-speaking monolingual and Spanish-English-speaking bilingual children were recruited for this study. To control for extraneous variables introduced by the diverse language experiences of each bilingual individual and to standardise bilingual participants’ experience with English, participants were excluded if they were first exposed to English in first grade or later. Twenty-five participants in the recruited sample were reported to be English-speaking monolingual children, and 14 were reported to be Spanish-English bilingual children.
2.2. Measure administration
Participants completed five measures: an NWR measure, an SR measure, a GJ measure containing of two separate tasks, a PC measure, and the comprehensive language measure (Clinical Evaluation of Language Fundamentals-4 Core Language Scale, CELF-4; Semel et al., Reference Semel, Wiig and Secord2003). The order of measure administration was randomised with the CELF-4 as the first or last measure administered. All task activities were completed by members of the Language Learning Lab and were conducted remotely via the Zoom® platform. Each child completed two remote research sessions from his or her home. Parents received written and spoken instructions regarding how to access the video conferencing application (Zoom®).
2.3 Index measures
2.3.1 Nonword repetition
NWR task involves the immediate exact repetition of monosyllabic and multisyllabic nonwords. In this study, we administered Dollaghan and Campbell’s (Reference Dollaghan and Campbell1998) NWR task, which consists of 16 items that increase in syllable length from one to four syllables (see Supplementary Appendix A for stimuli). Participants heard audio recorded instructions “I’m going to play some made up words. The first few words will be practice. Say the words exactly the way that I say them” followed by three practice words. For practice words, examiners provided feedback but did not repeat nonwords or offer feedback for experimental words. Following Dollaghan and Campbell (Reference Dollaghan and Campbell1998), each phoneme (consonant or vowel) was scored as correct or incorrect and used to calculate percent phonemes correct (PPC). Phoneme substitutions and omissions were scored as incorrect whereas instances of phoneme distortion were scored as correct. We calculated each participant’s PPC on the NWR task in line with previous studies investigating NWR as a predictor of child language ability (Archibald & Joanisse, Reference Archibald and Joanisse2009; Redmond et al., Reference Redmond, Thompson and Goldstein2011).
2.3.2 Sentence repetition
The SR task was developed for this study. This task requires the immediate repetition of 20 sentences. Experimental sentences increased in length and complexity over the course of task administration, with the first sentences consisting of eight morphemes (e.g., “Did the girl ride her bike to school?”) and the last sentences consisting of 18–20 morphemes (e.g., When the children finished their homework, they decided to eat dessert before going to bed.). The sentence types for this task included simple questions (two items), simple declarative sentences in the active voice (two items), simple declarative sentences in the passive voice (four items), multiclausal questions (two items), and complex declarative sentences in the active voice (10 items). We calculated each participant’s raw score on the SR task, in line with previous studies investigating SR as a predictor of child language ability (Archibald & Joanisse, Reference Archibald and Joanisse2009; Redmond et al., Reference Redmond, Thompson and Goldstein2011). See Supplementary Appendix B for the SR task stimuli.
Participants were instructed that “I am going to read some sentences to you. I want you to repeat them back to me, exactly as I say them.” Two practice sentences were administered to ensure that participants understood the demands of the task. For practice sentences, examiners provided feedback to participants who did not repeat sentences. For experimental sentences, examiners presented each sentence once and did not provide feedback.
Examiners scored sentences using a three-point scoring scheme. A score of “3” indicated that there were no errors in repetition (i.e., no lexical substitution, no changes in syntax, and no errors in grammatical morphology). A score of “2” indicated that there was one error in repetition (i.e., one lexical substitution, change in syntax, or error in grammatical morphology). A score of “1” indicated two to three errors in repetition, and a score of “0” indicated four or more errors in repetition. Scores were summed and reported as the participant’s SR score (max available = 60 points). The graduated scoring scheme aligns with the scoring procedures from other sentence recall studies of children with and without language weakness (Archibald & Joanisse, Reference Archibald and Joanisse2009; Oetting et al., Reference Oetting, McDonald, Seidel and Hegarty2016; Redmond et al., Reference Redmond, Thompson and Goldstein2011). Notably, similar three-point scoring schemes have been used in studies that have also used the CELF-4 as the comprehensive language measure (Christopulos & Redmond, Reference Christopulos and Redmond2023; Redmond et al., Reference Redmond, Ash, Christopulos and Pfaff2019).
2.3.3 Grammaggio A’ (GJWH and GJYN)
Grammaggio is a computerised app-based screener for grammatical impairments based on research on finiteness and morphosyntactic markers in English (Rice, Reference Ricen.d.). Each participant completed two 20-item GJ tasks (GJWH and GJYN), in which children heard spoken phrases and were asked to judge their grammaticality. GJWH included Wh- BE/DO questions (e.g., When are you eating; What do you like to do?) and GJYN included Yes/No BE/DO questions (e.g., Are you afraid? Do you like to fish?). Both tasks included grammatically correct questions and questions with omitted tense and agreement markers (e.g. Where the bug sleeping? The bear mean? see Supplementary Appendix C for stimuli).
Participants were instructed that “You’re going to hear some phrases. Sometimes they’ll sound right, but sometimes they won’t sound so good. If it sounds good, you can tell me “good” or give me a thumbs up. If it doesn’t sound so good, you can tell me “not good” or give me a thumbs down. I can only read the sentences one time, so I want you to listen carefully.” All participants answered 11 warm-up questions to orient participants to the task and allow examiners to explain the expectations for the task. For the warm-up items, examiners provided feedback to participants who did not respond appropriately. No feedback was provided for the experimental items, although examiners could repeat items as requested.
Examiners scored participant responses as “good” (G) or “not good” (N). A correct participant response was awarded a value of “1” and an incorrect response a value of “0.” Correct response scores were summed and reported as the participant’s A’ score as calculated by the Grammaggio app A’ is the proportion of correct responses in a two-alternative, forced-choice task (Rice et al., Reference Rice, Wexler and Redmond1999, Reference Rice, Hoffman and Wexler2009). Perfect discrimination of grammatical and ungrammatical would result in an A′ value of 1.0 and an A′ value of 0.5 represents chance performance.
A′ is a nonparametric sensitivity measure from signal detection theory that accounts for response bias and is robust to violations of normality and equal variance (Stanislaw & Todorov, Reference Stanislaw and Todorov1999). Furthermore, it is commonly used to quantify signal detection in two-alternative forced choice tasks, particularly in memory research (e.g., Abel et al., Reference Abel, Sharp and Konja2020). A′ reflects the proportion of true positive responses (hits) relative to false-positive responses (false alarms), yielding values from 0.5 (chance performance) to 1.0 (perfect performance; Stanislaw & Todorov, Reference Stanislaw and Todorov1999).
2.3.4 Passage comprehension
The PC task was also developed for this study. The PC task consists of spoken paragraphs and related comprehension questions that tapped inferencing ability and nonliteral language comprehension. Participants were read a brief spoken paragraph and were then asked between 5 and 10 comprehension questions, requiring knowledge of nonliteral language (e.g., “What does the man mean when he tells Maria, “do not be blue?”) and/or requiring that the participant be able to interpret clues from part of the text to determine the answer. There were six paragraphs and 43 questions total (see Supplementary Appendix D for stimuli).
Participants were instructed that “I am going to read you a story and then I’m going to ask you some questions about the story. So, listen carefully and answer the questions the best you can.” All participants heard each experimental passage once. Each comprehension question could be repeated one time. Examiners scored sentences using a two-point scoring scheme. A score of “2” indicated that participant response was accurate and specific (e.g., inferences were correctly made within the context of the passage). A score of “1” indicated that participant response was partially correct or too general (e.g., either nonliteral content was correctly interpreted in a general sense, but inferences were not correctly made in the context of the passage). A score of “0” indicated that participant response was not correct. Correct response scores were summed and reported as the participant’s PC score (max available = 86 points).
2.4 Comprehensive language measure
Consistent with past research investigating predictors of English language proficiency (Eadie et al., Reference Eadie, Nguyen, Carlin, Bavin, Bretherton and Reilly2014; Jacobson & Miller, Reference Jacobson and Thompson Miller2019), the CELF-4 (Semel et al., Reference Semel, Wiig and Secord2003) served as the comprehensive language measure for language proficiency. The Core Language Scale standard scores were calculated for each child following the CELF-4 manual.
2.5 Scoring
Data collection sessions were recorded using the HIPAA-complaint Zoom platform and were deidentified. All measures (NWR, SR, GJ, PC, and CELF-4) were scored online by the examiner administering the measures. An independent examiner re-watched the recorded session and re-scored each item. Any discrepancies were resolved by the two examiners with the assistance of the second author (AA). All examiners were students in a graduate speech-language pathology program.
3. Results
Table 1 provides descriptive statistics (means, standard deviations) for the index measures by language group. CELF-4 values are presented as standard scores, GJWH and GJYN are A’ scores, and all other index measure values are presented as proportion correct.
Descriptive statistics by language group
Note: M = mean, SD = standard deviation; CELF-4 = Clinical Evaluation of Language Fundamentals-4 Core Language Scale; NWR = Nonword Repetition Task; SR = Sentence Repetition Task; GJWH = Grammaggio WH- Questions; GJYN = Grammaggio Yes/No Questions; PC = Passage Comprehension.
Table 2 presents correlations between index measures and the standard scores for the CELF-4 for monolingual English-speaking participants. The weakest association was between the GJYN measure and the NWR measure (r = .34), and the strongest was between the SR measure and PC (r = .90). Additional strong associations were between the SR measure and the CELF-4 (r = .87), the GJWH measure and the SR measure (r = .83), and the GJWH measure and GJYN measure (r = .83). Moderate associations were between the GJYN measure and the SR measure (r = .76), the GJWH measure and the CELF-4 (r = .71), and the GJWH measure and the CELF-4 (r = .71). Supplementary Appendix G provides scatterplots depicting the association between each index measure and CELF-4 scores for monolingual, English-speaking participants.
Intercorrelations between the index measures and the comprehensive language measure for monolingual English-speaking participants
Note: CELF-4 = Clinical Evaluation of Language Fundamentals-4 Core Language Scale; NWR = Nonword Repetition Task; SR = Sentence Repetition Task; GJWH = Grammaggio WH- Questions; GJYN = Grammaggio Yes/No Questions; PC = Passage Comprehension.*Coefficient is significant at p < .01;**Coefficient is significant at p < .001.
Table 3 presents correlations between index measures and the standard scores for the CELF-4 for bilingual Spanish-English speaking participants. The weakest association was between the GJYN measure and the NWR measure (r = .50), and the strongest was between the SR measure and the CELF-4 (r = .89). Additional strong associations were between the PC measure and the CELF-4 (r = .88), GJYN measure and the SR measure (r = .84), the GJWH measure and the SR measure (r = .84), and the GJWH measure and GJYN measure (r = .81). Moderate associations were between the GJWH measure and the CELF-4 (r = .76) and the GJWH measure and the CELF-4 (r = .76).
Intercorrelations between the index measures and the comprehensive language measure for bilingual Spanish-English speaking participants
Note: CELF-4 = Clinical Evaluation of Language Fundamentals-4 Core Language Scale; NWR = Nonword Repetition Task; SR = Sentence Repetition Task; = Grammaggio WH-Questions; GJYN = Grammaggio Task 2; PC = Passage Comprehension.*Coefficient is significant at p < .01; **Coefficient is significant at p < .001.
To examine whether any of the index measures (NWR, SR, GJWY, GJYN, and PC) predict English language proficiency, for each participant group, a stepwise regression analysis was conducted with CELF-4 Core Language Scale Score standard score as the dependent variable and each of the index measures entered as predictor variables. For the monolingual English-speaking group, the optimal regression model is presented in Table 4. The final model included one variable; SR was a positive predictor of English language proficiency. NWR, GJWH, GJYN, and PC were eliminated from the model. The model was significant, F(1, 24) = 73.504, p < .001, and accounted for 76.2% of the variance in English language proficiency scores. Variables excluded from the stepwise regression model included NWR (p = .18), GJWH (p = .81), GJYN (p = .63), and PC (p = .59). Single predictor regression models for each index measure are presented in Supplementary Appendix E.
Regression model predicting English language proficiency for the monolingual English-speaking group
Note: SR = sentence repetition task.
The optimal regression model for the Spanish-English bilingual group is presented in Table 5. Following stepwise regression analysis, the final model included two variables; SR and PC were positive predictors of English language proficiency. NWR, GJWH, and GJYN were eliminated from the model. The model was significant, F(2, 13) = 41.42, p < .001, and accounted for 88.3% of the variance in English language proficiency scores. Variables excluded from the stepwise regression model included NWR (p = .18), GJWH (p = .68), and GJYN (p = .81). Single predictor regression models for each index measure are presented in Supplementary Appendix F.
Regression model predicting English language proficiency for the English-Spanish bilingual group
Note: SR = sentence repetition task, PC = passage comprehension.
4. Discussion
In a linguistically diverse sample of older, school-age children, we evaluated how well four language task – NWR, SR, two GJ tasks, and a PC task, predicted English language proficiency in older monolingual and bilingual children. We aimed to determine which tasks best predicted the English language proficiency of our sample. Our objectives were to (1) evaluate each task’s ability to predict English language proficiency in older monolingual English-speaking children and (2) evaluate each task’s ability to predict English language proficiency in older bilingual Spanish-English speaking children.
Counter to predictions, the only significant predictor of English proficiency for our monolingual group was SR, a measure sensitive to language ability in younger monolingual English and multilingual populations. None of the other tasks (NWR, GJ, and PC) were significant predictors in the optimal model.
The literature on whether NWR predicts language proficiency with both monolingual and bilingual children is mixed. We identified three possible explanations for our finding that NWR did not predict language proficiency in our monolingual group. First, successful completion of NWR and SR tasks likely requires many of the same mental processes that support general language ability, including engagement of phonological working memory and accessing long-term linguistic knowledge (Pawłowska, Reference Pawłowska2014). Because the single predictor regression model for NWR reached statistical significance, it is possible that the NWR task contributed to the prediction of general language ability but did not explain unique variance beyond the SR-only model. Second, unlike the majority of NWR studies predicting language proficiency in monolingual English speakers, our participants were older, school-age children (Hodges et al., Reference Hodges, Munro, Baker, McGregor and Heard2017; Ortiz, Reference Ortiz2021). It is possible that, during the later school-age years, NWR becomes less sensitive to language proficiency in monolingual English speakers. Finally, our study focused on the measures’ prediction of general language proficiency, whereas the majority of NWR studies are focused on vocabulary and/or word learning. Indeed, our reported intercorrelations between NWR and measures of GJ (GJWH and GJYN) were among the weakest correlations obtained in our sample. NWR may predict performance in the areas of semantics and phonological development but not general language level.
The GJWH and GJYN tasks did not predict language proficiency in our monolingual group. Several notable studies of GJ continued to find performance differences into the later school-age years across children with and without language impairment (Rice et al., Reference Rice, Wexler and Redmond1999, Reference Rice, Hoffman and Wexler2009). One possible explanation could be based on the greater variability in the global language proficiency of our participant sample. For example, Rice et al. (Reference Rice, Hoffman and Wexler2009) included participants with CELF-3 scores (i.e., M = 108, SD = 7) that were less variable than the CELF-4 scores obtained for our monolingual English-speaking participants (i.e., M = 101, SD = 21). Additionally, participants in Rice et al. (Reference Rice, Hoffman and Wexler2009) obtained GJ A’ scores (GJWH = 93%, GJYN = 95%) that were higher than those of our monolingual English-speaking participants (GJWH = 88%, GJYN = 89%). While modest in size, differences in participants’ global language proficiency and GJ scores may have played contributing roles in the reduced sensitivity of GJ in predicting our monolingual English-speaking children’s English language proficiency.
Contrary to our hypothesis, language proficiency was not predicted by PC task performance in the monolingual group. While we predicted that, due to additional years of exposure to academic language in the schools, children with lower language proficiency would differ from children with greater language proficiency only the PC task, which requires higher-level language skills. It is possible that, by later elementary school years, monolingual school-age children have received adequate exposure to tasks requiring non-literal language and inferencing (Adams et al., Reference Adams, Clarke and Haynes2009). It is possible that, having acquired the foundational semantic, syntactic, and morphological skill to complete PC tasks, the monolingual group also brought adequate exposure to inferencing tasks such that PC tasks could no longer differentiate higher and lower language proficiency. Such a finding suggests the need to identify a task that taps the memory processes that support language learning generally (as we propose in our discussion of the SR task below).
Turning to the Spanish-English bilingual group, unlike the monolingual group, both the SR task and PC task predicted a significant amount of variance in English language proficiency. Exclusion of the NWR measure from the optimal regression model aligns with previous studies of older bilingual Spanish-English school-age children (Gutiérrez-Clellen & Simon-Cereijido, Reference Gutiérrez–Clellen and Simon–Cereijido2010; Windsor et al., Reference Windsor, Kohnert, Lobitz and Pham2010). As with the model predicting language ability in the monolingual group, three possibles may explain the exclusion of NWR from the optimal regression model. Like the monolingual group, it may be that the NWR task simply does not explain unique variance beyond the SR-only model. Second, it may be possible that NWR is less predictive of general English language proficiency than vocabulary or phonological development in older bilingual children. Finally, for younger bilingual Spanish-English children, differing degrees of English exposure may make it easier or harder to draw analogies with the phonological forms of real words in English (Gutiérrez-Clellen & Simon-Cereijido, Reference Gutiérrez–Clellen and Simon–Cereijido2010), whereas older school-age children have received adequate exposure to language domains outside of vocabulary knowledge such that NWR is no longer predictive of general English language proficiency (Ortiz, Reference Ortiz2021).
As with the monolingual group, the GJ measures, GJWH and GJYN, were not strong predictors of language proficiency for the bilingual Spanish-English group. This finding differs from previous studies of younger bilingual Spanish-English children that found that GJ tasks predict language proficiency in English and Spanish (Davidson et al., Reference Davidson, Vanegas, Hilvert, Rainey and Misiunaite2019; Simard et al., Reference Simard, Labelle and Bergeron2017). We propose two possible explanations for this departure from earlier findings. It is possible that, for older Spanish-English bilingual children, GJ is likely frequently a part of daily life when code switching or language brokering for family members (Adesope et al., Reference Adesope, Lavin, Thompson and Ungerleider2010). Such activities may heighten bilingual children’s awareness of the syntactic and morphological differences between their languages, potentially improving their ability to detect grammatically correct and incorrect syntactic structures on GJ tasks. Similarly, the syntactic and morphological overlap between languages may increase bilingual children’s awareness of the well-formedness of utterances in both Spanish and English (Paradis, Reference Paradis2010). For these reasons, it is possible that older Spanish-English bilingual children with various levels of English language proficiency have ample exposure to GJ, limiting the task’s utility as a predictor of English language proficiency. A second possible explanation could be based on a potential bilingual advantage in meta-linguistic awareness skills that act as a protective factor for children with lower language proficiency when completing GJ tasks. Older children may reach a threshold of bilingual exposure that imparts meta-linguistic awareness skill which in turn supported overall performance on GJ tasks (Laurent & Martinot, Reference Laurent and Martinot2010; Reder et al., Reference Reder, Marec‐Breton, Gombert and Demont2013).
In line with our initial hypothesis, PC may predict English language proficiency in bilingual Spanish-English speakers because it taps semantic, syntactic, and morphological skills that ultimately support superior performance on higher-order language tasks. However, another possible explanation is that, in daily life, tasks associated with PCs may be encountered less frequently by older Spanish-English bilingual children than tasks associated with other tasks. Prevoo et al. (2014) state that the strength of cross-language transfer depends on the type of language proficiency task presented to the child. If cross-language transfer is a crucial component of language-general knowledge for older Spanish-English bilingual children, it is possible that inferencing, required for success on the PC task, occurs less frequently in children’s daily English-speaking environments than language skills tapped by our other measures.
The finding that SR predicted English proficiency in both groups is important for the ultimate goal of developing a screening measure for identifying subclinical language weakness in older monolingual and bilingual children. One possible explanation for SR’s strength as a predictor is based not only on SR’s direct engagement of the attention and memory mechanisms that support language acquisition but also SR’s ability to tap the linkages between those mechanisms and children’s language knowledge in LTM (Baddeley et al., Reference Baddeley, Hitch and Allen2009). Unlike NWR, SR requires the formation of “chunks” from the spoken message that link to language knowledge in LTM (Jefferies et al., Reference Jefferies, Ralph and Baddeley2004). Baddeley et al.’ (Reference Baddeley, Hitch and Allen2009) account of SR claims that performance on SR partially depends on two underlying mechanisms: (1) a phonological loop, or a short-term memory mechanism that stores quickly decaying verbal information and (2) an episodic buffer that “chunks” received information by linking overheard words to semantic and syntactic information stored in LTM (Jefferies et al., Reference Jefferies, Ralph and Baddeley2004). Notably, the episodic buffer requires the involvement of linguistic knowledge in LTM to chunk verbal information. Unlike the short word-like stimuli used in NWR tasks, SR stimuli may simultaneously (a) tax older children’s STM via engagement of the phonological loop and episodic buffer and (b) tap children’s syntactic and semantic linguistic knowledge in LTM. It is plausible that SR’s ability to tap both STM and LTM resulted in its strong prediction of English language proficiency.
Given our results, a major challenge in the development of a single screening protocol for older, linguistically diverse children is the unique contribution that PC makes in predicting language proficiency in older bilingual Spanish-English-speaking children. Our SR + PC model provides marginal but meaningful improvement on the amount of variance explained (88.3) when compared with our model containing only SR (79.7). Conversely, PC does not increase the amount of variance explained for older monolingual English-speaking children. A possible solution to this challenge is the development of a long-form screening tool for older bilingual Spanish-English-speaking children that includes both SR and PC tasks and a short-form screening tool for older monolingual English-speaking children (Gillam et al., Reference Gillam, Peña, Bedore, Bohman and Mendez-Perez2013; Peña et al., Reference Peña, Gillam, Bedore and Bohman2011).
We argue that the relative parsimony of the models across language groups also provides a major opportunity for the development of a future single screening tool. As described above, the SR task was brief and easy to administer to our older school-age participants. Compared to our other index measures, SR appears to be especially sensitive to the gradient differences in language proficiency into the later school-age years. Because SR tasks require the repetition of a variety of syntactic structures, potential SR screening measures could be further developed into tools for progress monitoring as children develop academic language proficiency (Yan et al., Reference Yan, Maeda, Lv and Ginther2016). Beyond the identification of general language proficiency, future SR tasks could provide suggestions regarding the types of syntactic structures that would best support the language enrichment of children with subclinical language weaknesses (Morton & Schuele, Reference Morton and Schuele2021; Rioux & Thordardottir, Reference Rioux and Thordardottir2023). Future studies may explore the clinical utility of SR as both a screening measure for subclinical language weakness as well as a tool for the identification of specific syntactic structures and grammatical forms not yet mastered by children with language weakness.
4.1. Limitations
Our study has limitations that require caution when interpreting results. Our results are based upon a relatively small number of participants, all of whom are based in the United States. Additionally, although we have attempted to recruit a diverse group of bilingual Spanish-English-speaking participants, we acknowledge that not all bilingual children in the U.S. are first exposed to English in first grade or earlier. Our decision to require significant English exposure necessarily limited the representativeness of our bilingual Spanish-English-speaking sample. These factors affect the generalisability of our results.
Our results are also limited to the comprehensive language measure that we selected (CELF-4; Semel et al., Reference Semel, Wiig and Secord2003). Even though this comprehensive language measure was consistent with previous language proficiency studies, we did not include a larger range of empirical and/or clinical instrumentals to index English language proficiency. Furthermore, it is possible that CELF-4 performance that is below age-expectations but in the subclinical level may not result in language deficits that impede success in school and beyond (McGregor et al., Reference McGregor, Ohlmann, Eden, Arbisi-Kelm and Young2023). Future studies evaluating the use of SR and other index measures may benefit from including additional measures of English language proficiency (Redmond et al., Reference Redmond, Ash, Christopulos and Pfaff2019).
Finally, we acknowledge that all research tasks were conducted remotely via the Zoom® platform, potentially impacting the generalisability of our results. Although empirical evidence suggests comparable performance on remote CELF-4 administration and in-person CELF-4 administration (Campbell et al., Reference Campbell, Lawrence and Goldstein2024), we acknowledge that future in-person studies should replicate our findings to ensure alignment between in-person and remote performance on our index and comprehensive language measures.
5. Conclusion
In this study, we evaluated measures that tap children’s language knowledge and mechanisms that support language learning within a linguistically diverse sample. Our findings identified that SR, but not NWR, may serve as a noncontrastive measure for English language proficiency within older, linguistically diverse student populations. Neither GJ task was a strong language proficiency predictor for either of our language groups. However, we found that a combination of a paragraph comprehension and SR tasks better predicted English language proficiency in older Spanish-English bilingual children than the SR task alone. These findings serve as an important first step towards the development of a screening measure for subclinical language weakness in older, linguistically diverse children.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0305000926100592.
Acknowledgements
Our study was supported by a NIH/NIDCD R21 grant DC018865. We thank the children and families who participated in this study.
Statement of ethical approval
Our study was approved by the San Diego State University Institutional Review Board (approval number HS-2021-0167). The types of consent obtained included written parental consent and verbal and written child assent.
Open access
