Measuring language comprehension in young children

Word comprehension as well as word production are usually measured using parental vocabulary checklists such as the MacArthur-Bates Communicative Development Inventory (MB-CDI, Fenson, Dale, Reznick, Bates, Thal, Pethick, Tomasello, Mervis & Stiles, Reference Fenson, Dale, Reznick, Bates, Thal, Pethick and Stiles1994) in early childhood. While parental reports are relatively easy to administer and are widely used to capture individual variation (e.g., Frank, Braginsky, Yurovsky & Marchman, Reference Frank, Braginsky, Yurovsky and Marchman2017), they are not a direct measure of word comprehension, opening the possibility to differences in parents’ interpretations of what it means for the child to understand a word and, more generally, for reporting biases. Additionally, unlike direct tests, checklists cannot reflect children’s speed of word processing. To date, there are two types of direct experimental procedures to measure word comprehension that have been used to study SES, lingual status, and age effects on lexical development. We introduce each in turn.

The first involves visual paradigms mainly represented by the looking-while-listening (LwL) procedure (Fernald, Pinto, Swingley, Weinberg & McRoberts, Reference Fernald, Pinto, Swingley, Weinberg and McRoberts1998), an audiovisual two-alternative forced choice task, where the child is expected to look at the image corresponding to the auditory speech prompt. In this work, accuracy is calculated as the proportion of time children look at the target picture compared to the time they gaze at the distractor: longer looking to the target than to the distractor indicates better performance. In its typical implementation, this paradigm also includes a measure of speed of online word processing: in trials when the child happens to be looking at the distractor when hearing the prompt, we can calculate the time the child needs to shift their gaze from the distractor to the target. When thus defined, accuracy and response time cannot be mathematically decorrelated. This is because longer response times necessarily entail less looking time to the target and, as a result, longer response times also imply lower accuracy. In order to assess the relation between speed of processing and some measure of accuracy or vocabulary size, researchers turn to additional measures of lexical knowledge, such as the MB-CDI.

The second type of direct experimental procedures to measure word comprehension involves tactile choice paradigms represented by procedures like the Computerized Comprehension Task (CCT, e.g., Friend & Keplinger, Reference Friend and Keplinger2003), the NIH Picture Vocabulary Test (NPVT, e.g., Koenig, Arunachalam & Saudino, Reference Koenig, Arunachalam and Saudino2020), and the Quick Interactive Language Screener (QUILS, e.g., Levine, Pace, Luo, Hirsh-Pasek, Golinkoff, de Villiers, Iglesias & Wilson Reference Levine, Pace, Luo, Hirsh-Pasek, Golinkoff, de Villiers and Wilson2020) among others. The CCT, for example, implements an intermodal two-alternative forced choice onto a touch screen, where the child is expected to touch the image corresponding to the auditory speech prompt (Friend & Keplinger, Reference Friend and Keplinger2003, Reference Friend and Keplinger2008). In this context, an advantage of the tactile paradigm is that any correlation between accuracy and response times is more easily attributed to true covariation in the child population, rather than a mathematical by-product.

We briefly review the literature using these paradigms (visual and tactile choice), bearing on individual differences as a function of SES, lingual status, and age.

Socioeconomic status and direct word comprehension measures

A large literature documents that socio-economic status (SES) is a strong predictor of language, including vocabulary outcomes, among monolingual English-learning American children. In a seminal study, Hart and Risley (Reference Hart and Risley1995) mapped the growing divergence in vocabulary size among monolingual children whose parents varied in socio-economic status. Previous work using the LwL visual paradigm has found clear differences in accuracy and response times as a function of SES among 18- to 24-month-old English learners (e.g., Fernald, Marchman & Weisleder, Reference Fernald, Marchman and Weisleder2013), such that infants from higher SES backgrounds have faster response times and higher accuracies than lower SES peers (see also Hurtado, Marchman & Fernald, Reference Hurtado, Marchman and Fernald2008).

In terms of response times, no study using a tablet-based measure found faster responses for children whose mothers are more educated. The literature is too large to summarize here, but further information can be found in the Supplementary Materials (Supplementary Materials, SM1). For example, in a large study with over 100 3-year-old Argentinean participants, Rosemberg and Alam (Reference Rosemberg and Alam2021) found similar response times among children tested in daycares located in what they describe as marginalized urban slums and children tested in daycares located in residential neighborhoods. For accuracy, tactile choice paradigms show mixed results. Some studies found significant associations between monolingual children’s accuracy and maternal education (Friend, Schmitt & Simpson, Reference Friend, Schmitt and Simpson2012; Rosemberg & Alam, Reference Rosemberg and Alam2021), but others did not find differences, notably for a sample of children learning Spanish (De Anda, Arias-Trejo, Poulin-Dubois, Zesiger & Friend, Reference De Anda, Arias-Trejo, Poulin-Dubois, Zesiger and Friend2016; Friend, De Anda, Arias-Trejo, Poulin-Dubois & Zesiger, Reference Friend, De Anda, Arias-Trejo, Poulin-Dubois and Zesiger2017) and a sample of Swiss French learners (Patrucco-Nanchen, Friend, Poulin-Dubois & Zesiger, Reference Patrucco-Nanchen, Friend, Poulin-Dubois and Zesiger2019; although note that the latter sample was very homogeneous). Mixed findings have also been reported on the correlation between SES and accuracy for non-monolingual learners (De Anda, Hendrickson, Zesiger, Poulin-Dubois & Friend, Reference De Anda, Hendrickson, Zesiger, Poulin-Dubois and Friend2018; Friend, Smolak, Liu, Poulin-Dubois & Zesiger, Reference Friend, Smolak, Liu, Poulin-Dubois and Zesiger2018; Legacy, Zesiger, Friend & Poulin-Dubois, Reference Legacy, Zesiger, Friend and Poulin-Dubois2018; Patrucco-Nanchen et al., Reference Patrucco-Nanchen, Friend, Poulin-Dubois and Zesiger2019).

Lingual status and direct word comprehension measures

A second important factor that affects vocabulary development is hearing multiple languages. In controlled studies focusing on bilinguals, vocabulary size and speed of word comprehension in a given language correlate with exposure to that language (Hurtado, Gruter, Marchman & Fernald, Reference Hurtado, Gruter, Marchman and Fernald2014; Place & Hoff, Reference Place and Hoff2011). Most often, studies on children exposed to multiple languages measure exposure in relative terms, asking parents to estimate the proportion of time the child hears language A versus language B, rather than attempting to measure absolute quantities of exposure (but see Marchman, Martínez, Hurtado, Grüter & Fernald, Reference Marchman, Martínez, Hurtado, Grüter and Fernald2017; Ramírez-Esparza, García-Sierra & Kuhl, Reference Ramírez-Esparza, García-Sierra and Kuhl2017).

In this study, we discuss children’s performance only in the dominant language distinguishing between three categories – monolinguals, children having minimal exposure to other languages, and children who have more significant exposure to other languages (see Methods for a detailed description of these three lingual groups). To avoid confusion between exposure to a given language and the general concept of language experiences, we call this grouping factor “lingual status”. The choice for these categories is justified by our research approach, detailed in the next subsection.Footnote ¹

In visual paradigms, individual variation in the proportion of time exposed to one or another language, as well as the sheer quantity of that language heard, predicts both accuracy and response times among children varying in lingual status (e.g., Hurtado et al., Reference Hurtado, Gruter, Marchman and Fernald2014; Marchman et al., Reference Marchman, Martínez, Hurtado, Grüter and Fernald2017).

If we now turn to tactile choice paradigms, several studies find a significant effect on accuracy, with better performance for children with greater exposure to the tested language (De Anda et al., Reference De Anda, Arias-Trejo, Poulin-Dubois, Zesiger and Friend2016, Reference De Anda, Hendrickson, Zesiger, Poulin-Dubois and Friend2018; Legacy, Zesiger, Friend & Poulin-Dubois, Reference Legacy, Zesiger, Friend and Poulin-Dubois2016; Legacy et al., Reference Legacy, Zesiger, Friend and Poulin-Dubois2018), but no study reported a significant effect on response times.

Child age and direct word comprehension measures

A third important factor explaining significant variation among young children is age. Child age is correlated with a host of factors internal to the child. All else being equal, older children perform better than younger children in a variety of tasks ranging from those measuring visual attention (Richards & Casey, Reference Richards, Casey and Erlbaum1992) to executive functions (Zelazo, Müller, Frye, Marcovitch, Argitis, Boseovski, Chiang, Hongwanishkul, Schuster, Sutherland & Carlson, Reference Zelazo, Müller, Frye, Marcovitch, Argitis, Boseovski and Carlson2003). The claim that children’s vocabulary, processing skills, and overall knowledge improves with age is uncontroversial, with important effects particularly in the case of lexical development (e.g., Fenson et al., Reference Fenson, Dale, Reznick, Bates, Thal, Pethick and Stiles1994). Turning now to the experimental evidence on visual and tactile choice paradigms, both setups show that older children score higher in accuracy (e.g., Friend et al., Reference Friend, De Anda, Arias-Trejo, Poulin-Dubois and Zesiger2017; Rosemberg & Alam, Reference Rosemberg and Alam2021; Weisleder & Fernald, Reference Weisleder and Fernald2013) and show shorter response times (Friend et al., Reference Friend, De Anda, Arias-Trejo, Poulin-Dubois and Zesiger2017; Rosemberg & Alam, Reference Rosemberg and Alam2021; Weisleder & Fernald, Reference Weisleder and Fernald2013).

Taken together, these results hint that the relation between SES and direct language comprehension is not settled. Age seems to be the only consistent factor across paradigms. When studying SES or language exposure, it seems that direct comprehension scores are sensitive to the type of task used, and/or it varies across human populations (see also Fernald, Kariger, Hidrobo & Gertler, Reference Fernald, Kariger, Hidrobo and Gertler2012, and discussion in Frank, Braginsky, Yurovsky & Marchman, Reference Frank, Braginsky, Yurovsky and Marchman2021, section 6.3).

The relation between response times (processing speed) and accuracy (cumulative knowledge)

There is substantial evidence in the visual paradigms that lexical speed of processing (indexed as reaction times) is correlated with vocabulary size (or cumulative knowledge, often indexed as accuracy) in early childhood (Fernald, Perfors & Marchman, Reference Fernald, Perfors and Marchman2006; Hurtado et al., Reference Hurtado, Marchman and Fernald2008; Weisleder & Fernald, Reference Weisleder and Fernald2013). The case can be made that greater accuracy and lower response times go conceptually hand in hand. For example, in two separate LwL studies exploring SES differences, it has been found that the effect of language experiences on children’s vocabulary size (measured with the CDI, and thus not subject to our argument regarding mathematical correlations between accuracy and response times) was rendered non-significant when speed of processing was considered (Hurtado et al., Reference Hurtado, Marchman and Fernald2008; Weisleder & Fernald, Reference Weisleder and Fernald2013). This result is consistent with the hypothesis that language experiences impact speed of processing: children exposed to more language are faster at language processing, which allows them to learn more words. In Hurtado et al. (Reference Hurtado, Marchman and Fernald2008), the opposite analysis was also carried out: the correlation between language experiences and speed of processing was mediated by vocabulary size. In this case, the results are consistent with the hypothesis that exposure to more language input enables infants to learn more words, and this greater vocabulary prompts faster processing of the speech signal. Similar interpretations ensue from analyses considering language experience in bilinguals, and associated response times as well as later lexicon in Marchman et al. (Reference Marchman, Martínez, Hurtado, Grüter and Fernald2017). Despite the fact that the causal relationship between lexical processing and vocabulary size has not fully been determined, together, these results suggest that the effects of SES and lingual status on processing speed (measured via response times) and cumulative knowledge (which within a specific task can be indexed by accuracy) are largely overlapping.

To our knowledge, there are no mediation analyses (trying to explain away response times differences across SES or lingual status groups with accuracy differences, or vice versa) like those described for the visual choice literature in the tactile choice setups. In a recent longitudinal study, Smolak, Hendrickson, Zesiger, Poulin-Dubois, and Friend (Reference Smolak, Hendrickson, Zesiger, Poulin-Dubois and Friend2021) report on 41 monolingual American English learners whose CCT data was collected at 2 years of age. The authors extracted accuracy as well as two response time measures from the CCT: a visual response time (time to look at the target) and a haptic response time (time to touch the target). The authors find distinct associations between accuracy and the two speed of processing measures: accuracy and visual response times were significantly correlated, whereas accuracy and haptic response times were not. Additionally, individual variation in later PPVT scores was better predicted by accuracy than visual response times, and haptic response times were uncorrelated to children’s later PPVT scores. These interesting results show that the modality of the measure for speed of processing (visual or tactile choice) is an important factor to consider for studying individual variation in direct language comprehension paradigms. It can be further extended that tactile choice assessments (requiring explicit decisions from children) may lead to emerging speed-accuracy trade-offs. Under certain conditions participants slow down their reply in order to answer more accurately or have a more erratic performance when responding quickly (e.g., Schneider & Frank, Reference Schneider and Frank2016). Furthermore, insofar as tactile choice reaction times are dependent on motor development and/or executive function, this adds complexity to the interpretation of the speed of processing measure. Visual paradigms (requiring less of the child, merely eye movement) might be more robust to potential individual differences, so that response times can reveal finer-grained and more specific variation in speed of lexical processing. This can be particularly relevant in the context of studying potential differences as a function of age and SES, which are both related to executive function (a meta-analysis of SES and children’s executive function in Lawson, Hook & Farah, Reference Lawson, Hook and Farah2018). That said, in typical visual preference paradigms, accuracy and response times are defined in such a way that they are mathematically correlated, leading researchers to use indirect or additional vocabulary assessments instead. In this context, an advantage of the tactile choice paradigms is to provide in a single assessment both accuracy and response time, making it easier to attribute true covariation in the child population. In sum, processing speed differences as a function of modality and their specific relation with language comprehension scores throughout early childhood are not yet fully understood. Our study aims to contribute to this body of research.

The present study

In this report, we measured lexical development using a two-alternative forced choice on a touch screen, similar to the CCT. We opted for a portable iPad (R) touch screen, rather than the original 17-inch CCT touch screen (Friend & Keplinger, Reference Friend and Keplinger2003), for two main reasons. First, a portable tablet allowed us to go to children’s everyday environment, lowering the bar for participation and reducing unfamiliarity effects. Second, this tablet is cheaper than the large touch screens, facilitating wider replication in the future. Our test was nonetheless strongly inspired by CCT research: we drew from their task the general structure of the test, the number of lexical items, and their distribution into word categories and difficulty levels, as well as the use of a touch screen to collect responses (divergences from their procedure are detailed in the Methods section). By collecting measures of both cumulative knowledge (operationalized as the proportion of trials in which the correct picture is touched) and processing speed (measured as the average response time in accurate trials), we introduce mediation analyses to the tactile choice literature. In addition, we contribute data collected in a country that had not been studied in either the visual or tactile choice literature, France. French socioeconomic policy and cultural caregiving practices may lead to a smaller relationship between parental education and childhood outcomes, especially when comparing it to studies from the United States (De Anda et al., Reference De Anda, Hendrickson, Zesiger, Poulin-Dubois and Friend2018; Friend et al., Reference Friend, Schmitt and Simpson2012) and Argentina (Rosemberg & Alam, Reference Rosemberg and Alam2021), where higher levels of childhood poverty (Thévenon, Manfredi, Govind & Klauzner, Reference Thévenon, Manfredi, Govind and Klauzner2018) and lower state support of disadvantaged families may lead to greater differences as a function of maternal education (see Supplementary Materials, section SM2 for further discussion).

To our knowledge, only one research group has reported on the effects associated with SES in early lexical development in France. A cohort has been followed up from before birth to late childhood (EDEN: Etude de cohorte generaliste menée en France sur les Déterminants pre- et post natals précoces du développement psychomoteur et de la santé de l’ENfant; Heude et al., Reference Heude, Forhan, Slama, Douhaud, Bedel and Saurel-Cubizolles2016), and early lexical acquisition results have been measured using the French version of the MB-CDI at 2 years of age (Peyre, Galera, Van Der Waerden, Hoertel, Bernard, Melchior & Ramus, Reference Peyre, Galera, Van Der Waerden, Hoertel, Bernard, Melchior and Ramus2016). Their results found a small but significant effect of maternal education. However, this result doesn’t integrate language experiences nor a direct measure of language comprehension, thus does not reflect speed of processing.

Research questions and general predictions

In summary, we collected direct measurements of children’s lexical development using a two-alternative forced choice task implemented on a portable iPad (R) touch screen. This task yields our two key online measures of interest, accuracy and response times. Most parents completed a background questionnaire, allowing us to derive a proxy for SES (maternal education) and children’s lingual status (as a function of their exposure to French).

From the research summarized above, we predicted the following patterns. First, children from higher SES have higher accuracy than children from lower SES (observed in both visual and tactile choice paradigms) and may have shorter response times (as found in visual tasks, with little tactile choice data reported on it). We predict also that the effects of maternal education on children’s accuracy are smaller in our French sample in comparison to previous work on American and Argentinean samples. Second, children with greater exposure to French have greater accuracy and may have faster response times than children with less exposure to French (with both measures showing this effect in visual tasks, and mixed findings from the tactile choice literature). Third, older children have higher accuracy and shorter response times than younger children. Finally, we made the prediction that speed of processing mediates SES, lingual status, and age effects on accuracy, and vice versa.

Participants

This study was approved by an ethics committee (CERES IRB 20132600001072). We collected data in three daycares located close to the south of Paris intra muros, a cosmopolitan neighborhood characterized by a relatively diverse ethnic composition (although it should be noted ethnicity is not an acceptable construct in French law). The specific area targeted contained several “social housing” buildings, which are allocated by the State to families of highly diverse economic means, with rent adapted to the family’s income. We hoped that, by testing in daycares located in this neighborhood, we would be able to sample from a population that was varied in parental education and language background.

A total of 134 families signed a consent form allowing their child to participate, but ultimately only 91 children had exploitable data (for full inclusion/exclusion details see Supplementary Materials; Section SM3.1). To begin with, 6 children were excluded because they were part of the pilot and 2 were absent on all visits. The majority of excluded children were excluded because they failed to return or did not fully complete the questionnaire (18) or because they refused to play (6). An additional 6 were excluded because the protocol was not followed (1 was tested by parents, 1 by a daycare staff member, and 1 was influenced by a daycare staff member; 2 failed to understand the task and completed only 2 or 4 trials, and only one failed the warm-up game). A total of 7 children were excluded because parents answered in the questionnaire that their children had a diagnosis that would affect their language (5) and a further 2 families did not answer this key question. Finally, 3 families were excluded because they did not provide information on maternal education plus 1 family who responded “0” years of education (which seems highly unlikely and is an outlier for the distribution of years of education).To provide an idea of attrition, we summed the number of children who (a) refused to play (6) and (b) did not produce enough data to be included in analyses (3), which is 7.8% of the children having tablet data.

The final sample is composed by 91 children (38 girls), averaging 2.59 years of age (range 1.49-3.36). There were 42 children in the monolingual group (over 95% exposure to French), 34 in the minimal exposure group (70-95% exposure to French), and 15 in the moderate exposure group (less than 70% exposure to French; see Questionnaire section below for a detailed description of these three language groups). Children exposed to other languages heard a wide variety of them (for further information see section 3 SM3.2). Regarding maternal education, it varied from 9 (meaning that high school was not completed) to 21 (Ph.D. level), with an average of 15.40 (corresponding to completing some post-secondary education). More specifically: 4% had not completed high school; 10% completed high school; 33% had done some college or pursued professional training; 13% had completed a university degree; 38% had a masters-level degree; the remaining 1% had a Ph.D. or similar. In 2008, 10% of the French population had not completed high school; and a further 8% completed high school without studying further; 32% had done some college (15%) or pursued professional training (17%); 23% had completed a university degree; 15% had a masters-level degree; and the remaining 12% had a Ph.D. or similar (Kabla-Langlois, Reference Kabla-Langlois2010). Thus, our sample covered the whole range, and although the precise distribution did not match perfectly that found in the general French population in 2008, it is fairly comparable and not particularly skewed towards higher SES families.

Stimuli design and implementation

The vocabulary task was an adaptation of the laboratory-based Computerized Comprehension Test, which has been previously used with English-, Spanish-, and French-learning toddlers (Friend & Keplinger, Reference Friend and Keplinger2008). Stimuli selection was carried out considering a similar test that was being developed by colleagues in Argentina (Rosemberg & Alam, Reference Rosemberg and Alam2021). To have more varied items, Rosemberg and Alam (Reference Rosemberg and Alam2021) drew a third of the items from the English CCT, a third from the Peabody Picture Vocabulary Test, and the final third from corpora. As in the CCT, there were three lexical categories, with more nouns than verbs and adjectives (44 items were nouns, 22 verbs, and 16 adjectives). We adapted these items to French, checking that they varied in terms of frequency in a French corpus.Footnote ² Items were arranged into 41 pairs (within class and gender), designed so that images were matched in salience, shape, color, and animacy, and the lexical items on word class and gender. The list of stimuli is available in Supplementary Materials (Supplementary Materials, SM4).

All nouns were embedded in the phrase “Touch the X” Touche le(s)/la X (using the appropriate article for each noun). All verbs were embedded in the phrase “Touch the one who X” Touche celui qui X. Finally, adjectives appeared post-nominally, making sure that the noun used did not give cues to the picture identity; for example, in the pair “tired-sad” fatigué-triste, the phrases were Touche l’homme fatigué versus Touche l’homme triste and the pictures showed men with appropriate facial expressions. Reaction time was calculated from the onset of the sentence.

Procedure and equipment

Data collection occurred over two academic years (2015-2016; and 2016-2017). Each year, we carried out 3-4 visits to each daycare, and each time 2-4 members of our team were present. Typically, two of us would administer the task and the rest would walk the children in and out, take notes, and liaise with the daycare staff. Children were tested one or two at a time in a quiet room, usually the library, while sitting on age-appropriate chairs or cushions on the floor.

After welcoming the child, the experimenter interacting with the child would show him/her a warm-up game (Bubble blasting for babies by Ali Tanriverdi, available from https://appsto.re/fr/9YpO_.i in 2017). This game was played until the experimenter felt the child was comfortable with her and the equipment, usually about 1-2 minutes. Only then would the experimenter propose the main game. At this point, both the child and the experimenter would put on headphones, with the same sound being delivered via a splitter. Most children agreed to this; in a few exceptions of refusal, the experimenter would hold the headphones next to the child’s ear so that she/he could hear the sounds.

Our touch-screen game starts with three training trials followed by 41 test trials, all with the same structure. A video exemplifying the procedure is available at our OSF repository. First, both images are presented to the child. Once the child is familiarized with both visual stimuli, a semi-transparent gray layer covers both images and an animated child-friendly character appears on the screen, which is used as an attention-getter. Then, the child (or the experimenter) must tap on the character to be able to hear the prompt phrase that includes the target word. This auditory prompt is delivered by the touch screen over the headphones (and not spoken by the experimenter), while the animated character’s mouth moves simultaneously to the sound, creating the illusion of directly speaking to the child. At the end of the prompt phrase, the gray layer automatically disappears and the child is able to produce a response. Note that all touches prior to this moment in the trial are ignored by the application, discouraging random touching that does not take into account the command. At this point then, a touch to one of the stimuli, regardless of whether it is the target or the distractor, causes the untouched image to disappear and highlights the touched image by adding a yellow border to the outside of the image, accompanied by a "plop" sound. Additionally, if the tap was on the target, the character jumps up and down happily, providing positive feedback, else the trial finishes (with no feedback). If no touch is produced 4 seconds after the prompt phrase, the animated character whistles and acts impatient to get the child’s attention back to the task. This was also used by the experimenter as a cue to repeat the prompt phrase out loud. Afterward, the child had 4 more seconds to produce a response. If the child still did not touch any of the stimuli, the experimenter said “the X was here” and touched the target image. Although we did not keep a record, this happened very rarely. This procedure was followed for all trials (familiarization and test). Since response times are over 7 seconds long in these cases, all of these trials are excluded from analyses. Both criteria were inspired from the CCT task in which the experimenter repeats the prompt phrase after 3.5 seconds without response and, after 7 seconds from trial onset without response, the screen goes blue and the trial is considered “not attempted”.

As in the CCT, the three initial training trials used highly familiar words, and the 41 test pairs included different word types and difficulties. In our task, there are two main differences between the training and test trials. First, the experimenter provided a great deal of feedback during training to make sure the child felt comfortable and understood the task. In contrast during test trials, the experimenter merely provided positive feedback infrequently at random points. Second, during training (but not during the test) if the child touched the distractor, the trial was immediately repeated. To proceed to the test phase, the participant had to perform all training trials correctly.

The 41 test image pairs were shown in a fixed pseudo-randomized order. The order was designed with the following constraints: no more than 3 trials in a row were moderate or hard; no more than two times in a row the correct response was on a given side. We then created 2 pseudo-random conditions in which each participant was asked to touch only one word of the pair (counterbalanced across conditions). This implied that a child was only tested in one of the words of that image pair. The experiment ended when the last trial was completed, or when the child indicated that they wanted to stop. Prior to testing, we had also decided to follow the CCT procedure of declaring the test over if the child failed to produce a response 4 trials in a row, but this criterion was never met.Footnote ³ The whole task lasted approximately 30 minutes.

Questionnaire

Two questions were relevant to the SES analyses, one asking about the education level of the three primary caregivers (be it parents, other family members, or unrelated others), and another about their occupation. Certainly, SES is a complex construct and typically many other indices, such as profession and income, could be collected to have a more accurate or fuller impression of a family’s SES. However, in a systematic review (Scaff & Cristia, Reference Scaff and Cristian.d.), over 70% of studies on infant word comprehension used maternal education as the SES proxy (the remaining 30% evenly split between using income or a composite). Because of this, and prior to data inspection, we decided that only maternal education would be employed here to increase comparability with previous work. Analyses with father’s education and the average of mother’s and father’s education are provided in Supplementary Materials (Supplementary Materials, SM6).

Another group of questions covered exposure to French versus other languages. We used parental report of percent of language exposure, to measure lingual status from birth until the child’s current age. We calculated the percent of exposure to each language based on three age periods: 0-12 months, 12-24 months, and after 24 months of age. For each age period, parents were asked to choose between the following options: a) 100% French; b) 70% French or more; c) 30-70% French; d) less than 30% French; e) none of the above). Although there is no consensus in the bilingual literature regarding what percentage of time children should be exposed to each language to be considered bilingual (see Rocha-Hidalgo & Barr, Reference Rocha-Hidalgo and Barr2021 for a discussion), bilingual studies often use the range between 30% (minimum) to 70% (maximum) of exposure to each language as the maximum each language to be considered bilingual (see Byers-Heinlein, Reference Byers-Heinlein2015 for a review of the methods used in bilingual literature; and Fennell, Byers-Heinlein & Werker, Reference Fennell, Byers‐Heinlein and Werker2007 for an example of a study using this cutoff). Parents additionally had a dedicated text box to provide further information in case of bilingual or multilingual exposure -- if they wanted to provide more fine-grained percentages for each language. Thus, in our questionnaire parents could report language exposure using two answer modalities. If parents answered categorically with options a-d, we converted these responses to numerical ones within each of the age ranges separately. If parents specified in the text box what percentage of each language the child was exposed to (e.g., parents first checked option c) 30-70% French and then provided their exact estimate of each language – for example, 65% French – 35% Japanese for the same age period), then we used this more precise percentage instead. Overall, percentages were calculated as the mean across the reports for the three separate age ranges, to obtain a global estimate of the child’s language exposure over the first years of life. Lingual status was then assigned in the following way. Monolingual children were those exposed to French between 100% and 96% of the time. Children were classified as having “minimal exposure to foreign languages” if they had between 95% and 70% exposure to French. Finally, children were classified as having “moderate exposure to foreign languages” if they were exposed less than 70% to French (minimum exposure to French: 20%). Notice that this classification is based on French exposure, regardless of how many other languages children were exposed to. Although precise tracking of different languages is crucial for research on bilingualism and multilingualism, the goal here was only to measure the effect of variation in exposure to the dominant language. Analyses with alternative implementations of our lingual status variable can be found in Supplementary Materials (Supplementary Materials, SM7).

Analyses

Data processing and statistical analyses were carried out in R (R Core Team, 2017). If no response was recorded during the first 7 seconds, then the trial was removed. This affected 352 or 11% of the trials, out of the 3191 trials across participants. Alternative analyses treating these trials as incorrect answers can be found in the Supplementary Materials (Supplementary Materials, SM9).

A mixed binomial logistic regression was used to predict accuracy, whether the response in a given trial was accurate or not, from the child’s age (centered), lingual status (monolingual French set as the baseline level), and maternal years of education (centered) as independent fixed factors, and the child’s unique ID as a random factor. We also declared all 2-way and 3-way interactions among the fixed factors. A similarly structured mixed linear model was fit to response times after the logarithm was applied, which resulted in response times being fairly normally distributed, and residuals being normally distributed.

All statistical models were submitted to an Analysis of Variance model comparison (Type III) to estimate to what extent a given fixed factor improved model fit. This is a tool that compares a model with versus without a given factor or interaction and reports the chi-square (binomial models) or F-value (linear models) for that comparison. This metric considers all sub-levels, allowing us to report a single number for all fixed factors and interactions. All regressions estimate a beta for each level of the factor; thus, for the main effects of lingual status as well as all interactions involving lingual status, there will be two betas (one comparing monolingual against minimal exposure, another comparing monolingual against moderate exposure).

Checking for mediation effects

Previous work suggests that SES and lingual status effects on cumulative knowledge (indexed by accuracy) are mediated by processing skills. Insofar as our response time measure indexes online processing skills, we should observe that the effects of maternal education and lingual status on accuracy should diminish once individuals’ response times are taken into account, and the converse, with accuracy explaining away response times.

Notice that mediation analyses can only be defined at the child level, not at the trial level, and thus these are simple regressions (not mixed models at the trial level). One mediation analysis was performed by adding the logarithm of the response time as an additional predictor to a logistic model that predicts the proportion of trials that are answered accurately from maternal education, lingual status, and age, as well as lingual status by age interaction (other interactions were not included because they were not significant in the main analysis reported on above, see Table 3). The other mediation analysis added proportion accurate as a predictor in a linear regression predicting the logarithm of the child-level average response time from maternal education, lingual status, and age (interactions were not included because they were not significant in the main analysis reported on above, see Table 4). In both cases, we followed up with a type III ANOVA.

Table 3.

Betas (standard errors) in a logistic regression predicting individual infants’ proportion of trials that were accurate from maternal education, lingual exposure (MinExp is the contrast between monolingual and minimal exposure; ModExp the same for moderate exposure); and age. DF stands for degrees of freedom. Level of significance: * p <.05; ** p < .01; *** p < .001.

Table 4.

Betas (standard errors) in a logistic regression predicting individual infants’ reaction times (log-transformed) from maternal education, lingual exposure (MinExp is the contrast between monolingual and minimal exposure; ModExp the same for moderate exposure); and age. DF stands for degrees of freedom. Level of significance: * p <.05; ** p < .01; *** p < .001.

Table 3 shows results for the regressions predicting accuracy. When response times are added to the regression, the coefficients for the main effects of maternal education and lingual exposure are virtually unaltered; the coefficient for age is reduced somewhat, and those for the interactions between lingual status and age are nearly halved. Nonetheless, all of these effects and interactions remain significant. Thus, response times do not explain away all variation associated with maternal education and lingual status.

Table 4 shows results for the regressions predicting the logarithm of the average response times at the individual child level. When proportion accurate child-level estimates are added to the regression, the coefficients for the main effect of age is reduced by a third, although it remains significant. Those for maternal education and lingual exposure were very small and remain so. In sum, neither reaction times nor accuracy mediate the effects of SES and exposure to other languages.

Maternal education (a proxy for SES) effects

We confirmed our first prediction, that children from higher SES have higher accuracy than children from lower SES in our word comprehension task. We found a significant correlation of r = .33 between maternal education and accuracy.Footnote ⁵ As reviewed in the Introduction, the literature assessing tactile choice paradigms does not yield a unique straightforward answer about the relation between accuracy and SES. In this study, we offer an additional data point that shows that tactile paradigms can capture individual variance attributed to SES. In contrast, results pertaining to response times and maternal education are more stable across samples, with the handful of tactile studies (De Anda et al., Reference De Anda, Hendrickson, Zesiger, Poulin-Dubois and Friend2018; Legacy et al., Reference Legacy, Zesiger, Friend and Poulin-Dubois2018; Rosemberg & Alam, Reference Rosemberg and Alam2021) investigating this relation reporting non-significant differences (as do we, r = 0).

We also confirm our prediction of smaller effects of SES on child lexical development in this French sample than in previously reported work on American toddlers using a visual choice paradigm (Fernald et al., Reference Fernald, Marchman and Weisleder2013) or Argentinean toddlers using a touch-screen study (Rosemberg & Alam, Reference Rosemberg and Alam2021), where effects are moderate. Thus, the idea that the strength of the association between SES and word comprehension accuracy may depend on how well families are supported in a given country needs to be further investigated.

Lingual status effects

Regarding lingual status, we found that children with greater exposure to French have higher accuracy in this French-only test than children with less exposure (r = .26; confirming our prediction) but not shorter response times (r = -.04, counter our prediction). The accuracy results are aligned with the visual and tactile work cited in the Introduction (e.g., De Anda et al., Reference De Anda, Arias-Trejo, Poulin-Dubois, Zesiger and Friend2016, Reference De Anda, Hendrickson, Zesiger, Poulin-Dubois and Friend2018; Hurtado et al., Reference Hurtado, Gruter, Marchman and Fernald2014; Legacy et al., Reference Legacy, Zesiger, Friend and Poulin-Dubois2016, Reference Legacy, Zesiger, Friend and Poulin-Dubois2018; Marchman et al., Reference Marchman, Martínez, Hurtado, Grüter and Fernald2017). Results converge with previous published tactile work, where no difference in reaction times between lingual status have been found (Poulin-Dubois, Bialystok, Blaye, Polonia & Yott, Reference Poulin-Dubois, Bialystok, Blaye, Polonia and Yott2013; De Anda et al., Reference De Anda, Hendrickson, Zesiger, Poulin-Dubois and Friend2018; Legacy et al., Reference Legacy, Zesiger, Friend and Poulin-Dubois2016, Reference Legacy, Zesiger, Friend and Poulin-Dubois2018).

The results also showed an interaction between lingual status and age in predicting lexical knowledge: monolinguals’ accuracy increased with age more rapidly in comparison to the other two lingual status groups. Previous results using indirect methods have yielded mixed results (e.g., using the MB-CDI, Silven, Voeten, Kouvo & Lunden, Reference Silven, Voeten, Kouvo and Lunden2014 find faster growth in the dominant language for monolinguals than non-monolinguals; but Pearson, Fernández & Oller, Reference Pearson, Fernández and Oller1993 and Hoff, Rumiche, Burridge, Ribot & Welsh, Reference Hoff, Rumiche, Burridge, Ribot and Welsh2014 do not). Therefore, we believe it may be relevant to first see this result replicated using a tactile choice paradigm before attempting to find an interpretation for it.Footnote ⁶

We want to stress again that this task does not provide a full picture of non-monolingual children’s language skills, since their abilities in the other language(s) they are exposed to are not reflected. Although our data are not relevant for research on strictly defined bilingualism and multilingualism, they remain important not only for the specific research topic we address here (for which such data are sufficient) but also for the description of language acquisition in any site where there is a dominant language.

Age effects and sensitivity of our measures to individual variation

As children become older, their executive functions, motor skills, and linguistic skills mature (Collins, Reference Collins2008). Since our touch-screen-based vocabulary task may load on all of these, in the Introduction we predicted larger effects on performance for older than younger children. The fact that most of our dependent measures correlated with age can be used as an argument for these very measures being sensitive to individual variation. That said, in our data, accuracy was correlated with all three factors (maternal education, lingual status, and age) whereas reaction times are predicted only by age, which may indicate differences in sensitivity between these two measures (visible as well in small differences in their split-half reliabilities).

Mediation effects

Finally, we made the prediction that reaction time – as a proxy of speed of processing – mediates the predictive value of SES, lingual status, and age with respect to accuracy; and that, conversely, accuracy – as a proxy of cumulative knowledge – mediates their predictive value on response times. Our results did not support either hypothesis, since the predictive value of SES, lingual status, and age was virtually unaffected by these additions. We conclude that our results are inconsistent with the proposal that the effects of SES and lingual status on accuracy and reaction time are largely overlapping and thus unlikely to stem from a similar cause (e.g., exposure to more language leads to faster language processing that then leads to higher accuracy and reciprocally, that exposure to more language leads to more vocabulary knowledge that then leads to faster processing). However, we acknowledge that this result might be tightly bound to the modality of our response times measure. We further discuss this possibility in the section below.

Tactile response times and limitations

Comparison across studies that do not use the same stimuli nor study the same population is a complicated task. We acknowledge that we are unable to fully understand why response times in our French-only task were not predicted by SES or lingual status, nor why we do not find mediation effects in our analyses. Furthermore, to our knowledge, no study has found reaction time differences as a function of SES or lingual status among the tactile choice paradigms. The available evidence seems to point that tactile response times are not well suited for studying individual variation in relation to SES and lingual status. Indeed, perhaps tactile response times are less specific to lexical processing than visual response times, as we explained in the Introduction, and as suggested by Smolak et al. (Reference Smolak, Hendrickson, Zesiger, Poulin-Dubois and Friend2021). Tactile response times are considered to be more “computationally costly”, primarily by the interaction of motor responses and executive functions. To specifically test this hypothesis, researchers could perform studies using the two modalities and measuring the impact of factors known to affect lexical processing (e.g., presence of visually or phonologically similar competitors). Both Smolak et al. (Reference Smolak, Hendrickson, Zesiger, Poulin-Dubois and Friend2021) and Koenig et al. (Reference Koenig, Arunachalam and Saudino2020) report on visual and tactile choice paradigms, but neither has interrogated their data as we described.

It is worth mentioning that the evidence showing reaction differences is scarce. To our knowledge, only the work based in the United States has shown so far differences in reaction times as a function of SES or lingual status. The study of SES and lingual status should further be replicated and extended to more diverse populations, so we can better understand how speed of processing measures generalized. Therefore, we look forward to re-analyses and further research to shed further light on this question.

Using a tablet-based paradigm

Given this variability in results, we hope additional work provides some much-needed data on both SES and differences in exposure to the tested language. Our findings indicate that variation in how accuracy and reaction times are defined, the stimuli chosen, the setting in which data are collected (lab, daycare, or at home), the SES range includedFootnote ⁷, the children’s age, and other methodological and conceptual parameters are all crucial factors adding individual variation in the study of word comprehension. Thanks to using portable tablets, we were able to test in public daycares in a neighborhood with a great deal of variability in both dimensions of interest: SES and lingual status. Regarding lingual status, portable tablets allowed us to test a large variety of learning language pairs that are mostly understudied (see Rocha-Hidalgo & Barr, Reference Rocha-Hidalgo and Barr2021 for a discussion on the need to expand bilingual research to more diverse samples). Even though in this study we only focus on the dominant language of the place of study, the ease of use of portable tablets opens a possibility to diversify samples and language groups from which we could obtain bilingual data in multiple languages if adapting the test to other languages. In terms of SES, our location choice resulted in a sample that broadly matches national statistics for education length in terms of SES distribution.Footnote ⁸ We believe the full range of SES is better captured in Rosemberg and Alam (Reference Rosemberg and Alam2021)’s work on Argentinean Spanish learners because they tested in daycares located in varied neighborhoods.

However, our sampling method may have reduced the variance among the children, many of whom have spent 3-5 days a week for at least the previous 6 months in a common environment, where children with both lower and higher SES levels, and with different levels of exposure to French, attend the same physical daycare, and thus benefited from common experiences that could reduce effects of prior and concurrent experiences outside this setting. This is an active area of research and intervention, as governments and populations may be interested in reducing the impact of economic inequality via educational experiences (although current recommendations suggest that doing so via interventions targeting parents is more effective than by investing in daycares; Busso, Cristia, Hincapie, Messina & Ripani, Reference Busso, Cristia, Hincapie, Messina and Ripani2017).

Finally, we draw attention to Lo, Rosslund, Chai, Mayor, and Kartushina (Reference Lo, Rosslund, Chai, Mayor and Kartushina2021)’s use of a browser-based version, which should make it easy to program and adapt the study to a variety of platforms, unlike our relatively expensive choice of iPads. Such approaches may allow us to better represent the full range of SES, but the question of how to fairly represent language development in cosmopolitan sites remains challenging.