Introduction
Today’s advances in digital learning and access to a wide variety of communication platforms have made the ability to comprehend multiple speakers of the L2 essential for achieving L2 proficiency. Therefore, multivoice L2 training should benefit learners who wish to improve their L2 skills for real-world interactions. However, empirical support of the effect of multivoice input on L2 acquisition has been largely limited to phonemic or lexical tone distinction and isolated word learning (e.g., Barcroft & Sommers, Reference Barcroft and Sommers2005, Reference Barcroft and Sommers2014; Bradlow et al., Reference Bradlow, Pisoni, Akahane-Yamada and Tohkura1997; Hardison, Reference Hardison2003, Reference Hardison2005; Lively et al., Reference Lively, Logan and Pisoni1993; Logan et al., Reference Logan, Lively and Pisoni1991; Sommers & Barcroft, Reference Sommers and Barcroft2007, Reference Sommers and Barcroft2011; Wiener et al., Reference Wiener, Ito and Speer2018, Reference Wiener, Chan and Ito2020, Reference Wiener, Ito and Speer2021: see following text). Critically, we do not know whether multivoice input is also beneficial to L2 sentential processing. The present study investigates the effect of voice variability on the acquisition of a novel structure in French. Specifically, learners receive processing instruction (PI), an established L2 intervention that focuses on providing structured input (SI) without demanding drill-like repetitive productions (VanPatten & Cadierno, Reference VanPatten and Cadierno1993), with either multivoice or single-voice input to learn the French causative structure. The improvement of input processing is assessed by comparing eye movements and accuracy scores from a picture-selection task before and after training.
Additionally, the present study examines how the number of training items used in PI interacts with phonetically variable input (PVI). Research has demonstrated that while facilitative effects of PVI may surface in learners who have had more exposure to the L2 (Wang et al., Reference Wang, Spence, Jongman and Sereno1999, Reference Wang, Jongman and Sereno2003), learners with little previous exposure to the L2 may benefit less from PVI (Wiener et al., Reference Wiener, Ito and Speer2018, Reference Wiener, Chan and Ito2020, Reference Wiener, Ito and Speer2021). In the PI research framework, it has been speculated that the number of training items may impact how successfully a target form is processed (White, Reference White2015).
The present study investigates whether PVI affects syntactic processing and whether PVI interacts with different training sizes for the acquisition of a syntactic structure. Specifically, we examine the effect of voice variability (single vs. four speakers) with PI training of three input training sizes: short (24 items) versus medium (48 items) versus long (96 items).
Background and Motivation
Effect of Voice Variability on Language Acquisition
The benefit of exposure to PVI has been investigated over decades, and the facilitative as well as debilitating effects have been demonstrated in first language (L1) word retrieval (Goldinger et al., Reference Goldinger, Pisoni and Logan1991; Mullennix et al., Reference Mullennix, Pisoni and Martin1989), L2 phonemic distinction (Bradlow et al., Reference Bradlow, Pisoni, Akahane-Yamada and Tohkura1997, Reference Bradlow, Akahane-Yamada, Pisoni and Tohkura1999; Lively et al., Reference Lively, Logan and Pisoni1993; Logan et al., Reference Logan, Lively and Pisoni1991; Sadakata & McQueen, Reference Sadakata and McQueen2014; Thomson, Reference Thomson2012), L2 lexical tone production (Perrachione et al., Reference Perrachione, Lee, Ha and Wong2011; Wang et al., Reference Wang, Spence, Jongman and Sereno1999, Reference Wang, Jongman and Sereno2003; Wiener et al., Reference Wiener, Chan and Ito2020, Reference Wiener, Ito and Speer2021), L2 lexical input processing (Barcroft & Sommers, Reference Barcroft and Sommers2005, Reference Barcroft and Sommers2014; Sommers & Barcroft, Reference Sommers and Barcroft2007), and foreign accent adaptation (Bradlow & Bent, Reference Bradlow and Bent2008). With the cumulative empirical validation of multivoice input effects, researchers have proposed accounts as to how PVI facilitates acquisition of sound categories (phonemes or tones) and lexical processing. In the 1990s, exemplar theory of human category formation was applied to speech perception and word learning. It was assumed that features such as the indexical information conveyed by talker-specific phonetic characteristics, as well as speech style and speaking rate are all traced and stored as clusters of exemplars, and that such episodic traces can facilitate the retrieval of information stored in memory upon processing new tokens of input (see Johnson & Mullennix, Reference Johnson and Mullennix1996; Pierrehumbert, Reference Pierrehumbert, McQueen and Cutler2001 for a review). Within this framework, researchers have argued that PVI enables more distributed, richer representations of exemplar clusters or categories across multiple dimensions that consist of phonemic, lexical, phrasing, register, and dialectal characteristics (Bradlow & Bent, Reference Bradlow and Bent2008; Goldinger, Reference Goldinger, Johnson and Mullennix1997; Hardison, Reference Hardison2003; Sommers & Barcroft, Reference Sommers and Barcroft2011. See Barriuso and Hayes-Harb, Reference Barriuso and Hayes-Harb2018 for a historical development of the research on high variability phonetic training in L2 acquisition.) In contrast, single-voice or less variable input is assumed to provide cues that are too narrowly specified for representing categories. Perceptual systems that have been trained with less variable input cannot effectively map characteristics of novel input onto category-general (and voice-nonspecific) features (Bradlow & Bent, Reference Bradlow and Bent2008; Bradlow et al., Reference Bradlow, Akahane-Yamada, Pisoni and Tohkura1999; Lively et al., Reference Lively, Logan and Pisoni1993).
Barcroft and Sommers conducted a series of lexical input processing studies and tested the “phonetic relevance hypothesis”—the idea that phonetic cues are retained with varying weights according to their relevance to the target of processing (Barcroft & Sommers, Reference Barcroft and Sommers2005, Reference Barcroft and Sommers2014; Sommers & Barcroft, Reference Sommers and Barcroft2007). This hypothesis was supported by the facilitatory effect of multispeaker (and variable speaking style) input on L2 word learning (Barcroft & Sommers, Reference Barcroft and Sommers2005), and the null effect of overall amplitude and fundamental frequency (F0) that are not relevant to lexical contrasts in the target L2 (Spanish) and in the learners’ L1 (English) (Sommers & Barcroft, Reference Sommers and Barcroft2007). Critically, Barcroft and Sommers (Reference Barcroft and Sommers2014) confirmed that F0 variability in L2 (Russian) input was useful for speakers of Zapotec, which has lexical tones, but not for speakers of Spanish, which does not have tonal contrast. This result echoes Perrachione et al. (Reference Perrachione, Lee, Ha and Wong2011) who found that PVI facilitates tone learning only in learners with high F0 sensitivity but not in those with no previous experience with categorical pitch processing. Sadakata and McQueen (Reference Sadakata and McQueen2014) also report that Dutch speakers with lower tone perceptual aptitude performed worse than those with higher aptitude in learning Mandarin pseudowords. These studies together illustrate that the processing efficacy of PVI may depend on cue validity and on learners’ perceptual experience and aptitudes. A recent study by Mora-Plaza et al. (Reference Mora-Plaza, Ortega and Morain press) also shows that individual differences in the ability to control attention affects the processing of PVI.
Given the complexity of the processing mechanism underlying PVI, the examination of L2 syntactic acquisition demands careful consideration of the cognitive cost for constructing rich representations of L2 structures. Evidence for higher processing costs for PVI comes from studies that demonstrate greater cortical activation (Uddin et al., Reference Uddin, Reis, Heald, Van Hedger and Nusbaum2020; Wong et al., Reference Wong, Nusbaum and Small2004), difficulty in word learning in infants (Jusczyk et al., Reference Jusczyk, Pisoni and Mullennix1992), less accurate word recognition (Mullennix et al., Reference Mullennix, Pisoni and Martin1989; Wiener et al. Reference Wiener, Ito and Speer2018, Reference Wiener, Chan and Ito2020), and word recall in adults (Martin et al., Reference Martin, Mullennix, Pisoni and Summers1989). In Wiener et al. (Reference Wiener, Chan and Ito2020), the effect of PVI on the production of an artificial tone language was detected for one of the four tones but only when the learners received explicit instruction on lexical tones. As for word recognition, both early L2 learners (Wiener et al., Reference Wiener, Ito and Speer2018) and novice learners (Wiener et al., Reference Wiener, Ito and Speer2021) who were trained with PVI relied more on the probability of syllable-tone co-occurrence than on the actual acoustic input. This led to a higher number of errors especially when the input had the least probable tone. Wiener et al. suggest that PVI may lead to more active use of distributional information in L2 processing. With respect to the manner of variability presentation, Lim et al. (Reference Lim, Qu, Tin and Perrachoine2019) reported that the more talkers to which participants were exposed, the more difficult it was for them to identify words accurately. Taken together, processing PVI seems to demand higher attention and flexible executive control and may evoke processing strategies (e.g., reliance on statistical information) more actively than phonetically less variable input.
Previous research has also shown that bilinguals’ short-term memory functions less efficiently in their L2 than in their L1 (Thorn & Gathercole, Reference Thorn and Gathercole1999, Reference Thorn and Gathercole2001; Thorn et al., Reference Thorn, Gathercole and Frankish2002). A meta-analysis of 79 studies by Linck et al. (Reference Linck, Osthus, Koeth and Bunting2014) demonstrated a solid relationship between executive function and verbal working memory and L2 performance. Given the general cost of PVI processing and the extra memory load for L2 processing, nonproficient L2 learners may experience a higher cognitive burden processing PVI, and thus may require a larger amount of training to construct rich representations of a novel L2 structure. Extracting a syntactic structure across multivoice input demands tracing additional layers of information, and parsing a novel grammatical structure with richer input may require a greater number of exemplars. Thus, we may hypothesize that the facilitatory effect of PVI may not surface with a smaller training size while a larger amount of training with PVI may benefit nonproficient L2 learners. At the same time, because the SI (see the following text) activities in PI are known to be effective for facilitating L2 learners’ input processing (VanPatten, Reference VanPatten, Loewen and Sato2017), these activities may help learners compensate for any processing costs that PVI might induce.
Effect of Number of Items in PI Training
Research on PI has consistently demonstrated that L2 learners can be trained to process input better by engaging in SI activities that give them practice with correct input processing (VanPatten & Cadierno, Reference VanPatten and Cadierno1993; VanPatten et al., Reference VanPatten, Collopy and Price2013). However, the number of training items has varied largely across these studies. For example, Fernandez (Reference Fernandez2008) used 30 aural target items (i.e., 20 target OVS sentences and 10 distractors sentences of the type SOV and SVO) while White (Reference White2015) included a condition that contained up to 140 items (aural and written).
White (Reference White2015) exposed six groups of participants to SI activities. Each group received either 40, 60, 80, 100, 120, or 140 items of third-person Spanish accusative clitics. All participants completed a pretraining sentence-level picture-selection interpretation test, an immediate posttraining test, and a delayed posttraining test that took place 3 weeks later. Participants were not provided with explicit instruction on the target structure. In the immediate posttraining test, all groups made significant improvements as compared to the pretraining performances and maintained the equivalent accuracy level in the delayed posttest. No effect of number of training items was observed on the immediate posttest. On the delayed posttest, the 140-item group outperformed the 100-item group (but not the other groups). Given the overall null effect of the number of training items across the two posttests, White (Reference White2015) concluded that any training size within the range of 40–140 items can positively impact participants’ interpretation of the target form (in this case, third-person Spanish accusative clitics) and the effects can be maintained after 3 weeks.
Because the 140-item group outperformed the 100-item group on the delayed posttest, but not the other groups with a lower number of items (i.e., 40, 60, and 80), White (Reference White2015) argues that the overall positive outcome of the interpretation task cannot be attributed to the number of training items. White reviews previous PI studies that investigated Spanish accusative clitics and remarks that both VanPatten and Oikkenon (Reference VanPatten and Oikkenon1996), with 138 training items, and White and DeMil (Reference White and DeMil2013), with 38 training items, made significant improvements on an interpretation task with mean scores similar to his own study. White claims that the robustness of the results on interpretation tasks must be due to the nature of SI activities and to their ability to teach learners to adopt more effective processing strategies.
To date, White (Reference White2015) is the only study that directly examined the effects of different training sizes within the same experimental design and measure. While its results confirmed the robust effect of PI with 40-plus training items, it remains unclear whether and how the number of training items impacts the acquisition of the target structure. Particularly, we do not know how small the size of training can be to lead to significant improvements in the comprehension of a new structure. It also remains unclear whether PVI—which may be effective for categorical feature extraction but with an extra burden on cognitive processing, will require more training to help learners achieve the level of performance that they would achieve with phonetically less variable input. The present study exposes participants to either 24 items—the shortest of all past PI studies (Fernandez, Reference Fernandez2008; White, Reference White2015), or 48 items—the most common range of intervention size, or 96 items—four times more than the shortest training. If PVI demands a high cognitive load, then nonproficient L2 learners may not benefit from the 24-item or even 48-item trainings, while these relatively shorter trainings may benefit learners if they are given phonetically less variable input. In contrast, PVI may become less burdensome and more beneficial once the training size exceeds a certain level, and thus the positive effect of PVI may appear only after the longer trainings.
Methodological Framework for Testing French Causative
The present study employs the methodological framework of two preceding studies that demonstrated the effect of PI on L2 spoken sentence processing (Wong and Ito [2018] and Ito and Wong [2019]).
Both studies examined participants’ eye movements to two quasiidentical scenes that exchanged the roles of two people while they listened to causative sentences in French, for example, Pierre fait faire un sandwich à Marie. “Pierre makes Marie make a sandwich.” This structure contains two cues that signal that the sentence is causative: the first is the verb fait followed by another verb in the infinitive form (fait faire); the second is the prepositional phrase (à Marie). Despite these cues, L2 learners of French tend to think the person making a sandwich is Pierre rather than Marie because Pierre is the first noun of the sentence (VanPatten & Wong, Reference VanPatten, Wong and VanPatten2004: this general tendency to assign the agent role to the first animate noun is termed “the First-Noun principle (FNP)” by VanPatten, Reference VanPatten1996, Reference VanPatten, VanPatten, Keating and Wulff2020).
In Wong and Ito (Reference Wong and Ito2018), intermediate French learners received either traditional training in which sentence fragments were mechanically pieced together to form a causative structure or PI training where they read a sentence (e.g., Patrick fait jouer du piano à Jean. “Patrick makes Jean play the piano.”) and answered the content related question (e.g., Who plays the piano?). Only the PI group exhibited robust improvement in the picture selection and reduced looks to the incorrect picture, which depicted the first noun mentioned in the sentence as the agent of the action (e.g., Pierre making a sandwich). This reduction of incorrect looks was observed after the second cue, that is, à + noun. Wong and Ito speculated that the size of their training at 48 items may have been sufficient to destabilize participants reliance on the FNP, but perhaps a more intense training may be needed to allow them to make use of the earlier cue.
Ito and Wong (Reference Ito and Wong2019) used the same eye-tracking method and tested the effect of voice familiarity with auditory input for the PI training. Participants either heard the same voice during training and testing or they heard one voice during training and a different voice during testing. Results confirmed the robust effect of PI on picture-selection accuracy, while the effect of voice familiarity depended on the clarity of the cue: while the female voice had a slower speech rate for the earlier cue (e.g., fait faire), the male voice had slower speech rate for the later cue (à Marie/Pierre). The effect of familiarity was shown only with the male voice in both picture-selection accuracy and reduction of incorrect looks during the final PP. Because the male voice did not show a baseline advantage as compared to the female voice in the before-training test block, Ito and Wong (Reference Ito and Wong2019) concluded that participants benefitted from becoming familiar with the voice that provided perceptually more robust structural cues in the prosodically prominent sentence-final position throughout the training.
The Present Experiment
While Ito and Wong (Reference Ito and Wong2019) demonstrated the effect of auditory PI training on the processing of a novel L2 structure, their findings could not speak to the general effect of PVI because they used only two speakers with confounding speech-rate differences. The present study uses four speakers for auditory PI training (two males and two females) that are different from the test voices. If a single-voice training benefits L2 learners who are not yet capable of using structural cues that appear earlier in spoken sentences, and if multi-voice input makes it harder to extract cues in running speech, the reduction of incorrect looks may happen only after a single-voice training but not after a multivoice training. In contrast, if multivoice input facilitates richer structural representations and improves L2 structural processing, the reduction in incorrect looks may appear in multivoice groups only. We also speculate that the effect of training voice may interact with the size of training: single-voice advantage (for providing consistent cues) and multivoice disadvantage (due to higher cognitive demand) may surface with shorter trainings whereas single-voice disadvantage (or the difficulty to generalize to a new voice) and multivoice advantage (due to richer representations) may become more evident with a longer training.
The present eye-tracking experiment tested the effect of PVI (single- vs. multivoice) training of three different sizes (24, 48, and 96 items) on L2 learners’ processing of the French causative structure. Third-semester French learners were tested with a picture- selection task before and after an auditory PI training with either a single voice or four voices.
Research Questions
RQ1. Does PVI used in PI training facilitate the processing of a novel causative structure in French learners? Or does PVI hinder learning a novel syntactic structure?
RQ2. How does the size of auditory PI training modulate the degree of learning? Do French learners benefit from the smallest training size (24 items) to the same degree as when they are trained with a larger number of items? Does the training size predict the degree of improvement in picture-selection accuracy?
RQ3. Does the effect of PVI interact with training size? Do learners benefit from a multivoice training only if they receive a sufficient amount of input? Does multivoice input hinder learning if the training is very short?
Method
Participants
A total of 185 French learners in a third-semester college course of French were recruited at a university in the Midwest across three semesters (autumn 2018, spring 2019, autumn 2019). Participants received three extra credit points upon completion of their participation. No participant had been formally instructed on the target causative structure prior to the experiment. Within each semester we ran one experiment with a fixed number of training items (24, 48, or 96). Participants were randomly assigned to either the single-voice training group or the multivoice training group. Within each training type, they were further assigned to one of the four presentation lists. A total of 16 participants spoke language(s) other than English and French. Because their performances did not pattern differently from the rest of the groups, their data are included in the analyses reported in the text that follows. Data from 11 participants were excluded due to technical failure (e.g., calibration errors, presentation software failure, data-saving errors) and nontechnical issues (e.g., task misunderstanding and incompletion). A total of 174 participants were included in the final data pool.
Materials
Eye-Tracking Stimuli
For the picture-selection task, a total of 96 picture pairs of quasiidentical scenes were presented with prerecorded auditory sentences. All picture scenes had the same two characters: one male (Pierre) and one female (Marie). For each pair, the positions of the two characters were exchanged. For example, Marie is making the sandwich for Pierre on one side while Pierre is making the sandwich for Marie on the other side (Figure 1).
Example of visual stimulus.
Twenty-four slides (each showing two scenes side by side) were presented as critical items in the causative and noncausative conditions. Two causative sentences (e.g., Pierre/Marie fait faire un sandwich à Marie/ Pierre. “Pierre/Marie makes Marie/Pierre make a sandwich.”) and two noncausative sentences (e.g., Pierre/ Marie fait un sandwich pour Marie/ Pierre. “Pierre/Marie makes a sandwich for Marie/Pierre.”) were presented with the same picture pair across four presentation lists. Among the 72 filler items, 24 slides were shown with active (e.g., Pierre/Marie réprimande Marie/Pierre. “Pierre/Marie scolds Marie/Pierre.”) and passive (e.g., Pierre/Marie est réprimandé(e) par Marie/Pierre. “Pierre is scolded by Marie/Pierre.”) sentences. The remaining 48 items presented contrastive actions (e.g., Pierre/Marie dort, et Marie/Pierre danse. “Pierre/Marie sleeps, and Marie/Pierre dances.”) and contrastive objects (e.g., drinking lemonade vs. water). To introduce variation in syntactic complexity among the stimuli, one third of these items presented an object-relative clause (e.g., Pierre/Marie regarde le film que Marie/Pierre a loué. “Pierre/Marie watches the film that Marie/Pierre rented.”).
A female native speaker in her early thirties from Rennes and a male native speaker in his early fifties from Marseille each recorded a total of 288 sentences (48 causative, 48 noncausative, 48 active, 48 passive, 64 conjoined, 32 object-relative clause) at 44.1 KHz sampling rate using Praat (Boersma & Weenink, Reference Boersma and Weenink2016). The duration measures of the critical causative sentences showed that the female speaker produced the earlier cues (i.e., fait followed by an infinitive form of the verb) with longer duration than the male speaker while the male speaker produced the rest of the sentence (i.e., subject, the rest of the verb phrase, and the final PP [à Marie/ Pierre]) with longer duration than the female speaker (Table 1). In each of the three experiments of the present study, participants in all eight presentation lists (2 voice training types × 4 voice counterbalancing) were tested with the female voice before the training and with the male voice after the training, that is, the novelty of the test voice was fixed across training types. The only difference between the presentation lists was the voice participants heard during training.
Duration analysis of causative test stimuli (for eye-tracking): lmer(Duration~Speaker+(1|item),data=testitem)
*** p < .001; **p < .01; *p < .05; ^p < .1 (for all the tables)
Training Materials
A total of 96 sentences (48 causative and 48 noncausative sentences) were created using words from participants’ textbooks and/or other course materials. Four native speakers of French (Male1 from Rennes, France in his mid-twenties; Male2 from Rennes, in his early thirties; Female1 from Brussels, Belgium in his late thirties; Female2 from Toulon, in her early thirties) each recorded all 96 sentences in a soundproof room using Praat (Boersma & Weenink, Reference Boersma and Weenink2016). Three of them had experience teaching French. Female1 did not teach French but was a faculty member at the same university. They were all instructed to speak at a comfortable rate for third semester students. The clarity of all 384 (96 × 4) sentences were cross-checked between the second author who is fluent in French and the research assistant with more than five years of training in French.
The durational analysis of the four speakers’ voices showed that females speakers spoke slower than male speakers throughout the target causative sentences (Figure 2a): Female1 had the longest average duration for all five regions of the utterances, and the interspeaker differences became larger after the infinitive verb (see Table 2 for the statistical analysis). While Female1 showed significantly longer duration than each one of other speakers in almost all regions, this durational difference did not lead to differences in the effect of PI. Further discussion on the durational difference among the training voices is therefore omitted hereafter.
(a) Duration of analysis of causative sentences for PI training and (b) F0 analysis of causative sentences for PI training.
Duration analysis of causative sentences for PI training: lmer(Duration~Speaker+(1|item),data=testitem); Female1 as the reference of Speaker
The analysis of mean fundamental frequency (F0) of the critical causative items showed the expected gender differences: female speakers had distinctively higher F0 than the male speakers throughout the sentence (Figure 2b; Table 3). Between the two speakers of the same gender, the average F0 did not differ much for each region. All speakers started the target sentences with a relatively high F0 range and finished the sentences at a relatively low F0 range, demonstrating similar tunes for the declarative sentences.
F0 analysis of causative sentences for PI training: lmer (F0~ Gender +(1|item), data = training item); where Gender is sum-coded
These acoustic analyses of the training items confirm that there were some differences in the speech rate and expected gender-oriented pitch range differences among the four voices, yet no speaker had peculiar changes in speech rate or in pitch levels during the target training sentences.
Procedure
Participants were tested individually. Each session consisted of a pretraining picture-selection task with eye-tracking, an auditory PI training, a posttraining picture-selection task with eye-tracking, and a background questionnaire. At the beginning of each session, the participant was seated 50 to 60 cm away from the eye-tracker monitor and calibrated using the Clearview 5-point calibration system. In a pretraining testing block, 48 items (6 critical causative, 6 noncausative, 12 passives, and 24 other filler items) were presented with the Tobii 1750 monitor using EPrime 1.0. In each trial, a paired picture (e.g., Figure 1) was presented for 1.5 sec before a sentence in the female voice was played through a noise-canceling headset. Participants were asked to click on the picture that they thought better matched the sentence. Upon the click, the screen switched to a crosshair in the center that signaled the beginning of the next trial. The coordinates of the participants’ gazes were continuously coded at 50 Hz throughout the trials.
In the PI training, participants were again seated in front of a monitor with a headset. They were told to listen carefully to each sentence (in French), read a question (in English) that appears on the screen, and answer the question using a button box. Participants had either 24, 48, or 96 trials total. Half the training items were in the causative structure, whereas the other half were in a noncausative structure. For each trial, “Listen” first appeared on the screen for one second. Following this, participants heard a sentence (e.g., Karen fait raconter l’histoire à Marc. “Karen makes Marc tell a story.”) in one of the four voices (f1, f2, m1, m2) with a blank screen. Then, a question (e.g., Who tells a story?) appeared on the screen with choices (e.g., a. Karen b. Marc). Participants pressed a button on a button box, and saw the correct answer highlighted in red. Then, the display automatically switched to “Listen” for the next trial. Participants who were assigned to the single-voice presentation lists heard all the trials in the same voice. Participants in the multivoice presentation lists heard all four voices an equal number of times (i.e., 6, 12, or 24 times each) in a randomized order.
The procedure of the posttraining eye-tracking task was identical to that of the pretraining task. Participants were calibrated again before testing. No items were repeated from the pretraining block within any list. All sentences were presented in the male voice, that is, participants were all tested with a novel voice.
Results
Data from 58 participants (with 24-item training), 65 participants (with 48-item training), and 51 participants (with 96-item trainings) were analyzed in the following manners. Their responses to the picture-selection task were coded as either 1 (for accurate) or 0 (for inaccurate) and were submitted to mixed-effects logistic regression analyses that tested the effects of Training Size (short [24], medium [48] and long [96]), Voice (single or multivoice), and Block (before or after the training). The model structure was simplified step by step until the output was generated without a conversion error (the final model structure is given in each table). The participants’ fixation data were expressed as the log of the ratio between the looks to the correct picture and those to the incorrect picture for each time bin and were aggregated for each analysis window. These log ratios were submitted to the mixed-effects regressions with the same set of predictor factors as the picture-selection responses using the R 3.5.1 lmer package.
Picture-Selection Accuracy
To our surprise, PI led to a robust increase in picture-selection accuracy to the comparable degree regardless of whether participants had voice variability and regardless of the number of training items. Figure 3 shows the picture-selection accuracy before (top) and after (bottom) PI training for causative and noncausative items. Replicating the results of Wong and Ito (Reference Wong and Ito2018) and Ito and Wong (Reference Ito and Wong2019), participants responded to the noncausative sentences at the ceiling accuracy level before the PI training, and the responses remained at a similar level after training. In contrast, the causative items had less than 10% accuracy before training across all groups. These findings confirm the strong bias toward interpreting the first NP as the agent of the visible action.
Picture-selection accuracy before (top) and after (bottom) PI training.
After the PI training, all groups exhibited a robust increase in picture-selection accuracy, yet to the similar degree (to 40–50%). The mixed-effects logistic regression model revealed a main effect of Block (z = 10.31, p < .0001), null effect of Training Size (Short: z = –1.27, p = .2; Long: z = –1.47, p = .14), null effect of Voice (z = .71, p = .47), a significant interaction between Block and Voice (z = –2.6, p < .01), a significant interaction between Short Training and Block (z = –2.28, p < .05), and a significant three-way interaction between Short Training, Block, and Voice (z = –2.23, p < .05).
Post-hoc comparison between the short (24) and the medium (48) training groups confirmed the robust Block effect, an interaction between Training Size and Block with a negative coefficient estimate, and an interaction between Training Size, Block, and Voice with a negative coefficient estimate (Table 4). The two-way interaction between Training and Block, together with the numerical trends shown in Figure 3, indicates that the degree of increase in the accuracy was smaller after the medium-size PI training than after the short PI training. The three-way interaction Training Size, Block, and Voice indicates that the difference in the increase in the accuracy level due to voice variability was smaller after medium than after short training.
Post-hoc comparison of picture-selection accuracy between Short (24) and Medium (48) size training
Response~Training+Block+Training:Block+Block:Voice+Training:Block:Voice+(1|Subj)+(1|Item), family=binomial, data=short-vs-med
The comparisons between medium (48) and long (96) training groups and between short (24) and long (96) groups confirmed only the robust effects of Block (Tables 5 and 6, respectively). There was no consistent effect of Training Size and no consistent effect of Voice. The mouse-click accuracy data therefore suggest that an increase in the amount of PI training beyond 24 items does not enhance the effect of training, and the multivoice input is equally effective as single-voice input.
Post-hoc comparison of picture-selection accuracy between Medium (48) and Long (96) size training
Response~Training+Block+Training:Block+Block:Voice+Training:Voice+Training:Block:Voice+(1|Subj)+(1|Item), family=binomial, data=med-vs-long
Post-hoc comparison of picture-selection accuracy between Short (24) and Long (96) size training
Response~Block+Voice+Training:Block+Block:Voice+Training:Block:Voice+(1|Subj)+(1|Item), family=binomial, data = short-vs-long
Eye-Tracking Data
The eye-tracking data were coded as the log of the ratio between looks to the correct picture and looks to the incorrect picture for each time bin for each participant, and then aggregated to generate a function of time during the critical sentences for each treatment group (Training Size × Voice). Figure 4 shows the changes in the eye-gaze patterns between before (top) and after (bottom) the PI training for each Training Size (24 vs. 48 vs. 96) × Voice variability (Single vs. Multi). The midline ( = 0) indicates that the ratio of looks is 50:50. An upward deviation from this midline would indicate more looks to the correct picture, whereas a downward deviation would indicate more looks to the incorrect picture.
Eye-gaze patterns before (top) and after (bottom) PI training.
As in Wong and Ito (Reference Wong and Ito2018) and Ito and Wong (Reference Ito and Wong2019), all groups showed an increase in looks to the incorrect picture (reflecting the bias to interpret the sentence-initial subject as the agent) while hearing the causative sentences before the PI training. The upper panels of Figure 4 show a steady downward deviation from 0 throughout the sentences. As in studies by Wong and Ito, the functions after the PI training showed a recovery from this bias, and this trend was visible across training sizes (Figure 4, bottom). Closer inspection of the functions suggests that the degree of upward movement (i.e., less looks to the incorrect picture) may have been earlier and larger for the multivoice training group (blue functions) than for the single-voice training group (green functions) within each training size. The degree of recovery appeared rather small for the single-voice groups who received medium and long trainings.
Results of linear mixed-effects regression models (LMER) that included Training Size (short, medium, and long, where the reference level is set to medium), Block (Before vs. After), and Voice (Single vs. Multi) as predictor factors showed a main effect for Short Training (z = 2.32, p < .05), a marginal effect for Voice (z = 1.96, p = .05), an interaction between Short Training and Voice (z = –2.38, p < .05), and an interaction among Short Training, Block, and Voice (z = –2.13, p < .05) for the 500 ms after the sentence offset, where the reduction of incorrect looks was most prominent. Thus overall, the short training and multivoice training independently led to relatively more correct looks immediately after the target sentence was heard, while the change in looks for this window as a result of PI had complex interactions. Pairwise comparisons showed that increasing the training size from short (24) to medium (48) led to overall less correct looks (Table 7, Training); that the effect of Voice was larger for the medium group (Table 7, Training:Voice), and that the difference in the change in the looks due to Voice was larger after medium-size training (Table 7, Training:Block:Voice). Increasing training from medium (48) to long (96) made the effect of Voice smaller (Table 8, Training:Voice). Interestingly, changing the training size from short (24) to long (96) did not affect the effects of predictor factors (Table 9). Analyses of Block and Voice effects within each training group revealed the effect of Block in the last window in the short training groups (Table 10), the effect of Voice in the medium groups (Table 11), and null effects of these two factors in the long groups (Table 12).
Post-hoc comparison of Window6 (Sentence Offset+500ms) between Short (24) and Medium (48) size training
Elog_w6~Training+Block+Voice+Training:Block+Training:Voice+Voice:Block+Training:Block:Voice+(1|Subj)+(1|Item)
Post-hoc comparison of Window6 (Sentence Offset+500ms) between Medium (48) and Long (96) size training
Elog_w6~Training+Block+Voice+Training:Block+Training:Voice+Voice:Block+Training:Block:Voice+(1+Block|Subj)+(1|Item)
Post-hoc comparison of Window6 (Sentence Offset+500ms) between Short (24) and Long (96) size training
Elog_w6~Training+Block+Voice+Training:Block+Training:Voice+Voice:Block+Training:Block:Voice+(1+Block|Subj)+(1|Item)
Short Training (24) eye-tracking data analysis: Elogit~Voice*Block+(1+Block|subj)+(1+Voice|item)
Medium Training (48) eye-tracking data analysis:
Long Training (96) eye-tracking data analysis:
To check whether voice variability in the training modulated the changes in looks from the prepositional phrase (PP) à Marie/Pierre), the data were compared between the last two windows (PP vs. 500 ms after sentence offset) of the after PI-training block within each group. The results of LMER models confirmed the robust effect of Window in the short training group, a robust interaction between Window and Voice in the medium training group, null effects of Window and Voice, and a null interaction between the two in the long training group (Table 13). Thus, looks to the incorrect picture decreased (i.e., the looks to the correct picture increased) significantly from the sentence final PP to the next 500 ms window regardless of voice variability after a short PI training, whereas the increase in looks to the correct picture was much larger for the multivoice-trained group after the medium size training. After the long PI training, looks to the incorrect picture did not decrease to a statistically reliable degree regardless of voice variability.
Reduction of incorrect looks from PP (á Marie/Pierre) to Window6 (500ms after sentence offset): lmer(Elog~Voice+Window+Voice:Window+(1+Window|Subj)+(1+Voice|Item), data=x, weights=Weight)
In sum, the eye-tracking patterns collected across the groups showed rather surprising effects of training size. Strikingly, the training with 96 items showed neither the effect of PI training (as the effect of Block) nor the effect of exposure to multiple voices, and the looks to the incorrect picture did not decrease after training. The effects of PI training and voice variability were more visible in the shorter training groups: importantly, the reduction of incorrect looks (i.e., Block effect) was confirmed with only 24 items of training. In addition, both training groups showed the recovery from the bias toward incorrect looks, that is, Short PI training was effective regardless of the voice variability. Participants who received 48 items seemed to have benefitted from the training with multiple voices to recover from the bias. Those who received 96 items in multiple voices showed a numerical trend of recovery from the bias, yet overall, long PI training did not lead to statistically reliable changes in responses to the target sentences.
Discussion
The present study was designed to test the interaction between voice variability and the number of PI training items, based on the assumption that learners may need more training items to benefit from PVI because PVI may require more cognitive resources for processing input when learning a novel L2 structure. Our results demonstrated a set of rather unpredicted patterns. In what follows we examine the present data in light of each of our three research questions.
RQ1. Does PVI used in PI training facilitate the processing of a novel causative structure in French learners? The picture-selection accuracy data showed no clear evidence of a facilitative effect of PVI for the target causative sentence comprehension. The numerical trends of the voice effect were not consistent across PI sizes and statistically unreliable. The eye-tracking data analyses, however, may have revealed the effect of exposure to PVI that could not be captured by the mouse-clicking accuracy data. The gaze functions exhibited a consistent trend of reduction in incorrect looks toward the end of the sentence after PI of different sizes, and the reduction was statistically confirmed after the Short and Medium training. Importantly, after the Short training, those who received multivoice training exhibited a comparable degree of recovery as those who received single-voice training. Furthermore, those who had a Medium-size training in multiple voices showed a larger reduction of the incorrect looks than those who received the same amount of training in a single voice. The present data suggest that multivoice training can be beneficial to L2 learners for learning a new syntactic structure if it is given in a relatively shorter training.
Importantly, the mouse-click data and the eye-tracking data analyses demonstrate that PVI in PI training does not hinder learning a novel syntactic structure. Even with the shortest training of just 24 items (12 in causative and 12 in noncausative) with only six sentences in each voice, the intermediate-level L2 French learners could benefit from PI training with PVI at least to the equivalent degree as those who received training in a single voice. Even after the large amount of training with PVI, participants achieved the equivalent level of picture-selection accuracy as those who were trained with a single voice. Using PVI in PI training per se does not burden L2 learners. In fact, it may facilitate online comprehension.
RQ2. How does the size of auditory PI training modulate the degree of learning? To our surprise, the mouse-click data showed no linear relationship between PI training size and picture-selection accuracy. It was rather striking that the degree of improvement in the picture selection was equivalent between the Short training group and the Long training group (each with feedback for each response). The present data suggest that a rather small amount of training of just 24 items, with only a half containing the target structure, was beneficial to the L2 French learners as much as was a longer training with 96 items. In the context of the present study, the size of training items was certainly not a predictor of the degree of improvement in picture-selection accuracy.
As for the eye-tracking data, the present data only showed the potential hindering effect of a long training. The data from Short and Medium size training groups replicated the general effect of PI training on the reduction of incorrect looks, which was previously shown with both written (Wong & Ito, Reference Wong and Ito2018) and auditory input (Ito & Wong, Reference Ito and Wong2019). This effect was absent in the Long training groups; either in the before versus after PI-training comparison or in the sentence-final PP versus post-sentence-offset comparison. We speculate that a long PI training (96) may have washed away the potential changes in real-time sentence processing. If PI training above a certain amount can lead to less sensitive responses to incoming auditory input, it may negatively impact the acquisition of the target structure.
RQ3. Does the effect of PVI interact with the size of training? While the picture-selection accuracy did not show a consistent pattern of interaction between PVI and PI training size, the eye-tracking data showed the interactions between the two factors in rather unpredicted directions. L2 learners did not benefit from PVI only if they received larger amounts of structured input. Learners also did not perform worse with PVI when the training was short. On the contrary, learners seemed to benefit from both single-voice input and multivoice input and showed the reduction of the incorrect looks when the PI training was short, whereas when the training size was doubled to 48, a clear recovery was exhibited only when learners had multivoice input. When the training was long, learners who received multivoice input only showed a numerical trend of recovery while those who had single-voice training did not show this trend at all. Taken together, the present data suggest that the benefit of PVI may be diluted when the amount of the training exceeds the cognitive burden threshold in L2 learners.
The present study aimed to advance our understanding of two critical factors for L2 acquisition: voice variability in training materials and the amount of training. Our results confirm the effectiveness of PI and offer some insight into implementing this intervention more efficiently.
First, the current results demonstrated facilitative rather than hindering effects of PVI in the eye-movement patterns. Our interpretation of the present data is generally in line with the previous claims about the positive effect of PVI (Bradlow & Bent, Reference Bradlow and Bent2008; Goldinger, Reference Goldinger, Johnson and Mullennix1997; Hardison, Reference Hardison2003; Sommers & Barcroft, Reference Sommers and Barcroft2011): phonetically variable training leads to richer representation of the target structure, resulting in faster recovery from the bias toward interpreting the sentence-initial subject as the agent of the visibly executed action. Importantly, the present data showed no negative impact of PVI, which would have been predicted from some previous studies (Lim et al., Reference Lim, Qu, Tin and Perrachoine2019; Mullennix et al., Reference Mullennix, Pisoni and Martin1989; Wiener et al., Reference Wiener, Ito and Speer2018, Reference Wiener, Ito and Speer2021).
One important factor that differentiates the present study from studies that reported hindering effects of PVI is the level of processing: syntactic parsing versus lexical access. We speculate that PVI does not impact structural representations as directly as it affects lexical semantic representations. When people learn new words for symbols or objects, PVI may increase the burden for tracing the multiple associations between the phonetic traits and the linked symbol/object. In this many-to-one mapping process, the higher number of traces may make it harder to find the exact trace to reach the associated symbol or object until the common phonetic properties are extracted and the activation of those traits becomes automatic. Wiener et al. (Reference Wiener, Ito and Speer2021) explain probability-based errors in word recognition as a result of the uncertainty in bottom-up phonetic processing compensated by the reliance on top-down statistical processing.
When a novel syntactic structure is introduced with PVI across various propositions or scenes, the association between the voice and the target structure is more abstract because the sentential semantics change for each phonetically variable input: that is, the mapping is many-to-many. Studies have demonstrated that the exact syntactic structure is not traced in detail or it tends to flee rather quickly even in native speakers’ sentential processing (Christianson et al., Reference Christianson, Williams, Zacks and Ferreira2006; Ferreira, Reference Ferreira2002; Ferreira & Patson, Reference Ferreira and Patson2007). We speculate that L2 syntactic representation can be more unstable, especially for a novel structure: while the learners may have noticed that their initial interpretation was incorrect, they may not have completed a detailed analysis of the actual syntactic structure. This is indicated by the timing of the eye movement. If learners correctly traced the syntactic details for causative interpretations, the upward curve in the gaze functions would have appeared right after the earlier critical cue fait + infinitive verb. However, they appeared only after the later cue. We suspect that PI training evoked the revision of initial sentential interpretation, and PVI may have assisted tracing such revisions at a rather abstract level without retaining the structural details.
We also speculate that the processing cost for the French causative structure may be high for even native speakers. An eye-tracking study by Lee and Doherty (Reference Lee and Doherty2019) found that processing passive sentences in Spanish was more challenging than processing active sentences for both nonnative and native speakers. They showed that the native speakers’ interpretation accuracy was 95% for the active and 88% for the passive sentences. Like the French causative structure, the passive construction in Spanish is also affected by the FNP. Thus, we may observe similar processing costs to French native speakers processing the French causative structure. Our future study plans to test whether native speakers display a reduction of looks to the incorrect picture when they hear the verb fait followed by an infinitive verb or if this shift of looks happen more slowly due to the processing cost associated with this structure.
As for the amount of training, the present study provides informative data for PI application in the auditory mode. Our picture-selection accuracy data showed no difference in the degree of improvement across the three sizes of PI training, which partially echoes the results by Fernandez (Reference Fernandez2008) and White (Reference White2015). Our Short 24-item training was slightly smaller than Fernandez’s (who had 30 items) yet this Short training led to a robust improvement in picture selection accuracy. Our Long training (96) was similar to White’s second-longest training (n = 100), and this led to the same level of improvement. Our data further revealed that the longest training failed to lead to changes in real-time responses to speech input. Given that increasing the number of training items from Short to Medium and from Medium to Long did not affect the magnitude of accuracy improvement, we predict that auditory training with items beyond 96 would not benefit L2 learners of French. Auditory processing can impose a qualitatively different cognitive burden on L2 learners as compared to text processing, and therefore the optimal amount of PI training may differ across modalities. While White (Reference White2015) suggests that learners may benefit from PI training with 140 items presented in both auditory and written form, our data suggest that auditory training beyond some amount between 48 and 96 items may not yield effective L2 syntactic learning.
This finding also has implications for the implementation of PI. One of the original guidelines by Lee and VanPatten (Reference Lee and VanPatten2003) recommended the use of PI in the classroom with both aural and written SI activities. The present results show that making L2 learners process a large quantity of aural input could tax their processing resources and lead them to become less sensitive to incoming cues. What our study cannot address is whether the negative effect of a long auditory PI training would disappear if written SI activities were also used. It is also necessary to investigate whether a large amount of exclusively written SI activities would also have negative effects on online input processing. We do know from Wong and Ito (Reference Wong and Ito2018) that exclusively written SI activities with 48 items has no negative effect on auditory online processing. We also speculate if we would have found the same negative effect for the 96-item training if we had split up the training into two parts of 48 items. Splitting up the large training over 2 days or on the same day with a distractor task might have lessened the cognitive load. These questions point to the need to examine more closely how the modality and spacing of training might impact the effectiveness of PI.
Because this study was not able to include delayed posttests, we do not know how the training size and voice variability may impact the learning of this structure in the longer term. While the 96-item training did not yield greater improvement in the short term, could it lead to more durable results in the longer term? Norris and Ortega (Reference Norris and Ortega2000) (among others) point out that the effects of more implicit types of training may need more time to show up. While the eye-tracking results showed that the larger training size made participants less sensitive to incoming auditory input, would this negative effect still appear if participants were tested a few weeks later? Future PI research needs to explore the span between treatment and testing.
The present study showed a rather low ceiling level of improvement in the picture-selection accuracy across different sizes. This merits further discussion. While PI training of any of the three sizes improved the picture-selection accuracy from 5–10% to 40–50% ranges, the after-training performance was not robustly higher than the chance level of the dichotomous picture selection. Thus, it is reasonable to question whether the intermediate L2 French learners learned the novel causative structure. Or were they simply confused?
Based on accuracy levels in the after-training blocks and the lack of earlier real-time responses to the critical grammatical cue (fait + infinitive verb) in the eye-tracking data, we argue that the effect of FNP remained robust and participants did not achieve a solid understanding of the target causative structure throughout the course of the study. They most likely began noticing the structural novelty of the causative items and started attending to their sentential meaning. We reject the possibility that learners were confused by the PI training because the ceiling level of performances for the noncausative sentences was maintained after the PI training. If participants simply thought that they could be wrong for interpreting the subject as the agent all the time, then such confusion would have led to lower performances for the noncausative sentences after training. Clearly, this was not the case. Participants were still correctly interpreting the sentential initial noun as the agent for the noncausative sentences after the PI training, that is, the changes were only for the target causative sentences. We take this as evidence for the effectiveness of PI.
In addition to not being able to measure the precise degree of syntactic analysis that the participants might have engaged in, the present study is also limited in two other ways. First, unlike Barcroft and Sommers (Reference Barcroft and Sommers2005, Reference Barcroft and Sommers2014), Perrachione et al. (Reference Perrachione, Lee, Ha and Wong2011), Sadakata and McQueen (Reference Sadakata and McQueen2014), and Mora-Plaza et al. (Reference Mora-Plaza, Ortega and Morain press), the present study did not test cue validity for the target structure or the effect of individual perceptual aptitude and cognitive ability such as attentional control. To test these aspects of PVI processing, future experiments need to manipulate the acoustic clarity of the critical cues and include a measure of individual perceptual sensitivity or cognitive functions. Second, we could not identify the exact amount of auditory training that would start to dilute the effect of PVI. Although we do not have a straightforward answer to how much is too much, our data offers at least one piece of good news to L2 learners: listening to multiple speakers is effective, even when the training is short.
Conclusion
The present data confirmed that the tendency to interpret the sentence-initial noun as the agent is resilient in L2 spoken sentence processing. While the exact syntactic details of the novel structure may not be fully grasped, a short PI training can trigger the revision of incorrect interpretations of the causative structure in French. PVI is shown to be effective to help the structural differentiation of a novel structure. For auditory PI training, longer training may not be efficient, or it can even hinder the learning process because listening to many voices over many sentences may burden L2 processing capacity and weaken the benefit of PVI. Thus, sometimes less is more.
