Learning a new language in adulthood can present challenges. One challenge that often arises is learning to perceive and produce the new language's sounds. A well-studied example is the difficulty native Japanese (NJ) speakers have with the English sounds /r/ as in rock and /l/ as in lock. Theories about the source of this difficulty vary (e.g., Flege, Reference Flege, Burmeister, Piske and Rhode2002; Kuhl, Reference Kuhl, deBoysson-Bardies, de Schonen, Jusczyk, McNeilage and Morton1993; Lenneberg, Reference Lenneberg1967), furthering interest in this topic as a means of identifying constraints on adult language learning (Flege, Takagi & Mann, Reference Flege, Takagi and Mann1996; Guion, Flege, Akahane-Yamada & Pruitt, Reference Guion, Flege, Akahane-Yamada and Pruitt2000; Takagi & Mann, Reference Takagi and Mann1995).

Additionally, a considerable amount of effort has been directed at targeted interventions that aim to teach participants to differentiate non-native contrasts reliably (e.g., Jamieson & Morosan, Reference Jamieson and Morosan1986; Strange & Dittman, Reference Strange and Dittman1984). These studies, while demonstrating that improvement is possible, have also served to highlight the difficulty NJ listeners have with English /r–l/ (Bradlow, Akahane-Yamada, Pisoni & Tohkura, Reference Bradlow, Akahane-Yamada, Pisoni and Tohkura1999; Bradlow, Pisoni, Akahane-Yamada & Tohkura, Reference Bradlow, Pisoni, Akahane-Yamada and Tohkura1997; Iverson, Hazan & Bannister, Reference Iverson, Hazan and Bannister2005; Lively, Logan & Pisoni, Reference Lively, Logan and Pisoni1993; Lively, Pisoni, Yamada, Tohkura & Yamada, Reference Lively, Pisoni, Yamada, Tohkura and Yamada1994; Logan, Lively & Pisoni, Reference Logan, Lively and Pisoni1991; McCandliss, Fiez, Protopapas, Conway & McClelland, Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002).

Strange and Dittman (Reference Strange and Dittman1984) made an early attempt to train NJ listeners to distinguish /r–l/. Their stimuli were synthetic items from rake–lake and rock–lock continua (MacKain, Best & Strange, Reference MacKain, Best and Strange1982). NJ listeners were trained to discriminate stimuli drawn from one of the two continua, then were tested on their ability to discriminate pairs from both continua. Listeners showed evidence of learning via improved discrimination on both continua but failed to reliably discriminate untrained natural speech /r–l/ minimal pairs (e.g., right–light).

McCandliss et al. (Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002) also used synthetic stimuli, but these were instances of rock–lock and road–load produced by one male native English (NE) speaker that were modified to emphasize the initial contrast. Participants in this task were trained to identify stimuli on one continuum then tested on their ability to identify and discriminate stimuli from both continua. The NJ listeners in this study better identified and discriminated both the trained and untrained continua at the post-test. Generalization to natural speech was not assessed, but improvement is unlikely (see Strange & Dittman, Reference Strange and Dittman1984).

The improvement seen in the above training studies has generally not been viewed as truly general speech perception learning given that the NJ listeners (in cases where this was tested) were unable to reliably differentiate natural speech /r–l/ minimal pairs. One response to this has been to raise the possibility that NJ listeners would be better able to learn the characteristics of English /r–l/ categories via training using natural speech /r–l/. If these natural speech stimuli were produced by a variety of NE speakers in a large number of contexts, the greater acoustic variability might enable the NJ listeners to learn those acoustic properties that reliably differentiate /r–l/. This approach was pursued by Pisoni and colleagues (Bradlow et al., Reference Bradlow, Akahane-Yamada, Pisoni and Tohkura1999; Bradlow et al., Reference Bradlow, Pisoni, Akahane-Yamada and Tohkura1997; Lively et al., Reference Lively, Logan and Pisoni1993; Lively et al., Reference Lively, Pisoni, Yamada, Tohkura and Yamada1994; Logan et al., Reference Logan, Lively and Pisoni1991). The stimuli in their studies were naturally produced instances of /r–l/ minimal pairs spoken by several NE speakers, both male and female. NJ listeners were trained to identify the words with feedback. Testing occurred via identification of trained and untrained tokens produced by talkers used in training and talkers not used in training (both types of talkers produced instances of trained and untrained tokens). Unlike the Strange and Dittman (Reference Strange and Dittman1984) study, NJ listeners in these studies showed both improvement on trained materials and generalization to untrained natural speech stimuli – both untrained words produced by talkers used in training and untrained words produced by talkers not used in training. However, even after training their identification performance still fell well below NE levels.

One possibility for these incomplete success stories is an inherent limitation in the ability of adult NJ listeners to learn the distinction (Takagi, Reference Takagi2002; Takagi & Mann, Reference Takagi and Mann1995), possibly reflecting a broad, age-dependent cessation of plasticity for this aspect of language learning (Johnson & Newport, Reference Johnson and Newport1989). While certainly this possibility is consistent with results to date, there remains an alternative: the reason for the incomplete success may lie in the fact that the cues NJ speakers learn to utilize may not be the crucial cue NE speakers use to differentiate /r–l/. In studies using synthetic speech (McCandliss et al., Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002; Strange & Dittman, Reference Strange and Dittman1984) listeners may learn to rely on cues that distinguish the training stimuli but are not robust cues to the /r–l/ contrast across the full range of natural /r–l/. This would account for high levels of performance on the training stimuli but poor generalization to natural speech. In studies using natural speech from a range of speakers (Lively et al., 1991; Logan et al., 1993; Logan et al., 1994) there may be a similar difficulty. NJ participants may learn to rely on a variety of partial cues that weakly covary with the /r–l/ contrast but which are nevertheless imperfect cues to the /r–l/ distinction. This would explain why trained participants show a real and persistent generalizable learning effect (Bradlow et al., Reference Bradlow, Pisoni, Akahane-Yamada and Tohkura1997; Bradlow et al., Reference Bradlow, Akahane-Yamada, Pisoni and Tohkura1999), but where final attainment is non-native-like. Similar to natural-speech training, long-term immersion may also result in reliance on a variety of partial cues, explaining why NJ speakers with extensive immersion experience with English also show improved but non-native levels of performance (Aoyama, Flege, Guion, Akahane-Yamada & Yamada, Reference Aoyama, Flege, Guion, Akahane-Yamada and Yamada2004; Gordon, Keyes & Young, Reference Gordon, Keyes and Yung2001).

There are several acoustic cues to /r–l/ category membership, the most well documented being the onset frequency of the third formant, F3, and the closure and transition duration of the first formant, F1, with F3 onset frequency being the most consistently reliable indicator of category membership (Espy-Wilson, Reference Espy-Wilson1992; O'Connor, Gertsman, Liberman, Delattre & Cooper, Reference O'Connor, Gerstman, Liberman, Delattre and Cooper1957). Instances of /r/ are typified by F3 onsets below the vowel steady state and long F1 closures followed by short transitions to the vowel steady state; instances of /l/ are typified by F3 onsets equal to or above the vowel steady state and short F1 closures followed by longer transitions to the vowel steady state. NE speakers make use of both of these cues when perceiving /r–l/ (Gordon et al., Reference Gordon, Keyes and Yung2001; O'Connor et al., Reference O'Connor, Gerstman, Liberman, Delattre and Cooper1957; Polka & Strange, Reference Polka and Strange1985; Underbakke, Polka, Gottfried & Strange, Reference Underbakke, Polka, Gottfried and Strange1988). However, NE listeners place the greatest weight on F3 and changes in F3 alone are sufficient to shift NE listeners’ responses from /r/ to /l/ (Iverson, Kuhl, Akahane-Yamada, Diesch, Tohkura, Ketterman & Siebert, Reference Iverson, Kuhl, Akahane-Yamada, Diesch, Tohkura, Kettermann and Siebert2003; Miyawaki, Strange, Verbrugge, Liberman, Jenkins & Fujimura, Reference Miyawaki, Strange, Verbrugge, Liberman, Jenkins and Fujimura1975; O'Connor et al., Reference O'Connor, Gerstman, Liberman, Delattre and Cooper1957; Yamada & Tohkura, Reference Yamada and Tohkura1990). NE speakers also weight F3 heavily in production, emphasizing this cue's importance (Lotto, Sato & Diehl, Reference Lotto, Sato, Diehl and Slifka2004).

Conversely, NJ listeners appear to rely more heavily on less reliable cues, most notably the onset frequency of the second formant, F2, in both perception and production (Iverson et al., Reference Iverson, Kuhl, Akahane-Yamada, Diesch, Tohkura, Kettermann and Siebert2003; Lotto et al., Reference Lotto, Sato, Diehl and Slifka2004; Yamada & Tohkura, Reference Yamada and Tohkura1990). NJ listeners have also been shown to be sensitive to closure duration and transition duration when perceiving /r–l/ (Aoyama et al., Reference Aoyama, Flege, Guion, Akahane-Yamada and Yamada2004; Hattori & Iverson, Reference Hattori and Iverson2009; Underbakke et al., Reference Underbakke, Polka, Gottfried and Strange1988). Importantly, NJ listeners’ accurate identification of natural speech /r–l/ is best predicted by their use of the F3 cue (Gordon et al., Reference Gordon, Keyes and Yung2001; Hattori & Iverson, Reference Hattori and Iverson2009), suggesting that greater reliance on this cue might result in more NE-like performance.

Iverson and colleagues (Iverson et al., Reference Iverson, Hazan and Bannister2005) sought to correct this disparity in perceptual cue weightings. Their stimuli were based on the high variability natural speech stimuli described above (Bradlow et al., Reference Bradlow, Pisoni, Akahane-Yamada and Tohkura1997; Bradlow et al., Reference Bradlow, Akahane-Yamada, Pisoni and Tohkura1999; Lively et al., Reference Lively, Logan and Pisoni1993; Lively et al., Reference Lively, Pisoni, Yamada, Tohkura and Yamada1994; Logan et al., Reference Logan, Lively and Pisoni1991). They manipulated these tokens to increase the salience of F3 onset frequency. Despite these efforts to emphasize F3 onsets, the results were very similar to those found in earlier work: individuals were better able to identify /r–l/ tokens by all talkers, but not at the levels of NE listeners. They also found no changes in F3 sensitivity following training. This may be due to the continued presence of cues in the training stimuli that may be more salient to NJ listeners – cues that are partially consistent with but not the most reliable indicators of /r–l/ category membership (Lotto et al., Reference Lotto, Sato, Diehl and Slifka2004; O'Connor et al., Reference O'Connor, Gerstman, Liberman, Delattre and Cooper1957; Yamada & Tohkura, Reference Yamada and Tohkura1990).

It is apparent that though F3 onset frequency is the most reliable cue to /r–l/ category membership, NJ listeners have difficulty relying on this cue to differentiate /r–l/ even after training (Iverson et al., Reference Iverson, Hazan and Bannister2005) or after immersion in English (Aoyama et al., Reference Aoyama, Flege, Guion, Akahane-Yamada and Yamada2004; Gordon et al., Reference Gordon, Keyes and Yung2001). It may be that the presence of cues other than F3 (e.g., F1 and F2) in training stimuli and in natural speech allows NJ listeners to rely on less reliable cues. In this article we consider the possibility that the presence of non-F3 cues in other training studies (and in natural immersion) might allow NJ speakers to rely on cues other than F3. From that perspective, we examine whether focusing all variation amongst /r–l/ tokens on F3 onset frequency will allow NJ speakers to learn to rely on F3. Specifically, if all remaining cues are equivalent among training stimuli, leaving only F3 onset frequency as a cue to differentiate among instances of /r/ and /l/, this may lead NJ learners to learn to rely on this cue, and this in turn should result in NE-like identification and discrimination performance. Our hope was that we might then see both the high levels of improvement on trained stimuli and generalization to untrained natural speech /r–l/ that have eluded previous efforts.

Experiments 1a and 1b

We constructed four synthetic /r–l/ series differing only on vowel: /ra–la/, /ræ–læ/, /ri–li/ and /ru–lu/, training participants with stimuli drawn from the /ra–la/ series and testing all participants with all four vowel contexts to assess generalization.Footnote ¹ Within each series, only the F3 onset frequency varied; all other formants and transition durations were held constant.

We relied on a training procedure similar to that used in the condition of McCandliss et al. (Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002) in which NJ participants exhibited the greatest improvement in English /r–l/ perception. This condition relied simply on repeated presentations of two fixed, moderately difficult stimuli with feedback. Based on the native-listener identification curves shown in Figure 2 below, we adopted Stimulus 4 and Stimulus 12 (circled in the figure) as moderately difficult /r/ and /l/ stimuli in Experiment 1a. As we shall see, participants in Experiment 1a did not improve from pre- to post-test. Therefore, in Experiment 1b, we used Stimulus 1 and Stimulus 16 as training stimuli. Since the results were similar in these two experiments, we present them together in the following section.

In addition to testing for improvements in identification for trained and untrained vowel contexts and natural speech, we looked for a shift to more NE-like discrimination post-training. This would be marked by an increase in discrimination accuracy at the /r–l/ category boundary (series middle) relative to within-category discrimination (series end).

Method

Participants

Sixteen NJ volunteers were recruited from the Pittsburgh area and participated in return for payment (see also footnote 1). As described in Footnote 1, data from those four individuals trained on /ri–li/ are not reported here. All reported normal hearing. There is no information regarding musical ability or years English was studied.

Participant eligibility was judged by performance in an English /r–l/ discrimination pre-test. Those participants scoring greater than 70% correct were excluded from participating in the remainder of the experiment (McCandliss et al., Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002). Four participants were excluded on this basis, noting that no participant performed at NE-like levels (ceiling). The first four eligible participants were used in Experiment 1a; the next four were used in 1b. Comparisons of eligible versus ineligible participants on age (31.75 vs. 30.25 years, t(9) = 0.47, p = .65), length of residency in North America (2.48 vs. 1.56 years, t(9) = 0.77, p = .46), age of first learning English (12.62 vs. 12.5 years old, t(6) = 0.36, p = .73), and self-reported ratios of spoken English to spoken Japanese (1.81 vs. 0.84 English/Japanese ratio, t(9) = 0.59, p = .57) revealed no across-group differences. Within each experiment, participants were divided equally into trained or untrained groups; group assignments were random.

Materials

Synthesized speech stimuli

Four 16-step synthesized consonant–vowel (CV) speech series varying from English /r/ to /l/ were created. The series were distinguished by the vowel, /a/, /æ/, /i/, and /u/. Within a series, only the third formant (F3) onset frequency distinguished members of the series. Stimuli were sampled at 11025 Hz and RMS matched in amplitude.

Syllables were synthesized using the parallel branch of the Klatt synthesizer (Klatt, Reference Klatt1980; Klatt & Klatt, Reference Klatt and Klatt1990). Each stimulus was 330 ms in total duration, with silence for the first 10 and last 5 ms. The fundamental frequency (f0) was a constant 110 Hz. The first and second formants (F1 and F2) had onset frequencies of 478 and 1088 Hz, respectively, and held these values across 85 ms at which time they linearly transitioned to the vowel steady-state frequency across 95 ms. F1 amplitude transitioned linearly from 0 to 50 dB across 35 ms whereas F2 amplitude transitioned linearly from 0 to 55 dB across 70 ms.Footnote ² The fourth formant (F4) had a steady-state value of 3850 Hz across the duration of the sound. The amplitude of F4 transitioned linearly from 0 to 20 dB across 125 ms.

Within series, stimuli were distinguished by the F3 onset frequency, which varied from 1601 to 3400 Hz in increments of 43 Mel steps. F3 was steady-state at these values for 65 ms, then linearly transitioned to 2530 Hz across 115 ms. It remained at this frequency for the duration of the stimulus. F3 amplitude at stimulus onset covaried with onset frequency, varying from 60 (at 3400 Hz) to 45 dB (at 1601 Hz) in 1 dB steps. F3 onset amplitude began linearly transitioning to the vowel steady state (60 dB) at 65 ms and reached 60 dB at 180 ms.

The four /r–l/ CV series were distinguished by the final vowel. The vowels /a/ and /æ/ shared an F1 frequency of 705 Hz whereas /i/ and /u/ shared an F1 frequency of 205 Hz. F2 frequency for /i/ and /æ/ was 2005 Hz. The vowels /a/ and /u/ shared a F2 steady-state frequency of 1035 Hz, but /u/ F2 began at 1450 Hz (180–210 ms) before linearly transitioning over the next 50 ms to the steady-state value.Footnote ³ All steady-states and transitions had identical durations both across and within CV series, removing duration as a possible cue to category membership (Aoyama et al., Reference Aoyama, Flege, Guion, Akahane-Yamada and Yamada2004; Iverson et al., Reference Iverson, Hazan and Bannister2005). Thus, /a/ and /æ/ vowel contexts differed from one another along the F2 dimension whereas /æ/ and /i/ differed from one another along the F1 dimension. This orthogonality provided the opportunity to examine the effects of each dimension on generalization. Note that /a/ and /i/ differ from one another along both dimensions (as do /æ/ and /u/). Pseudo-spectrograms of the synthesis parameters can be found in Figure 1.Footnote ⁴

Figure 1.

Endpoints of the /r–l/ series for each of the four (/a/, /æ/, /i/, and /u/) vowel contexts representing onsets and trajectories of the first through fourth formants. Stimuli within a vowel context differed from one another in only F3 onset frequency.

To assure that these synthesized stimuli were reliably labeled as /r/ and /l/, 13 NE monolingual listeners responded to 15 repetitions of each of the 64 stimuli (4 series × 16 stimuli) as “r” or “l”, presented in random order mixed across vowel context. Identification curves are shown in Figure 2, demonstrating reliable, if imperfect, identifications as /r/ and /l/ by English listeners for these stylized synthetic speech stimuli. The imperfect identifications at the /l/ end of the series may be due in part to the high F3 onset frequencies for /l/ (3400 Hz at the most extreme). This value is within the range found in natural productions (Lotto et al., Reference Lotto, Sato, Diehl and Slifka2004) but is higher relative to the vowel than is typical (O'Connor et al., Reference O'Connor, Gerstman, Liberman, Delattre and Cooper1957). However, an examination of the data at the individual level indicates that most of the listeners reliably divided the stimuli into /r/ and /l/ categories and what appears to be imperfect categorization at the /l/-end of the series is driven by two listeners who identified most of the stimuli as /r/.

Figure 2.

Proportion /r/ response for each stimulus by native English (NE) listeners. Stimuli are presented here separated by vowel context; participants heard all stimuli intermixed. Those stimuli that were used for training in Experiment 1a are marked with large circles. [Editorial Note: See pages 434–5, this issue.]

We used the native English listeners’ identification curves’ to identify their /ra–la/ category boundary (the /ra–la/ curves being the steepest). A proportion /r/ response difference of .50 or greater between two members of a discrimination pair was indicative of a category boundary and called the series middle (stimuli pairings 6–10 and 7–11). The remaining pairings were classified as the series end. Position assignment from the /ra–la/ series was extrapolated to the other vowel contexts. These classifications were used when analyzing the discrimination tests to determine if NJ listeners showed better between-category than within-category discrimination, as would be expected by native English listeners (Liberman, Harris, Hoffman & Griffith, Reference Liberman, Harris, Hoffman and Griffith1957).

Natural speech stimuli

Two lists of 16 /r–l/ English minimal pair words were created, resulting in 32 total pairs. Two native English speakers, one male and one female, produced all words in each list, for a total of four speakers. One male speaker was fluent in German, which he began learning at age 18, with English his only language prior to this. The remaining three speakers identified themselves as monolingual English speakers. The full list of minimal pairs is shown in Table 1. It is divided into four pair types, based on Logan et al. (Reference Logan, Lively and Pisoni1991), based on /r–l/ position: initial singleton (lock–rock; 9 pairs), initial cluster (flesh–fresh; 8 pairs), intervocalic (elect–erect; 7 pairs), and final singleton (file–fire; 8 pairs). Within each list of 16 words, each member of a minimal pair was spoken by a different talker; all 32 words were presented at test.

Table 1.

Minimal pair words to test participants’ perception of natural speech, divided by the position of the /r–l/ contrast.

Talkers produced two exemplars of each word in the sentence, “The next word is _____, _____”. Words were recorded at 11025 Hz on a PC desktop running Windows XP. The second production of each word was chosen as the stimulus. Stimuli were RMS matched in amplitude.

Procedure

McCandliss et al. (Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002) found the greatest improvement when training consisted of repeated presentations of two moderately difficult stimuli combined with performance feedback. This procedure served as the basis for Experiment 1 training, using stimuli in which instances of /r–l/ differed only on F3 onset frequency. Based on the native-listener identification curves shown in Figure 2, we adopted Stimuli 4 and 12 as moderately difficult /r/ and /l/ stimuli, respectively (Experiment 1a, N = 4). These stimuli are marked with circles in Figure 2. Following these listeners’ failure to learn, in a subsequent effort (Experiment 1b; N = 4), we used Stimuli 1 and 16 as training stimuli.

NJ participants began with an eligibility AX discrimination test using the training stimuli (Stimulus 4 versus Stimulus 12 in Experiment 1a; Stimulus 1 versus Stimulus 16 in Experiment 1b) across 25 repetitions. Individuals with discrimination accuracy greater than 70% were deemed ineligible for training (N = 4 ineligible individuals, accuracy M = 74%).

Eligible participants (N = 8, accuracy M = 51%) next completed an identification test of the synthesized speech stimuli, followed by a discrimination test of the same stimuli, then a test of natural speech identification. All participants would repeat these three tests on Day 11, the final post-test day. Additionally, on Day 6 all participants repeated the identification test of synthesized speech and the discrimination test of synthesized speech. The intervening days, Day 2–Day 5 and Day 7–Day 10, were the training days. Those individuals assigned to a training condition received a laptop computer and trained at home. Details of these testing and training procedures are described below. Logs of time-on-task were monitored to assure participants’ compliance with the training program.

All tests were administered under the control of E-Prime (Psychological Software Tools, Pittsburgh PA) on a PC laptop running Windows XP. Stimuli were presented diotically over Beyer DT-150 headphones at approximately 70 dB.

Identification

The identification test included all four synthesized /r–l/ series and was blocked by vowel context; no feedback was given. Participants in Experiment 1a heard 12 presentations of Stimuli 4, 6, 8, 10, and 12 (bounded by training stimuli 4 and 12) along each series and indicated whether the syllable began with /r/ or /l/ by pressing a labeled key on the keyboard. Short breaks were provided between blocks. Participants in Experiment 1b completed the same test for even-numbered series members (Stimuli 0–16; Stimulus 0 was an additional stimulus created to equalize the number of presented stimuli on either side of the series midpoint, Footnote 3). All participants took the identification test on Day 1 (pre-test), Day 6, and Day 11 (post-test).

Discrimination

Immediately following the identification test, an AX discrimination task of the synthesized speech series was administered. On each trial, listeners heard a pair of stimuli made up of items 4 steps apart along the series (e.g. stimuli 4 and 8 were used to create two same pairs: 4–4 and 8–8; and two different pairs: 4–8; 8–4). Listeners heard four presentations of each pair; pair members were separated by a 750 ms ISI. Stimuli 3–13 from all CV series were used in the discrimination test in Experiment 1a; Stimuli 1–16 were used in Experiment 1b. Participants indicated if the pair was “Same” by pressing “1” and “Different” by pressing “0” on the keyboard. Tests were blocked by vowel. Short breaks were provided between blocks. All participants took the discrimination test on Day 1, Day 6, and Day 11.

Natural speech identification

Identification of the natural speech English /r–l/ minimal pairs was assessed at pre- and post-test; identifications were made without feedback. On a given trial, orthographic representations of each word were presented on the computer monitor while the acoustic stimulus was presented once. Listeners used the keyboard to indicate which pair member was presented. Stimuli were not repeated within a test and each member of a minimal pair was spoken by a different talker.

Training

Individuals participating in training conditions completed daily 30-minute sessions at home using PC laptops running Windows XP and Beyer DT-100 headphones provided to them. Participants were given a sheet explaining the details of the daily training regimen in both English and Japanese and each training session was checked for completeness at the mid-test and post-test.

NJ participants trained with /ra–la/ synthesized speech. The training task was a 2AFC identification task. On each trial, listeners heard one of the two training stimuli (Stimulus 4 or 12 in Experiment 1a; Stimulus 1 or 16 in Experiment 1b, differentiated exclusively by F3 onset frequency) and indicated whether it sounded like “r” or “l” by pressing a labeled button on the keyboard. Sounds were not repeated within a trial and training did not advance until a response had been made. Response accuracy feedback was visually presented for 500 ms following each response. In each training session, participants heard 10 repetitions of each training stimulus (Stimuli 4 and 12 in Experiment 1a; Stimuli 1 and 16 in Experiment 1b) in 25 blocks for a total of 500 training trials per day (McCandliss et al., Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002). Each training session took approximately 30 minutes to complete, with short breaks given between blocks.

Participants trained across eight days, not training on the days they took the mid-test and post-test. Thus, they attempted to categorize to 4,000 repetitions of the two training stimuli (2,000 repetitions of each stimulus), with accuracy feedback on each trial.

Experiment 2

It is possible that the presence of other cues (e.g., closure duration and F2), even if they are held constant between instances of /r/ and l/, somehow prevents NJ listeners from being able to attend to the F3 cue (Iverson et al., Reference Iverson, Hazan and Bannister2005; Takagi, Reference Takagi2002; Takagi & Mann, Reference Takagi and Mann1995). In support of this possibility, it is interesting to note that NJ listeners are able to discriminate single, isolated F3 formants like those from minimal /ra–la/ syllables (Miyawaki et al., Reference Miyawaki, Strange, Verbrugge, Liberman, Jenkins and Fujimura1975). These non-speech sounds lack the harmonic structure of lower and higher frequencies to cause them to be heard as speech and are instead heard as non-speech beeps. These data demonstrate that NJ listeners’ difficulty in using F3 onset as a cue to the English /r–l/ contrast does not arise from complete insensitivity to F3 onset frequency. More generally, it is possible that the training paradigm employed in Experiment 1 did not optimally promote learning. Even though Experiment 1b used series endpoints, it may be that these stimuli were not sufficiently distinct for our NJ participants to use as a starting point for learning. Our aim in Experiment 2 was to determine if starting with F3-only stimuli that listeners could differentiate and gradually restoring the remaining formants as performance improved could effect learning to use F3 in the /r–l context; McCandliss et al. (Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002) found improvement using adaptive training both with and without the presence of feedback, leading us to also manipulate this variable. Comparing the results of this experiment to the total lack of improvement in Experiment 1 – which used full-spectrum /r–l/ throughout – would reveal whether F3-only training was beneficial relative to F1–F4 conjunction training where only F3 varied.

Method

Participants

Twenty-six NJ individuals from the Pittsburgh area, all of whom reported having normal hearing, participated in exchange for payment. There is no information regarding musical experience or years of English study. Ten participants (length of residency M = 1.38 years; age at test M = 28 years) were ineligible to participate in training because they were able to discriminate at least one set of the training stimuli from Experiment 1b at an accuracy level of greater than 70% (M = 73%); again, no participant performed at NE-like levels. Comparisons of eligible versus ineligible NJ speakers on age (30.56 vs. 27.70 years, t(16) = 0.95, p = .36), length of residency in North America (1.62 vs. 1.38 years, t(15) = 0.44, p = .67), age of first learning English (12.0 vs. 12.4 years old, t(20) = 0.76, p = .45), and self-reported ratios of spoken English to spoken Japanese (3.31 vs. 0.81 English/Japanese ratio, t(26) = 1.93, p = .06) revealed no significant differences.

Of the remaining 16 participants (length of residency M = 1.62 years; age at test M = 31 years), four were assigned to the untrained group. The testing conditions for Experiment 2 were identical to those of Experiment 1b, so the two Untrained participants from Experiment 1b were also included in the Untrained group for Experiment 2 (we did not include the Untrained participants from Experiment 1a since their testing stimuli excluded the extreme end-points).

The remaining 12 participants were equally divided into feedback and no-feedback groups; group assignment was random. Thus, at test, six participants were trained on /ra–la/ with feedback, six participants were trained on /ra–la/ without feedback, and six individuals received no training. The number of participants per condition, while small, is within the range in earlier work, ranging from 6 trained listeners (Logan et al., 1993; Logan et al., 1994) to 19 trained listeners (Lively et al., Reference Lively, Pisoni, Yamada, Tohkura and Yamada1994).

Materials

The training stimuli were based on the training stimuli in Experiment 1b and created using the parallel branch of the Klatt speech synthesizer (Klatt, Reference Klatt1980; Klatt & Klatt, Reference Klatt and Klatt1990). Stimuli consisting of single-formant F3 trajectories were created from series endpoints (Stimuli 1 and 16). These were created by setting the amplitudes of all non-F3 formants to 0 dB, leaving one stimulus with an F3 trajectory that started at 1601 Hz and another that started at 3400 Hz. These stimuli thus possessed the same F3 as the full-spectrum speech stimuli of Experiment 1, but did not have acoustic energy in frequencies other than F3 and sounded like non-speech beeps.

To implement the adaptive training, twenty additional stimuli were created. For these stimuli, the amplitude of the speech spectrum that was damped to 0 dB in the single-formant F3 trajectories was gradually restored by increasing the amplitudes of F1, F2, and F4 in increments of 10% of their final values until they reached the values of Experiment 1 stimuli (making 10 increases in 10% steps of the amplitude values of Experiment 1 stimuli for each endpoint, resulting in 20 stimuli). Figure 3 shows frequency × amplitude plots when the non-F3 formants are at 0%, 25%, 50%, and 100% of their final values. All modifications were limited to the endpoints of the series.

Figure 3.

Frequency × amplitude plots of the /ra–la/ adaptive training stimuli from Experiment 2 when the other formants are at 0%, 25%, 50%, and 100% of their final values.

Stimuli were sampled at 11025 Hz and RMS matched in overall amplitude to Experiment 1 stimuli. Stimuli were presented to participants in the same manner as Experiment 1.

Procedure

Testing procedure and the number of training sessions and trials per session were identical to Experiment 1b.

Training

As in Experiment 1, each training trial consisted of an auditory stimulus, followed by a 2AFC response. In line with McCandliss et al.'s (Reference McCandliss, Fiez, Protopapas, Conway and McClelland2002) adaptive procedure, half the participants received feedback after each response and half did not receive feedback.

Training began with the presentation of single-formant F3 trajectories that modeled Stimuli 1 and 16 of Experiment 1. Participants were told to respond “r” to the most /r/-like stimulus (F3 onset below the vowel) and to respond “l” to the most /l/-like stimulus (F3 onset above the vowel). Because these stimuli were non-speech and may have been difficult for participants to map onto the “r” and “l” response options, all participants, regardless of training condition (Feedback or No-Feedback) were given feedback when hearing these stimuli to ensure the response options were correctly mapped. From this point forward the stimuli were determined by the participants’ response accuracy. If the participant made six consecutive correct responses, the stimuli became progressively more speech-like. If the participant made one incorrect response, the stimuli became progressively less speech-like. Starting with single-formant F3-trajectories, stimuli were made more speech-like by increasing the amplitude of the non-F3 spectrum by 10%, edging the stimuli toward full-spectrum speech; stimuli were made less speech-like by decreasing the amplitude of non-F3 spectrum by 10%. If the participant made six correct responses to full-spectrum /r–l/, stimuli were then made progressively more difficult by moving closer to the center of the series (Figure 1 above). For example, the first full-spectrum stimuli would be the endpoints, Stimuli 1 and 16. Six consecutive correct responses to these stimuli would move the participant to Stimuli 2 and 15; the participant would then progress to stimuli 3 and 14 or 1 and 16, depending on performance. Overall, the adaptive procedure allowed the stimuli to range from pure isolated F3 formants to full spectrum stimuli in which the difference between F3 onsets of /r/ and /l/ tokens was reduced to only 55.05 Hz; Stimuli 7 vs. 9.

Trials were initiated with a visual fixation cross that preceded acoustic presentation of the sound. Participants identified the sound as “/r/-like” or “/l/-like”. No limits were placed on response time, but participants were encouraged not to deliberate at length. Stimuli were randomized with the constraint that low and high F3 onsets were presented equally often across the six trials required for advancement. Following the response, a blank screen appeared for 500 ms for participants in the No-Feedback condition. Those receiving feedback received an indicator of their accuracy for 500 ms. Each participant completed approximately 500 trials each day (with some variability because the task was adaptive) and began training in the next session with stimuli based on performance at the end of the preceding session.

Results

There are three main findings from this experiment. First, participants in both the Feedback and No-Feedback groups improved over the course of training, with considerable within and between subject variability. Second, three Feedback participants and one No-Feedback participant showed clear evidence of a pre- to post-test transition to NE-like identification on the trained stimulus series, while no Untrained participants showed such changes. Of the four participants showing a training effect, two showed evidence of an acquired discrimination peak at the /ra–la/ category boundary. There was evidence of generalization of the training effect to identification on the /ræ–læ/ series in only one of these participants, but three of the four showed evidence of an improvement from pre- to post-test in the natural speech identification test. Overall, training was a partial success for some participants. We now proceed to present these findings in detail, beginning with the effect of training.

Training

We assigned a difficulty score to each pair of training stimuli, using the percentage of the non-F3 formants present in the stimulus for the ten pairs in which this percentage varied (assuming that the inclusion of additional /r–l/ information would make the stimuli more difficult for NJ listeners), and then assigning difficulty scores incrementing in 10-point steps for the eight remaining pairs of stimuli, which stepped inward from the series endpoints. Thus, the maximum difficulty level was 180, assigned to the stimuli just one step apart the middle stimulus. The average difficulty level across different points in training is shown in Figure 4. Performance improved within sessions and tended to drop from the end of one session to the beginning of the next. The between-session drop shows signs of getting smaller in the later sessions, indicating maintenance of identification performance between sessions later in training.

Figure 4.

Level of difficulty for participants trained either with or without feedback as a function of training day. The y-axis is an indicator of difficulty: contexts where it is more difficult for native Japanese (NJ) listeners to differentiate high and low F3 onset frequencies are further from the origin. The portion of the axis marked “Amplitude” refers to those stimuli where non-F3 formants amplitudes are manipulated; values here indicate the percentage of the non-F3 formants present in the stimuli. The portion of the axis marked “Series” refers to those stimuli where non-F3 formants are fully present but the difference between low and high F3 onset frequencies has been reduced; values here indicate the difference (in Hz) between the onset frequencies.

We submitted the data to a Session (1–8) × Group (Feedback or No-Feedback) mixed-model ANOVA. We found a main effect of session, F(7,27) = 5.17, p < .001, η² = 0.124, indicating that the overall improvement is reliable across participants. Although the No-Feedback group appears to have reached higher difficulty levels on average over the last five sessions, neither the main effect of group nor the group by session interaction were significant. We supplemented the global ANOVA with an ANOVA for a linear trend, performed separately for each training group. Both the Feedback (F(1,27) = 4.36, p < .05, η ² = 0.076) and No-Feedback (F(1,27) = 25.57, p < .001, η ² = 0.449) groups significantly improved over the course of training.

Trained vowel series

Identification

Initial inspection of the pre- and post-test identification data revealed large individual differences among participants, obscuring possible training effects at the individual level. Furthermore, we often found large changes from pre to post-test that differed from the expected pattern of change from a relatively flat pre-test identification function to a relatively steep sigmoidal identification function. We therefore considered the data of each participant separately, looking for evidence of learning at the level of the individual, employing three separate criteria. Only participants meeting all three criteria were treated as showing a true shift from initial insensitivity to F3 to NE-like sensitivity.

The first criterion relied on the chi-square statistic to determine whether there was a reliable difference between pre- and post-test, combining the evidence of a pre- vs. post-test difference at each point along the 16-step stimulus series into a single value of chi square (Agresti, Reference Agresti1992).Footnote ⁵ The second criterion relied on the logistic regression slope to assess whether the change from pre- to post-test could be characterized as an increase in the slope of the identification function. A single z-score quantified the difference between the pre- and post-test slopes for each individual listener:

Comparing these z-scores to the normal distribution provides a significance test for the difference between these slopes. Chi-square test results, pre- and post-test slopes, standard errors, and the resulting z-scores for all listeners on all series can be found in Table S.5. Listeners’ identification curves at the pre- and post-test can be found in Figure S.1 (in supplementary materials), which shows both the extensive variability amongst listeners and the myriad identification patterns seen. Using just these two criteria, three Feedback listeners – listeners Feedback-1 (χ ² = 63.13, p < .001; z = –2.86, p < .05), Feedback-3 (χ ² = 43.13, p < .001; z = –4.94, p < .05), and Feedback-5 (χ ² = 30.96, p < .05; z = –3.58, p < .05) – and two No-Feedback listeners – listeners No-Feedback-4 (χ ² = 32.86, p < .05; z = –3.77, p < .05) and No-Feedback-6 (χ ² = 52.70, p < .001; z = –3.08, p < .05) – all showed an effect of training identifying stimuli in the /ra–la/ series.

Our final criterion is needed to address the pattern seen in participant No-Feedback-6 (Figure S.1). To visual inspection, both the pre- and post-test results show a steep transition in identification performance across the /ra–la/ series. The pre-test logistic regression slope is less extreme than the post-test value, but this is an artifact of the uptick in /r/ responses at the extreme /l/-like end of the series.Footnote ⁶ While such changes may well reflect an influence of training, it is not clear that in this case there has been a change in the participant's sensitivity to differences among stimuli along the /ra–la/ series. To allow separation of such cases, we adopted a final subjective filter to determine if participants who met the first two criteria showed a discernable increase in identification function slope from pre- to post-test. Visual inspection of the other four cases passing the first two criteria – participants Feedback-1, Feedback-3, Feedback-5, and No-Feedback-4 – shows that the canonical pattern is observed in all of these cases. In all four cases, pre-test identification performance is relatively uniform across all stimuli, while post-test performance shows the canonical NE-like pattern.

It is important to note that none of the Untrained participants showed changes that met both the first two criteria. One participant, Untrained-4, did show a significant change in the logistic regression slope (χ² = 21.54, p > .05; z = –3.49, p < .05), meeting one of the criteria. However, this participant's data showed a similarly steep transition in the pre-test, with an uptick in the proportion or /r/ responses at the most extreme /l/-like end of the series. Thus, there is no evidence that any of the Untrained participants showed a significant increase in sensitivity to F3 from pre- to post-test.

Discrimination

We used two criteria to determine whether each participant learned to discriminate across the NE listeners’ /r–l/ category boundary on the trained stimuli: (a) we compared discrimination performance on the middle of the /ra–la/ series at pre- and post-test; and (b) we asked whether post-test discrimination was better at the middle of the series – where a category boundary is spanned – than at the end of the series – where the tokens are within-category. Using paired t-tests to compare pre- and post-test performance on midpoint stimuli (Table S.6), several listeners showed significant improvement: Feedback-2 (t(31) = –2.34, p = 0.02, d = 0.461, pre-test accuracy = 53%, post-test accuracy 75%), Feedback-3 (t(31) = –2.74, p = .03, d = 0.577, pre-test accuracy 38%, post-test accuracy 66%), Feedback-5 (t(31) = –2.74, p = .01, d = 0.577, pre-test accuracy 50%, post-test accuracy 81%), No-Feedback-4 (t(31) = –4.98, p < .001, d = 1.317, pre-test accuracy 47%, post-test accuracy 97%), No-Feedback-5 (t(31) = –2.74, p = .01, d = 0.649, pre-test accuracy 38%, post-test accuracy 69%), and Untrained-2 (t(31) = –2.18, p = .02, d = 0.585, pre-test accuracy 44%, post-test accuracy 72%). Comparing middle vs. end-point stimuli (Table S.7) revealed listeners Feedback-5 (t(190) = –1.86, p = .03, d = 0.383 middle accuracy 81%, end accuracy 64%), No-Feedback-4 (t(190) = –4.55, p < .001, d = 1.088, middle accuracy 97%, end accuracy 56%), and Untrained-6 (t(190) = –1.81, p = .04, d = 0.356, middle accuracy 66%, end accuracy 48%) all showed better discrimination of middle than endpoint stimuli.

Combining the two tests, two of the four participants who showed clear signs of a transition from relative insensitivity to NE-like identification performance – Feedback-5 and No-Feedback-4 – showed the expected pattern of discrimination performance on both measures. No untrained participants showed this pattern.

Untrained vowel series

Identification

Using the first two criteria for a training effect on identification performance on the untrained vowel continua, listeners Feedback-1 (χ ² = 37.33, p < .01; z = –2.79, p < .05), No-Feedback-1 (χ ² = 108.93, p < .001; z = –4.99, p < .05), No-Feedback-5 (χ ² = 79.34, p < .001; z = –4.39, p < .05), and Untrained-2 (χ ² = 32.78, p < .05, z = –3.45, p < .05) showed a significant change from pre- to post-test on the /ræ–læ/ series. Inspection of the pre- and post-test curves reveals that two of the participants (Feedback-1 and No-Feedback-5) met the third criterion, showing initially flat identification functions that became more NE-like with training. For these participants, then, we have some evidence that the benefit of training generalized from the trained /ra–la/ series to the untrained /ræ–læ/ series. Participant Untrained-2 showed some evidence of improvement from pre- to post-test, but the final pattern is quite weak in comparison to the changes seen in participants Feedback-1 and No-Feedback-5. Overall, the data hint at the possibility that some trained participants were able to generalize an effect of training on the /ra–la/ series to produce NE-like identification performance on the untrained /ræ–læ/ series (Figure S.2). There was little if any sign of improvement on any of the other series. Several participants showed significant pre- to post-test differences on the /ru–lu/ series; however, none of these listeners showed NE-like identification curves at the post-test (Figures S.3 and S.4). None of the participants who showed a significant improvement identifying the trained vowel context also showed improvement in identification with either the /ru–lu/ or /ri–li/ series. Thus, there is no basis for thinking the training effect generalized to these series. Because only two participants showed any evidence of a generalization effect, we hesitate to speculate why generalization may have occurred in the /ræ–læ/ context but not in the /ri–li/ or /ru–lu/ contexts.

Discrimination

When assessing discrimination performance on the untrained vowel contexts via paired t-tests, no trained listener showed a significant improvement across the NE boundary from pre- to post-test together with evidence of NE-like discrimination (greater between than within categories) at the post-test.

Identification of natural speech

As in Experiment 1, we assessed differences in performance on natural speech identification collapsing across /r–l/ position. Each listener's performance as a function of /r–l/ position can be seen in Table S.8.

Two of the four participants who showed a transition to more NE-like identification on the trained /ra–la/ series, Feedback-1 and Feedback-5, also showed a significant improvement from the pre- to the post-test (Feedback-1: t(63) = 2.81, p = .003, d = 0.390, pre-test accuracy 66%, post-test accuracy 82%); Feedback-5: t(63) = 2.68, p = .005, d = 0.416, pre-test accuracy 62%, post-test accuracy 81%). No other participants showed such clear evidence of improvement from pre- to post-test, although two participants, No-Feedback-5 and Untrained-3, showed some evidence of improvement (No-Feedback-5: t(63) = 2.01, p = .02, d = 0.307, pre-test accuracy 63%, post-test accuracy 77%; Untrained-3: t(63) = 1.72, p = .04, d = 0.276 pre-test accuracy = 76%, post-test accuracy 87%).

Summary

Positive evidence of a pre- to post-test improvement is largely restricted to the participants who were trained with feedback. Three of these six participants – Feedback-1, Feeback-3, and Feedback-5 – showed a transition to NE-like performance on the trained /ra–la/ series, and Feedback-1 and Feedback-5 showed further signs of improvement. Feedback-1 showed generalization of the NE-like transition to another synthetic speech series, /ræ–læ/, and Feedback-5 showed a transition to a NE-like pattern of discrimination on the trained /ra–la/ series. These two participants also showed robust evidence of a post-test improvement in identification of /r–l/ from natural speech.

Training without feedback produced unequivocal evidence of a transition to native-like identification performance in only one of the six participants, No-Feedback-4. This participant also showed evidence of a transition to NE-like discrimination on the trained series, but did not show generalization to untrained stimuli. One other participant in this group – No-Feedback-5 – showed a transition to a NE-like pattern of identification on the untrained /ræ–læ/ series, along with a statistically borderline improvement in natural speech identification. Inspection of this participant's pre- and post-test identification curves for the trained series shows some evidence of a training effect there as well, but it is not clear-cut. Thus, it appears that three participants trained with feedback and one or two participants trained without feedback showed a benefit of training; of these participants, two of those trained with feedback showed fairly clear evidence of an improvement in natural speech identification. As with any experiment utilizing repeated stimulus presentations, it is possible that the apparent learning effects are not the result of training but are instead the result of repeated stimulus presentations mitigated by attentional and motivational factors. However, the fact that only one untrained participant – Untrained-4 – showed any of the positive signs of improvement from pre- to post-test, in the form of a borderline improvement on identification of natural speech stimuli, suggests to us that this is not the case. Thus, there is reason to believe that the pre- to post-test changes, where observed, are due to the training regime, and not simply to an improvement due to test practice.

We now consider the relationship between the efficacy of the training and the performance of the participants during training, shown in Table S.9. Two participants who showed steady progress over training (Feedback-1 and No-Feedback-4) also showed a transition to NE-like identification. However, several participants who showed good progress during training nevertheless failed to show a transition to native-like performance, whereas other participants whose performance during the training phase was quite variable nevertheless appeared to benefit from training. For example, participant Feedback-5 showed clear pre- to post-test improvement while showing quite variable performance during training; participant Feedback-3 also showed a transition to NE-like identification after variable performance during training. Both of these participants did have some sessions, however, during which their training task performance was near the upper end of the range, while none of the three participants whose performance during training stayed uniformly below difficulty level 100 showed a transition to NE-like performance.