Teachers and researchers have had high hopes for computer-assisted pronunciation teaching, or CAPT, for several decades (Molholt, Reference Molholt1988). Yet it remains in its infancy in many ways. Chun (Reference Chun2007), for example, did not classify pronunciation among the topics published in two prominent computer-assisted language learning (CALL) journals from 2001 to 2006, except perhaps as a subset of other topics such as speaking, listening, or computer-mediated communication (CMC). This is odd, because the use of computers is almost ideally suited to learning pronunciation skills. Computers can provide individualized instruction, frequent practice through listening discrimination and focused repetition exercises, and automatic visual support that demonstrates to learners how closely their own pronunciation approximates model utterances. In a foreign language teaching environment in which few teachers receive adequate training in teaching pronunciation (Breitkreutz, Derwing, & Rossiter, Reference Breitkreutz, Derwing and Rossiter2002; Burgess & Spencer, Reference Burgess and Spencer2000; Kawai & Hirose, Reference Kawai and Hirose2000; MacDonald, Reference MacDonald2002; Murphy, Reference Murphy1997), and in which little class time appears to be available for focused work on pronunciation because of emphasis given to other skills, CAPT seems tailor-made to meet a critical need. Despite this great promise, effective commercial CAPT applications are less innovative either in pedagogy or use of computer technology than one might expect.

A variety of experimental software applications and studies into how various features of pronunciation might best be taught have demonstrated the flexibility and value of CAPT. A wide variety of pronunciation-related features have been examined, including general pronunciation quality (Seferoğlu, Reference Seferoğlu2005); speech rate, fluency, and liveliness (Hincks, Reference Hincks2005); vowels and consonants (Lambacher, Reference Lambacher1999; Neri, Cucchiarini, & Strik, Reference Neri, Cucchiarini and Strik2006a; Wang & Munro, Reference Wang and Munro2004); vowel lengthening and pitch accents (Hirata, Reference Hirata2004; Kawai & Hirose, Reference Kawai and Hirose2000); intonation (Cauldwell, Reference Cauldwell2002; Chun, Reference Chun1998; Hardison, Reference Hardison2004a; Kaltenboeck, Reference Kaltenboeck2002; Levis & Pickering, Reference Levis and Pickering2004); and English stress timing (Coniam, Reference Coniam2002). CAPT seems to also work for children, if designed with that audience in mind (Mich, Neri, & Giuliani, Reference Mich, Neri and Giuliani2006). The great majority of these studies demonstrate that CAPT, when constructed wisely, can be both effective and flexible in addressing pronunciation instruction.

Yet difficulties remain, and CAPT rarely makes its way into books that present state-of-the-art views of CALL. Some of the difficulties that CAPT has faced are pedagogical, some are technological, and some are related to teacher preparedness. Pedagogically, a significant gap often exists between CAPT applications and goals advocated by current pronunciation theory and pedagogy, such that CAPT applications can look suspiciously like traditional, drill-oriented pedagogy in new clothing. Although key researchers have strongly called for pedagogy to be based on empirical findings (Derwing & Munro, Reference Derwing and Munro2005), such a connection is all too infrequent in CAPT applications. Applications that are explicitly grounded in theory, like Streaming Speech, are rare. Because applications are often not grounded in appropriate descriptions of pronunciation (Pennington, Reference Pennington1999), they are not consistently able to measure improvement in meaningful ways. Technologically, CAPT systems often suffer from difficulties in giving learners adequate, accurate feedback and an inability to provide accurate and automatic diagnosis of pronunciation errors. Both of these areas relate to the use of automatic speech recognition (ASR) for accented speech. Finally, a substantial number of teachers are not able to make effective use of applications because of both a lack of training in pronunciation and in the use of technology.

This article begins by examining the importance of pedagogical goals. Next, I examine whether CAPT actually works and in what ways it works. In other words, do learners actually improve when using CAPT? Then I look at research on feedback and the use of ASR in CAPT before turning to the question of teacher preparation. In doing so, I offer suggestions of areas of technology that teachers and software developers must be able to understand to be more intelligent users of CAPT.

A writer's own experience and perspective always influences how a review such as this is written. Because I am far more expert on the pronunciation side of this topic and am primarily a consumer of CAPT rather than a developer, my review will necessarily take the perspective of a pronunciation practitioner who is interested in what CAPT can do for students as well as for teachers who will be unlikely to author their own software. There is a great deal more research available on critical areas such as speech recognition technology that I could have reported on, but did not, partly because much of it is more technical than I found useful, and partly because much of it asks questions (such as how spoken dialogue systems can be made more reliable) that are not directly relevant to CAPT. On the pronunciation side of this review, I try to avoid pronunciation issues that I do not see as directly relevant to CAPT design. I also have not specifically reviewed many individual applications. Detailed reviews of these can be regularly found in journals such as Calico Journal, Language Learning and Technology, and Computer Assisted Language Learning. A recent worthwhile review of CAPT that takes a wider scope than my review, including reviews of many individual applications, can be found in O'Brien (Reference O'Brien, Ducate and Arnold2006), described in the annotated references at the end of this article.

Appropriate Pedagogical Goals

The great psychologist Abraham Maslow was reported to have said, “If the only tool you have is a hammer, you tend to see every problem as a nail.” Similarly, CAPT applications are tools to meet instructional goals, and the tool should be appropriate to the job. Before CAPT applications can be effective, the job they need to do should be specified through appropriate pedagogical goals, a requirement that is not always given much emphasis by designers (Neri, Cucchiarini, Strik, & Boves, Reference Neri, Cucchiarini, Strik and Boves2002). CAPT, like CALL in general, should be based on an explicit theoretical approach to language teaching. Although general CALL principles may be useful, specific applications to pronunciation are still needed. Neri, Cucchiarini, Strik, and Boves (Reference Neri, Cucchiarini, Strik and Boves2002) suggested there is a critical need for specific CAPT principles for two reasons. First, many CAPT applications do not show any evidence of such principles, and second, general CALL principles often do not seem to have been written with pronunciation in mind. They argue that applications should do the following:

1. 1. Provide a substantial amount of meaningful input, including the use of multiple models and accurate articulatory instruction.

2. 2. Give learners a reason and desire to practice through rule-oriented practice and realistic materials.

3. 3. Provide immediate, useful feedback, especially for those features that are most important for intelligibility.

Likewise, Pennington (Reference Pennington1999, p. 434) wrote that most CAPT programs showed very little understanding of phonology, the range of acceptable variation in pronunciation, or how to apply this knowledge to teaching. Some of Pennington's principles for the design of CAPT materials are that CAPT systems should

1. 1. Establish baseline, reference accents for instruction.

2. 2. Set measurable goals and performance targets.

3. 3. Be designed to build skills from easier to more challenging exercises.

4. 4. Link pronunciation to other aspects of communication.

5. 5. Raise users' awareness of how their L1 phonological systems differ from the system of the target language.

These recommendations call for pedagogical explicitness in CAPT design so that applications do not simply use a new medium to do the same old exercises or employ technology without pedagogical grounding. Neri, Cucchiarini, and Strik (Reference Neri, Cucchiarini and Strik2002a) reported that “many authors describe commercially available programs as fancy-looking systems that may at first impress student and teacher alike, but eventually fail to meet sound pedagogical requirements. . . . These systems . . . look more like the result of a technology push, rather than a demand pull” (p. 442). In other words, the programs exploit the possibilities of the computer interface without considering whether the tool is appropriate to the job.

Some studies explore new ways to teach that exploit the computer's strengths. Wang and Munro (Reference Wang and Munro2004) suggested expanding the range of pedagogical techniques to include those typically employed for research but not usually for teaching. In training learners to recognize difficult vowel contrasts, they used identification tasks (rather than discrimination of minimal pairs) and synthesized speech to expand learners' perceptual space, making the learners more sensitive to the range of phonetic variation in the pronunciation of vowel phonemes. Such an expansion of perceptual space usually requires the use of multiple models and voices, a practice advocated by many researchers. However, in the absence of multiple voices in CAPT applications, the researchers say that speech synthesis may be very useful.

Multiple speech models, another way to provide variation in input, is evident in some recent software for English, such as Connected Speech (Protea Textware, 2007), Streaming Speech (Cauldwell, Reference Cauldwell2002), and Listening to American Accents (http://www.speechinaction.com/). Connected Speech, a program originally published in Australia, is available with nine regionally varied speech models (both male and female voices) for three markets: the United Kingdom, North America, and Australia/New Zealand. Streaming Speech is available for British and American English.

Pronunciation teaching is subject to two overlapping and conflicting foci (Levis, Reference Levis2005): a focus on accuracy (involving instruction on all possible aspects of pronunciation, with little attempt to prioritize attention) and a focus on intelligibility (involving instruction for only the elements of pronunciation that are critical for communicative success, while ignoring or deemphasizing those that are not). Traditionally, pronunciation instruction has been strongly accuracy oriented, assuming that all possible vowel and consonant sounds in the target language should be taught. Not only does this comprehensive coverage increase the teaching and learning burden, there is very little evidence that such accuracy is either possible for most adult learners or that it is desirable. All errors of pronunciation are not equal in their impact on listeners (Jenkins, Reference Jenkins2000; Munro & Derwing, Reference Munro and Derwing2006).

Because some errors impact listener understanding more than others, it is now generally agreed among pronunciation researchers that the goal of instruction is not perfection but rather being good enough to achieve communicative goals. “Good enough” is usually referred to as intelligibility, that is, whether a speaker is understandable (e.g., Dalton & Seidlhofer, Reference Dalton and Seidlhofer1994; Morley, Reference Morley1991). Intelligibility in the general sense, however, includes two main types of understanding, which researchers call intelligibility and comprehensibility. In the more specialized sense, intelligibility refers to whether listeners can adequately decode the words pronounced by a speaker in context. Comprehensibility, on the other hand, refers to whether listeners can understand (or believe they can understand) the message communicated by a speaker (see Munro & Derwing, Reference Munro and Derwing1999, for discussion of these terms). Applied to the teaching of pronunciation, both intelligibility and comprehensibility address listeners' ability to understand at the micro level of word identification and at a communicative level of meaning processing.

Intelligibility as a Principle for CAPT

The goals of accuracy and intelligibility are often incompatible in non-CAPT instruction but there is no reason for them to be so in CAPT applications. Although certain errors may hinder understanding no matter what the language background (such as word stress; see Field, Reference Field2005), many deviations that cause misunderstandings are language specific, such as /l/ and /r/ mispronunciations for Japanese speakers but not for speakers of French. In addition, some causes of misunderstanding may even be dependent on the listeners' perceptual backgrounds. For example, Jenkins (Reference Jenkins2000) discussed how, in communication in English between a Swiss German and Japanese speaker, the perceptual system of the German speaker and the pronunciation of the Japanese speaker both were critical in communication breakdowns.

Applied to CAPT design, these findings suggest that CAPT applications should, unless built for learners of a particular language group, be complete enough to allow teachers or learners to pick pronunciation features most likely to affect their pronunciation needs. Suprasegmental features such as word stress, length variations, and intonation should be included for all learners (Derwing & Rossiter, Reference Derwing and Rossiter2003). For segmentals, CAPT systems should guard against the tendency to do everything. The system should assist learners and teachers in prioritizing pronunciation topics by channeling learners toward typical vowel and consonant errors for their language backgrounds. For example, a Korean learner of English would, after setting up a user profile, be directed to pronunciation topics that are problematic for Korean learners. Such information about typical errors for a variety of language backgrounds can be found (for learners of English) in a variety of sources, such as Swan and Smith (Reference Swan and Smith2001) and Avery and Ehrlich (Reference Avery and Ehrlich1992).

Even better than this rather crude channeling mechanism would be an error diagnostic informed by language specific filtering. A diagnostic component in a CAPT system should include perception elements in which learners identify and discriminate among problematic sounds. It should also include production elements in which learners have to produce potentially problematic elements. Their pronunciation of words and phrases could be scored by means of ASR technology, which works relatively well for very limited tasks. Finally, the diagnostic could also include a prediction component. Some pronunciation errors are a result of confusion about the connection between sound and spelling (e.g., the -ed endings, multisyllabic word stress, vowel pronunciations). A prediction component could identify the cause of these types of errors and provide appropriate remediation that addresses the cognitive basis of the problems.

Improvement and CAPT

A key question is whether CAPT works. In non-CAPT settings, this has often been asked about pronunciation instruction. Adult learners of foreign languages rarely achieve native-like accents. Because most nonnative speakers will have a noticeable accent with or without instruction, should pronunciation be taught? Asking about improvement is actually asking multiple questions. First, does instruction lead to improvement on the features trained in the CAPT system? Second, does learning transfer to novel contexts? Third, does learning last over time? And finally, does learning in one pronunciation topic lead to improvement in other areas? Is there a positive spillover?

Regarding the first question, CAPT seems to be effective in improving pronunciation accuracy. For a wide variety of pronunciation skills, learners improve through the use of well-designed CAPT instruction. Hirata (Reference Hirata2004) taught learners Japanese pitch and duration contrasts. One experimental group learned the contrasts at the word level, while another learned the same words embedded in sentence-level practice. Both groups improved their production of these contrasts, with improvement being greater for those who learned the contrasts embedded in sentence-level practice. She stated that “developing abilities to produce and perceive L2 contrasts in sentences instead of only in an isolated word contest, is an important factor for L2 learners coping with real world situations” (p. 372). In another study of Japanese, Kawai and Hirose (Reference Kawai and Hirose2000) found that subjects more successfully made phonemic contrasts between short and long segments, with the greatest improvement on the short segments. Wang and Munro (Reference Wang and Munro2004) trained learners to pay attention to vowel quality rather than durational cues for the perception improvement of three vowel contrasts among learners of English, /i/-/ɪ/, /u/- /ʊ/, and /ɛ/- /æ/. They found that subjects increased their perception performance on all the contrasts. Hardison (Reference Hardison2004a) found that learners who practiced French intonation, with NS models provided as feedback, improved their intonation production. Neri, Cucchiarini, and Strik (Reference Neri, Cucchiarini and Strik2006b) found that learners of Dutch improved their production of the trained Dutch segments.

The second and third questions, regarding transfer to untrained contexts and improvement over time, also point to the benefits of CAPT. Hirata (Reference Hirata2004) found that pitch and duration training for learners of Japanese transferred to words that were not part of the training, that is, that “subjects who participated . . . acquired a generalized ability to produce and perceive Japanese words contrasting in pitch and duration” (p. 372). Kawai and Hirose (Reference Kawai and Hirose2000) found that phonemic lengthening, especially for vowels, transferred to novel minimal pairs. Wang and Munro (Reference Wang and Munro2004) found that the perceptual improvements in distinguishing vowel contrast generalized to new words. In contrast, Neri et al. (Reference Neri, Cucchiarini and Strik2006b) found that training on vowels and consonants did not cause any change in segments that were not part of the training. For vowels and consonants, at least, this suggests that changes follow training, and that without training, changes are less likely to occur. In regard to changes lasting over time, only one study of CAPT addressed this. Wang and Munro (Reference Wang and Munro2004) retested the subjects' perceptions 3 months later and found that the improvement had lasted.

The final question is the most intriguing. Given that CAPT instruction works for the trained items, is there any spillover to improvement in other areas? There is evidence that improvements in perception can lead to improvements in production (Bradlow, Pisoni, Akahana-Yamada, & Tohkura, Reference Bradlow, Pisoni, Akahana-Yamada and Tohkura1997). Hirata (Reference Hirata2004), however, found evidence that production improvements also can lead to improved ability to perceive contrasts. It is possible that improvement in one area spills over to improvements in the other, although the extent to which this is so needs to be tested for various pronunciation features. Hardison (Reference Hardison2004a), in more evidence of spillover, found that not only did learners improve their intonation, but they also showed improvement in two areas that were peripheral to the treatment: lexical recall and segmental accuracy. She suggested that “as they [the subjects] became more confident with this aspect [prosody] of their language production, they were able to notice other elements such as liaison and their production of specific sounds” (p. 48).

Visual Feedback

Perhaps the central issue in CAPT is the provision of adequate feedback. Although this can be done in various ways, the most common methods are visualization and through automatic speech recognition. Visual feedback includes “the graphical display of a native speaker's face . . . [and] the vocal tract” both of which seem to improve learners' ability to identify words and produce more native-like speech timing (Ehsani & Knodt, Reference Ehsani and Knodt1998, p. 63). This kind of feedback is usually found in the study of phonetics, as in the phonetics page at the University of Iowa Web site (http://www.uiowa.edu/~acadtech/phonetics/). However, the best-known CAPT visual displays are spectrograms, waveforms, and pitch tracings. These visual displays have been advocated for a long time (Anderson-Hsieh, Reference Anderson-Hsieh1994; Cranen, Weltens, de Bot, & van Rossum, Reference Cranen, Weltens, de Bot and van Rossum1984; Spaai & Hermes, Reference Spaai and Hermes1993) and are still being used in research and teaching.

Applications of many phonetic displays to pedagogy have not always been successful because the displays are not transparently interpretable and may require more specialized training than a teacher is able to provide. Spectrograms and waveforms are widely argued to not be worth the pedagogical time and energy, at least when used as the sole source of feedback (Ehsani & Knodt, Reference Ehsani and Knodt1998; Neri, Cucchiarini, & Strik, Reference Neri, Cucchiarini and Strik2002b). Pitch tracings, however, are relatively iconic, with rising, falling, and level lines on the display usually corresponding to rises, levels, and falls in a speaker's voice pitch. Although they require some training to interpret, they appear to be useful pedagogically (Chun, Reference Chun2007; Hardison, Reference Hardison2004a) despite an ongoing need to define which intonational features are most important for intelligibility (Chun, Reference Chun2007).

Nonetheless, spectrograms are still being advocated by some recent work. Coniam (Reference Coniam2002) used spectrograms to raise language teachers' awareness of the relatively stress-timed nature of American English, and by extension, other inner circle Englishes. The teachers that Coniam described were native speakers of Cantonese in Hong Kong, and the outer-circle English variety spoken there, Hong Kong English, is relatively syllable-timed. Coniam used spoken dialogues from local TV shows involving well-known characters speaking both Hong Kong and American English. The utterances he used for analysis were analyzed through spectrograms, which demonstrated that Hong Kong English has a greater frequency of energy peaks than American English speech. The visual representation of the energy peaks in the spectrograms was successful in helping teachers understand this difference in the two varieties. The success of this awareness raising exercise appears to happen because Coniam did not ask much of the spectrogram, focusing only on the relatively iconic visual displays of syllable energy while also providing a significant amount of scaffolding that identified the location of the words associated with each peak. That his subjects were language teachers was also significant. His goals were that these expert users begin to understand the nature of rhythmic differences rather than produce those differences. This more restrictive goal contributed to the successful use of spectrograms.

Lambacher (Reference Lambacher1999) indicated success with teaching English consonants to Japanese learners using spectrograms. For difficult consonant features and contrasts (i.e., aspiration, r/l, nasals, s/ʃ, f/h, and s/ɵ), he illustrates how the contrasts look on visual displays built into the CAPT program. Lambacher offered no data on the effectiveness of his approach, but it is interesting to note that he did not advocate using spectrograms alone. For example, Lambacher taught aspiration with reference to a spectrogram but also through standard pronunciation teaching tricks like blowing away a piece of paper.

Pitch tracings, or models of the fundamental frequency of speech, while also requiring training to interpret, appear to be much easier to use successfully. Commercially available software such as Kay Elemetrics VisiPitch or its more technically oriented cousin, the Computerized Speech Laboratory (CSL), as well as free programs such as WASP and PRAAT, all provide the technology to represent intonation visually. Most also allow learners to compare their own production to a model utterance by overlaying their utterance's pitch tracing on that of the model. Pitch tracings are ubiquitous in CAPT studies and recommendations. Although pitch is most often modeled for sentence intonation (e.g., Hardison, Reference Hardison2004a, Reference Hardison2004b), pitch tracings have also been used to teach word-level pitch accents in Japanese (Hirata, Reference Hirata2004). Levis and Pickering (Reference Levis and Pickering2004) recommended using pitch visualization to teach paratones, that is, initial extra-high pitch levels that mark discourse topic shifts in connected speech. Streaming Speech uses the discourse intonation model of David Brazil to teach the intonation and pronunciation of natural connected speech, and discourse uses of pitch are advocated by Chun (Reference Chun1998; Reference Chun2007).

ASR Feedback

Despite ongoing issues regarding the use of visual feedback tools, the central question in CAPT feedback is whether automatic speech recognition (ASR) can effectively provide immediate feedback that allows learners to know which parts of their pronunciation are correct and which are not. Conceptually, the problem is this: Do the words and phrases pronounced by nonnative speakers (NNSs) match the models of native speaker speech on which the ASR is based? If not, can the NNS be given global feedback on the mispronunciations (e.g., “Your score is 62%. This means that you often have words that are mispronounced.”) or specific feedback on particular sounds or prosodic categories? Finally, how should feedback be given about the nature of the mispronunciation? Assuming these questions are successfully answered, another concern is whether ASR can be configured to provide accurate instruction to remedy errors. This last question, however, is a CAPT design issue and not unique to ASR. If automatic feedback is adequate, the provision of instructional remedies should be relatively straightforward.

Neri, Cucchiarini, Strik, and Boves (Reference Neri, Cucchiarini and Strik2002) wrote that “ideal systems should always include an option to provide feedback by means of ASR technology, so that the user can receive immediate information on his/her performance” (p. 458). Because ASR applications have been successful with native speakers of English, especially in the use of voice recognition for word processing, it might be assumed that they could be adjusted to provide feedback nonnative speakers. Some applications, such as recent versions of Dragon Naturally Speaking (ScanSoft, 2005), are 95% or more accurate for native-speaking English users (Ehsani & Knodt, Reference Ehsani and Knodt1998; Pogue, Reference Pogue2004). However, accuracy for these programs, when used by advanced proficiency but accented nonnative speakers of English, drops to near 70% (Coniam, Reference Coniam1999; Derwing, Munro, & Carbonaro, Reference Derwing, Munro and Carbonaro2000). The reason for this drop is that commercial ASR dictation programs are not built for nonnative speakers. In fact, “much of the progress in the last 15 years in acoustic modeling [for ASR] is based on more detailed modeling, creating sharper and sharper distributions for narrower and narrower classes. This is diametrically opposite of the tolerance and robustness required for non-natives” (Van Compernolle, 2000, p. 76). Research on improving ASR effectiveness for nonnative speech continues to be of interest (e.g., Gorozny, Rapp, & Kompe, Reference Gorozny, Rapp and Kompe2004; Oh, Yoon, & Kim, Reference Oh, Yoon and Kim2007), but much of this research is currently of little direct relevance to CAPT. The failure of ASR systems (developed for native speakers) to successfully handle nonnative speech does not mean that ASR is useless in pronunciation training. Rather, it means that new ASR systems with goals more appropriate to language learning contexts are needed.

Feedback for CAPT should be consistent with human feedback (Kim, Reference Kim2006; Neri, Cucchiarini, & Strik, Reference Neri, Cucchiarini and Strik2003), be immediate (Neri, Cucchiarini, Strik, & Boves, Reference Neri, Cucchiarini, Strik and Boves2002), be pertinent and correct (Eskenazi, Reference Eskenazi1999), be given in a form that students can make use of (Kawai & Hirose, Reference Kawai and Hirose2000), include information about when goals have been reached (Kawai & Hirose, Reference Kawai and Hirose2000), and possibly suggest ways to address errors (Neri et al., Reference Neri, Cucchiarini and Strik2002a).

These requirements reflect a real problem. It is difficult for ASR systems to pinpoint learner errors in read speech even where the system knows what words are intended (Truong, Neri, de Wet, Cucchiarini, & Strik, Reference Truong, Neri, de Wet, Cucchiarini and Strik2005). Precise feedback is even harder for suprasegmental errors because all languages make use of the same acoustic categories of pitch, duration, and intensity for differently organized prosodic systems. Also, disagreements continue about how the prosodic systems should be measured (for a still relevant discussion of this, see Ladd, Reference Ladd1980). ASR systems measure prosody without reference to linguistic organization, leaving a gap between human perceptions of correctness and what the systems can do.

The problem of wrong feedback is enormous in CAPT ASR systems. Neri, Cucchiarini, Strik, and Boves (Reference Neri, Cucchiarini, Strik and Boves2002) wrote that “erroneous feedback is a common problem. . . . Patently wrong error detection can be so frustrating for the student that [some] recommend using implicit rather than explicit judgemental feedback” (p. 458). In other words, because of the difficulty of ASR pinpointing particular errors, feedback that does not try to be too precise is likely to be more successful (e.g., “The word system was not pronounced correctly.”) than feedback that singles out particular sounds for correction (e.g., “The first vowel in system was pronounced wrongly.”). This is the difference between implicit and explicit feedback. By telling the truth at a sufficient level of generality, the feedback will at least not be wrong (Delmonte, Reference Delmonte2000). ASR feedback is best when its strengths are used and its limitations are recognized (Ehsani & Knodt, Reference Ehsani and Knodt1998). Neri et al. (Reference Neri, Cucchiarini and Strik2003) reflected the current state of ASR-based feedback when they stated that to be effective, ASR-based feedback should “keep the recognition task as simple and as limited as possible, by carefully designing the learning activities” (p. 1158).

Does ASR-based feedback work, given all its limitations? Feedback appears to be effective, but the picture may be complicated by the complex relationship between segmental accuracy and global ratings of improvement (cf. Derwing & Rossiter, Reference Derwing and Rossiter2003). Neri et al. (Reference Neri, Cucchiarini and Strik2006b) did an experimental study of CAPT effectiveness. Two experimental groups, one using CAPT and receiving feedback and one using CAPT without feedback (and one control group) were tested using a CAPT system focused on Dutch segmental contrasts that are difficult for L2 learners of Dutch. Global pronunciation quality after training was rated by six expert raters. All three groups improved after the treatment period, but there was no significant difference in the global ratings. In further analysis, the researchers found that the accuracy of the target sounds was significantly better for the group receiving feedback over the other groups, and that the group simply using the CAPT training was significantly better than the control group. They concluded that “the automatic feedback . . . was effective in improving the quality of the targeted phonemes” (p. 1985). The mismatch between the global ratings and specific improvement occurred because the targeted phonemes were much less frequent than other sounds, and presumably their impact on the global score was not great enough to be noticed in the global ratings.

Automatic Diagnosis of Pronunciation Errors

Automatic diagnosis of pronunciation, both globally and related to specific errors, is a hopeful application of ASR technology. One study says that “speech recognition technology is key to the automatic evaluation of pronunciation quality” (Neumeyer, Franco, Digalakis, & Weintraub, Reference Neumeyer, Franco, Digalakis and Weintraub2000, p. 83). The trick is to know which door is opened by the key, a door to general pronunciation scores or a door that specifies the errors made by the nonnative speaker. Research indicates that the first door can be opened, while the second door remains tightly shut in search of the right key.

The standard against which ASR systems should be measured when diagnosing pronunciation quality is whether they sufficiently correlate with human judgments (Franco, Neumeyer, Digalakis, & Ronen, Reference Franco, Neumeyer, Digalakis and Ronen2000). From this perspective, the accuracy of automatic scoring varies greatly depending on the type of evaluation task. At one end, Neri, Cucchiarini, Strik, and Boves (Reference Neri, Cucchiarini, Strik and Boves2002) found in examining one ASR-based system that only 25% of the actual errors were detected, and that a small number of correct pronunciations were called errors. These two types of errors, where errors are not detected and where correct pronunciations are labeled as errors, seem to be common to all ASR systems. At the more optimistic end of the evaluation spectrum, the use of certain automatic scores such as duration, log-posterior, and speech rate allow some ASR evaluation algorithms to approach the correlations found to exist between human raters, at least when there is enough speech material being tested (Cucchiarini, Strik, & Boves, Reference Cucchiarini, Strik and Boves2000; Neumeyer et al., Reference Neumeyer, Franco, Digalakis and Weintraub2000; Rypa & Price, Reference Rypa and Price1999). Usually, performance is better when a combination of parameters is measured rather than only one (Cucchiarini et al., Reference Cucchiarini, Strik and Boves2000; Franco et al., Reference Franco, Neumeyer, Digalakis and Ronen2000; Witt & Young, Reference Witt and Young2000). Although it appears possible to predict human ratings with ASR, the combination of automatic measures used and the quality and representativeness of the speech database to which the nonnative speech is compared cause the performance of different ASR studies to vary greatly. Cucchiarini et al. (Reference Cucchiarini, Strik and Boves2000), for example, found that two of their measures correlated well to human ratings of fluency and speech rate but not to ratings of overall pronunciation. Witt and Young (Reference Witt and Young2000) found that their Goodness of Pronunciation score had to be modified by several other measures and models of the learners' L1 to get within shouting distance of human raters. These problems with predicting human ratings reflect the fact that machine scores attempt to correlate with humans, but machines do not, and perhaps cannot, hear like humans. Ladd, in discussing rhythmic organization in language, quoted a 1950s era researcher, Fred Householder:

Progress on the disconnect between how humans perceive speech and how ASR models perception needs to be made before ASR can make more than incremental changes in accuracy. Scharenborg (Reference Scharenborg2007), in a comparison of the goals and methods of research in human and automatic speech recognition, stated that “human listeners are able to use all information that is present in the acoustic signal to differentiate between phones and thus words, while ASR systems can only use the information that is encoded in the acoustic features. . . . ASR systems thus do not have available all information that is available to human listeners” (p. 341). If Householder is correct, human listeners also have other information available, the “things that aren't there.” All of this makes human perception robust, allowing humans to be “far better at dealing with accents, noisy environments, differences in speaking style, speaking rate, etc.” (Scharenborg, Reference Scharenborg2007, p. 344). ASR, on the other hand, is not robust, and ASR systems for nonnative speech can currently, at best, only approach human raters. That closeness, however, may be enough for many uses of automatic pronunciation evaluation.

Pinpointing Specific Errors

The key has not yet been found for the door that opens to automatically evaluating particular pronunciation errors, although this would be far more useful to language teachers than global measures of pronunciation quality. Telling a learner that their pronunciation is worth a grade of “C” is almost guaranteed to bring the question, “What should I work on?” Global measures offer no answer to this kind of question. Neri et al. (Reference Neri, Cucchiarini and Strik2003) stated that “recent research on ASR-based CAPT has . . . shown that this technology is not yet mature enough to provide relatable detailed diagnoses of pronunciation errors” (p. 1159).

Cucchiarini et al. (Reference Cucchiarini, Strik and Boves2000) correlated human ratings of Segmental Quality with ASR measures (which worked well for predicting human ratings of fluency and speech rate). They found that “segmental quality . . . is predicted most poorly on the basis of automatic scores, which is not a positive finding if we consider that segmental quality is the best predictor [for human raters] of overall pronunciation” (p. 116). Truong et al., (Reference Truong, Neri, de Wet, Cucchiarini and Strik2005) constructed an ASR system that automatically measured the accuracy of three Dutch phonemes commonly mispronounced by learners of Dutch, /ɑ/, /ʏ/ and /x/. They found that errors in the pronunciation of /x/ were automatically detected most successfully, but that the ASR system was less successful identifying difficulties with the vowels. Nonetheless, the approach seems on target for automatic evaluation. Human raters don't need to hear every instance of a phonemic error to make the judgment that there is a systematic problem. In addition, even though learners do not always mispronounce sounds that are difficult, a system that can identify frequent errors could be very useful for learners and teachers. What is clear is that a lot of work needs to be done defining the parameters of the segmentals in a language and including them in ASR systems before precise feedback on individual segments is possible (for one example of this process, see Neri, Cucchiarini, & Strik, Reference Neri, Cucchiarini and Strik2006a).

The success of an ASR system depends on the uses to which it is put. Global measures of pronunciation may be useful for some contexts, such as testing (e.g., Versant Testing, the test previously called PhonePass and Set-10). For the classroom teacher and the self-directed learner, however, there is a strong need for ASR systems that can pinpoint specific errors and provide help in addressing the errors (Neri et al., Reference Neri, Cucchiarini and Strik2003).

Teacher Preparation and CAPT

What do teachers need to know to get the best use of CAPT applications? It seems clear to me that all teachers need to have a basic understanding of technology that is used in phonetics research. Teachers should be able to understand, at the very least, spectrograms, waveforms, and fundamental frequency contours, and they should be familiar with terms like low pass filtering, if only to be intelligent consumers. Freely available programs like PRAAT and WASP, or more costly options like the Computerized Speech Lab (CSL), should supplement any pronunciation training course. Teachers should also understand the strengths and limitations of ASR technology and be able to critique the use of ASR in a variety of language learning applications, perhaps through their own or their students' use of the applications. Third, teachers should become familiar with exercises that appear to be effective or not effective with a computer interface, and they should have the opportunity to develop and test their own computer-based pronunciation exercises through the use of basic CALL authoring tools.