Skip to main content Accessibility help
×
Hostname: page-component-74656f9884-6grmk Total loading time: 0 Render date: 2026-11-04T10:23:22.697Z Has data issue: false hasContentIssue false

Exploring Multi-Word Combinations as Measures of Linguistic Accuracy in Second Language Writing

Published online by Cambridge University Press:  29 December 2020

By
Charlene Polio  and
Hyung-Jo Yoon
Edited by
Bert Le Bruyn  and
Magali Paquot
Bert Le Bruyn
Affiliation:
UIL-OTS, Utrecht University
Magali Paquot
Affiliation:
FNRS – Centre for English Corpus Linguistics, UCLouvain
Get access

Summary

While written language development involves reducing erroneous expressions, traditional error-based measures are problematic for several reasons, including low inter-coder reliability for lexical errors, limited sensitivity for capturing development within a short time period, and the questionable separation of lexical and grammatical errors. Given these problems, we explore automated accuracy measures rooted in a usage-based theory of Second Language Acquisition, which views language as a set of constructions or chunks. For this study, we examined 139 essays in terms of using traditional measures of complexity, accuracy, lexical sophistication, and fluency, as well as novel corpus-based n-gram measures. A factor analysis was conducted to explore how traditional measures grouped with corpus-based measures, and regression analyses were used to examine how corpus-based measures predicted error counts and holistic accuracy scores. With the results of these analyses, we suggest that automated n-gram based measures are a viable alternative to traditional accuracy measures.

Keywords

accuracyn-gramsautomated analysis of writingusage-based approacheserrors

Information

Type
Chapter
Information
Learner Corpus Research Meets Second Language Acquisition , pp. 96 - 121
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Abdel Latif, M. (2013). What do we mean by writing fluency and how can it be validly measured? Applied Linguistics 34(1), 99–105.CrossRefGoogle Scholar
Bestgen, Y. & Granger, S. (2014). Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing 26, 28–41.CrossRefGoogle Scholar
Canagarajah, A. (2006). The place of world Englishes in composition: Pluralization continued. College Composition and Communication 57, 586–619.Google Scholar
Connor-Linton, J. & Polio, C. (2014). Comparing perspectives on L2 writing: Multiple analyses of a common corpus: Introduction. Journal of Second Language Writing 23, 1–9.CrossRefGoogle Scholar
Cook, V. (1999). Going beyond the native speaker in language teaching. TESOL Quarterly 33(2), 185–209.Google Scholar
DeKeyser, R. (2015). Skill acquisition theory. In VanPatten, B. & Williams, J. (eds.), Theories in Second Language Acquisition: An Introduction, 94–112. New York, NY: Routledge.Google Scholar
Derwing, T. & Munro, M. (2005). Second language accent and pronunciation teaching: A research-based approach. TESOL Quarterly 39(3), 379–397.CrossRefGoogle Scholar
Ellis, R. & Barkhuizen, G. (2005). Analyzing Learner Language. Oxford: Oxford University Press.Google Scholar
Ellis, N., Römer, U., & O'Donnell, M. (2016). Constructions and usage-based approaches to language acquisition. Language Learning 66(S1), 23–44.Google Scholar
Ellis, N. & Wulff, S. (2015). Usage-based approaches to SLA. In VanPatten, B. & Williams, J. (eds.), Theories in Second Language Acquisition: An Introduction, 75–93. Mahwah, NJ: Erlbaum.Google Scholar
Evans, N., Hartshorn, K., Cox, T., & Martin de Jel, T. (2014). Measuring written linguistic accuracy with weighted clause ratios: A question of validity. Journal of Second Language Writing 24, 33–50.CrossRefGoogle Scholar
Foster, P. & Skehan, P. (1996). The influence of planning on performance in task‐based learning. Studies in Second Language Acquisition 18(3), 299–324.CrossRefGoogle Scholar
Friedman, D. (2010). Speaking correctly: Error correction as a language socialization practice in a Ukrainian classroom. Applied Linguistics 31(3), 346–367.Google Scholar
Goldberg, A. E. (2006). Constructions at Work: The Nature of Generalization in Language. Oxford: Oxford University Press.Google Scholar
Granger, S. (2015). Contrastive interlanguage analysis: A reappraisal. International Journal of Learner Corpus Research 1(1), 7–24.Google Scholar
Granger, S. & Bestgen, Y. (2017). Using collgrams to assess L2 phraseological development: A replication study. In de Haan, P., van Vuuren, S., & de Vries, R. (eds.), Language, Learners and Levels: Progression and Variation, 385–408. Louvain-la-Neuve: Presses universitaires de Louvain.Google Scholar
Grant, L. & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing 9(2), 123–145.CrossRefGoogle Scholar
Hartshorn, K., Evan, N., Merrill, P., Sudweeks, R., Strong-Krause, D., & Anderson, N. (2010). Effects of dynamic corrective feedback on ESL writing accuracy. TESOL Quarterly 44(1), 84–109.CrossRefGoogle Scholar
Hymes, D. (1972). On communicative competence. In Pride, J. B. & Holmes, J. (eds.), Sociolinguistics, 269–293. Harmondsworth: Penguin.Google Scholar
Knoch, U., Rouhshad, A., & Storch, N. (2014). Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? Assessing Writing 21, 1–17.Google Scholar
Kuiken, F. & Vedder, I. (2008). Cognitive task complexity and written output in Italian and French as a foreign language. Journal of Second Language Writing 17(1), 48–60.Google Scholar
Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics 27(4), 590–619.CrossRefGoogle Scholar
Laufer, B. (1994). The lexical profile of second language writing: Does it change over time? RELC Journal 25(2), 21–33.CrossRefGoogle Scholar
Laufer, B. & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics 16(3), 307–332.CrossRefGoogle Scholar
Laufer, B. & Waldman, T. (2011). Verb–noun collocations in second language writing: A corpus analysis of learners’ English. Language Learning 61(2), 647–672.Google Scholar
Lennon, P. (1991). Error: Some problems of definition, identification, and distinction. Applied Linguistics 12(2), 180–196.CrossRefGoogle Scholar
Lu, X. (2010). Automatic measurement of syntactic complexity in child language acquisition. International Journal of Corpus Linguistics 14(1), 3–28.CrossRefGoogle Scholar
Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal 96(2), 190–208.Google Scholar
Mukherjee, J. (2005). The native speaker is alive and kicking: Linguistic and language-pedagogical perspectives. Anglistik 16(2), 7–23.Google Scholar
Norris, J. & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics 30(4), 555–578.Google Scholar
Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied Linguistics 30(4), 590–601.Google Scholar
Paquot, M. (2017). The phraseological dimension in interlanguage complexity research. Second Language Research 35(1), 1–25.Google Scholar
Paquot, M. & Naets, H. (2017). The role of the reference corpus in studies of EFL learners’ use of statistical collocations. Paper presented at ICAME38. Prague, May 24–28, 2017.Google Scholar
Pett, M., Lackey, N., & Sullivan, J. (2003). Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. California: SAGE Publications Inc.CrossRefGoogle Scholar
Pienemann, M. & Lenzig, A. (2015). Processability theory. In VanPatten, B. & Williams, J. (eds.), Theories in Second Language Acquisition: An Introduction, 159–179. New York, NY: Routledge.Google Scholar
Polio, C. (1997). Measures of linguistic accuracy in second language writing research. Language Learning 47(1), 101–143.CrossRefGoogle Scholar
Polio, C. (2012). The relevance of second language acquisition theory to the written error correction debate. Journal of Second Language Writing 21(4), 375–389.Google Scholar
Polio, C., Fleck, C., & Leder, N. (1998). “If I only had more time”: ESL learners’ changes in linguistic accuracy on essay revisions. Journal of Second Language Writing 7(1), 43–68.CrossRefGoogle Scholar
Polio, C. & Shea, M. (2014). An investigation into current measures of linguistic accuracy in second language writing research. Journal of Second Language Writing 26, 10–27.Google Scholar
Polio, C. & Yoon, H.-J. (2018). The reliability and validity of automated tools for examining variation in syntactic complexity across genres. International Journal of Applied Linguistics 28(1), 165–188.Google Scholar
Read, J. (2000). Assessing Vocabulary. Cambridge: Cambridge University Press.Google Scholar
Römer, U. (2009). The inseparability of lexis and grammar: Corpus linguistic perspectives. Annual Review of Cognitive Linguistics 7(1), 140–162.Google Scholar
Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. Paper presented at the International Conference on New Methods in Language Processing. Manchester, UK.Google Scholar
Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing 21(3), 239–263.CrossRefGoogle Scholar
Wei, N. & Li, J. (2013). A new computing method for extracting contiguous phraseological sequences from academic text corpora. International Journal of Corpus Linguistics 18(4), 506–535.Google Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity. Honolulu, HI: University of Hawaii Press.Google Scholar
Wulff, S. & Gries, S. Th. (2011). Corpus-driven methods for assessing accuracy in learner production. In Robinson, P. (ed.), Second Language Task Complexity: Researching the Cognition Hypothesis of Language Learning and Performance, 61–87. Amsterdam: John Benjamins.Google Scholar
Yang, W., Lu, X., & Weigle, S. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing 28, 53–67.Google Scholar
Yoon, H. (2017). Investigating the interactions among genre, task complexity, and proficiency in L2 writing: A comprehensive text analysis and study of learner perceptions. Unpublished Ph.D. dissertation, Michigan State University.Google Scholar
Yoon, H. & Polio, C. (2017). ESL students’ linguistic development in two written genres. TESOL Quarterly 51(2), 275–301.Google Scholar

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the HTML of this chapter is currently unknown and may be updated in the future.

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×