Hostname: page-component-7b7567c8c-2n9c9 Total loading time: 0 Render date: 2026-11-15T06:17:41.159Z Has data issue: false hasContentIssue false

Evaluating State-of-the-Art Treebank-style Parsers for Coh-Metrix and Other Learning Technology Environments

Published online by Cambridge University Press:  22 May 2006

CHRISTIAN F. HEMPELMANN
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
VASILE RUS
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
ARTHUR C. GRAESSER
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
DANIELLE S. MCNAMARA
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.

Information

Type
Papers
Copyright
2006 Cambridge University Press