Hostname: page-component-74656f9884-j7cmm Total loading time: 0 Render date: 2026-11-08T15:26:52.949Z Has data issue: false hasContentIssue false

The Talent system: TEXTRACT architecture and data model

Published online by Cambridge University Press:  11 October 2004

MARY S. NEFF
Affiliation:
IBM T.J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA e-mail: maryneff@us.ibm.com roybyrd@us.ibm.com bran@us.ibm.com
ROY J. BYRD
Affiliation:
IBM T.J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA e-mail: maryneff@us.ibm.com roybyrd@us.ibm.com bran@us.ibm.com
BRANIMIR K. BOGURAEV
Affiliation:
IBM T.J. Watson Research Center, 19 Skyline Drive, Hawthorne, NY 10532, USA e-mail: maryneff@us.ibm.com roybyrd@us.ibm.com bran@us.ibm.com
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

We present the architecture and data model for TEXTRACT, a robust, scalable and configurable document analysis framework. TEXTRACT has been engineered as a pipeline architecture, allowing for rapid prototyping and application development by freely mixing reusable, existing, language analysis plugins and custom, new, plugins with customizable functionality. We discuss design issues which arise from requirements of industrial strength efficiency and scalability, and which are further constrained by plugin interactions, both among themselves, and with a common data model comprising an annotation store, document vocabulary and a lexical cache. We exemplify some of these by focusing on a meta-plugin: an interpreter for annotation-based finite state transduction, through which many linguistic filters can be implemented as stand-alone plugins. The framework and component plugins have been extensively deployed in both research and industrial environments, for a broad range of text analysis and mining tasks.

Information

Type
Papers
Copyright
© 2004 Cambridge University Press