2. Evolution of metadata and annotation practices

The history of metadata and annotation in historical text corpora reflects broader transformations in computing technology, scholarly communication and humanities research methodologies. This evolution can be understood as a progression from ad hoc, project-specific approaches toward standardised, interoperable frameworks, driven by the dual imperatives of accommodating diverse scholarly needs and enabling large-scale data sharing and reuse (Cummings Reference Cummings, Siemens and Schreibman2008; Burnard Reference Burnard2014). The development has not been linear but rather characterised by periods of experimentation, consolidation and innovation, with each phase building upon and sometimes challenging the assumptions of earlier work.

The origins of digital text corpora can be traced to the 1960s, when pioneering computational linguists and literary scholars began creating machine-readable versions of texts for quantitative analysis. Father Roberto Busa’s Index Thomisticus project, which began in 1949 and continued for decades, is often cited as the first major humanities computing project, producing a lemmatised concordance of Thomas Aquinas’ works through computational methods (Hockey Reference Hockey2000; Jones Reference Jones2016). This ambitious undertaking demonstrated the feasibility of applying computational techniques to humanities materials, though it also revealed substantial technical and intellectual challenges. Early projects like the Brown Corpus (1961–4), a carefully balanced collection of one million words of American English, established principles of corpus design including sampling strategies, genre balance and documentation practices that remain relevant today (Kučera & Francis Reference Kučera and Francis1967; Leech Reference Leech, Hundt, Nesselhauf and Biewer2007). The Brown Corpus represented a significant advance in systematic corpus construction, introducing the notion that corpora should be representative samples of language use rather than arbitrary collections of texts.

These early efforts faced significant technical constraints that shaped their methodologies and limited their scope. Computing resources were expensive and scarce, storage was limited, and there were no standard formats for representing textual data (Hockey Reference Hockey2000). Projects typically developed custom encoding schemes, often using punched cards or magnetic tape as storage media. Documentation was minimal by contemporary standards, and interchange between projects was virtually impossible due to incompatible formats and the absence of networking infrastructure. Character encoding was particularly problematic, as no universal standard existed for representing non-ASCII characters, making work with historical texts in languages other than English extremely difficult (Ide & Sperberg-McQueen Reference Ide and Sperberg-McQueen1995). Despite these limitations, early digitisation efforts demonstrated the potential of computational approaches to textual analysis and established the need for systematic approaches to text encoding and documentation. They also revealed a fundamental tension that would persist throughout the field’s development: the need to balance the flexibility required for diverse scholarly purposes against the standardisation necessary for interoperability and long-term preservation (DeRose et al. Reference DeRose, Durand, Mylonas and Renear1990).

The 1980s saw critical developments in text encoding standards with the emergence of markup languages, particularly the Standard Generalised Markup Language (SGML), published as ISO 8879 in 1986. SGML provided a formal framework for defining document structures through markup, representing a significant conceptual advance over earlier approaches (Coombs, Renear & DeRose Reference Coombs, Renear and DeRose1987; Goldfarb Reference Goldfarb1990). SGML’s key innovation was the separation of document content from presentation and the use of descriptive markup, that is, tags that identify the structural or semantic role of text elements rather than specifying formatting. This approach enabled documents to be processed by different systems and for different purposes without modification, supporting what came to be known as the principle of format independence (DeRose et al. Reference DeRose, Durand, Mylonas and Renear1990; Renear, Durand & Mylonas Reference Renear, Durand, Mylonas, Ide and Hockey1996). The development of SGML reflected insights from earlier work in document processing and artificial intelligence, particularly the recognition that explicit representation of document structure facilitates automated processing and supports multiple uses of the same content (Coombs, Renear & DeRose Reference Coombs, Renear and DeRose1987).

The Text Encoding Initiative (TEI) was founded in 1987 to develop guidelines for encoding literary and linguistic texts in SGML, bringing together scholars, librarians and technologists from multiple disciplines and institutions (Sperberg-McQueen & Burnard Reference Sperberg-McQueen and Burnard1990; Ide & Sperberg-McQueen Reference Ide and Sperberg-McQueen1995). The TEI consortium represented an unprecedented collaborative effort to create a comprehensive, flexible encoding standard for humanities texts. The motivations for establishing the TEI were multiple: the recognition that individual projects were duplicating effort by developing incompatible encoding schemes, the desire to facilitate data sharing and reuse and the understanding that standardisation would be essential for long-term preservation of digital texts (Hockey Reference Hockey2000; Burnard Reference Burnard2014). The first version of the TEI Guidelines was published in 1994 (TEI P1 through P3), providing detailed specifications for encoding a wide range of textual features, e.g. structural elements, editorial interventions, linguistic annotations and manuscript descriptions (Sperberg-McQueen & Burnard Reference Sperberg-McQueen and Burnard1994). The Guidelines represented an enormous intellectual achievement, synthesising scholarly requirements from diverse disciplines and translating them into formal encoding specifications.

The TEI’s significance extended beyond technical specifications to encompass broader principles of encoding practice that have shaped the field. The Guidelines established the importance of explicit documentation, arguing that encoding decisions should be transparent and well documented to support scholarly interpretation and reuse (Sperberg-McQueen & Burnard Reference Sperberg-McQueen and Burnard1994; Renear Reference Renear, Schreibman, Siemens and Unsworth2004). They emphasised the value of interoperability, recognising that texts encoded according to common standards could be more easily shared, compared and integrated (Ide & Romary Reference Ide and Romary2004). The Guidelines stressed the need for extensibility to accommodate diverse text types and research needs, providing mechanisms for projects to define project-specific elements and attributes while remaining within the TEI framework (Burnard & Sperberg-McQueen Reference Burnard and Sperberg-McQueen2006). Finally, the TEI established the role of community governance in standards development, creating a model of collaborative, consensus-based decision-making that has proven remarkably durable (Cummings Reference Cummings, Siemens and Schreibman2008; Flanders & Jannidis Reference Flanders and Jannidis2019).

The TEI also introduced the concept of the TEI Header, a structured metadata section that accompanies encoded texts and provides essential descriptive, bibliographic and technical information (Ide & Sperberg-McQueen Reference Ide and Sperberg-McQueen1995; Pouchard Reference Pouchard1998). The Header represented a significant innovation in metadata practice, providing a rich, extensible framework for documenting electronic texts that went far beyond the bibliographic description typical of library catalogues. The Header’s four main sections, covering file description, encoding description, profile description and revision description, established a model for comprehensive metadata that has influenced subsequent metadata standards (Burnard & Bauman Reference Burnard and Bauman2013).

The late 1990s brought another transformative development with the emergence of XML (Extensible Markup Language) as a simplified, web-compatible subset of SGML. XML’s adoption by the World Wide Web Consortium (W3C) as a web standard in 1998 provided a more accessible markup framework and enabled integration with web technologies (Bray et al. Reference Bray, Jean Paoli, Maler and Yergeau1998; Harold & Means Reference Harold and Means2004). XML retained SGML’s key principles of descriptive markup and format independence while eliminating some of SGML’s more complex and rarely used features, making it easier to implement and more suitable for web-based applications Bray et al. (Reference Bray, Jean Paoli, Maler and Yergeau1998). The emergence of XML coincided with the rapid expansion of the World Wide Web and the growing importance of web-based delivery of digital resources, creating strong incentives for digital humanities projects to adopt XML-based encoding (Hockey Reference Hockey2000; Rahtz Reference Rahtz, Burnard, O’Keeffe and Unsworth2006).

The TEI Consortium responded to these developments by developing TEI P4 (2002), which supported both SGML and XML encoding, and subsequently TEI P5 (2007), which adopted XML as the primary format and introduced the ODD (One Document Does it all) system for schema customisation (Burnard & Bauman Reference Burnard and Bauman2007; Rahtz & Burnard Reference Rahtz and Burnard2013). TEI P5 represented a major consolidation and modernisation of the Guidelines, incorporating lessons learned from more than a decade of practical experience with TEI encoding. It introduced a modular architecture allowing projects to select relevant components and customise schemas through ODD specifications, providing a formal mechanism for what had previously been ad hoc customisation (Burnard & Rahtz Reference Burnard and Rahtz2004). This flexibility enabled TEI to accommodate diverse text types, from medieval manuscripts to historical newspapers to born-digital materials, while maintaining a common core that supported interoperability (Pierazzo Reference Pierazzo2015; Driscoll & Pierazzo Reference Driscoll and Pierazzo2016). Projects can now formally define project-specific encoding practices while remaining within the TEI framework, balancing local needs with community standards (Erjavec & Ogrin Reference Erjavec and Ogrin2005; Bauman Reference Bauman2011).

During this period, numerous large-scale digitisation projects adopted TEI as their encoding standard, establishing it as the de facto standard for scholarly text encoding. Libraries and archives began systematic digitisation of special collections, often partnering with scholars to develop appropriate encoding practices (Deegan & Tanner Reference Deegan and Tanner2002; Hughes Reference Hughes2004). Projects like the Electronic Text Center at the University of Virginia, the Women Writers Project at Brown University and the Oxford Text Archive established models for sustainable, TEI-based text collections (Flanders et al. Reference Flanders, Bauman, Caton and Caton1998; Price Reference Price2009). The period also saw the development of tools and infrastructure supporting TEI workflows, including XML editors such as oXygen and Emacs with specialised TEI modes, transformation tools based on XSLT, and publication platforms that could render TEI texts in various formats (Rahtz Reference Rahtz, Burnard, O’Keeffe and Unsworth2006). The development of this ecosystem of tools and resources was essential for making TEI practical for working scholars and for establishing it as a sustainable standard rather than merely an academic exercise (Cummings Reference Cummings, Siemens and Schreibman2008).

The past two decades have seen the maturation and diversification of digital text corpus practices, with TEI P5 becoming the de facto standard for scholarly text encoding while undergoing regular updates and refinements responding to community needs (Burnard Reference Burnard2014; Flanders & Jannidis Reference Flanders and Jannidis2019). The TEI Consortium supports active special interest groups addressing specific domains such as manuscript description, correspondence, music notation and ontologies, whose activities can contribute to the extension and modification of the Guidelines in response to specialised scholarly requirements (TEI Consortium 2023). The consortium has also expanded international participation, with growing involvement from non-English-speaking countries and attention to multilingual and non-Western textual traditions (Romary & Wegstein Reference Romary and Wegstein2012).

Several significant trends characterise contemporary practice in metadata and annotation. First, the scale of digitisation has increased dramatically, with projects now routinely involving millions of pages rather than hundreds or thousands (Terras Reference Terras2011; Prescott Reference Prescott2012). This scale has driven the development of automated and semi-automated workflows combining OCR, handwritten text recognition (HTR) and machine learning with human correction and validation (Springmann & Lüdeling Reference Springmann and Lüdeling2017; Rubinstein Reference Rubinstein2019; Milioni Reference Milioni2020). The Transkribus platform, for example, employs recurrent neural networks for HTR and has been widely adopted by libraries and archives for historical document transcription, demonstrating the potential of machine learning to scale digitisation efforts (Kahle et al. Reference Kahle, Colutto, Hackl and Mühlberger2017; Muehlberger et al. Reference Muehlberger, Seaward, Terras, Oliveira, Bosch, Bryan, Colutto, Déjean, Diem, Fiel, Gatos, Greinoecker, Grüning, Hackl, Haukkovaara, Heyer, Hirvonen, Hodel, Jokinen, Kahle, Kallio, Kaplan, Kleber, Labahn, Lang, Laube, Leifert, Louloudis, McNicholl, Meunier, Moreux, Nieminen, Retsinas, Sluyter-Gäthje, Souibgui, Stamatopoulos, Strauß, Terbul, Toselli, Ulreich, Villegas, Gorène, Walcher, Weidemann, Wurster and Zagoris2019). However, these automated approaches have also raised questions about quality control, the role of human expertise and the appropriate balance between efficiency and accuracy (Terras Reference Terras2006; Smith & Cordell Reference Smith and Cordell2018).

Second, collaborative and distributed workflows have become standard, supported by version control systems, web-based annotation tools and platforms that enable multiple contributors to work on shared corpora (Siemens & Burr Reference Siemens and Burr2013; Kuhry Reference Kuhry2020): tools such as the CoBaLT editor provide web-based interfaces for corpus annotation and lexicon construction with TEI P5 import and export, supporting efficient manual correction of automatically generated annotations while building reusable linguistic resources (Kenter et al. Reference Kenter, Erjavec, Dulmin, Fišer, Calzolari, Choukri, Declerck, Doğan, Maegaard, Mariani, Moreno, Odijk and Piperidis2012). The Annotonia tool enables browser-based annotation for editorial notes that can be compiled back into TEI files, supporting multi-editor workflows and status tracking (Kuhry Reference Kuhry2020). These collaborative tools reflect a broader shift in digital humanities toward distributed, team-based research that crosses institutional boundaries (Siemens Reference Siemens2009; Nowviskie Reference Nowviskie2011).

Third, there has been growing emphasis on semantic enrichment and linked data, with projects integrating TEI encoding with ontologies, authority files and semantic web technologies (Beretta Reference Beretta2016; Ciotti & Tomasi Reference Ciotti and Tomasi2016; Eide & Ore Reference Eide, Ore, Flanders and Jannidis2019). The Symogih project, for example, integrates historical ontologies with TEI encoding to produce semantically enriched transcriptions that support complex queries about historical actors, events and relationships (Beretta Reference Beretta2016). Such approaches reflect the influence of semantic web technologies on digital humanities practice and the recognition that explicit representation of semantic relationships can enhance the research value of digital texts (Ore & Eide Reference Ore and Eide2009; Ciula & Eide Reference Ciula and Eide2017).

Recent years have also seen increased attention to sustainability and preservation, driven by the recognition that many early digital projects have become inaccessible due to technological obsolescence or loss of institutional support (Warwick et al. Reference Warwick, Galina, Rimmer, Terras, Blandford, Gow and Buchanan2009; Schöch Reference Schöch2013). Repositories and platforms such as LAUDATIO, TextGrid and CLARIN have been developed to provide long-term hosting, discovery and access for digital text corpora (Neuroth et al. Reference Neuroth, Lohmeier and Smith2011; Odebrecht, Belz & Zeldes Reference Odebrecht, Belz and Zeldes2017). These initiatives emphasise comprehensive metadata documentation, format migration strategies and persistent identifiers to ensure corpora remain accessible and usable over decades. The adoption of FAIR principles (Findable, Accessible, Interoperable, Reusable) in research data management has reinforced the importance of robust metadata and standardised formats (Wilkinson et al. Reference Wilkinson, Dumontier, Aalbersberg, Appleton, Axton, Baak, Blomberg, Boiten, da Silva Santos, Bourne, Bouwman, Brookes, Clark, Crosas, Dillo, Dumon, Edmunds, Evelo, Finkers, Gonzalez-Beltran, Gray, Groth, Goble, Grethe, Jaap, Hoen, Rob, Tobias, Ruben, Joost, Lusher, Martone, Albert, Packer, Persson, Philippe, Marco, van Schaik, Sansone, Erik, Thierry, Ted, George, Swertz, Mark, van der Lei, van Mulligen, Jan, Andra, Peter, Katherine, Jun and Barend2016; Mons et al. Reference Mons, Neylon, Velterop, Dumontier, da Silva Santos and Wilkinson2017). The FAIR principles, originally developed in the sciences, have been increasingly adopted in digital humanities as a framework for ensuring the long-term value of digital resources (Borgman Reference Borgman2015; Schöch et al. Reference Schöch, Döhl, Rettinger, Gius, Trilcke, Leinen, Jannidis, Hinzmann and Röpke2020).

3. Metadata standards and frameworks

Metadata standards for historical text corpora serve multiple functions, enabling resource discovery, documenting provenance, supporting interoperability and facilitating long-term preservation (Greenberg Reference Greenberg2005; Riley Reference Riley2017). The landscape of metadata standards reflects both general information science developments and domain-specific requirements of humanities scholarship, with standards ranging from simple, broadly applicable schemas to rich, specialised frameworks designed for particular types of materials (Zeng & Qin Reference Zeng and Qin2016).

Descriptive metadata provides information about the intellectual content and bibliographic characteristics of texts, enabling discovery and supporting scholarly citation. The Dublin Core Metadata Initiative (DCMI), established in 1995, developed a simple, flexible metadata standard initially consisting of 15 core elements, i.e. title, creator, subject, description, publisher, contributor, date, type, format, identifier, source, language, relation, coverage and rights (Weibel et al. Reference Weibel, Kunze, Lagoze and Wolf1998; Hillmann Reference Hillmann2005). Dublin Core’s simplicity and broad applicability have made it one of the most widely implemented metadata standards across libraries, archives and digital repositories (Park Reference Park2009). For historical text corpora, Dublin Core provides a baseline level of description suitable for resource discovery, supporting harvesting via the OAI-PMH (Open Archives Initiative Protocol for Metadata Harvesting) protocol and enabling cross-domain searching (Lagoze & Van de Sompel Reference Lagoze and Van de Sompel2001). However, Dublin Core’s simplicity limits its expressiveness for detailed scholarly description, lacking specialised elements for manuscript description, editorial apparatus, linguistic annotation, or the complex bibliographic relationships common in historical texts (Greenberg et al. Reference Greenberg, Pattuelli, Parsia and Robertson2001). Consequently, Dublin Core is often used alongside more specialised metadata schemas in corpus projects, providing a simple, interoperable discovery layer while richer metadata resides in domain-specific formats (Haslhofer & Klas Reference Haslhofer and Klas2010). A simple example of descriptive metadata for a historical corpus text might include fields such as author (‘John Hart’), title (An Orthographie), publication date (‘1569’), genre (‘grammatical treatise’), repository identifier (e.g. EEBO ID) and language (‘Early Modern English’). Such metadata enables researchers to filter texts chronologically, by genre, or by provenance, while also supporting interoperability across repositories and corpus platforms.

METS (Metadata Encoding and Transmission Standard), developed by the Library of Congress and the digital library community, provides an XML framework for encoding descriptive, administrative and structural metadata for digital library objects (McDonough Reference McDonough2006). For complex digital objects like digitised historical books, which may consist of hundreds of page images, transcribed text files and derivative formats, METS provides a container format that organises all components and their relationships (Gartner Reference Gartner2003). MODS (Metadata Object Description Schema), also developed by the Library of Congress, is an XML schema for bibliographic metadata that provides a middle ground between the simplicity of Dublin Core and the complexity of MARC (Machine-Readable Cataloging) records (Guenther & McCallum Reference Guenther and McCallum2003). For historical text corpora, MODS can provide bibliographic descriptions appropriate for scholarly use while remaining more accessible than full MARC records (Dushay & Hillmann Reference Dushay and Hillmann2003).

Administrative metadata documents the technical characteristics, provenance, and rights information for digital objects. For historical text corpora, administrative metadata is essential for long-term preservation, reproducibility and rights management (Caplan Reference Caplan2003; Lavoie & Gartner Reference Lavoie and Gartner2005). Technical metadata describes the digital files themselves, including file formats, character encodings, image resolution, compression methods and other technical specifications. For digitised historical texts, technical metadata should document scanning parameters, OCR software and settings, and any post-processing applied (Kenney & Rieger Reference Kenney and Rieger2000). Provenance metadata records the history of digital objects, documenting who created them, when, using what processes and tools, and what modifications have been made over time (Buneman, Khanna & Tan Reference Buneman, Khanna and Tan2001). For scholarly corpora, provenance metadata supports reproducibility and enables users to assess the reliability of digital texts (Odebrecht, Belz & Zeldes Reference Odebrecht, Belz and Zeldes2017). Rights metadata specifies intellectual property rights, usage restrictions and access conditions, enabling users to understand what they can legally do with corpus materials (Besser Reference Besser and Sitts2001).

The TEI Header represents a domain-specific metadata schema tailored for scholarly text encoding, with four main sections providing comprehensive metadata about encoded texts (Pouchard Reference Pouchard1998; Burnard & Bauman Reference Burnard and Bauman2013). The File Description contains bibliographic information about the electronic file and the source text(s) from which it was derived. The Encoding Description documents the relationship between the electronic text and its source(s), specifying editorial principles, normalisation practices, sampling methods and the encoding scheme used. The Profile Description provides non-bibliographic metadata including languages used, text classification and contextual information. The Revision Description records the revision history of the electronic file, documenting changes over time and supporting version control. The TEI Header’s richness and flexibility have made it the primary metadata standard for scholarly text encoding projects, with projects routinely customising TEI Headers through ODD specifications to accommodate project-specific metadata needs while maintaining interoperability with the broader TEI ecosystem (Erjavec & Ogrin Reference Erjavec and Ogrin2005; Hagen, Ketzan & Jannidis Reference Hagen, Ketzan, Jannidis and Witt2020). For example, XML tags identify persons, titles and dates explicitly, enabling both human-readable transcription and machine-readable processing. Such markup facilitates searching, linking and semantic enrichment across corpora and digital repositories.

Repository projects have also developed metadata schemas to support corpus sharing and reuse. The LAUDATIO repository, designed for historical corpora, implemented a metadata schema covering corpus-level, document-level and annotation-level metadata, with explicit documentation of preparation steps, formats and relationships among corpus components (Odebrecht, Belz & Zeldes Reference Odebrecht, Belz and Zeldes2017). The HistCorp platform similarly emphasises standardised metadata elements to enable cross-corpus compatibility and discovery (Pettersson & Megyesi Reference Pettersson, Megyesi, Calzolari, Choukri, Cieri, Declerck, Goggi, Hasida, Isahara, Maegaard, Mariani, Mazo, Moreno, Odijk, Piperidis and Tokunaga2018).

4. Annotation frameworks and methodologies

Linguistic annotation adds information about the linguistic properties of texts, supporting computational analysis and linguistic research. For historical texts, linguistic annotation faces challenges arising from language change, orthographic variation and the limited availability of language resources for historical language varieties (Piotrowski Reference Piotrowski2012; Schneider Reference Schneider2017). Part-of-speech (POS) tagging assigns grammatical categories to words in texts, providing foundational information for many computational linguistic analyses (Leech, Garside & Bryant Reference Leech, Garside and Bryant1994). For contemporary languages, POS tagging is highly automated, with accuracy exceeding 97 per cent for English, but historical texts present greater challenges due to orthographic variation, morphological differences and lexical changes (Rayson et al. Reference Rayson, Archer, Baron and Smith2007; Scheible, Whitt & Dunning Reference Scheible, Whitt and Dunning2011).

Projects working with historical corpora typically employ hybrid approaches combining automated tagging with manual correction. Tools developed for contemporary language are adapted through training on historical data or through normalisation strategies that map historical spellings to modern equivalents before tagging (Baron, Rayson & Archer Reference Baron, Rayson and Archer2009; Bollmann Reference Bollmann2019). The CoBaLT (Corpus Building and Lexicon Tool) editor exemplifies this approach, providing a web-based interface for correcting automatically generated POS tags while simultaneously building project-specific lexicons (Kenter et al. Reference Kenter, Erjavec, Dulmin, Fišer, Calzolari, Choukri, Declerck, Doğan, Maegaard, Mariani, Moreno, Odijk and Piperidis2012). Syntactic parsing analyses sentence structure, identifying grammatical relationships among words and representing them as dependency structures or phrase-structure trees (Kübler, McDonald & Nivre Reference Kübler, McDonald and Nivre2009). Parsing historical texts is more challenging than POS tagging due to the greater complexity of the task and the scarcity of syntactically annotated historical training data (Pettersson, Megyesi & Nivre Reference Pettersson, Megyesi and Nivre2013).

Semantic annotation adds information about meaning, including word senses, semantic roles, named entities and conceptual categories (Ide & Pustejovsky Reference Ide and Pustejovsky2010). For historical texts, semantic annotation often focuses on domain-specific concepts and entities relevant to particular research questions. Projects may develop custom annotation schemes to capture historically specific categories such as social roles, administrative units and historical events that do not map neatly onto contemporary taxonomies (Ciotti & Tomasi Reference Ciotti and Tomasi2016). The Symogih project exemplifies semantic annotation for historical research, integrating a historical ontology with TEI encoding to produce semantically enriched transcriptions that support complex queries about historical actors, events and relationships (Beretta Reference Beretta2016).

Named Entity Recognition (NER) identifies and classifies named entities in texts, typically persons, locations, organisations and sometimes dates, quantities and other types (Nadeau & Sekine Reference Nadeau and Sekine2007). Historical NER faces distinctive challenges, as entity names may appear in variant spellings or in languages different from the main text, historical entities may not exist in contemporary knowledge bases, and geographic and administrative entities may have changed names, boundaries, or status over time (Ehrmann, Romanello & Clematide Reference Ehrmann, Romanello and Clematide2016; Boroş et al. Reference Boroş, Besançon, Ferret and Grau2018). TEI provides extensive facilities for encoding named entities through elements such as <persName>, <placeName>, <orgName> and <date>, which can include attributes specifying entity types, linking to authority records and providing normalised forms (Beretta Reference Beretta2016; Kurz & Fischer-Nebmaier Reference Kurz and Fischer-Nebmaier2020).

Temporal and spatial annotation marks time expressions and place references in texts, potentially linking them to formal timelines, chronologies, or gazetteers (Grover et al. Reference Grover, Tobin, Byrne, Woollard, Reid, Dunn and Ball2010; Leidner & Lieberman Reference Leidner and Lieberman2011). TEI provides elements for temporal annotation such as <date>, <time> and <event>, which can include machine-readable temporal attributes alongside the original text. For complex historical materials, projects may employ event-based modelling that represents historical events with their participants, times, places and relationships (Kurz & Fischer-Nebmaier Reference Kurz and Fischer-Nebmaier2020).

A fundamental design choice in annotation frameworks is whether annotations are embedded directly in text through inline annotation or stored separately with pointers to text locations through standoff annotation (Ide & Romary Reference Ide and Romary2004; Schmidt Reference Schmidt2010). Inline annotation, exemplified by TEI XML, embeds annotations directly in text using markup tags, offering simplicity and clarity but facing challenges with multiple overlapping annotation layers (Sperberg-McQueen & Huitfeldt Reference Sperberg-McQueen and Huitfeldt1999; DeRose Reference DeRose2004). Standoff annotation stores annotations separately from base text, using pointers to indicate what text is annotated, enabling multiple independent annotation layers and preservation of original text unchanged but introducing complexity in maintaining alignment (Thompson & McKelvie Reference Thompson and McKelvie1997; Witt Reference Witt2004). In practice, many projects use hybrid approaches, combining inline TEI annotation with standoff annotations for additional layers (Banski Reference Bański2010).

5. Challenges in historical text processing

Historical texts exhibit far greater orthographic variation than contemporary texts, as before the standardisation of spelling in the eighteenth and nineteenth centuries, writers employed variable spellings even within single texts (Nevalainen & Raumolin-Brunberg Reference Nevalainen and Raumolin-Brunberg2003; Stein & Stiles Reference Stein and Stiles2009). This variation poses significant challenges for computational processing, complicating searching, impeding POS tagging and lemmatisation, and affecting frequency analysis and other quantitative methods (Baron, Rayson & Archer Reference Baron, Rayson and Archer2009). Projects have developed several strategies to address orthographic variation, including normalisation that maps historical spellings to modern equivalents, multiple layers of transcription providing both diplomatic and normalised transcriptions, flexible search and matching algorithms employing fuzzy matching or edit distance, and lexicon-based approaches building project-specific lexicons mapping attested historical forms to lemmas and modern forms (Piotrowski Reference Piotrowski2012; Bollmann Reference Bollmann2019). The Jerusalem Corpus of Emergent Modern Hebrew illustrates these challenges and solutions, as Hebrew texts from the late nineteenth and early twentieth centuries exhibit enormous orthographic variation due to the rapid development of the language during this period (Rubinstein Reference Rubinstein2019).

Beyond orthographic variation, historical texts reflect earlier stages of language evolution with different vocabulary, morphology and syntax (Traugott & Dasher Reference Traugott and Dasher2001; Rissanen Reference Rissanen2008). Many historical language varieties lack the linguistic resources, such as dictionaries, grammars and annotated corpora, that support automated processing of contemporary languages (Piotrowski Reference Piotrowski2012). Language change affects lexical semantics, as words may have had different meanings in the past or may not yet have had modern meanings (Traugott & Dasher Reference Traugott and Dasher2001). Morphological systems may differ, with inflectional paradigms being richer or different, and derivational processes being more or less productive (Rissanen Reference Rissanen2008). These differences necessitate historical language resources and tools, with projects working with historical corpora often needing to create language-specific resources as part of corpus development (Scheible, Whitt & Dunning Reference Scheible, Whitt and Dunning2011).

Optical Character Recognition (OCR) is the primary method for converting printed historical texts to digital form, but OCR accuracy on historical materials is substantially lower than on contemporary printed texts (Holley Reference Holley2009; Strange et al. Reference Strange, McNamara, Wodak and Wood2014). Factors affecting OCR quality include historical typefaces and unusual characters, degraded paper and uneven inking, multi-column layouts and complex page structures, and the absence of language models for historical language varieties (Smith Reference Smith2002; Springmann & Lüdeling Reference Springmann and Lüdeling2017). OCR errors propagate through subsequent processing, affecting search, annotation and analysis (Tanner, Muñoz & Hemy Ros Reference Tanner, Munoz and Ros2009). Projects must balance the cost of manual correction against the impact of errors on research uses, employing strategies such as selective correction focusing manual effort on high-value materials, double-keying or crowdsourcing distributing correction labour, iterative improvement using corrected texts to retrain OCR systems, and OCR post-correction tools employing pattern-based or statistical methods (Ayuso Reference Ayuso2017; Rubinstein Reference Rubinstein2019).

Recent developments in handwritten text recognition (HTR) and deep learning-based OCR show promise for historical materials. The Transkribus platform, which employs recurrent neural networks for HTR and OCR, has been widely adopted by libraries and archives for the transcription of historical documents (Muehlberger et al. Reference Muehlberger, Seaward, Terras, Oliveira, Bosch, Bryan, Colutto, Déjean, Diem, Fiel, Gatos, Greinoecker, Grüning, Hackl, Haukkovaara, Heyer, Hirvonen, Hodel, Jokinen, Kahle, Kallio, Kaplan, Kleber, Labahn, Lang, Laube, Leifert, Louloudis, McNicholl, Meunier, Moreux, Nieminen, Retsinas, Sluyter-Gäthje, Souibgui, Stamatopoulos, Strauß, Terbul, Toselli, Ulreich, Villegas, Gorène, Walcher, Weidemann, Wurster and Zagoris2019; Milioni Reference Milioni2020). Curated training data for specific historical periods and languages can substantially improve OCR accuracy (Springmann et al. Reference Springmann, Fink and Schulz2018).

Historical texts often lack complete bibliographic information, with manuscripts potentially being undated or of uncertain authorship and early printed books sometimes lacking title pages or publication information (Deegan & Tanner Reference Deegan and Tanner2002). Projects must decide how to represent uncertainty and incomplete information in metadata, with TEI providing mechanisms for encoding uncertain or alternative attributions, approximate dates and other forms of incomplete information (Pierazzo Reference Pierazzo2015). Crowdsourcing and collaborative approaches can help fill metadata gaps, as projects like the Chinese historical archives crowdsourcing initiative have successfully engaged volunteers in metadata creation tasks (Liang, Wang & Li Reference Liang, Liang, Wang and Li2020).

The proliferation of digital text projects has created interoperability challenges, as projects use different metadata schemas, encoding standards, file formats and annotation frameworks (Ide & Romary Reference Ide and Romary2004). This heterogeneity impedes data integration, cross-corpus searching and comparative analysis. Standardisation through TEI and other community standards addresses some interoperability challenges, but projects inevitably make different choices in applying standards (Burnard Reference Burnard2014). Several approaches support interoperability despite heterogeneity, including metadata crosswalks mapping elements between different schemas, repository platforms providing infrastructure for documenting corpus formats, standoff annotation and Web Annotation standards enabling annotations to be layered on existing texts, and linked data approaches using semantic web technologies to link entities and concepts across datasets (Kuster Reference Kuster2011; Beretta Reference Beretta2016).

Digital preservation is a critical concern for historical text corpora, as digital materials are fragile with file formats becoming obsolete, storage media degrading, software dependencies changing and institutional commitments wavering (Warwick et al. Reference Warwick, Galina, Rimmer, Terras, Blandford, Gow and Buchanan2009; Schöch Reference Schöch2013). Best practices for digital preservation include format selection using open, well-documented, non-proprietary formats; redundancy maintaining multiple copies in geographically distributed locations; comprehensive documentation and metadata; planning for format migration; ensuring long-term institutional support; and depositing materials in trusted repositories (Caplan Reference Caplan2008). The FAIR principles provide a framework for assessing and improving the long-term value of digital resources (Wilkinson et al. Reference Wilkinson, Dumontier, Aalbersberg, Appleton, Axton, Baak, Blomberg, Boiten, da Silva Santos, Bourne, Bouwman, Brookes, Clark, Crosas, Dillo, Dumon, Edmunds, Evelo, Finkers, Gonzalez-Beltran, Gray, Groth, Goble, Grethe, Jaap, Hoen, Rob, Tobias, Ruben, Joost, Lusher, Martone, Albert, Packer, Persson, Philippe, Marco, van Schaik, Sansone, Erik, Thierry, Ted, George, Swertz, Mark, van der Lei, van Mulligen, Jan, Andra, Peter, Katherine, Jun and Barend2016).

6. Best practices, emerging trends and future directions

Effective metadata schemas for historical text corpora balance expressiveness, simplicity, interoperability and extensibility (Greenberg Reference Greenberg2005). Key principles include building on established standards like TEI, Dublin Core, or domain-specific schemas rather than creating entirely custom schemas; documenting customisations explicitly through formal mechanisms such as TEI ODD specifications; separating concerns by using appropriate metadata schemas for different purposes; recording provenance comprehensively by documenting sources, processes, tools and decisions; and planning for incompleteness and uncertainty by providing mechanisms for representing uncertainty and gaps in knowledge (Erjavec & Ogrin Reference Erjavec and Ogrin2005; Hagen, Ketzan & Jannidis Reference Hagen, Ketzan, Jannidis and Witt2020).

The long-term success of TEI in historical corpus and digital editing projects can largely be attributed to its balance between standardisation and flexibility. Unlike more rigid or highly specialised encoding systems, TEI accommodates a wide range of textual traditions and scholarly objectives while remaining extensible through customisation mechanisms such as ODD (Burnard & Bauman Reference Burnard and Bauman2007; Burnard Reference Burnard2014). Equally important has been the existence of an active international community that continuously maintains guidelines, tools and documentation. At the same time, recent years have seen the growing coexistence of lightweight workflows alongside rich XML-based approaches. While TEI offers expressive and semantically detailed encoding, lightweight architectures based on plain text, CSV, or simplified metadata schemas often provide greater ease of use, lower technical barriers, and improved compatibility with corpus analysis platforms and machine-learning pipelines. Consequently, unresolved tensions remain between interoperability and usability: highly structured encoding may enhance preservation and semantic richness but can also increase complexity, reduce accessibility for smaller projects, and complicate integration with analytical tools not designed for richly marked-up texts.

Consistent annotation requires clear guidelines and quality control procedures (Fort, Sagot & Adda Reference Fort, Sagot and Adda2011). Annotation guidelines should specify scope and definitions, decision procedures for handling ambiguous cases, encoding conventions and quality control procedures (Ide & Pustejovsky Reference Ide and Pustejovsky2010). For collaborative annotation projects, inter-annotator agreement measures provide quantitative assessment of consistency, with common measures including percent agreement, Cohen’s kappa and Krippendorff’s alpha (Artstein & Poesio Reference Artstein and Poesio2008). The medieval glosses project illustrates the importance of clear guidelines and tool support for complex annotation tasks (Kuhry Reference Kuhry2020).

Quality control is essential throughout corpus creation, with strategies including automated validation using schema validation and project-specific validation rules, manual review sampling and reviewing materials at multiple stages, iterative refinement treating corpus creation as iterative, error tracking documenting known issues and limitations, and version control using systems to track changes and enable rollback (Burnard Reference Burnard2014). Comprehensive documentation is essential for corpus usability and reuse, covering corpus design, sources, processing, encoding, technical details and limitations (Pouchard Reference Pouchard1998).

Digital humanities is an inherently collaborative discipline, and corpus projects benefit from engagement with community standards and initiatives (Siemens Reference Siemens2009). Best practices include participating in standards development, sharing resources such as encoding guidelines and tools, using shared infrastructure, including community repositories and authority files, collaborating across institutions, and adopting open practices using open licenses, formats and tools (Borgman Reference Borgman2015).

Several projects exemplify best practices in historical corpus creation. The EncycNet project encoded 22 historical German encyclopaedias in TEI, developing detailed encoding guidelines and proposing adaptations to TEI lexical modules (Hagen, Ketzan & Jannidis Reference Hagen, Ketzan, Jannidis and Witt2020). The CoBaLT editor provides a web-based interface for corpus annotation and lexicon construction with TEI P5 import and export (Kenter et al. Reference Kenter, Erjavec, Dulmin, Fišer, Calzolari, Choukri, Declerck, Doğan, Maegaard, Mariani, Moreno, Odijk and Piperidis2012). The Transkribus platform has been widely adopted for HTR and OCR of historical documents (Milioni Reference Milioni2020). The LAUDATIO repository provides infrastructure for long-term hosting and access to historical corpora (Odebrecht, Belz & Zeldes Reference Odebrecht, Belz and Zeldes2017). The Symogih project integrates historical ontologies with TEI encoding (Beretta Reference Beretta2016).

Three landmark historical corpora demonstrate the evolution of metadata and annotation practices over several decades and illustrate different approaches to addressing the challenges of historical language data. The Helsinki Corpus of English Texts, compiled in the 1980s and early 1990s, represents one of the earliest systematic attempts to create a diachronic corpus with comprehensive metadata (Rissanen et al. Reference Kytö, Rissanen, Rissanen, Ihalainen, Nevalainen and Taavitsainen1991; Kytö Reference Kytö1996). Covering Old, Middle and Early Modern English from roughly 750 to 1700, the Helsinki Corpus established important principles for historical corpus design including systematic period sampling, genre classification and detailed sociolinguistic metadata documenting text type, dialect, author information where available, and manuscript or print source details. The corpus’s metadata scheme, though developed before the widespread adoption of TEI, anticipated many features that would become standard in later projects, particularly the importance of documenting text provenance and sociolinguistic context (Rissanen Reference Rissanen and Svartvik1992). The Helsinki Corpus was annotated for part-of-speech tags using a tagset adapted for historical English morphology, demonstrating early recognition that contemporary linguistic tools required modification for historical materials (Kytö & Rissanen Reference Kytö, Rissanen, Rissanen, Ihalainen, Nevalainen and Taavitsainen1992). The corpus’s influence on subsequent historical corpus projects has been substantial, establishing methodological templates for sampling strategies, metadata documentation and linguistic annotation that continue to inform contemporary practice (Claridge Reference Claridge2008).

The Coruña Corpus of English Scientific Writing, developed in the early 2000s, built upon the Helsinki model while extending coverage and refining annotation practices (Moskowich & Crespo Reference Moskowich and Crespo2012). Focusing on scientific writing from the late seventeenth to the early nineteenth century, the Coruña Corpus demonstrates the value of domain-specific historical corpora and the role of detailed metadata in supporting specialised research questions. The corpus’s metadata scheme documents not only standard bibliographic information but also discipline-specific categories including scientific field, text function (theoretical, practical, educational) and intended audience (Moskowich Reference Moskowich, Moskowich and Crespo2016). This rich metadata enables researchers to investigate the development of scientific discourse across disciplines and to trace the emergence of specialised registers and terminologies. The Coruña Corpus also illustrates the challenges of working with early printed scientific texts, where complex layouts, mathematical notation, tables and diagrams require careful encoding decisions. The project’s documentation of these encoding choices and the rationale behind them exemplifies the principle of transparency in corpus construction (Crespo & Moskowich Reference Crespo and Moskowich2015).

Another influential resource closely connected to the Helsinki tradition is the Corpus of Early English Medical Writing (CEEM), developed between the late 1990s and early 2000s as part of the MEMT (Middle English Medical Texts) and EMEMT (Early Modern English Medical Texts) projects (Taavitsainen & Pahta Reference Taavitsainen and Pahta2010). Covering medical and scientific writing from approximately 1375 to 1800, CEEM extended many of the methodological principles established by the Helsinki Corpus while adapting them to the needs of specialised discourse analysis. The corpus adopted carefully balanced sampling procedures and detailed metadata categories that document authorship, genre, date and disciplinary subfields, thereby facilitating diachronic studies of the development of scientific and medical language. CEEM also became influential for later specialised corpora, including the Coruña Corpus, particularly regarding sample-size conventions and the organisation of domain-specific subcorpora. At the same time, methodological exchanges between the CEEM and Coruña teams contributed to the consolidation of best practices in the design of historical scientific corpora, including decisions concerning disciplinary coverage, metadata granularity and representativeness (Moskowich & Crespo Reference Moskowich and Crespo2012; Taavitsainen & Pahta Reference Taavitsainen and Pahta2010).

Early English Books Online (EEBO), representing a much larger scale of digitisation than either the Helsinki or Coruña corpora, encompasses over 125,000 Early Modern English texts printed between 1473 and 1700 (Gadd Reference Gadd2009; Bruster & Smith Reference Bruster and Smith2016). EEBO’s metadata, derived from bibliographic records in the English Short Title Catalogue, provides standardised descriptive information including author, title, imprint, physical description and subject headings (Lancashire Reference Lancashire and Burke2013). The availability of EEBO texts through platforms such as Sketch Engine (Kilgarriff et al. Reference Kilgarriff, Rychlý, Smrž and Tugwell2004; Kilgarriff et al. Reference Kilgarriff, Baisa, Bušta, Jakubíček, Kovář, Michelfeit, Rychlý and Suchomel2014) has enabled large-scale corpus linguistic analysis, though the quality and consistency of OCR remain variable due to the challenges of early modern typography and print quality (Baron & Rayson Reference Baron and Rayson2008; Bruster & Smith Reference Bruster and Smith2016).Footnote ⁴ The EEBO-TCP (Text Creation Partnership) project has produced corrected, TEI-encoded transcriptions of over 25,000 EEBO texts, with markup capturing structural features, typographic distinctionsand editorial interventions (Mueller Reference Mueller2013). These TEI texts include more granular metadata than the basic EEBO records, documenting encoding practices, source editions and revision history through TEI Headers (Gadd Reference Gadd2009). The linguistic annotation of EEBO texts has been undertaken through several initiatives, including the EEBO-TCP Phase I texts, which have been lemmatised and part-of-speech tagged, enabling grammatical and lexical searches not possible with raw text. The scale of EEBO and the variety of approaches to its annotation and enhancement illustrate both the opportunities and challenges of working with large historical text collections, particularly the tension between comprehensive coverage and depth of annotation, and the importance of metadata for managing and making discoverable such extensive resources (Lancashire Reference Lancashire and Burke2013; Mueller Reference Mueller2013).

Linked Open Data (LOD) and semantic web technologies offer powerful approaches to connecting historical text corpora with external knowledge bases and enabling sophisticated queries across datasets (Berners-Lee, Hendler & Lassila Reference Berners-Lee, Hendler and Lassila2001; Heath & Bizer Reference Heath and Bizer2011). For historical texts, LOD approaches enable linking persons, places, events and concepts mentioned in texts to authority files, gazetteers, prosopographies and other structured knowledge sources (Ciotti & Tomasi Reference Ciotti and Tomasi2016; Eide & Ore Reference Eide, Ore, Flanders and Jannidis2019). The Symogih project uses RDF to represent relationships among historical actors, events and sources while maintaining TEI as the primary text encoding format (Beretta Reference Beretta2016). Challenges remain in balancing the expressiveness of TEI with the more constrained modelling of RDF (Ciula & Eide Reference Ciula and Eide2017).

Machine learning, particularly deep learning, has transformed many text processing tasks, with neural network models achieving high accuracy on POS tagging, NER, syntactic parsing and other annotation tasks for contemporary languages (Manning Reference Manning2015; Goldberg Reference Goldberg2017). For historical texts, machine learning offers the potential to reduce manual annotation effort and scale corpus creation, though challenges remain including training data scarcity, domain mismatch and interpretability concerns (Piotrowski Reference Piotrowski2012; Bollmann Reference Bollmann2019). Despite challenges, ML-assisted workflows are increasingly common, with projects using ML for pre-annotation followed by human expert correction (Springmann & Lüdeling Reference Springmann and Lüdeling2017; Muehlberger et al. Reference Muehlberger, Seaward, Terras, Oliveira, Bosch, Bryan, Colutto, Déjean, Diem, Fiel, Gatos, Greinoecker, Grüning, Hackl, Haukkovaara, Heyer, Hirvonen, Hodel, Jokinen, Kahle, Kallio, Kaplan, Kleber, Labahn, Lang, Laube, Leifert, Louloudis, McNicholl, Meunier, Moreux, Nieminen, Retsinas, Sluyter-Gäthje, Souibgui, Stamatopoulos, Strauß, Terbul, Toselli, Ulreich, Villegas, Gorène, Walcher, Weidemann, Wurster and Zagoris2019).

Most digital humanities work has focused on Western European languages, particularly English, but there is growing recognition of the need for multilingual and cross-cultural approaches that encompass a broader range of languages, scripts and cultural traditions (Romary & Wegstein Reference Romary and Wegstein2012). Challenges in multilingual corpus creation include script diversity requiring appropriate fonts and rendering support, language resources being lacking for many languages, cultural context with encoding schemes potentially not accommodating different textual traditions, and technical infrastructure varying across languages (Thaller Reference Thaller2012; Rubinstein Reference Rubinstein2019).

Crowdsourcing has proven effective for large-scale digitization and annotation projects, with platforms like Zooniverse, FromThePage, and institutional crowdsourcing initiatives engaging thousands of volunteers (Ridge Reference Ridge2014; Causer & Terras Reference Causer and Terras2014). Successful crowdsourcing requires careful task design, breaking work into manageable units, providing clear instructions, implementing quality control mechanisms, and giving volunteers feedback and recognition (Liang, Wang & Li Reference Liang, Liang, Wang and Li2020). Crowdsourcing is particularly effective for tasks that benefit from human judgment but do not require specialized expertise (Carletti et al. Reference Carletti, Giannachi, Price and McAuley2013).

The FAIR principles (Findable, Accessible, Interoperable, Reusable) have gained widespread adoption in research data management (Wilkinson et al. Reference Wilkinson, Dumontier, Aalbersberg, Appleton, Axton, Baak, Blomberg, Boiten, da Silva Santos, Bourne, Bouwman, Brookes, Clark, Crosas, Dillo, Dumon, Edmunds, Evelo, Finkers, Gonzalez-Beltran, Gray, Groth, Goble, Grethe, Jaap, Hoen, Rob, Tobias, Ruben, Joost, Lusher, Martone, Albert, Packer, Persson, Philippe, Marco, van Schaik, Sansone, Erik, Thierry, Ted, George, Swertz, Mark, van der Lei, van Mulligen, Jan, Andra, Peter, Katherine, Jun and Barend2016). For historical text corpora, FAIR principles reinforce best practices including use of standard formats like TEI, comprehensive metadata documentation, deposit in repositories with persistent identifiers, and adoption of open licenses (Borgman Reference Borgman2015; Schöch et al. Reference Schöch, Döhl, Rettinger, Gius, Trilcke, Leinen, Jannidis, Hinzmann and Röpke2020).

7. Case study: MetaLing Corpus

The MetaLing Corpus provides a valuable testbed for evaluating the feasibility and limitations of the metadata and annotation practices outlined in the previous sections. While modest in scale (the corpus comprises approximately one million tokens), it exemplifies many of the challenges that arise in the construction of historical linguistic corpora. Developed within a two-year publicly funded Italian research project (Andreani & Russo Reference Andreani and Russo2023, Reference Andreani and Russo2026), the corpus is designed to support the investigation of English metalanguage between 1500 and 1700, with a focus on how linguistic self-reflection and conceptual vocabulary evolved over this period.

A defining feature of the MetaLing project was its selection of historical sources. Rather than aiming for exhaustive coverage, the project made a strategic decision to concentrate on a curated set of texts identified in Alston’s Bibliography of the English Language from the Invention of Printing to the Year 1800 (Reference Alston1974). Alston’s multi-volume bibliography remains a foundational reference in English historical linguistics and lexicography, offering a genre-diverse, chronologically structured inventory of works on English language matters. Given constraints in both time and access to certain volumes, the selection process was necessarily selective and thematic. The team adopted a phenomenon-driven approach, beginning with a focus on linguistic variation, a core theme in early modern language reflection.

Corpus construction began with the compilation of a detailed spreadsheet that served as both a metadata source and a working log. Fields included author name, title, publication date, format, repository location, associated Alston volume and entry number, and provenance identifiers such as OTA (Oxford Text Archive) codes and EEBO (Early English Books Online) references. This spreadsheet laid the groundwork for metadata collection and informed subsequent steps in encoding and publication.

Most digitised texts were sourced from the OTA, which provides materials in various formats, including plain text and TEI-XML. Some texts unavailable in digital form were transcribed using the Transkribus platform, which combines text recognition with machine learning. While Transkribus offered a functional entry point for processing texts, the results required extensive human verification. With only two to four team members available at any one time, this step proved labour-intensive, underscoring the resource demands of manual transcription and correction workflows, even when assisted by advanced tools.

An early experiment in orthographic normalisation using VARD 2 (Baron & Rayson Reference Baron and Rayson2008), a tool designed to standardise spelling variation in historical English, was ultimately abandoned. The system produced inconsistent results across texts, introducing irregularities (especially with the ‘long s’ ⟨ſ⟩) that risked obscuring valuable linguistic variation. Furthermore, extensive normalisation was found to undermine core research goals, such as spelling conventions.Footnote ⁵ This experience reaffirms a key principle in historical corpus construction: annotation and normalisation strategies must be tailored to the research objectives, and overly aggressive interventions may limit the interpretability and reuse of data.

Metadata from the editorial spreadsheet were converted into Dublin Core format using Omeka,Footnote ⁶ an open-source platform for managing digital collections, which was selected for its balance between simplicity and interoperability, facilitating basic discovery functions while remaining adaptable for export into richer metadata environments. While the TEI-XML format initially appeared ideal for text encoding, especially given its alignment with best practices described in earlier sections, its integration with Sketch Engine (Kilgarriff et al. Reference Kilgarriff, Rychlý, Smrž and Tugwell2004; Kilgarriff et al. Reference Kilgarriff, Baisa, Bušta, Jakubíček, Kovář, Michelfeit, Rychlý and Suchomel2014), the platform adopted by the research institution for corpus analysis, proved problematic. TEI markup was not fully compatible, often rendering XML tags as plain text within the corpus interface. Conversely, Sketch Engine’s tokenisation engine performed acceptably well on raw historical texts, albeit with several issues.Footnote ⁷

Faced with this incompatibility, the project adopted a minimalist strategy: the final corpus consists of plain text (.txt) files, paired with structured metadata delivered via Omeka. This lightweight architecture enables broad compatibility across analysis environments and preserves maximum flexibility for future annotation and transformation pipelines. The decision to decouple metadata from corpus files also reflects an increasing trend toward modular, standoff-compatible designs in corpus linguistics.

From a FAIR (Findable, Accessible, Interoperable, Reusable) perspective, the MetaLing Corpus exhibits several aligned practices. The use of Dublin Core facilitates discovery through widely supported protocols such as OAI-PMH. By avoiding tightly coupled, platform-specific encoding, the corpus remains interoperable and adaptable, ready for integration with TEI, CMDI, or other schemas should future research directions require more elaborate markup. Reusability is further supported by the project’s commitment to transparent documentation and the use of open standards and formats.

The MetaLing project also illustrates broader challenges and opportunities in historical corpus development. Its modest size allowed for close control and experimentation, highlighting trade-offs between expressiveness, interoperability and analytical usability. The difficulties encountered with TEI-to-Sketch Engine integration, VARD-based normalisation and transcription bottlenecks reflect common tensions in historical corpus work. Yet the project’s solutions (pragmatic data structuring, open-format design and metadata modularity) offer a replicable model for similarly scoped corpora. These decisions exemplify best practices in aligning corpus architecture with both research goals and platform constraints.

Finally, the MetaLing Corpus reinforces the importance of corpus-building as an interdisciplinary endeavour. Linguists, philologists, software developers and archivists must collaborate to ensure that corpora serve not just as textual repositories, but as structured, sustainable research infrastructures.