📄 Annotating 200 Million Words: The Bank of English Project

Summary

The paper reports how the Bank of English, a corpus planned at about 200 million words of written and spoken English, was tagged morphologically and syntactically. The corpus is sponsored by HarperCollins and compiled by the COBUILD team in Birmingham. The annotation was done at the University of Helsinki in 1993–94. Text arrived from COBUILD in monthly batches of roughly 10 million words, each with markup codes. Every batch went through a fixed cycle:

• standardising preprocessing,
• a filtering pass with the ENGTWOL analyser to find word forms missing from the lexicon,
• semi-automatic lexicon extension,
• full analysis with the English Constraint Grammar (ENGCG) pipeline: morphological disambiguation, then syntactic mapping and pruning.

By the time of writing, about 103 million words had been processed. Lexicon growth had slowed sharply, and the rule-based syntax had been largely rewritten. Syntactic tagging reached roughly 7% wrong tags, with notable leftover ambiguity, much of it from prepositional attachment.

The work matters because it shows that a hand-written, explainable rule system can annotate a very large, heterogeneous corpus at production speed. It also shows how routine maintenance keeps that system accurate.

Key points

• Pipeline stages: preprocessing, ENGTWOL lexical analysis, ENGCG morphological disambiguation, then ENGCG syntactic mapping and disambiguation. The whole chain runs at about 400 words per second on a SUN SPARCstation 10.
• Lexicon:
  • ENGTWOL holds about 75,000 entries.
  • Before updating, it covered 97–98% of tokens in incoming text.
  • The 11 batches annotated so far (103,636,771 words) added 18,834 entries, with 6,540 of them from the first batch.
  • Later batches needed very few additions.
• Morphological disambiguation: around 1,100 rules written by Voutilainen, combining grammar-based and heuristic rules. They leave little ambiguity and make very few errors.
• Syntax:
  • The grammar uses 282 mapping rules, 492 syntactic constraints and 204 heuristic constraints.
  • It assigns the wrong syntactic code to about 7 of 100 words on new text.
  • About 16% of words remain ambiguous, mostly due to PP attachment.
• Testing: a hand-disambiguated syntactic test corpus of 16 samples covering varied text types, almost all continuous unedited excerpts from the Bank.

Interesting ideas

• I1. Before each full analysis, run the analyser in a filter-only mode that lists unknown word forms, then triage the most frequent ones; coverage gaps become visible early and cheaply.
• I2. Raw token frequency misleads when the data repeats itself (daily broadcasts recycle text), so the threshold for adding a word is judged case by case rather than fixed.
• I3. Frequent informal or non-standard forms (wanna, veggie) get proper lexical entries, because a missing reading can wreck the analysis of a whole sentence. Unknown words fall back to an underspecified tag (SG/PL) instead of a forced guess.
• I4. The preprocessor fixes things like run-together words but inserts a code marking each correction, so no original information is lost.
• I5. Heuristic rules form a separate, optional layer from safe rules, so users can trade more remaining ambiguity for fewer errors, and the system keeps truly unresolvable attachments ambiguous.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): run this filter on every new text collection before parsing, so the en lexicon grows from real out-of-vocabulary lists rather than from parse failures.
• I2 (learn): when silver data is drawn from repetitive sources, deduplicate or count distinct documents before frequency decides what enters the lexicon or the training set.
• I3 (tokens): give colloquial forms in fables and dialogue real entries in crate en, and let the morphology output underspecified features rather than a false default.
• I4 (tokens): log every normalisation the tokenizer makes as an explicit annotation, which keeps the explainability chain intact back to the raw text.
• I5 (tree): split repair rules into reliable and heuristic tiers, and let downstream stages see genuine PP-attachment ambiguity rather than a forced single tree.