📄 CLAWS4: The Tagging of the British National Corpus

Summary

The paper presents CLAWS4, the part-of-speech tagger from Lancaster's UCREL group that was used to annotate the 100-million-word British National Corpus (BNC). The corpus mixes many written genres with 10 million words of speech, so the tagger had to be extremely robust. It also had to handle several tagsets and the TEI-based CDIF markup.

CLAWS4 keeps the older CLAWS pipeline:

• segmentation;
• lexicon- and suffix-based candidate tags;
• a rule-driven "idiom" (phraseological) stage built on regular-expression templates;
• HMM disambiguation over tag bigrams;
• a second deterministic rule pass that repairs typical errors.

When evidence is too weak, the tagger outputs a portmanteau tag that keeps two readings. Spoken data is handled by swapping in supplementary lexicons and idiom lists and by ignoring disfluent repetitions.

Measured on major categories, the error rate is about 1.5%, with 3.3% of tokens left ambiguous. The authors argue that such figures mean little unless they are tied to a published annotation scheme. They propose judging tagsets by consistency, size, discriminative value and linguistic quality.

Key points

• Scale: the BNC holds c.100M words, of which c.70M were tagged by April 1994. About 10M words are spoken, and c.4.6M of those are conversation.

• Idiom lexicons: these hold over 3000 entries in total. They cover:

  • complex prepositions and conjunctions (so that, according to);
  • multiword names (Mount Pleasant);
  • foreign phrases (ad hoc).

  Ditto tags such as CJS21/CJS22 mark the position of each word within a multiword unit.

• Disambiguation model: an HMM over tag bigrams. Trigrams were tried and gave little gain.

• Accuracy: on major categories, ~1.5% errors and ~3.3% unresolved portmanteau tags. Counting distinctions inside major categories (e.g. NP0-NN1), the figures rise to 1.78% and 4.60%.

• Tagsets: C5 (58 tags) was used for the whole corpus and C6 (138 tags) for the 2M-word Core Corpus. The richer tagset barely changed the error rate.

Interesting ideas

• I1. Run hand-written pattern rules twice: first to propose ambiguous readings for the statistical model, then deterministically afterwards to patch its known systematic mistakes (e.g. VVB→VVI after do).
• I2. Instead of forcing a guess, emit a two-way "portmanteau" tag when the evidence is insufficient, so downstream components see honest uncertainty.
• I3. Treat multiword units (complex prepositions, names, foreign phrases) as single grammatical items, with positional "ditto" tags on each word.
• I4. Swap resource files (lexicon, rule lists, transition statistics) by text type, and filter disfluent repetitions before tagging speech.
• I5. Evaluate accuracy only against an explicit annotation scheme, and weigh each tag distinction by its discriminative value and practical usefulness, not raw error rate alone.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): A post-tagger repair pass with explicit, logged rules would fix recurring errors like base-form vs. infinitive while keeping each fix explainable.
• I2 (tree): The parser could accept tokens with two candidate tags and resolve them using syntactic context, rather than inheriting a confident but wrong tagger decision.
• I3 (tokens): Marking units like according to or ad hoc with position-indexed tags would give the UD stage clean signals for fixed/flat relations.
• I4 (learn): Genre-specific resource slots, plus a repetition filter, would let Mova adapt to dialogue-heavy tales or transcripts without retraining the core model.
• I5 (learn): Evaluation reports could break errors down by tag class and annotation guideline, so a low error rate on trivial, low-discrimination tags does not hide weaknesses.