📄 Text Chunking using Transformation-Based Learning

Summary

Ramshaw and Marcus tackle text chunking, which means splitting sentences into flat, non-overlapping segments as a step before full parsing. They aim at two targets: non-recursive "baseNP" chunks, and a full partition of each sentence into N-type and V-type chunks loosely modelled on Abney's proposal. Instead of inserting brackets, they assign each word a chunk tag. This turns chunking into tagging and lets them reuse Brill's transformation-based learning. Training and test data were generated automatically from the Wall Street Journal parses in the Penn Treebank, and the text was POS-tagged with Brill's tagger. A baseline gives each word a chunk tag based on its POS tag. The learner then builds an ordered list of rules that rewrite chunk tags using nearby words, POS tags and chunk tags. Scored automatically against the Treebank-derived answers, the system reaches roughly 92% recall and precision on baseNPs and about 88% on the partitioning chunks. The study matters because earlier chunkers used narrower chunk definitions and were mostly checked by hand. This work offers a reproducible, corpus-based way to evaluate chunking.

Key points

• Data. Chunk annotations were derived algorithmically from Penn Treebank WSJ parses. BaseNPs are roughly NPs that contain no nested NP. They include determiners but stop before postmodifying PPs or clauses. A possessive marker starts a new baseNP.
• Partitioning task. Prepositions are grouped with their object NP inside an N chunk, so the system must tell prepositions apart from particles. Everything outside N chunks is labelled V-type, even non-verbal material such as adjective phrases. Tags such as BN, N, BV and V mark chunk starts and continuations.
• Results. About 92% recall/precision on baseNPs and about 88% on N/V partitioning.
• Ablation without word-specific templates (largest training size, 200K):
  • baseNP drops to 90.7% recall / 90.5% precision, a loss of about 1 point;
  • partitioning drops to 83.6% / 83.5%, a loss of about 5 points;
  • so lexical rules matter much more for partitioning.
• Error analysis. VBG/VBN words that occur inside NPs and conjunctions (CC, commas) together make up about half of the sampled baseNP errors. Many of these cases appear to need semantic knowledge beyond the local context.

Interesting ideas

• I1. Chunks as per-word tags, not brackets. Local rewrite rules can then never produce unbalanced bracketings. The same trick applies to NER, segmentation, or any span task handled by local models.
• I2. Baseline first, then learned corrections. Start from a trivial POS-based guess and learn an ordered sequence of readable fixes. This yields an interpretable model whose earliest rules show the most common systematic errors.
• I3. Automatic gold data from a treebank. Deriving chunk annotations from existing parses provides large training sets and fully automatic evaluation. This avoids the bias of judges checking a system's own output, and it is useful wherever a treebank exists but chunk data does not, for example deriving chunks from UD trees.
• I4. Feature ablation as a diagnostic. Rerunning without word-specific templates shows how much each task depends on lexical information versus POS patterns. It is a cheap way to decide whether a task needs a lexicalized model.
• I5. Ideas for later work. Rule templates could refer to chunk structure, such as the first word of the current chunk, rather than fixed offsets. Dependency-like tags such as "N+2" could encode attachment within the same tagging framework, extending chunking toward larger structures.