📄 Representing Text Chunks

Summary

The paper asks whether the way noun phrase chunks are encoded as per-word tags changes how well a learner can find them. Ramshaw and Marcus (1995) had turned baseNP chunking into a tagging problem. The authors compare seven encodings of that task:

• four complete schemes: IOB1, IOB2, IOE1 and IOE2;
• three combinations of partial schemes: separate opening- and closing-bracket taggers, and an inside/outside tagger paired with one bracket type.

As the learner they use memory-based learning (IB1-IG from TiMBL). Their tuning steps were:

• choosing word/POS context windows;
• adding a cascaded second stage that uses neighbouring predicted chunk tags;
• merging outputs from several first-stage runs;
• adjusting the number of nearest neighbours.

All tuning was done by cross-validation on WSJ section 15. The choice of encoding turned out to matter only slightly; the differences were not significant. Even so, the tuned IOB1 configuration beat the previous best result on the standard Ramshaw and Marcus benchmark. Different encodings also favoured precision or recall, which is relevant for downstream applications.

Key points

• Benchmark data: the Ramshaw and Marcus baseNP data, POS-tagged with the Brill tagger.
  • Training: 211,727 words from WSJ sections 15–18.
  • Test: 47,377 words from WSJ section 20.
  • Tuning: 5-fold cross-validation on section 15 (50,442 words).
• Main result: IOB1 reached F(β=1) = 92.37, against the earlier best of 92.03. All but two of the seven formats exceeded that earlier figure.
• Larger setting: the authors trained on WSJ sections 2–21 and tested on section 0, using only the first-stage IOB1 setup (three words and three POS tags on each side, k=3). This gave F = 93.81, compared with 93.3 reported by Ramshaw and Marcus.
• Precision/recall trade-off: the opening-plus-closing bracket combination had the highest precision (93.66%) but the lowest recall (90.81%).
• Feature sensitivity: IB1-IG weights features by information gain, yet redundant features still hurt it. Setting k=3 instead of the usual default also improved results.

Interesting ideas

• I1. Split the problem into partial taggers. Separate models can predict chunk starts, chunk ends and inside/outside status, and their outputs can then be merged. Each sub-model can use its own context window, for example more left context for starts and more right context for ends. This is a cheap way to get asymmetry in any span-detection task.
• I2. Drop the focus prediction in a cascade. When stacking a second classifier on first-stage predictions, the authors left out the predicted tag of the current word. That feature is right so often that it would dominate the weighting and drown out the other inputs, so only neighbouring predicted tags were kept. This is a useful guard against over-reliance on one strong feature in stacked models.
• I3. Combine first-stage runs with different context sizes. Feeding a second stage the outputs of several first-stage runs, each with a different context size, gives it information from different window widths. This works as a lightweight ensemble.
• I4. Treat the encoding as a choice in its own right. Picking the tagging scheme can serve as a precision/recall knob for downstream needs, rather than being fixed by convention. This is relevant to NER or span annotation in any language.
• I5. Feature selection matters even with learned weights. Feature weighting does not make a model immune to redundant inputs, so an explicit search over features and parameters is still worthwhile. The authors suggest hill-climbing for this search.