📄 Joint Word Segmentation and POS Tagging Using a Single Perceptron

Summary

Chinese text has no explicit word boundaries, so POS tagging usually runs after a separate segmentation step. Segmentation mistakes then pass into the tagger and cannot be fixed. Earlier joint systems shrank the combined search space by tagging at the character level (Ng and Low, 2004) or by reranking N-best segmentations (Shi and Wang, 2007). Both approaches limit how much word-level and POS evidence can interact. Zhang and Clark build a single discriminative linear model over full segmented-and-tagged outputs. Its feature set is simply the union of the templates from their baseline perceptron segmentor and baseline POS tagger. The model is trained with one averaged perceptron. Because a standard beam search decoder performs poorly on this joint space, they introduce a multiple-beam search algorithm. On the Chinese Treebank (CTB 3), evaluated with 10-fold cross validation, the joint model cuts segmentation errors by about 14.6% and overall tagging errors by about 12.2% relative to the pipeline. Its accuracy is competitive with systems that rely on outside resources, even though it is purely data-driven.

Key points

• With the 10-fold split, the pipeline baseline reaches 95.20 segmentation F-score (SF), 90.33 overall F-score (TF) and 92.17 tagging accuracy by character (TA). The joint model reaches 95.90 SF, 91.34 TF and 93.02 TA, and it beats the baseline in every one of the ten folds.
• The single-beam decoder from Zhang and Clark (2007) was enough for segmentation alone with an agenda of 8. In the joint model it lowered segmentation accuracy below the baseline segmentor, even with a beam size of 1024.
• Shi and Wang (2007) report slightly higher absolute scores (96.05 SF, 91.86 TF). However, they use special-character knowledge and a semantic net from outside CTB, and their data partition differs. The authors follow Ng and Low's even split of sentences ordered by sentence ID.
• In the error analysis, confusions between NN and VV (in both directions) and between NR and NN make up 14.92% of all errors. Roughly half of all errors (51.47%) are segmentation errors.
• The authors also tried SVM struct for training the parameters, but it was far too slow to be practical.

Interesting ideas

• I1. Fair joint-vs-pipeline comparison: the joint model uses only the features already present in the two pipeline components. Any gain can therefore be credited to joint inference rather than to extra information. This is a clean experimental design for any study of whether merging two tasks helps.
• I2. Rethinking which hypotheses compete in a beam: the authors asked which partial candidates are fairly comparable when items mix words and tags. Their answer was to keep several agendas instead of a single beam ranked across everything. This lesson carries over to other joint decoding problems where hypotheses differ in structure.
• I3. Pruning the search with training statistics: the decoder limits where a word may start using a maximum observed word length for each tag. A tag dictionary further restricts frequent words and closed-set tags to the tags seen in training. Both are cheap constraints that speed up decoding without hand-written rules.
• I4. Character categories as features: a character's category is defined as the set of tags it carried in the training data. This gives morphology-like features without needing external web corpora, which is attractive for languages with few resources.
• I5. POS features that also inform segmentation: in a joint model, a tagger feature such as a number word followed by a measure word also signals where word boundaries fall. Features from one task can thus supervise the other at no extra cost.