📄 SVMTool: A general POS Tagger Generator Based on Support Vector Machines

Summary

The paper presents SVMTool, a toolkit for building part-of-speech taggers. It aims to balance accuracy, speed, ease of use and language independence. Each tagging decision is made by linear Support Vector Machine classifiers trained with SVM-light in the primal formulation. They draw on a large, configurable set of contextual features: word and POS n-grams, ambiguity classes, "may be" indicators, and affix and orthographic cues for unknown words. The package has three parts: a learner (SVMTlearn), a tagger (SVMTagger) and an evaluator (SVMTeval), plus an API for embedded use. Users can choose greedy or Viterbi-based sentence-level decoding, the tagging direction, one or two passes, and a backup morphological lexicon. On the Wall Street Journal corpus the tool reaches 97.16% accuracy. On the Spanish LEXESP corpus it reaches 96.89%, and both scores beat TnT under identical conditions. The Perl prototype tags about 1,500 words per second. The work matters because it offers a freely available tagger generator that is competitive, flexible and adaptable to new languages using only an annotated corpus.

Key points

• English setup and results: WSJ sections 0–18 were used for training (912K words), 19–21 for validation and 22–24 for testing. SVMTool scored 97.16% overall versus 96.46% for TnT, and 89.01% versus 85.86% on unknown words. Unknown words made up 2.81% of the test set.
• Spanish setup and results: LEXESP (106K words) was split into 86K words for training and 20K for testing, with 12.21% unknown words. SVMTool scored 96.89% versus 96.50% for TnT. When a morphological analyzer supplied a backup lexicon, the scores rose to 98.86% versus 98.41%.
• Efficiency: Tagging runs at about 1,500 words per second on a Pentium IV at 2GHz. The heaviest English training run, using the full feature set and fine C tuning, took about 24 CPU hours.
• Model compression: Removing small weight components up to about 70% of all weights left accuracy unchanged. Accuracy dropped below the state-of-the-art range (97.0–97.2%) only after about 95% had been removed.
• Default settings: The feature window is 5 tokens with the target word at position 2. The feature space is capped at 100,000 dimensions, and features seen only once are discarded.

Interesting ideas

• I1. Bidirectional combination: The tagger can run left-to-right and right-to-left and then combine both outputs, which gave a significant gain at little cost. This is a cheap trick for any greedy sequence labeller where the processing order biases errors.
• I2. Pruning linear model weights: Most of the weight components can be zeroed out with little loss in accuracy. This is a simple way to shrink and speed up linear models before deploying them.
• I3. Ambiguity-class features: The tagger encodes which tags a word could take ("may be" features), not just which tags were already assigned. This lexicon-derived feature transfers well to morphologically rich languages, where dictionaries can narrow the candidate tags.
• I4. Breaking down evaluation results: SVMTeval reports accuracy separately for known and unknown words, for ambiguous words, and by ambiguity class or class size. This shows more clearly where a tagger fails than a single overall number does.
• I5. Exposing scores for all candidate tags: The API returns the score for every candidate tag, not only the winning one. Downstream components can then handle near-tied decisions themselves, for example by flagging them for review or passing several candidates to a parser.