📄 A Simple Rule-Based Part of Speech Tagger

Summary

Brill asks whether part-of-speech tagging really needs stochastic (Markov-model) methods, or whether a plain rule-based tagger with no syntactic knowledge can match them. His tagger starts by giving every word its most frequent tag from a tagged corpus. Unseen words get a guess: capitalized unknowns become proper nouns, and other unknowns take the tag most typical for their final three letters. The tagger then learns corrective "patches" from a fixed set of templates, such as conditions on neighboring tags or capitalization. Each patch is chosen greedily for the largest net error reduction on a held-out patch corpus. The experiments use the Brown Corpus. About 71 learned rules bring the error rate down to roughly 5%. That is close to the author's own reimplementation of a stochastic tagger, and it needs no dictionary or hand-built idiom lists. The work matters because it shows a compact, readable, and portable rule set can rival large probability tables. This later became the basis of transformation-based learning.

Key points

• Data: Brown Corpus, about 1.1M words with 192 tags, 96 of which occur more than 100 times. It was split into 90% for training the lexical tagger, 5% as a patch corpus for rule learning, and 5% as a test set covering all genres.
• Baseline and result: The baseline (most-frequent tag plus the unknown-word heuristics) has about 7.9% error. Applying 71 learned patches lowers this to 5.1%. Three different random splits all gave about 5%.
• Rule effects on the test set: Of the 71 patches, 66 reduced errors, 3 had no net effect, and 2 increased errors.
• Comparison: The author's dictionary-free reimplementation of the stochastic tagger from reference [1] reached about 4.5% error on the same splits. The rule tagger needs fewer than 80 rules, while a trigram model needs tens of thousands of statistics.
• Unknown words: When every word is known, the same patches give 4.1% error if the lexicon comes from the whole Brown Corpus, and 3.5% if it comes from the test text itself. This suggests a dictionary would help.

Interesting ideas

• I1. Net-gain rule scoring: Each candidate rule is scored by the errors it fixes minus the errors it creates. For example, one rule fixed 98 of 159 verb-for-noun errors but introduced 18 new ones, for a net gain of 80. This greedy, error-driven loop could be applied to many sequence-labeling or post-correction tasks, for instance as a fix-up layer on top of a Ukrainian tagger.
• I2. Templates are cheap to try: A useless template does no harm, because none of its instances will be selected. This makes it low-risk to experiment with new feature types, such as morphology-specific conditions for richly inflected languages.
• I3. Lexicon constraint on rewrites: A rule may change a word's tag only to a tag that word has actually carried in the training data. This simple guard against implausible corrections could be strengthened with an external dictionary.
• I4. Separate data for rule learning: Rules are induced on a dedicated patch corpus, kept apart from both the lexicon-training data and the test data. This is a clean way to fit a second-stage corrector without overfitting to the base model's training set.
• I5. Learned rules replace hand-written idioms: The system discovered on its own how to tag "as old as" correctly. Interpretable learned rules could likewise substitute for hand-curated exception lists when moving to a new tag set, genre, or language.