TnT – A Statistical Part-of-Speech Tagger

Summary

TnT ("Trigrams'n'Tags") shows that a carefully engineered second-order hidden Markov model is as accurate as more complex taggers of its time. Tag trigram probabilities are smoothed by linear interpolation of unigram, bigram and trigram estimates, with the interpolation weights set by deleted interpolation rather than tuned by hand. Unknown words are handled by a suffix model that estimates tag distributions from word endings of rare training words, with separate statistics for capitalised and lowercase words. Decoding is exact Viterbi search, with beam pruning for speed. The tagger is fast, language-independent and reaches state-of-the-art accuracy on English and German treebanks.

How Mova uses it

en/src/tag.rs — a direct reimplementation: trigram Markov model over tag classes with a dense transition table, deleted-interpolation smoothing, a word-to-tags lexicon for known words, and suffix models (up to 10 characters, separately for upper- and lowercase words, trained on words seen at most 10 times). Exact Viterbi over tag pairs.
en/src/ptag.rs — the main tagger is now an averaged perceptron with a beam; TnT's lexicon and suffix model still supply prior tag classes for unknown words.
TnT remains a baseline and a fast mode (e.g. greedy parsing with TnT tags, faster generation).

Effectiveness in Mova

Measured in the development notes of en: TnT alone scores 93,12% (EWT), 93,12% (PUD), 92,84% (GUM) tagging accuracy; the perceptron tagger that uses TnT priors for unknown words reaches 94,35 / 94,04 / 93,82%. Downstream, greedy parsing with TnT tags gives EWT UAS/LAS 82,3/77,3 versus 83,4/79,0 with the perceptron tags. TnT-based generation is about 4 times faster (~10 500 vs ~2 600 sentences/s) at slightly lower quality (54,6% vs 56,6% exact, chrF 92,4 vs 92,8).

👨‍🔬💥

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/A00-1031.pdf

Summary

The paper presents TnT (Trigrams'n'Tags), a part-of-speech tagger built on a second-order Markov model. It responds to claims that Markov-model taggers are weaker than alternatives such as Maximum Entropy. Brants argues that their reputation suffers because published descriptions leave many implementation choices unstated, and he documents each of these choices explicitly:

Sequence boundaries: begin- and end-of-sequence tags.
Smoothing: context-independent linear interpolation of unigram, bigram and trigram tag probabilities, with weights set by deleted interpolation.
Unknown words: suffix-based guessing smoothed by successive abstraction.
Capitalization: capitalization flags in the context.
Speed: beam search to accelerate Viterbi decoding.

TnT is evaluated on the German NEGRA corpus and the English Penn Treebank. Its results match those reported for the Maximum Entropy tagger of Ratnaparkhi (1996). An independent comparison of seven taggers also ranked TnT the most accurate and the fastest in both training and tagging.

The work matters because it shows that a simple, carefully engineered statistical model is competitive with more complex approaches. It also provides a practical, language-independent baseline that is freely available for research.

Key points

Model: TnT uses tag trigrams for transitions and word-given-tag emissions. The interpolation weights (λ1–λ3) are global rather than context-dependent. Frequency-grouped λ sets never outperformed this simpler variant.
Unknown words:
- Suffixes of up to 10 characters are used, taken only from training words seen 10 or fewer times.
- Separate suffix tries are kept for uppercase and lowercase words.
- The smoothing weights θ are set to the standard deviation of the tag probabilities, typically 0.03–0.10.
Known vs. unknown accuracy: on German newspaper text (NEGRA), accuracy is 97.7% for known tokens and 89.0% for unknown ones. About 1000 training tokens already yield 95–96% accuracy on known words.
Reliability estimates: the tagger's probabilities can flag a subset of reliable assignments, which reaches over 99% accuracy. This is useful for focusing human correction in annotation projects.
Comparison and availability: Maximum Entropy is identified as the only competing approach with comparable accuracy. TnT is distributed free of charge to universities for research.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/974147.974178 · arXiv cs/0003055

@inproceedings{brants2000tnt,
  title = {{TnT – A Statistical Part-of-Speech Tagger}},
  author = {Brants, Thorsten},
  booktitle = {Sixth Applied Natural Language Processing Conference},
  year = {2000},
  doi = {10.3115/974147.974178},
  eprint = {cs/0003055},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/974147.974178}
}

References

14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Thorsten Brants, Wojciech Skut, and Hans Uszkoreit. 1999. Syntactic annotation of a German newspaper corpus. In Proceedings of the ATALA Treebank Workshop, pages 69-76, Paris, France. find
Eric Brill. 1993. A Corpus-Based Approach to Language Learning. Ph.D. Dissertation, Department of Computer and Information Science, University of Pennsylvania. find
Eugene Charniak, Curtis Hendrickson, Neil Jacobson, and Mike Perkowitz. 1993. Equations for part-of-speech tagging. In Proceedings of the Eleventh National Con[erence on Artificial Intelligence, pages 784-789, Menlo Park: AAAI Press/MIT Press. find
Doug Cutting, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1992. A practical part-of-speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing (ACL), pages 133-140. find
Walter Daelemans, Jakub Zavrel, Peter Berck, and Steven Gillis. 1996. Mbt: A memory-based part of speech tagger-generator. In Proceedings of the Workshop on Very Large Corpora, Copenhagen, Denmark. find
Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330. find our card
Lawrence R. Rabiner. 1989. A tutorial on Hidden Markov Models and selected applications in speech recognition. In Proceedings o] the IEEE, volume 77(2), pages 257-285. find
Adwait Ratnaparkhi. 1996. A maximum entropy model for part-of-speech tagging. In Proceedings o] the Conference on Empirical Methods in Natural Language Processing EMNLP-96, Philadelphia, PA. find
Christer Samuelsson. 1993. Morphological tagging based entirely on Bayesian inference. In 9th Nordic Conference on Computational Linguistics NODALIDA-93, Stockholm University, Stockholm, Sweden. find
Helmut Schmid. 1995. Improvements in part-of-speech tagging with an application to German. In Helmut Feldweg and Erhard Hinrichts, editors, Lexikon und Text. Niemeyer, Tfibingen. find
Wojciech Skut, Brigitte Krenn, Thorsten Brants, and Hans Uszkoreit. 1997. An annotation scheme for free word order languages. In Proceedings of the Fifth Conference on Applied Natural Language Processing ANLP-97, Washington, DC. find
Hans van Halteren, Jakub Zavrel, and Walter Daelemans. 1998. Improving data driven wordclass tagging by system combination. In Proceedings of the International Conference on Computational Linguistics COLING-98, pages 491-497, Montreal, Canada. find
Martin Volk and Gerold Schneider. 1998. Comparing a statistical and a rule-based tagger for german. In Proceedings of KONVENS-98, pages 125-137, Bonn. find
Jakub Zavrel and Walter Daelemans. 1999. Eval-uatie van part-of-speech taggers voor bet corpus gesproken nederlands. CGN technical report, Katholieke Universiteit Brabant, Tilburg. find