Elamur Research⌕
☰
home / papers / 2011 / Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections

Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections

Summary

The paper tackles POS tagging for languages that have no annotated data and no tagging dictionary but do have parallel text with English. The authors build a bilingual similarity graph from a sentence-aligned corpus. Foreign-side vertices are trigram types, linked to each other by distributional similarity. English-side vertices are word types, linked to foreign trigrams through high-confidence word alignments. A supervised tagger labels the English side, and those label distributions seed the English vertices. Label propagation then moves tag distributions to aligned foreign trigrams and onward to unaligned ones. The resulting distributions become soft constraint features in the feature-based HMM of Berg-Kirkpatrick et al. (2010), not direct training labels. The twelve universal POS tags of Petrov et al. (2011) serve as the shared tagset. Across eight European languages, average accuracy reaches 83.4%, compared with 73.0% for the monolingual feature-HMM and 66.7% for EM-trained HMMs. This substantially narrows the gap to supervised tagging (96.6%) without any target-language annotation.

Key points

  • The evaluation covers eight languages (Swedish, Spanish, Greek, German, Portuguese, Danish, Italian and Dutch), with accuracy measured against the universal POS tags.
  • Macro-averaged accuracies:
    • EM-HMM: 66.7%
    • Feature-HMM: 73.0%
    • Direct "Projection": 78.8%
    • "No LP": 81.3%
    • Full "With LP": 83.4%
    • Supervised (Brants, 2000): 96.6%
  • The full model's improvements are statistically significant for seven of eight languages. On German it trails direct projection, but the difference is not statistically significant.
  • Graph details:
    • Foreign trigram vertices are scored with PMI over nine context feature concepts and connected as a 5-nearest-neighbour graph.
    • Bilingual edges come only from intersected alignments with confidence above 0.9.
    • The graph is built from 2 million trigrams.
  • The same hyperparameters were used for all languages, including threshold τ = 0.2. Label propagation greatly expands coverage; for German, the number of words with constraints rises from 59,534 to 107,249.

Interesting ideas

  • I1. Asymmetric vertex design: the side that is already labeled (English) is represented by plain word types. The unlabeled side gets trigram vertices that carry context for disambiguation. This pattern suits any transfer graph where knowledge flows in one direction only.
  • I2. Precision first, recall from propagation: only very confident alignments are kept, and the graph spreads labels to words that were never aligned. This is a useful way to combine a noisy but precise signal with broad coverage.
  • I3. Projected labels as soft constraints, not gold training data: feeding projected tags as features into an unsupervised model lets the target-language data correct projection noise. Training directly on noisy projections would not allow this.
  • I4. A coarse universal tagset as a bridge: mapping both languages to shared categories makes transfer feasible. It also allows direct accuracy evaluation without the contested many-to-one mapping metrics used for unsupervised induction.
  • I5. Hyperparameters chosen without a development set: the authors fixed one threshold value that held across languages. This is a practical recipe for truly low-resource languages, such as Ukrainian dialects or other under-resourced languages, where no validation data exists.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

165 papers of our library cite it (of the 140884 whose bibliographies we read); 21 of them have a page here, 144 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{das2011unsupervised,
  title = {{Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections}},
  author = {Das, Dipanjan and Petrov, Slav},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1061/}
}

References

29 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hiyan Alshawi, Srinivas Bangalore, and Shona Douglas. 2000. Head-transducer models for speech translation and their automatic acquisition from bilingual data. Machine Translation, 15. find
  2. Yasemin Altun, David McAllester, and Mikhail Belkin. 2005. Maximum margin semi-supervised learning for structured variables. In Proc. of NIPS. find
  3. Taylor Berg-Kirkpatrick, Alexandre B. Côté, John DeNero, and Dan Klein. 2010. Painless unsupervised learning with features. In Proc. of NAACL-HLT. find
  4. Thorsten Brants. 2000. TnT - a statistical part-of-speech tagger. In Proc. of ANLP. find our card
  5. Peter F. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: parameter estimation. Computational Linguistics, 19. find
  6. Sabine Buchholz and Erwin Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In Proc. of CoNLL. find our card
  7. Andrew Carnie. 2002. Syntax: A Generative Introduction (Introducing Linguistics). Blackwell Publishing. find
  8. Christos Christodoulopoulos, Sharon Goldwater, and Mark Steedman. 2010. Two decades of unsupervised POS induction: How far have we come? In Proc. of EMNLP. find
  9. Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39. find
  10. Kuzman Ganchev, Jennifer Gillenwater, and Ben Taskar. 2009. Dependency grammar induction via bitext projection constraints. In Proc. of ACL-IJCNLP. find
  11. Mark Johnson. 2007. Why doesn’t EM find good HMM POS-taggers? In Proc. of EMNLP-CoNLL. find
  12. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In MT Summit. find
  13. Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45. find
  14. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: the Penn treebank. Computational Linguistics, 19. find our card
  15. Tahira Naseem, Benjamin Snyder, Jacob Eisenstein, and Regina Barzilay. 2009. Multilingual part-of-speech tagging: Two unsupervised approaches. JAIR, 36. find
  16. Tahira Naseem, Harr Chen, Regina Barzilay, and Mark Johnson. 2010. Using universal linguistic knowledge to guide grammar induction. In Proc. of EMNLP. find
  17. Frederick J. Newmeyer. 2005. Possible and Probable Languages: A Generative Perspective on Linguistic Typology. Oxford University Press. find
  18. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. find
  19. The CoNLL 2007 shared task on dependency parsing. In Proceedings of CoNLL. find our card
  20. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. A universal part-of-speech tagset. ArXiv:1104.2086. find
  21. Sujith Ravi and Kevin Knight. 2009. Minimized models for unsupervised part-of-speech tagging. In Proc. of ACL-IJCNLP. find
  22. Libin Shen, Giorgio Satta, and Aravind Joshi. 2007. Guided learning for bidirectional sequence classification. In Proc. of ACL. find
  23. Benjamin Snyder, Tahira Naseem, and Regina Barzilay. 2009. Unsupervised multilingual grammar induction. In Proc. of ACL-IJCNLP. find
  24. Amar Subramanya, Slav Petrov, and Fernando Pereira. 2010. Efficient graph-based semi-supervised learning of structured tagging models. In Proc. of EMNLP. find
  25. UN. 2006. ODS UN parallel corpus. find
  26. Stephan Vogel, Hermann Ney, and Christoph Tillmann. 1996. HMM-based word alignment in statistical translation. In Proc. of COLING. find
  27. Chenhai Xi and Rebecca Hwa. 2005. A backoff model for bootstrapping resources for non-English languages. In Proc. of HLT-EMNLP. find
  28. David Yarowsky and Grace Ngai. 2001. Inducing multilingual POS taggers and NP bracketers via robust projection across aligned corpora. In Proc. of NAACL. find
  29. Xiaojin Zhu, Zoubin Ghahramani, and John D. Lafferty. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In Proc. of ICML. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/das-2011-unsupervised-part-speech-tagging-bilingual/