Elamur Research⌕
☰
home / papers / 2013 / Universal Dependency Annotation for Multilingual Parsing

Universal Dependency Annotation for Multilingual Parsing

Summary

The paper addresses the problem of heterogeneity among dependency treebanks. The CoNLL corpora share a common format, but their annotation schemes differ. As a result, cross-lingual comparison and parser evaluation yield distorted results. The authors created a set of treebanks for six languages (English, French, German, Korean, Spanish and Swedish) with a single unified scheme. The scheme is based on Stanford dependencies and the universal part-of-speech tags of Petrov et al. The English and Swedish data were obtained by automatic conversion of existing corpora. The remaining languages were annotated manually following language-specific guidelines, after which the labels were harmonized across languages. To demonstrate the value of the resource, the authors conducted experiments on cross-lingual transfer of a delexicalized parser. The best source language now mostly belongs to the same language group as the target language. Accuracy is noticeably higher than in previous studies, and for the first time it became possible to measure LAS correctly. The resource was released openly and became the direct predecessor of the Universal Dependencies (UD) project.

Key points

  • The final scheme has 40 syntactic labels, whereas there are only 12 universal POS tags. During harmonization, language-specific labels were merged: nn, aa, and vv were combined into compmod. Some labels were renamed; for example, prep became adpmod so as to cover postpositions as well.
  • The English portion was derived from the WSJ section of the Penn Treebank using Stanford parser v1.6.8. The data themselves are not distributed; the release contains only scripts for reproducing them. The Swedish portion was derived from Talbanken via deterministic rules.
  • German, Spanish, French, and Korean were annotated manually in TrEd. Sentences were drawn from news, blogs, and consumer reviews, and before annotation they were automatically tagged and parsed.
  • The experiments used a perceptron shift-reduce parser with beam 8 and the features of Zhang & Nivre (2011) without lexical features. UAS ranges from 55–75%, whereas McDonald et al. (2011) reported 38–68%. On the same Swedish test set, UAS rose from 58.3% to 70.6%.
  • For German and for English alike, Swedish proved the most useful language to transfer from. For the Romance languages, transfer nearly reaches the accuracy of a parser trained on data from the language itself. Korean lags noticeably behind the other languages whether it is the source or the target. The data are available at code.google.com/p/uni-dep-tb.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Universal Dependencies Multilingual & cross-lingual NLP

 Institutions

🏢GoogleMountain View · United States272 works · 255,532 citations🎓Bar-Ilan UniversityRamat Gan · Israel102 works · 17,587 citations🎓Uppsala UniversityUppsala · Sweden46 works · 11,454 citations

Cite

Cite as Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló and Jungmee Lee. 2013. Universal Dependency Annotation for Multilingual Parsing. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mcdonald2013universal,
  title = {{Universal Dependency Annotation for Multilingual Parsing}},
  author = {Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló and Jungmee Lee},
  booktitle = {Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2013},
  url = {https://aclanthology.org/P13-2017/}
}

References

34 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alena Böhmová, Jan Hajič, Eva Hajičová, and Barbora Hladká. 2003. The Prague Dependency Treebank: A three-level annotation scenario. In Anne Abeillé, editor, Treebanks: Building and Using Parsed Corpora, pages 103–127. Kluwer. ↗
  2. Sabine Brants, Stefanie Dipper, Silvia Hansen, Wolfgang Lezius, and George Smith. 2002. The TIGER Treebank. In Proceedings of the Workshop on Treebanks and Linguistic Theories. find
  3. Sabine Buchholz and Erwin Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In Proceedings of CoNLL. our card
  4. Miriam Butt, Helge Dyvik, Tracy Holloway King, Hiroshi Masuichi, and Christian Rohrer. 2002. The parallel grammar project. In Proceedings of the 2002 workshop on Grammar engineering and evaluation-Volume 15. our card
  5. Pi-Chuan Chang, Huihsin Tseng, Dan Jurafsky, and Christopher D. Manning. 2009. Discriminative reordering with Chinese grammatical relations features. In Proceedings of the Third Workshop on Syntax and Structure in Statistical Translation (SSST-3) at NAACL HLT 2009. find
  6. Dipanjan Das and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. In Proceedings of ACL-HLT. our card
  7. Marie-Catherine de Marneffe and Christopher D. Manning. 2008. The Stanford typed dependencies representation. In Coling 2008: Proceedings of the workshop on Cross-Framework and Cross-Domain Parser Evaluation. our card
  8. Marie-Catherine De Marneffe, Bill MacCartney, and Chris D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC. our card
  9. Tomaz Erjavec. 2012. MULTEXT-East: Morphosyntactic resources for Central and Eastern European languages. Language Resources and Evaluation, 46:131–142. ↗
  10. Kuzman Ganchev, Jennifer Gillenwater, and Ben Taskar. 2009. Dependency grammar induction via bitext projection constraints. In Proceedings of ACL-IJCNLP. find
  11. Douwe Gelling, Trevor Cohn, Phil Blunsom, and Joao Graça. 2012. The pascal challenge on grammar induction. In Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure. find
  12. Jan Hajič, Barbora Vidova Hladka, Jarmila Panevová, Eva Hajičová, Petr Sgall, and Petr Pajas. 2001. Prague Dependency Treebank 1.0. LDC, 2001T10. find
  13. Katri Haverinen, Timo Viljanen, Veronika Laippala, Samuel Kohonen, Filip Ginter, and Tapio Salakoski. 2010. Treebanking finnish. In Proceedings of The Ninth International Workshop on Treebanks and Linguistic Theories (TLT9). find
  14. Stephen Helmreich, David Farwell, Bonnie Dorr, Nizar Habash, Lori Levin, Teruko Mitamura, Florence Reeder, Keith Miller, Eduard Hovy, Owen Rambow, and Advaith Siddharthan. 2004. Interlingual annotation of multilingual text corpora. In Proceedings of the HLT-EACL Workshop on Frontiers in Corpus Annotation. ↗
  15. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: the 90% solution. In Proceedings of NAACL. our card
  16. Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Natural Language Engineering, 11(03):311–325. find
  17. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of ACL. our card
  18. Dan Klein and Chris D. Manning. 2004. Corpus-based induction of syntactic structure: models of dependency and constituency. In Proceedings of ACL. find
  19. Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency Parsing. Morgan and Claypool. find
  20. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: the Penn treebank. Computational Linguistics, 19(2):313–330. our card
  21. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proceedings of EMNLP. our card
  22. Jens Nilsson, Joakim Nivre, and Johan Hall. 2007. Generalizing tree transformations for inductive dependency parsing. In Proceedings of ACL. find
  23. Joakim Nivre and Beáta Megyesi. 2007. Bootstrapping a Swedish treebank using cross-corpus harmonization and annotation projection. In Proceedings of the 6th International Workshop on Treebanks and Linguistic Theories. find
  24. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan Mc-Donald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. find
  25. The CoNLL 2007 shared task on dependency parsing. In Proceedings of EMNLP-CoNLL. our card
  26. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. In Proceedings of LREC. our card
  27. Mojgan Seraji, Beáta Megyesi, and Nivre Joakim. 2012. Bootstrapping a Persian dependency treebank. Linguistic Issues in Language Technology, 7(18):1–10. ↗
  28. David A. Smith and Jason Eisner. 2009. Parser adaptation and projection with quasi-synchronous grammar features. In Proceedings of EMNLP. find
  29. Oscar Täckström, Dipanjan Das, Slav Petrov, Ryan McDonald, and Joakim Nivre. 2013. Token and type constraints for cross-lingual part-of-speech tagging. Transactions of the ACL. ↗
  30. Ulf Teleman. 1974. Manual för grammatisk beskrivn-ing av talad och skriven svenska. Studentlitteratur. find
  31. Reut Tsarfaty. 2013. A unified morpho-syntactic scheme of stanford dependencies. Proceedings of ACL. find
  32. Daniel Zeman, David Marecek, Martin Popel, Loganathan Ramasamy, Jan Štepánek, Zdeněk Žabokrtskỳ, and Jan Hajic. 2012. Hamledt: To parse or not to parse. In Proceedings of LREC. find
  33. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of ACL-HLT. find
  34. Yuan Zhang, Roi Reichart, Regina Barzilay, and Amir Globerson. 2012. Learning to map into a universal pos tagset. In Proceedings of EMNLP. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mcdonald-2013-universal-dependency-annotation-multilingual-parsing/