Elamur Research⌕
☰
home / papers / 2015 / Cross-Lingual Dependency Parsing with Universal Dependencies and Predicted PoS Labels

Cross-Lingual Dependency Parsing with Universal Dependencies and Predicted PoS Labels

Summary

This paper sets up monolingual and cross-lingual dependency parsing baselines for the first release (1.0) of the Universal Dependencies (UD) data set. It also measures how much results change when parsers receive automatically predicted PoS and morphological tags instead of gold annotation. Ten UD languages are included in the experiments: Czech, English, Finnish, French, German, Hungarian, Irish, Italian, Spanish and Swedish. Parsers were trained with the Mate tools and taggers with Marmot. The study compares three approaches:

  • delexicalized model transfer,
  • annotation projection over parallel data,
  • treebank translation.

With predicted tags, lexicalized monolingual parsers lose only a few LAS points. Delexicalized parsers degrade much more, because they rely entirely on PoS labels. Models trained on noisy projected or translated data also lose accuracy, and their parsing quality correlates strongly with tagger accuracy.

The main takeaway is that evaluating with gold tags overstates real-world performance, especially for delexicalized transfer. Developers should therefore improve tagging before trusting cross-lingual parsers. The paper also finds that current transfer methods work poorly for languages from different families.

Key points

  • Monolingual baselines: LAS ranges from 72.73 (Irish, about 0.7k-sentence treebank) to 89.58 (Italian). Removing morphology and lemmas costs Finnish and Hungarian roughly 10 LAS points and Czech about 5.
  • Effect of predicted tags (Marmot): With full features, monolingual LAS drops about 2–4 points, and 5–7 points for small treebanks (e.g., Hungarian 83.19 → 75.78, Irish 72.73 → 66.74).
  • Delexicalized parsers: These suffer the most from predicted tags, e.g., Finnish 59.54 → 49.79 and Czech 70.36 → 62.44 LAS.
  • Correlations with tagging: PoS tagger accuracy correlates strongly with cross-lingual LAS (r = 0.918 for projected models, r = 0.900 for translated models). It correlates negatively with the size of the LAS drop (r ≈ −0.67).
  • Treebank translation: This approach stays competitive with projection despite low BLEU scores, and BLEU correlates with LAS (r = 0.795). Its lexicalized models lose about 2 LAS points, sometimes more than 4, under predicted tags.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Dependency parsing · Universal Dependencies · Treebanks · Part-of-speech tagging

Cited by

30 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 24 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tiedemann2015cross,
  title = {{Cross-Lingual Dependency Parsing with Universal Dependencies and Predicted PoS Labels}},
  author = {Tiedemann, Jörg},
  booktitle = {Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015)},
  year = {2015},
  url = {https://aclanthology.org/W15-2137/}
}

References

26 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bernd Bohnet and Jonas Kuhn. 2012. The Best of Both Worlds – A Graph-based Completion Model for Transition-based Parsers. In Proceedings of EACL, pages 77–87. find
  2. Bernd Bohnet. 2010. Top Accuracy and Fast Dependency Parsing is not a Contradiction. In Proceedings of COLING, pages 89–97. find our card
  3. Sabine Buchholz and Erwin Marsi. 2006. CoNLL-X Shared Task on Multilingual Dependency Parsing. In Proceedings of CoNLL, pages 149–164. find our card
  4. Dipanjan Das and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. In Proceedings of ACL, pages 600–609. find
  5. Marie-Catherine De Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of LREC, pages 449–454. find our card
  6. Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping Parsers via Syntactic Projection across Parallel Texts. Natural Language Engineering, 11(3):311– 325. find
  7. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Christopher J. Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translatio… find
  8. Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Proceedings of MT Summit, pages 79–86. find
  9. Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency Parsing. Morgan & Claypool Publishers. find
  10. Ryan McDonald, Slav Petrov, and Keith Hall. 2011a. Multi-Source Transfer of Delexicalized Dependency Parsers. In Proceedings of EMNLP, pages 62–72. find our card
  11. Ryan McDonald, Slav Petrov, and Keith Hall. 2011b. Multi-source transfer of delexicalized dependency parsers. In Proceedings EMNLP, pages 62–72. find our card
  12. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló, and Jungmee Lee. 2013. Universal Dependency Annotation for Multilingual Parsing. In Proceedings of AC… find
  13. Thomas Müller and Hinrich Schütze. 2015. Robust morphological tagging with word representations. In Proceedings of NAACL. find
  14. Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. Selective Sharing for Multilingual Dependency Parsing. In Proceedings of ACL, pages 629– 637. find
  15. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan Mc-Donald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. find
  16. The CoNLL 2007 Shared Task on Dependency Parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 915–932. find
  17. Franz Josef Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics, 29(1):19–52. find
  18. Robert Östling. 2015. Bayesian Models for Multilingual Word Alignment. Ph.D. thesis, Stockholm University, Department of Linguistics. find
  19. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A Universal Part-of-Speech Tagset. In Proceedings of LREC, pages 2089–2096. find
  20. Oscar Täckström, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual Word Clusters for Direct Transfer of Linguistic Structure. In Proceedings of NAACL, pages 477–487. find our card
  21. Oscar Täckström, Ryan McDonald, and Joakim Nivre. 2013. Target Language Adaptation of Discriminative Transfer Parsers. In Proceedings of NAACL, pages 1061–1071. find
  22. Jörg Tiedemann, Željko Agić, and Joakim Nivre. 2014. Treebank Translation for Cross-Lingual Parser Induction. In Proceedings of CoNLL, pages 130–140. find
  23. Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. In Proceedings of LREC, pages 2214–2218. find
  24. Jörg Tiedemann. 2014. Rediscovering Annotation Projection for Cross-Lingual Parser Induction. In Proceedings of COLING, pages 1854–1864. find
  25. Min Xiao and Yuhong Guo. 2014. Distributed Word Representation Learning for Cross-Lingual Dependency Parsing. In Proceedings of CoNLL, pages 119–129. find
  26. Daniel Zeman and Philip Resnik. 2008. Cross-Language Parser Adaptation between Related Languages. In Proceedings of IJCNLP, pages 35–42. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/tiedemann-2015-cross-lingual-dependency-parsing-universal/