Elamur Research⌕
☰
home / papers / 2020 / Parsing in the absence of related languages: Evaluating low-resource dependency parsers on Tagalog

Parsing in the absence of related languages: Evaluating low-resource dependency parsers on Tagalog

Summary

The authors look for a way to build a dependency parser for a low-resource language that has no typologically close language with a large treebank. They study Tagalog, an Austronesian language of the Philippines. For this purpose, they manually annotated a new UD treebank, Ugnayan, based on educational texts. They then used WALS distance to select the UD languages closest to Tagalog: Indonesian, Ukrainian, Vietnamese, Romanian and Catalan. In UDPipe, they compared three approaches. The monolingual approach uses only the tiny Tagalog corpus and is evaluated with 10-fold cross-validation. The cross-lingual approach is trained only on source-language data. The multilingual approach combines both sources. They also separately tested the off-the-shelf Stanza (Indonesian model) and UDify. The best tagging and parsing came from the model trained on roughly 900 Tagalog tokens. Adding foreign-language data to it only lowered quality. The practical conclusion is this: when no related donor language is available, a hundred sentences annotated by hand are more useful than tens or hundreds of thousands of tokens from distant languages. For low-resource languages, this is a strong argument for investing in their own annotation.

Key points

  • Ugnayan contains 94 sentences (1011 tokens) with a variety of constructions: complex sentences, questions, inversion. The previous UD treebank, TRG, had only 55 simple sentences (292 tokens).
  • According to WALS, no UD language is closer to Tagalog than 0.44. Ukrainian (uk-iu) ranked second with 0.455, after Indonesian with 0.446. Conversely, Tagalog is not among the five languages closest to Ukrainian itself.
  • The monolingual UDPipe model achieved F1: UPOS 80.54, UAS 63.47, LAS 55.37. The best cross-lingual model (tl-trg) has an LAS of only 13.11. The best multilingual model (Ugnayan + vi-vtb) has an LAS of 48.65. UDify without Tagalog data gives an LAS of 32.09, but with gold tokenization.
  • On news texts (7 sentences, 265 tokens), the LAS of the Ugnayan model drops to 25.86, yet it still outperforms the rest. Spanish and English PUD did not help, even though Tagalog has many loanwords from these languages.
  • Given gold POS tags as input, the Ugnayan model reaches an LAS of 66.89. The closest source-language model to it (en-pud) scores 49.77.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Dependency parsing · Treebanks

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{aquino2020parsing,
  title = {{Parsing in the absence of related languages: Evaluating low-resource dependency parsers on Tagalog}},
  author = {Aquino, Angelina and Leon, Franz de},
  booktitle = {Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020)},
  year = {2020},
  doi = {10.48448/0jhv-jk62},
  url = {https://doi.org/10.48448/0jhv-jk62}
}

References

43 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Željko Agić, Anders Johannsen, Barbara Plank, Héctor Martı́nez Alonso, Natalie Schluter, and Anders Søgaard. 2016. Multilingual projection for parsing truly low-resource languages. Transactions of the Association for Computational Linguistics, 4:301–312. find
  2. Željko Agić. 2017. Cross-lingual parser selection for low-resource languages. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 1–10, Gothenburg, Sweden, May. Association for Computational Linguistics. find our card
  3. Wasi Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, and Nanyun Peng. 2019. On difficulties of cross-lingual transfer with order differences: A case study on dependency parsing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Ling… find
  4. Ani Rosa Almario and Yvette Tan. 2016. Leveled Readers in Filipino – Isang Kakaibang Araw – Alamin Natin ang mga Anyong-Tubig sa Pilipinas! Department of Education, Philippines. find
  5. Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2016. Many languages, one parser. Transactions of the Association for Computational Linguistics, 4:431–444. find our card
  6. Irshad Bhat, Riyaz A. Bhat, Manish Shrivastava, and Dipti Sharma. 2018. Universal dependency parsing for Hindi-English code-switching. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pap… find
  7. Huadong Chen, Shujian Huang, David Chiang, and Jiajun Chen. 2017. Improved neural machine translation with a syntax-aware encoder and decoder. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1936–1945, Vancouver, Canada, July.… find our card
  8. Mathieu Dehouck and Pascal Denis. 2019. Phylogenic multi-lingual dependency parsing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 192–203, Minneapolis, Minnes… find
  9. Yuan Ding and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 541–548, Ann Arbor, Michigan, June. Association for Computational Linguisti… find our card
  10. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 20–30, Vancouver, Canada, August. Association for… find our card
  11. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  12. Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. 2015. Low resource dependency parsing: Cross-lingual parameter sharing in a neural network parser. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lan… find
  13. David M. Eberhard, Gary F. Simons, and Charles D. Fennig, editors. 2020. Ethnologue: Languages of the World. SIL International, Dallas, Texas, 23 edition. find
  14. Agnieszka Falenska and Özlem Çetinoğlu. 2017. Lexicalized vs. delexicalized parsing in low-resource scenarios. In Proceedings of the 15th International Conference on Parsing Technologies, pages 18–24, Pisa, Italy, September. Association for Computational Linguistics. find
  15. Marcos Garcia, Carlos Gómez-Rodrı́guez, and Miguel A. Alonso. 2018. New treebank or repurposed? on the feasibility of cross-lingual parsing of Romance languages with Universal Dependencies. Natural Language Engineering, 24(1):91–122. find
  16. Dan Garrette and Jason Baldridge. 2013. Learning a part-of-speech tagger from two hours of annotation. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 138–147, Atlanta, Georgia, June. Association… find
  17. Ryan Georgi, Fei Xia, and William Lewis. 2010. Comparing language similarity across genetic and typologically-based groupings. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 385–393, Beijing, China, August. find
  18. Coling 2010 Organizing Committee. find
  19. Simon J. Greenhill and Russell D. Gray. 2009. Austronesian language phylogenies: myths and misconceptions about Bayesian computational methods. In Alexander Adelaar and Andrew Pawley, editors, Austronesian Historical Linguistics and Culture History: A festschrift for Robert Blust, chapter 22, pages… find
  20. Rebecca Hwa, Philip Resnik, Amy Weinberg, and Okan Kolak. 2002. Evaluating translational correspondence using annotation projection. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 392–399, Philadelphia, Pennsylvania, USA, July. Association for Compu… find
  21. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing universal dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–279… find our card
  22. Xuezhe Ma and Fei Xia. 2014. Unsupervised dependency parsing with transferring distribution via parallel guidance and entropy regularization. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1337–1348, Baltimore, Maryland, June… find
  23. Erlyn Manguilimotan and Yuji Matsumoto. 2011. Dependency-based analysis for Tagalog sentences. In Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation, pages 343–352, Singapore, December. Institute of Digital Enhancement of Cognitive Processing, Waseda University. find
  24. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 62–72, Edinburgh, Scotland, UK., July. Association for Computational Linguistics. find our card
  25. Ailsa Meechan-Maddon and Joakim Nivre. 2019. How to parse low-resource languages: Cross-lingual parsing, target language annotation, or both? In Proceedings of the Fifth International Conference on Dependency Linguistics (Depling, SyntaxFest 2019), pages 112–120, Paris, France, August. Association f… find
  26. Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis M. Tyers. 2017. Tutorial on Universal Dependencies: Adding a new language to UD. Presented at the 15th Conference of the European Chapter of the Association for Computational Linguistics, April. find
  27. Niko Partanen, Kyungtae Lim, Michael Rießler, and Thierry Poibeau. 2018. Dependency parsing of code-switching data with cross-lingual feature representations. In Proceedings of the Fourth International Workshop on Computational Linguistics of Uralic Languages, pages 1–17, Helsinki, Finland, January.… find
  28. Barbara Plank and Željko Agić. 2018. Distant supervision from disparate sources for low-resource part-of-speech tagging. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 614–620, Brussels, Belgium, October-November. Association for Computational Ling… find
  29. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Association for Computational Linguistics (ACL) System Demonstrations. find our card
  30. Logathan Ramasamy. 2014. Parsing under-resourced languages: Cross-lingual transfer strategies for Indian languages. Ph.D. thesis, Charles University, Prague, Czech Republic. find
  31. Lawrence Reid. 2018. Modeling the linguistic situation in the Philippines. Senri Ethnological Studies, 98:91–105. find
  32. Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1192–1202, Berlin, Germany, August. Association for Computational Linguistic… find
  33. Stephanie Dawn Samson. 2018. A treebank prototype of Tagalog. Undergraduate thesis, University of Tübingen, Germany. find
  34. Paul Schachter and Fe T. Otanes. 1972. Tagalog Reference Grammar. University of California Press, Berkeley and Los Angeles, California. find
  35. Milan Straka and Jana Straková. 2017. Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada, August. Association for Computational Linguistics. find our card
  36. Clara Vania, Yova Kementchedjhieva, Anders Søgaard, and Adam Lopez. 2019. A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Jo… find
  37. Daniel Zeman and Philip Resnik. 2008. Cross-language parser adaptation between related languages. In Proceedings of the IJCNLP-08 Workshop on NLP for Less Privileged Languages. find
  38. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gokirmak, Anna Nedoluzhko, Silvie Cinková, Jan Hajič jr., Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Ur… find
  39. CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–19, Vancouver, Canada, August. Association for Computational Linguistics. find our card
  40. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. find
  41. CoNLL 2018 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium, October. Association for Computational Linguistics. find our card
  42. Daniel Zeman, Joakim Nivre, et al. 2020. Universal Dependencies 2.6. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
  43. Yuhao Zhang, Peng Qi, and Christopher D. Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2205–2215, Brussels, Belgium, October-November. Association for Comput… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aquino-2020-parsing-absence-related-languages-evaluating/