Elamur Research⌕
☰
home / papers / 2025 / The UD-NewsCrawl Treebank: Reflections and Challenges from a Large-scale Tagalog Syntactic Annotation Project

The UD-NewsCrawl Treebank: Reflections and Challenges from a Large-scale Tagalog Syntactic Annotation Project

Summary

Tagalog is widely spoken, but syntactic resources for it remain scarce. The two existing UD treebanks, UD-Ugnayan and UD-TRG, together contain only 149 sentences. This is too little to train parsers. Moreover, they have no established split into subsets, so the results of earlier work are hard to compare. The authors created UD-NewsCrawl, the largest Tagalog treebank to date, with 15,619 sentences taken from the Leipzig Tagalog NewsCrawl corpus. Native speakers with linguistic training annotated it manually following Universal Dependencies. Quality was checked both manually and semi-automatically. Several transformer-based baseline parsers were trained on this data and will serve as a reference point for further research. The authors also explain why some features of Tagalog grammar are hard to capture with UD categories, which are in effect based on English. These features include the symmetrical voice system, the absence of a copula verb, and the contested part-of-speech status of roots. The work is valuable both as an open resource and as practical experience for building treebanks for other low-resource languages.

Key points

  • Size and split. Train has 12.4k sentences and 286.9k tokens; dev and test have about 1.56k sentences and ~37k tokens each. For comparison, UD-Ugnayan contains 94 sentences from educational texts, and UD-TRG contains 55 sentences from grammar references.
  • Data source and annotation. The texts come from news in Leipzig Tagalog NewsCrawl written no later than 2011. The annotation was done by staff of the Department of Linguistics at the University of the Philippines Diliman. Part of the work took place during the COVID-19 pandemic (January 2021 – April 2022).
  • Voice. The authors chose a symmetrical approach: sentences with different voice affixes are treated as equally basic transitive constructions. Voice is marked with a morphological feature with five values: Act, Pass, Bfoc, Lfoc, Cau.
  • Availability. The treebank and baseline models are open: hf.co/datasets/UD-Filipino/UD_Tagalog-NewsCrawl. All 17 UPOS tags and 39 DEPREL types occur in the train portion.
  • Limitations. The corpus consists only of news, dominated by entertainment and health topics. The texts are no newer than 2011. Annotation may be inconsistent in borderline cases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Universal Dependencies · Treebanks

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.357 · arXiv 2505.20428

@inproceedings{aquino2025ud,
  title = {{The UD-NewsCrawl Treebank: Reflections and Challenges from a Large-scale Tagalog Syntactic Annotation Project}},
  author = {Aquino, Angelina Aspra and Miranda, Lester James Validad and Or, Elsie Marie T.},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.357},
  eprint = {2505.20428},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2025.acl-long.357}
}

References

67 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassi-lyev, Jesujoba Alabi, Yanke Mao, Haonan Gao, and En-Shiun Lee. 2024. SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects. In Proceedings of the 18th Conference of the European Chapter of… find our card
  2. Željko Agić. 2017. Cross-lingual parser selection for low-resource languages. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 1–10, Gothenburg, Sweden. Association for Computational Linguistics. find our card
  3. Angelina Aquino and Franz de Leon. 2020. Parsing in the absence of related languages: Evaluating low-resource dependency parsers on Tagalog. In Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020), pages 8–15, Barcelona, Spain (Online). Association for Computational Linguistics. 6… find our card
  4. Angelina Aquino and Franz de Leon. 2022. Zero-shot and few-shot approaches for tokenization, tagging, and dependency parsing of Tagalog text. In Proceedings of the 36th Pacific Asia Conference on Language, Information and Computation, pages 190– 202, Manila, Philippines. Association for Computationa… find
  5. Ernesto H. Cubar. 1975. Topicalization and Some Related Processes in Philippine Languages. University of the Philippines Department of Linguistics. Reprinted in The Archive Classics (2019). find
  6. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2019. On the cross-lingual transferability of monolingual representations. arXiv preprint arXiv:1910.11856. find our card
  7. Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. Universal Dependencies. Computational Linguistics, 47(2):255–308. find our card
  8. Maria Bardají, Elsie Or, Angelina Aquino, and Nikolaus Himmelmann. 2024. The challenges of symmetrical voice languages for universal dependencies. In Proceedings of the 15th International Conference of the Association for Linguistic Typology. find
  9. Fiona de Vos. 2010. Essential Tagalog Grammar: A Reference for Learners of Tagalog. Learning Tagalog. find
  10. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA. Association for Computational Linguistics. find
  11. Shaily Bhatt and Fernando Diaz. 2024. Extrinsic evaluation of cultural competence in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16055–16074, Miami, Florida, USA. Association for Computational Linguistics. find
  12. Frank R Blake. 1917. Reduplication in tagalog. The American journal of philology, 38(4):425–431. find
  13. Robert Blust. 1991. The Greater Central Philippines Hypothesis. Oceanic Linguistics, 30(2):73–129. find
  14. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  15. Jill Carrier-Duncan. 1985. Linking of thematic roles in derivational word formation. Linguistic Inquiry, 16(1):1–34. find
  16. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  17. Jan Christian Blaise Cruz and Charibeth Cheng. 2022. Improving large-scale language models and resources for Filipino. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages V. P. De Guzman. 1978. Syntactic Derivation of Tagalog Verbs. University of Hawaii Press, Honolu… find
  18. Mathieu Dehouck and Pascal Denis. 2019. Phylogenic multi-lingual dependency parsing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 192–203, Minneapolis, Minnes… find
  19. Peter Dirix, Liesbeth Augustinus, Daniel van Niekerk, and Frank Van Eynde. 2017. Universal Dependencies for Afrikaans. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 38–47, Gothenburg, Sweden. Association for Computational Linguistics. find
  20. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 20–30, Vancouver, Canada. Association for Computa… find our card
  21. Matthew S. Dryer and Martin Haspelmath. 2013. WALS Online (v2020.4). Data set. find
  22. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. find
  23. David M. Eberhard, Gary F. Simons, and Charles D. Fennig, editors. 2024. Ethnologue: Languages of the World, 27 edition. SIL International, Dallas, Texas. find
  24. Leo James English. 1986. Tagalog-English dictionary. Congregation of the Most Holy Redeemer Manila. find
  25. William Foley. 2008. The place of philippine languages in a typology of voice systems. In P. K. Austin and S. Musgraves, editors, Voice and grammatical relations in Austronesian languages, pages 22–44. CSLI Publications. find
  26. Karën Fort. 2016. Collaborative Annotation for Reliable Natural Language Processing: Technical and Sociological Aspects. John Wiley & Sons. find
  27. Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. Building large monolingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC‘12), pages 759–765, Istanbul, Turkey. European… find our card
  28. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with Disentangled Attention. In International Conference on Learning Representations. find our card
  29. Santiago Herrera, Caio Corro, and Sylvain Kahane. 2024. Sparse logistic regression with high-order features for automatic grammar rule extraction from treebanks. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 202… find
  30. Nikolaus P. Himmelmann. 1991. The Philippine Challenge to Universal Grammar. Number N.F. 15 in Arbeitspapier / Institut für Sprachwissenschaft, Universität Köln. find
  31. Nikolaus P. Himmelmann. 2005. Tagalog. In K. Alexander Adelaar and Nikolaus P. Himmelmann, editors, The Austronesian Languages of Asia and Madagascar, 1st edition, page 27. Routledge. find
  32. Nikolaus P. Himmelmann. 2007. Lexical categories and voice in tagalog. In P. K. Austin and S. Mus-grave, editors, Voice and grammatical relations in Austronesian languages, pages 247–293. find
  33. Matthew Honnibal and Mark Johnson. 2015. An improved non-monotonic transition system for dependency parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1373–1378, Lisbon, Portugal. Association for Computational Linguistics. find our card
  34. Hiroshi Kanayama and Ran Iwamoto. 2020. How universal are Universal Dependencies? exploiting syntax for multilingual clause-level sentiment detection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4063–4073, Marseille, France. European Language Resources Associati… find
  35. Daniel Kaufman. 2009. Austronesian nominalism and its consequences: A tagalog case study. Theoretical Linguistics, 35(1):1–49. find
  36. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing Universal Dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–279… find our card
  37. Paul Kroeger. 1993. Phrase structure and grammatical relations in Tagalog. Center for the Study of Language (CSLI). find
  38. Shalom Lappin. 2024. Assessing the strengths and weaknesses of large language models. Journal of Logic, Language and Information, 33(1):9–20. find
  39. Percy Liang, Michael Jordan, and Dan Klein. 2011. Learning dependency-based compositional semantics. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 590–599, Portland, Oregon, USA. Association for Computational Linguistic… find
  40. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. find
  41. Lester James Miranda. 2023a. calamanCy: A Tagalog natural language processing toolkit. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pages 1–7, Singapore. Association for Computational Linguistics. find
  42. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  43. Lester James Miranda. 2023b. Developing a named entity recognition dataset for Tagalog. In Proceedings of the First Workshop in South East Asian Language Processing, pages 13–20, Nusa Dua, Bali, Indonesia. Association for Computational Linguistics. find
  44. Jem R Javier and Elsie Marie T Or. 2022. Tagalog linguistics: Historical development and theoretical trends. In The Routledge Handbook of Asian Linguistics, pages 33–46. Routledge. find
  45. Lester James Miranda, Ákos Kádár, Adriane Boyd, Sofie Van Landeghem, Anders Søgaard, and Matthew Honnibal. 2022. Multi hash embeddings in spacy. arXiv preprint arXiv:2212.09255. — [arXiv:2212.09255](https://arxiv.org/abs/2212.09255) find
  46. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Comp… find our card
  47. Kadri Muischnek, Kaili Müürisep, and Tiina Puolakainen. 2016. Estonian dependency treebank: from constraint grammar tagset to Universal Dependencies. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC‘16), pages 1558–1565, Portorož, Slovenia. European Lan… find
  48. Thomas Müller, Ryan Cotterell, Alexander Fraser, and Hinrich Schütze. 2015. Joint lemmatization and morphological tagging with lemming. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2268–2274, Lisbon, Portugal. Association for Computational Linguist… find
  49. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Language Resources and Eval… find
  50. Timothy Osborne and Kim Gerdes. 2019. The status of function words in dependency grammar: A critique of universal dependencies (ud). Glossa: a journal of general linguistics (2016-2021). find
  51. Philippine Statistics Authority. 2020 census of population and housing report no. 21 - demographic and housing characteristics (non-sample variables). Retrieved from https://library.psa.gov.ph/. ↗
  52. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online.… find our card
  53. Uwe Quasthoff and Matthias Richter. 1998. Projekt Der Deutsche Wortschatz. na. find
  54. Andrea Rackowski and Norvin Richards. 2005. Phrase edge and extraction: A tagalog case study. Linguistic Inquiry, 36(4):565–599. find
  55. Ria P. Rafael. 2016. Sinong pasimuno? paggamit ng subject at topic sa pag-aaral ng wika’ (who’s the initiator? the use of subject and topic in language studies). Daluyan: Journal ng Wikang Filipino, 22(1/2):165–180. find
  56. Lawrence Reid. 2005. Tagalog and philippine languages. In Philipp Skutch, editor, Encyclopedia of Linguistics. Routledge, New York. find
  57. Sonja Riesberg, Kurt Malcher, and Nikolaus P. Himmelmann. 2019. How universal is agent-first? evidence from symmetrical voice languages. Language, 95(3):523–561. find
  58. Michael J Ryan, William Held, and Diyi Yang. 2024. Unintended impacts of LLM alignment on global representation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16121–16140, Bangkok, Thailand. Association for Computational Lin… find
  59. Paul Schachter. 1976. The subject in philippine languages: Topic, actor, actor-topic, or none of the above. In Charles Li, editor, Subject and Topic, pages 491–518. Academic Press, New York. find
  60. Paul Schachter and Fe T. Otanes. 1972. Tagalog Reference Grammar. University of California Press. find
  61. Milan Straka, Jan Hajič, and Jana Straková. 2016. UD-Pipe: Trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, POS tagging and parsing. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4290– 4297,… find
  62. Roberto D Tangco and Ricardo Ma Nolasco. 2002. ‘taglish’verbs: How english loanwords make it into the philippine languages. In Tenth Annual Meeting of the Southeast Asian Linguistics Society, pages 391– 406. find
  63. Simone Tedeschi, Valentino Maiorca, Niccolò Campolungo, Francesco Cecconi, and Roberto Navigli. 2021. WikiNEuRal: Combined neural and knowledge-based silver data creation for multilingual NER. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2521–2533, Punta Cana, Domi… find
  64. Jianwei Wang, Tianyin Wang, and Ziqian Zeng. 2024. On the use of silver standard data for zero-shot classification tasks in information extraction. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1242… find
  65. Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, et al. 2024. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024 Conference on Empirical Methods in… find our card
  66. Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2022. A survey of active learning for natural language processing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6166–6190, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  67. Stephanie Dawn Samson. 2018. A treebank prototype of Tagalog. Bachelor’s thesis, University of Tübingen, Germany. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/aquino-2025-ud-newscrawl-treebank-reflections-challenges/