Elamur Research⌕
☰
home / papers / 2026 / Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages

Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages

Paper · 2026 · Zhonghao Zhang, Na Liu, Jiajia Ma et al.

Summary

Agglutinative languages build words by chaining many morphemes, so corpora contain huge numbers of rare surface forms. In low-resource settings, this weakens word-level models, while purely morpheme-based units lose the meaning of the whole word. The authors present MAGNet, a pre-training framework that encodes each word along two parallel paths. The fine-grained path segments words into morphemes, attaches a functional tag to each one, adds morpheme and tag embeddings, and pools them with intra-word attention. It is trained with a masking objective aimed at morphological regularities. The coarse path is a word-level Transformer that does not depend on segmentation, so segmentation errors cannot affect it. A learnable gate then weighs the two representations according to context. On Mongolian and Turkish dependency parsing and NER, MAGNet beats strong single-granularity baselines. Ablations show that every component contributes. This is a practical recipe for morphologically rich, low-resource languages, and the authors suggest extending it to Kazakh and Uyghur.

Key points

  • Preprocessing tools: Traditional Mongolian is segmented and tagged jointly by a fine-tuned multi-task model (MTLM, Liu et al., 2025). Turkish uses the Zemberek-NLP toolkit, with tags (case, number, person, tense/mood, voice) following the UD_Turkish-IMST scheme.
  • Main dependency parsing results: The full model reaches 85.0 LAS on Mongolian and 84.1 LAS on Turkish.
  • Ablation ranking: Removing the whole-word encoder hurts most, dropping LAS to 82.3 for Mongolian and 81.5 for Turkish. Removing segmentation is the next most harmful (82.8 / 82.1), and removing tags causes the smallest drop (83.4 / 82.7).
  • NER ablations: The same ordering holds for NER, where the full Turkish model scores roughly 84 on the reported metrics.
  • Stated limitations: The work covers only two languages and two tasks, and results may be affected by errors in the segmentation and annotation tools.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Inner Mongolia University of Technology.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Morphology · 420 works

Cite

Cite as Zhonghao Zhang, Na Liu, Jiajia Ma, Nier Wu and Guiping Liu. 2026. Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.92

@inproceedings{zhang2026morphology,
  title = {{Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages}},
  author = {Zhonghao Zhang, Na Liu, Jiajia Ma, Nier Wu and Guiping Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.92},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.92}
}

References

29 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmet Acar, Onur Güngör, and Zemberek Development Team. Zemberek-nlp: Turkish natural language processing library. GitHub. find
  2. Duygu Altinok. 2023. A diverse set of freely available linguistic resources for Turkish. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13739–13750, Toronto, Canada. Association for Computational Linguistics. find
  3. Catherine Arnett and Benjamin K. Bergen. 2025. Why do language models perform worse for morphologically complex languages? In Proceedings of the 31st International Conference on Computational Linguistics (COLING 2025), pages 6607–6623. Association for Computational Linguistics.
  4. P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov. 2017. Enriching word vectors with subword information. Transactions of the ACL.
  5. Ryan Cotterell and Hinrich Schütze. 2015. Morphological word-embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1287–1292, Denver, Colorado. Association for Computational Linguistics. find
  6. M. Creutz and K. Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. ACM Transactions on Speech and Language Processing. find
  7. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT.
  8. Benjamin Heinzerling and Michael Strube. 2018. BPEmb: Tokenization-free pre-trained subword embeddings in 275 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  9. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77.
  10. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
  11. Angeliki Lazaridou, Marco Marelli, Roberto Zamparelli, and Marco Baroni. 2013. Compositional-ly derived representations of morphologically complex words in distributional semantics. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa… find
  12. Bofang Li, Aleksandr Drozd, Tao Liu, and Xiaoyong Du. 2018. Subword-level composition functions for learning word embeddings. In Proceedings of the Second Workshop on Subword/Character LEvel Models, pages 38–48, New Orleans. Association for Computational Linguistics. find
  13. Na Liu, Jiajia Ma, Zhonghao Zhang, Aodengbala Aodengbala, Min Lu, and Guiping Liu. 2025. Pre-training and fine-turning multi-task learning model: An effective method for mongolian morphological tagging. In 2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8. find
  14. R. Liu, B. Sisman, F. Bao, J. Yang, G. Gao, and H. Li. 2021. Exploiting morphological and phonological features to improve prosodic phrasing for mongolian speech synthesis. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:274–285. Doi: 10.1109/TASLP.2020.3040523. — [doi:10.1109/taslp.2020.3040523](https://doi.org/10.1109/taslp.2020.3040523) find
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  16. T. Mikolov, K. Chen, G. Corrado, and J. Dean. 2013. Efficient estimation of word representations in vector space. Proceedings of ICLR.
  17. Muhan Na, Xiaolin Jin, and Weihua Wang. 2024a. Mongolian Pre-trained Language Model Incorporating Agglutinative Language Features. Journal of Minzu University of China (Natural Sciences Edition), 33(3):32–39. find
  18. Muhan Na, Rui Liu, Feilong Bao, and Guanglai Gao. 2024b. Pre-training language model for mongolian with agglutinative linguistic knowledge injection. In 2024 International Joint Conference on Neural Networks (IJCNN), pages 1–8. find
  19. J. Pennington, R. Socher, and C. Manning. 2014. Glove: Global vectors for word representation. Proceedings of EMNLP.
  20. M. E. Peters, M. Neumann, M. Iyyer, and 1 others. 2018. Deep contextualized word representations. Proceedings of NAACL-HLT.
  21. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
  22. Mohammad Sadegh Rasooli and Joel R. Tetreault. 2015. Yara parser: A fast and accurate dependency parser. Computing Research Repository, arXiv:1503.06733. Version 2. — [arXiv:1503.06733](https://arxiv.org/abs/1503.06733) find
  23. Sebastian Ruder, Anders Søgaard, and Ivan Vulić. 2019. Unsupervised cross-lingual representation learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 31–38, Florence, Italy. Association for Computational Linguistics. find
  24. Lao S. and et al. 2016. Mongolian named entity recognition system with rich features. In Proceedings of COLING 2016, pages 211–220. 33209PER/LOC/ORG. find
  25. Umut Sulubacak and Gülşen Eryiğit. 2018. Implementing universal dependency, morphology and multiword expression annotation standards for turkish language processing. Turkish Journal of Electrical Engineering & Computer Sciences, pages 1–23. find
  26. Y. Sun, S. Wang, Y. Li, and 1 others. 2020. Ernie: Enhanced representation through knowledge integration. Proceedings of ACL.
  27. Changbing Yang and Garrett Nicolai. 2025. Learning beyond limits: Multitask learning and synthetic data for low-resource canonical morpheme segmentation. ArXiv, abs/2505.16800. — [arXiv:2505.16800](https://arxiv.org/abs/2505.16800) find
  28. Chen Zhang, Mingxu Tao, Quzhe Huang, Jiuheng Lin, Zhibin Chen, and Yansong Feng. 2024. Mc 2 : Towards transparent and culturally-aware nlp for minority languages in china. arXiv preprint arXiv:2311.08348. find
  29. Xinsong Zhang, Pengshuai Li, and Hang Li. 2021. Ambert: A pre-trained language model with multi-grained tokenization. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 421–435. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-morphology-aware-multi-granularity-representation/