Elamur Research⌕
☰
home / papers / 2023 / CLASSLA-Stanza: The Next Step for Linguistic Processing of South Slavic Languages

CLASSLA-Stanza: The Next Step for Linguistic Processing of South Slavic Languages

Paper · 2023 · Luka Terčon and Nikola Ljubešic

Summary

South Slavic languages such as Macedonian and Bosnian remain under-served by NLP tools despite large speaker populations. This paper describes CLASSLA-Stanza, a fork of the Stanza neural pipeline from the CLASSLA knowledge centre, and documents how models for its 2.1 release were trained. The tool covers Slovenian, Croatian, Serbian, Bulgarian and Macedonian, with Internet-language variants for the first three. It handles tokenization, morphosyntactic tagging, lemmatization, dependency parsing, NER and semantic role labeling, though not every language has every task.

Key changes from Stanza include:

  • rule-based tokenizers instead of learned ones,
  • external inflectional lexicons,
  • training on data beyond UD treebanks,
  • handling of closed word classes,
  • CLARIN.SI word embeddings built from large web crawls.

The authors report that CLASSLA-Stanza matches or beats Stanza on every task, and it adds languages and tasks Stanza lacks. A manual comparison on web text led to a new "web" mode that pairs the standard tokenizer with non-standard models. The paper is meant as the reference for future releases and for users training their own models.

Key points

  • Tokenization: Stanza's learned tokenizer is replaced by two rule-based tools that also split sentences. Obeliks handles standard Slovenian, and the ReLDI tokenizer handles non-standard Slovenian and all other languages.
  • Lexicon lookup: Lemma lookup uses full XPOS (MULTEXT-East V6) tags rather than UPOS. For Slovenian, the lexicon also limits which tags the morphosyntactic tagger can predict.
  • Extra training data: Standard Croatian tagging and lemmatization models use about 290 thousand additional manually annotated tokens. The dependency parser is trained on roughly 210 thousand tokens.
  • Task coverage gaps: SRL is available only for Slovenian, since it relies on JOS dependencies. Macedonian has no dependency parsing or NER models.
  • Evaluation metrics: Lemmatization, tagging (UPOS, XPOS, UFeats) and SRL are scored with micro F1. Parsing is scored with LAS.
  • Web mode: Standard models misread non-standard forms, especially words missing diacritics. Non-standard models, in turn, segmented sentences worse. This motivated the combined "web" type.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Jožef Stefan InstituteLjubljana · Slovenia17 works · 2,165 citations🎓Jožef Stefan International Postgraduate SchoolLjubljana · Slovenia11 works · 1,776 citations🎓University of LjubljanaLjubljana · Slovenia13 works · 38,693 citations🔬Institute of Contemporary HistoryLjubljana · Slovenia4 works · 32 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Luka Terčon and Nikola Ljubešic. 2023. CLASSLA-Stanza: The Next Step for Linguistic Processing of South Slavic Languages. arXiv:2308.04255.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.04255 · arXiv 2308.04255

@misc{tervcon2023classla,
  title = {{CLASSLA-Stanza: The Next Step for Linguistic Processing of South Slavic Languages}},
  author = {Luka Tervcon and Nikola Ljubevsic},
  year = {2023},
  doi = {10.48550/arxiv.2308.04255},
  eprint = {2308.04255},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.04255}
}

References

35 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dirk Goldhahn, Maciej Janicki, and Uwe Quasthoff. Corpus collection for under-resourced languages with more than one million speakers. In Workshop on Collaboration and Computing for Under-Resourced Languages (CCURL), LREC, 2016. find
  2. Nikola Ljubešić, Tomaž Erjavec, Maja Miličević Petrović, and Tanja Samardžić. Together We Are Stronger: Bootstrapping Language Technology Infrastructure for South Slavic Languages with CLARIN.SI, pages 429– 456. De Gruyter, Berlin, Boston, 2022. ISBN 9783110767377. doi:10.1515/9783110767377-017… — [doi:10.1515/9783110767377-017](https://doi.org/10.1515/9783110767377-017) find
  3. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2020. URL https://nlp.stanfor… ↗
  4. Nikola Ljubešić and Kaja Dobrovoljc. What does Neural Bring? Analysing Improvements in Morphosyntactic Annotation and Lemmatisation of Slovenian, Croatian and Serbian. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, pages 29–34, Florence, Italy, August 2019. Associat… find
  5. Miha Grčar, Simon Krek, and Kaja Dobrovoljc. Obeliks: statisticni oblikoskladenjski oznacevalnik in lematizator za slovenski jezik [Obeliks: Statistical Morphosyntactic Tagger and Lemmatizer for Slovene]. In J. Žganec Gros T. Erjavec, editor, Proceedings of the Eighth Language Technologies Conferen… find
  6. Tanja Samardžić, Nikola Ljubešić, and Maja Miličević. Regional Linguistic Data Initiative (ReLDI). In The 5th Workshop on Balto-Slavic Natural Language Processing, pages 40–42, Hissar, Bulgaria, September 2015. INCOMA Ltd. Shoumen, BULGARIA. URL https://aclanthology.org/W15-5306. ↗
  7. Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. Universal Dependencies. Computational Linguistics, 47(2):255–308, July 2021. ISSN 0891-2017. doi:10.1162/coli_a_00402. URL https://doi.org/10.1162/coli_a_00402. ↗
  8. Tomaž Erjavec. MULTEXT-East: morphosyntactic resources for Central and Eastern European languages. Language Resources and Evaluation, 46(1):131–142, 2012. ISSN 1574020X, 15728412. URL http://www.jstor.org/ stable/41486069. ↗
  9. Tomaž Erjavec, Darja Fišer, Simon Krek, and Nina Ledinek. The JOS Linguistically Tagged Corpus of Slovene. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta, May 2010a. European Language Resources Association (ELRA). URL http://www… find
  10. Simon Krek, Polona Gantar, Kaja Dobrovoljc, and Iza Škrjanec. Označevanje udeleženskih vlog v učnem korpusu za slovenščino [Semantic Role Labeling in the Training Corpus for Slovene]. page 106–110. Znanstvena založba Filozofske fakultete [Ljubljana University Press, Faculty of Arts], 2016. URL ht… find
  11. Nikola Ljubešić and Tanja Samardžić. Croatian linguistic training corpus hr500k 2.0, 2023. ISSN 2820-4042. URL http://hdl.handle.net/11356/1792. Slovenian language resource repository CLARIN.SI. ↗
  12. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Depe…
  13. Luka Terčon, Nikola Ljubešić, and Tomaž Erjavec. Word embeddings CLARIN.SI-embed.sl 2.0, 2023. ISSN 2820-4042. URL http://hdl.handle.net/11356/1791. Slovenian language resource repository CLARIN.SI. ↗
  14. Luka Terčon and Nikola Ljubešić. Word embeddings CLARIN.SI-embed.hr 2.0, 2023a. ISSN 2820-4042. URL http://hdl.handle.net/11356/1790. Slovenian language resource repository CLARIN.SI. ↗
  15. Luka Terčon and Nikola Ljubešić. Word embeddings CLARIN.SI-embed.sr 2.0, 2023b. ISSN 2820-4042. URL http://hdl.handle.net/11356/1789. Slovenian language resource repository CLARIN.SI. ↗
  16. Luka Terčon and Nikola Ljubešić. Word embeddings CLARIN.SI-embed.mk 2.0, 2023c. ISSN 2820-4042. URL http://hdl.handle.net/11356/1788. Slovenian language resource repository CLARIN.SI. ↗
  17. Luka Terčon and Nikola Ljubešić. Word embeddings CLARIN.SI-embed.bg 1.0, 2023d. ISSN 2820-4042. URL http://hdl.handle.net/11356/1796. Slovenian language resource repository CLARIN.SI. ↗
  18. Marta Bañón, Miquel Esplà-Gomis, Mikel L. Forcada, Cristian García-Romero, Taja Kuzman, Nikola Ljubešić, Rik van Noord, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Peter Rupnik, Vít Suchomel, Antonio Toral, Tobias van der Werff, and Jaume Zaragoza. MaCoCu: Massive collection and curation of monolin… find
  19. Slavko Žitnik and Frenk Dragar. SloBENCH evaluation framework, 2021. ISSN 2820-4042. URL http://hdl.handle.net/11356/1469. Slovenian language resource repository CLARIN.SI. ↗
  20. Špela Arhar Holdt, Simon Krek, Kaja Dobrovoljc, Tomaž Erjavec, Polona Gantar, Jaka Čibej, Eva Pori, Luka Terčon, Tina Munda, Slavko Žitnik, Nejc Robida, Neli Blagus, Sara Može, Nina Ledinek, Nanika Holz, Katja Zupan, Taja Kuzman, Teja Kavčič, Iza Škrjanec, Dafne Marko, Lucija Jezeršek, and Anja… find
  21. Simon Krek, Kaja Dobrovoljc, Tomaž Erjavec, Sara Može, Nina Ledinek, Nanika Holz, Katja Zupan, Polona Gantar, Taja Kuzman, Jaka Čibej, Špela Arhar Holdt, Teja Kavčič, Iza Škrjanec, Dafne Marko, Lucija Jezeršek, and Anja Zajc. Training corpus ssj500k 2.3, 2021. ISSN 2820-4042. URL http://hdl.handl… ↗
  22. Jakob Lenardič, Jaka Čibej, Špela Arhar Holdt, Tomaž Erjavec, Darja Fišer, Nikola Ljubešić, Katja Zupan, and Kaja Dobrovoljc. CMC training corpus Janes-Tag 3.0, 2022. ISSN 2820-4042. URL http://hdl.handle.net/11356/ 1732. Slovenian language resource repository CLARIN.SI. ↗
  23. Nikola Ljubešić, Tomaž Erjavec, Vuk Batanović, Maja Miličević, and Tanja Samardžić. Croatian Twitter training corpus ReLDI-NormTagNER-hr 3.0, 2023a. ISSN 2820-4042. URL http://hdl.handle.net/11356/1793. Slovenian language resource repository CLARIN.SI. ↗
  24. Vuk Batanović, Nikola Ljubešić, Tanja Samardžić, and Tomaž Erjavec. Serbian linguistic training corpus SETimes.SR 2.0, 2023. ISSN 2820-4042. URL http://hdl.handle.net/11356/1843. Slovenian language resource repository CLARIN.SI. ↗
  25. Nikola Ljubešić, Tomaž Erjavec, Vuk Batanović, Maja Miličević, and Tanja Samardžić. Serbian Twitter training corpus ReLDI-NormTagNER-sr 3.0, 2023b. ISSN 2820-4042. URL http://hdl.handle.net/11356/1794. Slovenian language resource repository CLARIN.SI. ↗
  26. Tomaž Erjavec, Ana-Maria Barbu, Ivan Derzhanski, Ludmila Dimitrova, Radovan Garabík, Nancy Ide, Heiki-Jaan Kaalep, Natalia Kotsyba, Cvetana Krstev, Csaba Oravecz, Vladimír Petkevič, Greg Priest-Dorman, Behrang QasemiZadeh, Adam Radziszewski, Kiril Simov, Dan Tufiş, and Katerina Zdravkova. MULTEXT-… find
  27. Petya Osenova and Kiril Simov. Universalizing BulTreeBank: a Linguistic Tale about Glocalization. In The 5th Workshop on Balto-Slavic Natural Language Processing, pages 81–89, Hissar, Bulgaria, September 2015. INCOMA Ltd. Shoumen, BULGARIA. URL https://aclanthology.org/W15-5313. ↗
  28. Jaka Čibej, Kaja Gantar, Kaja Dobrovoljc, Simon Krek, Peter Holozan, Tomaž Erjavec, Miro Romih, Špela Arhar Holdt, Luka Krsnik, and Marko Robnik-Šikonja. Morphological lexicon Sloleks 3.0, 2022. ISSN 2820-4042. URL http://hdl.handle.net/11356/1745. Slovenian language resource repository CLARIN.SI. ↗
  29. Nikola Ljubešić. Inflectional lexicon hrLex 1.3, 2019a. ISSN 2820-4042. URL http://hdl.handle.net/11356/ 1232. Slovenian language resource repository CLARIN.SI. ↗
  30. Nikola Ljubešić, Filip Klubička, Željko Agić, and Ivo-Pavao Jazbec. New inflectional lexicons and training corpora for improved morphosyntactic annotation of Croatian and Serbian. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4264–4270,… find
  31. Nikola Ljubešić, Maja Miličević Petrović, and Tanja Samardžić. Borders and boundaries in bosnian, croatian, montenegrin and serbian: Twitter data to the rescue. Journal of Linguistic Geography, 6(2):100–124, 2018. find
  32. Nikola Ljubešić. Inflectional lexicon srLex 1.3, 2019b. ISSN 2820-4042. URL http://hdl.handle.net/11356/ 1233. Slovenian language resource repository CLARIN.SI. ↗
  33. Dimitar Popov, Kiril Simov, and Svetlomira Vidinska. Rechnik za pravogovor, pravopis, punktuatsiya [A Dictionary of Writing, Pronunciation and Punctuation of Bulgarian Language]. Atlantis KL, Sofia, 1998. find
  34. Kaja Dobrovoljc, Luka Terčon, and Nikola Ljubešić. Universal Dependencies za slovenščino: nadgradnja smernic, učnih podatkov in razčlenjevalnega modela [Universal Dependencies for Slovenian: An Upgrade to the Guidelines, Annotated Data and Parsing Model]. In Darja Fišer and Tomaž Erjavec, edito… find
  35. Marta Bañón, Malina Chichirau, Miquel Esplà-Gomis, Mikel L. Forcada, Aarón Galiano-Jiménez, Cristian García-Romero, Taja Kuzman, Nikola Ljubešić, Rik van Noord, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Peter Rupnik, Vít Suchomel, Antonio Toral, and Jaume Zaragoza-Bernabeu. Slovene web corpus MaC… — Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ter-vcon-2023-classla-stanza-next-step-linguistic/