Elamur Research⌕
☰
home / papers / 2025 / Comparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages

Comparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages

Summary

The paper asks how well three widely used transformer encoders handle sentiment classification across nine typologically different languages: Arabic and Bulgarian, the Uralic pair Hungarian and Finnish, and five Western European languages (Spanish, English, Italian, German and French). The authors fine-tune BERT (an English-oriented baseline), mBERT and XLM-R separately on corpora for each language. They use subword tokenization, normalization and stopword removal, a cross-entropy objective and early stopping. Each language is scored on its own with accuracy, precision, recall and macro F1, followed by per-class error analysis. BERT is strongest on English. XLM-R leads on Italian, Finnish and Arabic. mBERT trails slightly almost everywhere. The authors attribute XLM-R's edge on morphologically rich languages to its broad multilingual pretraining, and they argue that language-specific fine-tuning matters most for dialectal and informal text. The paper reports a single number per model and language, with no variance and no significance testing, and its prose and tables sometimes disagree. Its value is a rough per-language baseline rather than a definitive ranking.

Key points

  • Data: ArSAS for Arabic, the FinnSentiment Corpus for Finnish, and the Twitter-based Open Dataset for Sentiment Analysis for EN/ES/FR/IT/DE. Sources for the Hungarian and Bulgarian data are not stated.
  • Training set sizes range from 5,040,000 examples for English down to 12,000 for Finnish and 17,600 for Arabic. The validation and test splits are identical in size for every language.
  • Fine-tuning settings: learning rate 3×10⁻⁵, batch size 32, 5 epochs, maximum sequence length 128.
  • Accuracy:
    • BERT reaches 91.2% on English, against 90.0% for XLM-R and 89.5% for mBERT.
    • XLM-R reaches 89.0% on Italian, 88.0% on Finnish and 85.0% on Arabic.
    • The lowest score is mBERT's 82.0% on Bulgarian.
  • Inconsistency on Hungarian: the conclusion calls XLM-R superior on Hungarian, but its own table gives XLM-R 83.0% there, below BERT (85.0%) and mBERT (84.0%). The "above 88%" claim in the abstract therefore holds only for some complex languages.

Interesting ideas

  • I1. Running an English-centric model next to multilingual ones on every language gives a cheap reference point, so any gain from broader pretraining is measured against something concrete. — For Mova (learn): Mova's evaluation could always include a deliberately simple baseline reader, so that improvements claimed by the full tree-to-frames pipeline are measured against a fixed floor.
  • I2. Scoring each language separately, with macro averages and per-class error analysis, keeps a large language or a majority class from masking failures elsewhere. — For Mova (learn): Mova's test reports could be broken down by text genre and by question type, so that a weak category such as negation or irony questions cannot hide inside the overall accuracy.
  • I3. Pairing the corpus-size table with the results table exposes how strongly scores follow data volume, here roughly 5M versus 12k examples. — For Mova (tree): treebank sizes could be logged alongside per-dialect LAS in gparse experiments, to separate genuine parser weaknesses from plain data scarcity.
  • I4. The authors stress that subword segmentation matters most for agglutinative languages, where whole-word vocabularies fragment heavily. — For Mova (tokens): this supports measuring tagger and lemmatizer errors on rare, heavily inflected or compound English forms specifically, instead of only on the frequent vocabulary.
  • I5. The authors argue that fine-tuning on dialect-specific and informal data is what pushes performance on social-media text. — For Mova (frames): a separate slice of informal or colloquial input could be used to check whether irony and speech-act detection in prag hold up outside edited prose.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5121/csit.2024.150112 · arXiv 2501.12540

@misc{sidorov2025comparative,
  title = {{Comparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages}},
  author = {Sidorov;, Mikhail Krasitskii; Olga Kolesnikova; Liliana Chanona Hernandez; Grigori and Gelbukh, Alexander},
  year = {2025},
  doi = {10.5121/csit.2024.150112},
  eprint = {2501.12540},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.5121/csit.2024.150112}
}

References

27 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. L. Zhu and A. Koivisto, ”XLM-R for Finnish Sentiment Analysis: Capturing Morphological Complexity,” Journal of Computational Linguistics, 2024. Available: https://doi.org/10.1111/finsen2024. ↗
  2. A. Virtanen and H. Paloma¨ki, ”Multilingual BERT and FinBERT for Finnish Social Media Analysis,” Journal of Natural Language Processing, 2019. Available: https://doi.org/10.1234/finbert2019. ↗
  3. K. Kettunen, ”Challenges of Sentiment Analysis in Agglutinative Languages: Case of Finnish,” Journal of Language Technology, 2021. Available: https://doi.org/10.5678/lantech2021. ↗
  4. S. Schweter, ”Fine-Tuning Multilingual BERT for German Sentiment Analysis,” International Journal of Computational Linguistics, 2020. Available: https://doi.org/10.1002/gerbert2020. ↗
  5. L. Martin et al., ”CamemBERT: RoBERTa-Based Model for French,” Journal of French Language Processing, vol. 15, pp. 55-70, 2020. Available: https://doi.org/10.1093/jflp2020. ↗
  6. C. Guti´errez and J. Rivas, ”Optimizing BERT for Spanish Sentiment Analysis,” Arxiv P reprint, 2022. Available: https://arxiv.org/abs/2203.01567. ↗
  7. M. Polignano and P. Basile, ”AlBERTo: A Sentiment Analysis Model for Italian Social Media,” Pro - ceedings of the Italian NLP Conference, 2019. Available: https://doi.org/10.1002/alberto2019. ↗
  8. W. Antoun, R. Baly, and H. Hajj, ”AraBERT: Transformer-Based Model for Arabic Language Understanding,” Arxiv Preprint, 2020. Available: https://arxiv.org/abs/2003.00104. ↗ our card
  9. M. Abdul-Mageed and E. M. Nagoudi, ”ARBERT and MARBERT: Deep Bidirectional Transformers for Arabic NLP,” Journal of Computational Arabic Linguistics, 2021. Available: https://doi.org/10.1002/arbmbert2021. ↗
  10. G. Farkas, ”Sentiment Analysis in Hungarian: Exploring Morphological Adaptations,” Journal of Eastern European Computational Linguistics, 2022. Available: https://doi.org/10.5678/hunsent2022. ↗
  11. D. Ganchev, ”Exploring the Effectiveness of Multilingual Models for Bulgarian Sentiment Analysis,” Journal of Computational Linguistics, vol. 28, pp. 120-135, 2021. Available: https://doi.org/10.1001/jcl2021. ↗
  12. I. Ganchev, ”Adapting Multilingual BERT for Bulgarian Sentiment Classification,” Journal of Balkan Linguistics, 2023. Available: https://doi.org/10.1111/bulgsent2023. ↗
  13. A. Joshi et al., ”Unsupervised Learning for Low-resource Languages: Extending mBERT,” Journal of Low-resource Language Studies, vol. 25, pp. 35-50, 2022. Available: https://doi.org/10.1177/jlr2022. ↗
  14. J. Brown and T. Smith, ”BERT and Sentiment Analysis for English and Anal ytical Languages,” Language Processing Research, vol. 15, no. 3, pp. 225-240, 2023. Available: https://doi.org/10.1002/lpr2023. ↗
  15. X. Liu and R. Wang, ”Multilingual BERT in High Inflection Languages: A Comparative Study,” Journal of AI and Linguistics, 2022. Available: https://doi.org/10.1145/jail2022. ↗
  16. A. Conneau et al., ”Unsupervised Cross-lingual Representation Learning at Scale,” Proceedings of the 58th ACL, pp. 8440-8451, 2020. Available: https://doi.org/10.18653/v1/2020.acl-main.747. ↗
  17. C. Sun, X. Qiu, Y. Xu, ”How to Fine-Tune BERT for Text Classification?” CCF International Conference on NLP and Chinese Computing, 2019. Available: https://doi.org/10.1007/978-3-030-32381-3. ↗ our card
  18. S. Kim, ”Sentiment Analysis for Agglutinative Languages: A Cross-linguistic Study,” International Journal of Language and Computing, vol. 29, pp. 65-80, 2023. Available: https://doi.org/10.1177/ijlc2023. ↗
  19. M. Almutairi et al., ”AraBERT: Enhancing Sentiment Analysis for Arabic Dialects,” Computational Linguistics Journal, vol. 12, pp. 345-361, 2023. Available: https://doi.org/10.1056/arabert2023. ↗
  20. P. Gupta, ”Multilingual Transformers for Dialectal Sentiment Analysis,” ACM Transactions on Com - putational Logic, vol. 23, no. 4, 2022. Available: https://doi.org/10.1109/tcl2022. ↗
  21. L. Huang, ”Exploring mBERT in Underrepresented Languages: Performance and Challenges,” Linguistic Technology Review, vol. 18, pp. 112-130, 2024. Available: https://doi.org/10.1093/ltr2024. ↗
  22. H. Chen and M. Zhang, ”Multilingual Sentiment Classification Using Transformer Models,” Journal of Natural Language Engineering, vol. 32, pp. 102-118, 2023. Available: https://doi.org/10.1007/nle2023. ↗
  23. D. Lee, ”Fine-Tuning Transformer Models for Language-Specific Sentiment Analysis,” Computational Intelligence Journal, vol. 29, pp. 57-70, 2023. Available: https://doi.org/10.1016/ci2023. ↗
  24. K. Johnson, ”Cross-Lingual Sentiment Analysis Using XLM-R for Social Media Data,” Journal of Social Media Studies, vol. 11, pp. 92-107, 2022. Available: https://doi.org/10.1038/jsms2022. ↗
  25. A. dos Santos, ”Formal vs Informal Registers in Sentiment Analysis: A Study of French,” Language and Technology Review, vol. 17, pp. 95-110, 2021. Available: https://doi.org/10.1145/lt2021. ↗
  26. D. Miller, ”Transfer Learning for Sentiment Analysis in Low-resource Languages,” Journal of Language and AI, vol. 14, pp. 90-105, 2023. Available: https://doi.org/10.1001/jla2023. ↗
  27. J. Devlin et al., ”BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 NAACL, pp. 4171-4186, 2019. Available: https://doi.org/10.18653/v1/N19-1423. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/krasitskii-2025-comparative-approaches-sentiment-analysis-using/