Elamur Research⌕
☰
home / papers / 2026 / NASIMLab at SemEval-2026 Task 9: A Comparative Analysis of Fine-Tuned Small Language Models vs. Generative Large Language Models for Multilingual Polarization Type Detection

NASIMLab at SemEval-2026 Task 9: A Comparative Analysis of Fine-Tuned Small Language Models vs. Generative Large Language Models for Multilingual Polarization Type Detection

Paper · 2026 · Neel Sabhahit, Sanjeevan Selvaganapathy and Mehwish Nasim

Summary

The paper addresses Subtask 2 of SemEval-2026 Task 9: multi-label detection of polarization types (political, racial/ethnic, religious, gender/sexual, other) in social media posts from the multilingual POLAR dataset. The authors fine-tuned a separate encoder-only small language model for each language. They mostly used XLM-RoBERTa, but chose regional models where available: ruRoBERTa, MARBERTv2, Polish and Spanish RoBERTa variants, and AfroXLMR. Thresholds were tuned per label, and Asymmetric Loss was used for the XLM-R models. They compared this setup with few-shot prompting of large commercial and open LLMs (Gemini 3 Flash, Grok 4.1 Fast, Qwen 3.5 397B) accessed through OpenRouter. The fine-tuned SLMs outperformed the LLMs on nearly every low-resource language. LLM scores dropped sharply for low-resource languages, while the SLMs did not show this drop. The takeaway is that compact, language-tailored encoders are a cheap and effective option for polarization and hate-speech classification when resources are limited. Ukrainian is not among the covered languages, though Russian and Polish are.

Key points

  • Leaderboard results: the system ranked 1st in Burmese, 3rd in Swahili, 5th in Hindi and 6th in Punjabi. Twelve languages saw it finish among the ten best systems, and every remaining language saw it finish no lower than 20th. Chinese was excluded from the paper because of post-submission model errors.
  • Average Macro-F1 of the fine-tuned SLMs: 0.5833 on low-resource languages and 0.5336 on high-resource languages. Low- and high-resource groups follow the NLLB classification.
  • Best LLM: Gemini 3 Flash, with 0.4146 on low-resource and 0.5207 on high-resource languages. It topped the other LLMs on 14 of 21 languages, with Grok 4.1 Fast leading on the other 7.
  • Where LLMs won: Gemini 3 Flash beat the SLMs on Punjabi and on several high-resource languages, and Grok 4.1 Fast did so on German.
  • Per-label difficulty: the "other" label was hardest (average F1 0.4223) and political was easiest (0.6835). Results on "other" were very uneven, for example Khmer 0.8895 versus Hausa 0.0000.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of Western AustraliaPerth · Australia8 works · 1,707 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Neel Sabhahit, Sanjeevan Selvaganapathy and Mehwish Nasim. 2026. NASIMLab at SemEval-2026 Task 9: A Comparative Analysis of Fine-Tuned Small Language Models vs. Generative Large Language Models for Multilingual Polarization Type Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.413

@inproceedings{sabhahit2026nasimlab,
  title = {{NASIMLab at SemEval-2026 Task 9: A Comparative Analysis of Fine-Tuned Small Language Models vs. Generative Large Language Models for Multilingual Polarization Type Detection}},
  author = {Neel Sabhahit, Sanjeevan Selvaganapathy and Mehwish Nasim},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.413},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.413}
}

References

29 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro…
  2. Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022. Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic…
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and… find
  4. Jawad Chowdhury, Rezaur Rashid, and Gabriel Tere-janu. 2026. Measuring Social Media Polarization Using Large Language Models and Heuristic Rules. In Aijun An, Alfredo Cuzzocrea, and Hongxin Hu, editors, Social Networks Analysis and Mining, volume 16324, pages 429–444. Springer Nature Switzerland, Ch… find
  5. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  6. Sławomir Dadas, Michał Perełkiewicz, and Rafał Poświata. 2020. Pre-training Polish Transformer-Based Language Models at Scale. In Artificial Intelligence and Soft Computing, pages 301–314, Cham. Springer International Publishing. find
  7. Javier De la Rosa, Eduardo Ponferrada, Manu Romero, Paulo Villegas, Pablo González de Prado Salas, and María Grandury. 2022. BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling. Procesamiento del Lenguaje Natural, pages 13–23. find
  8. DeepSeek-AI. 2025. Deepseek-v3.2: Pushing the frontier of open large language models.
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  10. Joan-Maria Esteban and Debraj Ray. 1994. On the Measurement of Polarization. Econometrica, 62(4):819. find
  11. Google. 2026. Gemini 3 Developer Guide. https://ai.google.dev/gemini-api/docs/gemini-3. ↗
  12. Kati Kish Bar-On, Eugen Dimant, Yphtach Lelkes, and David G Rand. 2024. Unraveling polarization: Insights into individual and collective dynamics. PNAS Nexus, 3(10):pgae426. find
  13. Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems, volume 35, pages 22199–22213. Curran Associates, Inc.
  14. Emily Kubin and Christian Von Sikorski. 2021. The role of (social) media in political polarization: A systematic review. Annals of the International Communication Association, 45(3):188–206. find
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint.
  16. Alexandre Magueresse, Vincent Carles, and Evan Heetderks. 2020. Low-resource Languages: A Review of Past Work and Future Challenges. arXiv preprint. find
  17. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acarürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakra… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  18. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Ky… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  19. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, and 20 o… find
  20. OpenAI. GPT-4o mini Technical Documentation. https://developers.openai.com/api/docs/models/gpt-4o-mini. ↗
  21. Qwen Team. 2026. Qwen3.5: Accelerating productivity with native multimodal agents. find
  22. Tal Ridnik, Emanuel Ben-Baruch, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. 2021. find
  23. Asymmetric Loss For Multi-Label Classification. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 82–91, Montreal, QC, Canada. IEEE. find
  24. Jay J. Van Bavel, Steve Rathje, Elizabeth Harris, Claire Robertson, and Anni Sternisko. 2021. How social media shapes polarization. Trends in Cognitive Sciences, 25(11):913–916. find
  25. A.2 All fine-tuning experiments were conducted using the following common configuration: • Optimizer: AdamW with a weight decay coefficient of 0.01. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all yo…
  26. Learning Rate Schedule: Linear decay with a warmup ratio of 0.1. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing…
  27. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transf…
  28. Shih-Hung Wu and Tsai Tsung Hsun. 2025. Fine-Tuned Models for Hate Speech Detection: Assessing Generalization on Social Media. In 2025 IEEE International Conference on Information Reuse and Integration and Data Science (IRI), pages 250–255, San Jose, CA, USA. IEEE. xAI. Grok 4.1 Fast Developer Docum… find
  29. Dmitry Zmitrovich, Aleksandr Abramov, Andrey Kalmykov, Vitaly Kadulin, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Tatiana Shavrina, Sergei S. Markov, Vladislav Mikhailov, and Alena Fenogenova. 2024. A Family of Pretrained Transformer Language Models for R…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sabhahit-2026-nasimlab-semeval-2026-task-9/