Elamur Research⌕
☰
home / papers / 2026 / POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization

POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization

Paper · 2026 · Usman Naseem, Robert Geislinger, Juan Ren et al.

Summary

Computational studies of online polarization mostly cover English or other well-resourced languages, focus on a single event such as a U.S. election, and treat polarization as a yes/no label. POLAR addresses these gaps with a benchmark of more than 110K texts in 22 languages from seven language families. The texts come from social media and local news forums and are tied to real events in each culture, including wars, elections, migration, and public health crises. Native-speaker annotators labeled each text at three levels: whether it is polarized, what social dimension the polarization concerns, and which rhetorical strategies it uses. The annotation tools varied by context and included MTurk, Prolific, LabelStudio, and POTATO. The authors fine-tuned six small language models and tested open and closed LLMs in zero-shot and few-shot settings. Models handle binary detection fairly well but perform much worse on types and manifestations. Implicit hostility and low-resource languages are especially hard. The work matters because it gives a shared multilingual resource for culturally aware moderation and polarization research.

Key points

  • The benchmark defines three tasks:
    • POLARDETECT decides whether a text is polarized (binary).
    • POLARTYPE assigns the social dimension: political/ideological, racial/ethnic, religious, gender/sexual, or other.
    • POLARMANIFEST identifies the rhetorical strategy: stereotype, vilification, dehumanization, extreme language/absolutism, lack of empathy, or invalidation.
  • In reverse alphabetical order by code, the 22 languages are zho, urd, tur, tel, swa, spa, rus, pol, pan, ori, nep, mya, khm, ita, hin, hau, fas, eng, deu, ben, arb, and amh. They mix high-, medium-, and low-resource languages.
  • Some languages reuse and relabel instances from existing hate or toxic speech corpora, including ToxiCN, COLD, HaSpeede2, HODI, BAN-PL, and Turkish, Myanmar, and Bangla hate speech sets.
  • In error analysis, models tended to label text as polarized only when two opposing groups were named explicitly. They missed hostility that targeted one group or relied on implied opposition.
  • The data is released on Hugging Face (POLAR-SemEval2026) and served as SemEval-2026 Task 9, which drew over 1000 participants.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Alexandro Garrido Veliz, P Sam Sahil, Yiran Zhang, Idris Abdulmumin, Marco Antonio Stranisci, Özge Alacam, Cengiz Acarturk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Kritesh Rauniyar, Tanmoy Chakraborty, MD Arfeen Zeeshan, Dheeraj Kodati, Satya Keerthi, Sahar Moradizeyveh, Firoj Alam, Md Arid Hasan, Syed Ishtiaque Ahmed, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Diana Awuor Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Jane Wanjiru Kimani, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad and Seid Muhie Yimam. 2026. POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1433

@inproceedings{naseem2026polar,
  title = {{POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization}},
  author = {Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Alexandro Garrido Veliz, P Sam Sahil, Yiran Zhang, Idris Abdulmumin, Marco Antonio Stranisci, Özge Alacam, Cengiz Acarturk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Kritesh Rauniyar, Tanmoy Chakraborty, MD Arfeen Zeeshan, Dheeraj Kodati, Satya Keerthi, Sahar Moradizeyveh, Firoj Alam, Md Arid Hasan, Syed Ishtiaque Ahmed, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Diana Awuor Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Jane Wanjiru Kimani, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad and Seid Muhie Yimam},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1433},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1433}
}

References

40 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and Annie En-Shiun Lee. 2023. SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects. Preprint, arXiv:2309.07445.
  2. Dimosthenis Antypas and Jose Camacho-Collados. 2023. Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation. In The 7th Workshop on Online Abuse and Harms (WOAH), pages 231–242, Toronto, Canada. Association for Computational Linguistics. find
  3. Swapan Deep Arora, Guninder Pal Singh, Anirban Chakraborty, and Moutusy Maity. 2022. Polarization and Social Media: A Systematic Review and Research Agenda. Technological Forecasting and Social Change, 183:121942. find
  4. Lora Aroyo and Chris Welty. 2015. Truth is a lie: Crowd truth and the seven myths of human annotation. AI Magazine, 36(1):15–24.
  5. Zewen Bai, Liang Yang, Shengdi Yin, Junyu Lu, Jingjie Zeng, Haohao Zhu, Yuanyuan Sun, and Hongfei Lin. 2025. STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10206… find
  6. Juan M. Banda, Ramya Tekumalla, Guanyu Wang, Jingyuan Yu, Tuo Liu, Yuning Ding, Ekaterina Artemova, Elena Tutubalina, and Gerardo Chowell. 2021. A large-scale COVID-19 Twitter chatter dataset for open scientific research—an international collaboration. Epidemiologia, 2(3):315–324. find
  7. Pablo Barberá. 2020. Social Media and Democracy: The State of the Field, Prospects for Reform, chapter Social Media, Echo Chambers, and Political Polarization. Cambridge University Press Cambridge. find
  8. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter. In Proceedings of the 13th International Wor…
  9. Federico Cabitza, Andrea Campagner, and Valerio Basile. 2023. Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37-6, pages 6860–6868, Washington, DC, USA.
  10. Fernando Casal Bértoa and José Rama. 2021. Polarization: What Do We Know and What Can We Do About It? Frontiers in Political Science, 3:1–11. find
  11. Hyung Won Chung, Thibault Fevry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021. Rethinking Embedding Coupling in Pre-trained Language Models. In International Conference on Learning Representations, pages 1–17, Online.
  12. Çağrı Çöltekin. 2020. A Corpus of Turkish Offensive Language on Social Media. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6174–6184, Marseille, France. European Language Resources Association.
  13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  14. Dorottya Demszky, Nikhil Garg, Rob Voigt, James Zou, Jesse Shapiro, Matthew Gentzkow, and Dan Jurafsky. 2019. Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Comp… find
  15. Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, and Minlie Huang. 2022. COLD: A Benchmark for Chinese Offensive Language Detection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11580–11599, Abu Dhabi, United Arab Emirates. Ass… find
  16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  17. William Donohue and Mark Hamilton. 2022. A Framework for Understanding Polarizing Language. In The Routledge Handbook of Language and Persuasion, pages 207–223. Routledge. find
  18. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT Sentence Embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891, Dublin, Ireland.
  19. Kiran Garimella. 2018. Polarization on Social Media. Ph.D. thesis, Aalto University, Finland. find
  20. Todd Gitlin. 2016. The Outrage Industry: Political Opinion Media and the New Incivility By Jeffrey M. Berry and Sarah Sobieraj Oxford University Press. Social Forces, 95(1):e26–e26. find
  21. Md Arid Hasan, Firoj Alam, Md Fahad Hossain, Usman Naseem, and Syed Ishtiaque Ahmed. 2025. LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Target. arXiv preprint arXiv:2510.01995. find
  22. Valentin Hofmann, Xiaowen Dong, Janet Pierrehumbert, and Hinrich Schuetze. 2022. Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 536–550, Seattle,… find
  23. Luca Iandoli, Simonetta Primario, and Giuseppe Zollo. 2021. The impact of group polarization on the quality of online debate in social media: A systematic literature review. Technological Forecasting and Social Change, 170:1–12. find
  24. Anna Kolos, Inez Okulska, Kinga Gł abińska, Agnieszka Karlinska, Emilia Wisnios, Paweł Ellerik, and Andrzej Prałat. 2024. BAN-PL: A Polish Dataset of Banned Harmful and Offensive Content from Wykop.pl Web Service. In Proceedings of the 2024 Joint International Conference on Computational Linguisti… find
  25. Emily Kubin and Christian von Sikorski. 2021. The Role of (Social) Media in Political Polarization: A Systematic Review. Annals of the International Communication Association, 45(3):188–206. find
  26. Nang Aeindray Kyaw, Ye Kyaw Thu, Thazin Myint Oo, Hutchatai Chanlekha, Manabu Okumura, and Thepchai Supnithi. 2024. Enhancing Hate Speech Classification in Myanmar Language through Lexicon-Based Filtering. In 2024 21st International Joint Conference on Computer Science and Software Engineering (JCSS… find
  27. Raquel Martínez-España, Julio Fernández-Pedauye, José Giner-Pérez de Lucía, Jose Miguel Rojo-Martínez, Kaoutar Bakdid-Albane, and Juan José García-Escribano. 2024. Methodology for Measuring Individual Affective Polarization Using Sentiment Analysis in Social Networks. IEEE Access. find
  28. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection. In Proceedings of the AAAI Conference on Artificial Intelligence 2021, volume 35, pages 14867–14875, online. find
  29. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alaçam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakr… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  30. Debora Nozza, Andrea Teresa Cignarella, Greta Damo, Tommaso Caselli, and Viviana Patti. 2023. HODI at EVALITA 2023: Overview of the first Shared Task on Homotransphobia Detection in Italian. In Proceedings of the EVALITA 2023 Evaluation Campaign, pages 1–8, Parma, Italy. find
  31. Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2020. Misogyny Detection in Twitter: a Multilingual and Cross-Domain Study. Information Processing & Management, 57(6):102360. find
  32. Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, and David Jurgens. 2022. POTATO: The Portable Text Annotation Tool. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 327– 3…
  33. James A. Piazza. 2023. Political polarization and political violence. Security Studies, 32(3):476–504. find
  34. José Miguel Rojo Martínez. 2025. Unravelling Radicalisation: Exploring Concepts, Contexts, and Perspectives, chapter Affective polarisation, Dehumanisation of the Adversary and Political Violence. Springer. find
  35. Manuela Sanguinetti, Gloria Comandini, Elisa di Nuovo, Simona Frenda, Marco Stranisci, Cristina Bosco, Tomaso Caselli, Viviana Patti, and Irene Russo. 2020. HaSpeeDe 2 @ EVALITA2020: Overview of the EVALITA 2020 Hate Speech Detection Task. In Proceedings of the 7th Evaluation Campaign of Natural Lan… find
  36. Almog Simchon, William J Brady, and Jay J Van Bavel. 2022. Troll and divide: the language of online polarization. PNAS nexus, 1(1):pgac019. find
  37. Barea Sinno, Bernardo Oviedo, Katherine Atwell, Malihe Alikhani, and Junyi Jessy Li. 2022. Political Ideology and Polarization: A Multi-dimensional Approach. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  38. Felipe Bonow Soares and Raquel Recuero. 2021. Hashtag Wars: Political Disinformation and Discursive Struggles on Twitter Conversations During the 2018 Brazilian Presidential Campaign. Social Media+ Society, 7(2):1– 13. find
  39. Jan Strich, Adeline Scharfenberg, Chris Biemann, and Martin Semmann. 2025. EncouRAGe: Evaluating RAG Local, Fast, and Reliable. Preprint, arXiv:2511.04696. — [arXiv:2511.04696](https://arxiv.org/abs/2511.04696) find
  40. Isaac Waller and Ashton Anderson. 2021. Quantifying social organization and political polarization in online platforms. Nature, 600(7887):264–268. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/naseem-2026-polar-benchmark-multilingual-multicultural-multi/