Elamur Research⌕
☰
home / papers / 2026 / IReL_IIT(BHU) at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization

IReL_IIT(BHU) at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization

Paper · 2026 · Soumadip Majumder, Arjun Mukherjee, Krishna Tewari et al.

Summary

This system paper reports the IReL_IIT(BHU) entry for the Chinese track of SemEval-2026 Task 9, which covers online polarization. The team entered all three subtasks: binary detection of polarized text, multi-label classification of polarization type, and multi-label identification of how polarization manifests. They argue that earlier systems handled these tasks with separate pipelines, used fixed thresholds, and ensembled in an ad hoc way. Their alternative is one shared workflow. Chinese encoders (Erlangshen-DeBERTa and MacBERT) are fine-tuned with stratified 5-fold cross-validation. The out-of-fold (OOF) probabilities then serve two purposes: calibrating decision thresholds and choosing ensemble weights. Preprocessing on the official shared-task data was kept deliberately light. A single DeBERTa model performed best on the binary task, while weighted DeBERTa-heavy ensembles helped on the multi-label subtasks. Official test Macro-F1 was 0.9081, 0.7962 and 0.6484. The authors conclude that careful calibration and validation design matter. They also note that manifestation labels, which depend on implicit, context-dependent cues, remain hard to predict.

Key points

  • Backbones: IDEA-CCNL/Erlangshen-DeBERTa-v2-320M-Chinese was the strongest single model, and hfl/chinese-macbert-large added complementary signal. For Subtask 1, mDeBERTa-v3 and XLM-R were also tried.
  • Final configurations:
    • Subtask 1: a single DeBERTa model with an OOF-tuned threshold slightly under 0.5.
    • Subtask 2: a 0.6/0.4 DeBERTa/MacBERT ensemble over 5 labels.
    • Subtask 3: a 0.9/0.1 ensemble over 6 labels.
    • Both multi-label subtasks used per-label thresholds and BCEWithLogitsLoss with per-label pos_weight.
  • Training: AdamW with a learning rate of about 1.5e-5, FP16, early stopping with patience 2, and seed 42, run on an RTX 4060 laptop GPU. Maximum sequence length was 128, 192 and 256 for Subtasks 1–3.
  • Test results (Macro-F1 / rank):
    • Subtask 1: 0.9081 (10th)
    • Subtask 2: 0.7962 (8th)
    • Subtask 3: 0.6484 (7th)
  • Generalization gaps:
    • Subtask 3 fell from 0.7136 in the development phase to 0.6484 on test.
    • Per-label ensemble weighting raised OOF scores but overfit.
    • The hardest labels were "other" among polarization types and "lack of empathy" among manifestations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Technology BHUVaranasi · India10 works · 548 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksSocial Sciences · 1,954 worksSociology and Political Science · 626 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 882 works

Cite

Cite as Soumadip Majumder, Arjun Mukherjee, Krishna Tewari, Sanjaya Lenka and Sukomal Pal. 2026. IReL_IIT(BHU) at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.297

@inproceedings{majumder2026irel,
  title = {{IReL\_IIT(BHU) at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization}},
  author = {Soumadip Majumder, Arjun Mukherjee, Krishna Tewari, Sanjaya Lenka and Sukomal Pal},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.297},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.297}
}

References

18 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marco Antonio Stranisci, Idris Abdulmumin, "Ozge Alacam, Cengiz Acar"urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakraborty, and 15 others. 2026a. SemEval-2026 task 9: Detecting multilingual, multicultural and multieven…
  2. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 657–668, Online.
  3. Saeedeh Davoudi and Nazli Goharian. 2026. Online polarization detection in Persian (Farsi) social media. In The Proceedings of the First Workshop on NLP and LLMs for the Iranian Language Family, pages 50– 59, Rabat, Morocco. Association for Computational Linguistics. find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  5. Thomas G. Dietterich. 2000. Ensemble methods in machine learning. In International Workshop on Multiple Classifier Systems (MCS), pages 1–15. Springer. find
  6. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume…
  7. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In Proceedings of ICML. find
  8. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. Preprint, arXiv:2006.03654.
  9. Ron Kohavi. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI). find
  10. Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42:318–327.
  11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692.
  12. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwa… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  13. Usman Naseem, Juan Ren, Saba Anwar, Sarah Kohail, Rudy Alexandro Garrido Veliz, Robert Geislinger, Aisha Jabr, Idris Abdulmumin, Laiba Qureshi, Aarushi Ajay Borkar, Maryam Ibrahim Mukhtar, Abinew Ali Ayele, Ibrahim Said Ahmad, Adem Ali, Martin Semmann, Shamsuddeen Hassan Muhammad, and Seid Muhie Yim… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  14. Alexandru Niculescu-Mizil and Rich Caruana. 2005. Predicting good probabilities with supervised learning. In Proceedings of the 22nd International Conference on Machine Learning (ICML), pages 625–632. find
  15. Nakwon Rim, Joshua Conrad Jackson, Marc G Berman, and Yuan Chang Leong. 2026. Natural language reveals that political partisans are more affectively aligned over political issues than partisan identities. Communications Psychology, 4(1):65. find
  16. Pavlos Sermpezis, Stelios Karamanidis, Eva Paraschou, Ilias Dimitriadis, Sofia Yfantidou, Filitsa-Ioanna Kouskouveli, Thanasis Troboukis, Kelly Kiki, Antonis Galanopoulos, and Athena Vakali. 2026. Ago-raspeech: a multi-annotated comprehensive dataset of political discourse through the lens of humans… find
  17. Jiaxing Zhang, Ruyi Gan, Junjie Wang, Yuxiang Zhang, Lin Zhang, Ping Yang, Xinyu Gao, Ziwei Wu, Xiaoqun Dong, Junqing He, Jianheng Zhuo, Qi Yang, Yongfeng Huang, Xiayu Li, Yanghan Wu, Junyu Lu, Xinyu Zhu, Weifeng Chen, Ting Han, and 6 others. 2022. Fengshenbang 1.0: Being the foundation of chinese c… — [arXiv:2209.02970](https://arxiv.org/abs/2209.02970) find
  18. Min-Ling Zhang and Zhi-Hua Zhou. 2014. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 26(8):1819– 1837. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/majumder-2026-irel-iit-bhu-semeval-2026/