Elamur Research⌕
☰
home / papers / 2026 / NIT-Agartala-NLP-Team at SemEval-2026 Task 9: A Weighted Soft-Voting Ensemble Framework of Fine-Tuned LLMs for Binary and Multi-Label Polarization Detection

NIT-Agartala-NLP-Team at SemEval-2026 Task 9: A Weighted Soft-Voting Ensemble Framework of Fine-Tuned LLMs for Binary and Multi-Label Polarization Detection

Paper · 2026 · Shivam, Manish Kumar and Anupam Jamatia

Summary

The paper describes a system for SemEval-2026 Task 9, which targets polarization in online text. The task has two parts: deciding whether a post is polarized, and labeling polarized posts with one or more types (Political, Racial/Ethnic, Religious, Gender/Sexual, Others). The task corpus covers 22 languages, but the authors used only the English portion, which is small and heavily imbalanced. They first built a TF-IDF Logistic Regression baseline with L2 regularization. They then fine-tuned four base-size transformers with sigmoid output heads and binary cross-entropy loss: BERT, RoBERTa, XLM-RoBERTa and GPT-2. The final system averages the models' predicted probabilities, with weights set from each model's validation performance, and then tunes per-class decision thresholds to improve macro-F1. On the official test set, the ensemble placed in the middle of the leaderboard for both subtasks. Its main value is as a simple, reproducible ensemble recipe. The authors also show that class imbalance, rather than model choice alone, limits how well such systems generalize.

Key points

  • Data: English training split has 2047 non-polarized vs. 1175 polarized posts, and the dev split has 101 vs. 59. Among polarized training posts, Political labels (1150) far outnumber Gender/Sexual (72) and Religious (112).
  • Official test results: macro-F1 of 78.6 on Subtask 1 (26th of 44 teams) and 46.0 on Subtask 2 (18th of 29 teams).
  • Subtask 1 validation: the ensemble reached 81.66 macro-F1, compared with 76.02 for the best single model (GPT-2 base) and 72.07 for Logistic Regression.
  • Subtask 2 validation: the ensemble's gain was marginal, at 46.71 vs. 46.20 for RoBERTa base.
  • Errors and limitations: XLM-RoBERTa base collapsed to predicting mostly the non-polarized class (38.70 macro-F1). Scores dropped from validation to test, which the authors attribute to class and category imbalance. Further constraints were English-only data and base-size models only, the latter due to limited hardware.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: National Institute of Technology Agartala.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Shivam, Manish Kumar and Anupam Jamatia. 2026. NIT-Agartala-NLP-Team at SemEval-2026 Task 9: A Weighted Soft-Voting Ensemble Framework of Fine-Tuned LLMs for Binary and Multi-Label Polarization Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.398

@inproceedings{shivam2026nit,
  title = {{NIT-Agartala-NLP-Team at SemEval-2026 Task 9: A Weighted Soft-Voting Ensemble Framework of Fine-Tuned LLMs for Binary and Multi-Label Polarization Detection}},
  author = {Shivam, Manish Kumar and Anupam Jamatia},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.398},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.398}
}

References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mari Ashiga, Wei Jie, Fan Wu, Vardan Voskanyan, Fateme Dinmohammadi, Paul Brookes, Jingzhi Gong, and Zheng Wang. 2025. Ensemble learning for large language models in text and code generation: A survey. CoRR, abs/2503.13505. — [arXiv:2503.13505](https://arxiv.org/abs/2503.13505) find
  2. Imon Banerjee, Yuan Ling, Matthew C Chen, Sadid A Hasan, Curtis P Langlotz, Nathaniel Moradzadeh, Brian Chapman, Timothy Amrhein, David Mong, Daniel L Rubin, and 1 others. 2019. Comparative effectiveness of convolutional neural network (cnn) and recurrent neural network (rnn) architectures for radio… find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  4. Sorouralsadat Fatemi, Yuheng Hu, and Maryam Mousavi. 2025. A comparative analysis of instruction fine-tuning large language models for financial text classification. ACM Trans. Manage. Inf. Syst., 16(1). find
  5. Shervin Minaee, Nal Kalchbrenner, Erik Cambria, Narjes Nikzad, Meysam Chenaghlu, and Jianfeng Gao. 2021. Deep learning–based text classification: A comprehensive review. ACM Comput. Surv., 54(3). find
  6. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, "Ozge Alacam, Cengiz Acar"urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chak… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  7. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Ky… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  8. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn:… find
  9. Fabrizio Sebastiani. 2002. Machine learning in automated text categorization. ACM Computing Surveys, 34(1):1–47.
  10. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transf… find
  11. [0,0,0,0,0] [0,0,0,0,0] [0,0,0,0,0] [0,0,0,0,0] [0,0,0,0,0] *Table 9: Correct and incorrect predictions by the proposed ensemble model on Subtask 2 (Multi-label Polarization Type Classification).* find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shivam-2026-nit-agartala-nlp-team-semeval/