Elamur Research⌕
☰
home / papers / 2026 / transformer_1376 at SemEval-2026 Task 9: A Multi-Stage Pipeline with Calibrated Ensembles and Lexical Post-Processing for Online Polarization Detection in Bengali

transformer_1376 at SemEval-2026 Task 9: A Multi-Stage Pipeline with Calibrated Ensembles and Lexical Post-Processing for Online Polarization Detection in Bengali

Paper · 2026 · Shuvodwip Saha and Pritha Saha

Summary

The paper describes a system for Subtask 1 of POLAR @ SemEval-2026 Task 9, a binary task that labels Bengali social media comments as polarized or not. The authors used the POLAR dataset's Bengali split: 3,333 training, 166 development and 1,501 test comments, with a mild majority of non-polarized examples. They fine-tuned BanglaBERT Large with class-weighted cross-entropy and ran stratified 5-fold cross-validation. The fold models were combined by weighting each one by its squared validation macro F1. The combined probabilities were then calibrated with temperature scaling (T = 1.3), and the decision threshold was tuned on the development set. A final rule layer, driven by a hand-built 13-term Bengali hostility lexicon, adjusts some predictions. The best submission reached a macro F1 of 0.8463, placing 5th of 36 teams. It outperformed multilingual encoders but stayed slightly below the official LaBSE baseline. The ablation shows that class weighting gave most of the improvement. Ensembling and calibration slightly lowered test scores, which suggests overfitting to the development set. The work offers a reproducible reference pipeline for polarization detection in a low-resource language.

Key points

  • Test macro F1 by model:
    • bert-base-uncased: 0.7407
    • google/muril-base-cased: 0.8345
    • XLM-Roberta-large: 0.8355
    • this system: 0.8463
    • official LaBSE baseline: 0.8528
  • The ablation pipeline reproduced 0.8425. Ensembling plus threshold tuning scored ΔF1 = −0.0029 against a single-fold model, while the lexicon rules added +0.0008.
  • The three post-processing rules are:
    • an override to Polarized when the calibrated probability exceeds 0.85;
    • a flip to Polarized when the probability is below 0.65 and any lexicon term is present;
    • a borderline rule within ±0.05 of the threshold, based on whether at least two lexicon terms appear.
  • The confusion matrix shows 742 true negatives, 530 true positives, 126 false positives and 103 false negatives; ROC AUC is 0.9143.
  • False positives tend to be insults aimed at individuals, such as a single politician. False negatives tend to be implicit identity-based othering, for example questioning someone's gender or religion without overtly hostile words.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chittagong University of Engineering & TechnologyChittagong · Bangladesh25 works · 29 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Shuvodwip Saha and Pritha Saha. 2026. transformer_1376 at SemEval-2026 Task 9: A Multi-Stage Pipeline with Calibrated Ensembles and Lexical Post-Processing for Online Polarization Detection in Bengali. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.262

@inproceedings{saha2026transformer,
  title = {{transformer\_1376 at SemEval-2026 Task 9: A Multi-Stage Pipeline with Calibrated Ensembles and Lexical Post-Processing for Online Polarization Detection in Bengali}},
  author = {Shuvodwip Saha and Pritha Saha},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.262},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.262}
}

References

12 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adem Chanie Ali, Seid Muhie Yimam, Abinew Ali Ayele, Chris Biemann, and Martin Semmann. 2025. Silenced voices: social media polarization and women’s marginalization in peacebuilding during the northern ethiopia war. i-com, 24(2):407–432. find
  2. Guy Banim. 2025. Very large online platforms—how big is your polarization footprint? towards a metric to give eu citizens transparency around an online systemic risk driving conflict in our societies. Report, Build Up. find
  3. Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, and Rifat Shahriyar. 2022. BanglaBERT: Language model pretraining and benchmarks for low-resource language understanding evaluation in Bangla. In Findings of the Association for Comp…
  4. R. Kelly Garrett. 2009. Echo chambers online?: Politically motivated selective exposure among internet news users1. Journal of Computer-Mediated Communication, 14(2):265–285. find
  5. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1321–1330. PMLR. find
  6. Valentin Hofmann, Xiaowen Dong, Janet Pierrehumbert, and Hinrich Schuetze. 2022. Modeling ideological salience and framing in polarized online groups with graph neural networks and structured sparsity. find
  7. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 536–550, Seattle, United States. Association for Computational Linguistics. find
  8. Daniel Miehling, Daniel Dakota, and Sandra Kübler. 2025. Investigating polarization in YouTube comments via aspect-based sentiment analysis. In Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era, p… find
  9. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acarürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakra… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  10. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Ky… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  11. Donghee Yvette Wohn and Brian J Bowe. 2016. Micro agenda setters: The effect of social media on young adults’ exposure to and attitude toward news. Social Media + Society, 2(1):2056305115626750. find
  12. A *Figure 3: Receiver Operating Characteristic (ROC) Curve for the final ensembled model on the test set, demonstrating an AUC of 0.9143.* C Mathematical Formulation of Ensemble Calibration find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saha-2026-transformer-1376-semeval-2026-task/