Elamur Research⌕
☰
home / papers / 2026 / Multi-Label Polarization Classification with twHIN-BERT and SCUT Threshold Optimization

Multi-Label Polarization Classification with twHIN-BERT and SCUT Threshold Optimization

Paper · 2026 · Ilinca Vandici, Ådne Jøssing and Lukas Viestädt

Summary

The ILA-POLAR team from Universität Tübingen describes its system for SemEval-2026 Task 9, Subtask 2. The subtask asks systems to tag social-media posts with any combination of five polarization types: political, racial/ethnic, religious, gender/sexual and other. The shared data covers 21 languages, and the authors trained on nine of them. The labels are highly skewed, especially the "other" class.

The team fully fine-tuned TwHIN-BERT-base, a multilingual encoder pre-trained on tweets. They then replaced the default 0.5 decision cutoff with thresholds tuned per language using the SCUT algorithm. Local validation splits were created per language with iterative stratified sampling, so that rare label combinations stay in training.

Reported scores are inconsistent within the paper:

  • The abstract and introduction give macro F1 of about 0.60 on dev and 0.58 on test.
  • The results section gives 0.66 macro and 0.60 micro on dev, and 0.65 and 0.59 on test.

SCUT barely changed macro F1, but it led the model to predict labels it had previously never output. An analysis of hidden representations suggests that languages forming compact clusters tend to score better. Overall, label distribution appeared to matter more than how well-resourced a language is.

Key points

  • Model and training: TwHIN-BERT-base with all parameters updated, trained for 5 epochs with learning rate 2e-5, AdamW and batch size 16. mBERT, Canine and LaBSE were also tried but converged more slowly.
  • Ranking: the system placed second for Farsi in the test phase, but in the lower half for English and Urdu.
  • "Other" category: this was the hardest label. Per-language F1 ranged from 0.764 for Urdu down to 0.0 for English, German and Russian, which shows no link to resource level.
  • Label imbalance: 69.5% of Urdu texts carry at least one polarization label, whereas 63.5% of English texts carry none.
  • Representation analysis:
    • Before fine-tuning, layer-9 embeddings cluster in linguistically plausible groups. Urdu, Hindi and Nepali group together, as do Spanish, German and English with Polish nearby. Russian and Farsi are outliers.
    • After fine-tuning, boundaries between languages blur.
    • A logistic-regression language probe on final-layer embeddings indicates that the better-scoring languages have more distinct representations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of TübingenTübingen · Germany75 works · 41,081 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Ilinca Vandici, Ådne Jøssing and Lukas Viestädt. 2026. Multi-Label Polarization Classification with twHIN-BERT and SCUT Threshold Optimization. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.356

@inproceedings{vandici2026multi,
  title = {{Multi-Label Polarization Classification with twHIN-BERT and SCUT Threshold Optimization}},
  author = {Ilinca Vandici, Ådne Jøssing and Lukas Viestädt},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.356},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.356}
}

References

16 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Astha Agrawal, Herna L Viktor, and Eric Paquet. 2015. Scut: Multi-class imbalanced data classification using smote and cluster-based undersampling. In 2015 7Th international joint conference on knowledge discovery, knowledge engineering and knowledge management (IC3k), volume 1, pages 226–234. IEEE. find
  2. Jonathan H Clark, Dan Garrette, Iulia Turc, and John Wieting. 2022. Canine: Pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics, 10:73–91.
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  4. William Donohue and Mark Hamilton. 2022. A framework for understanding polarizing language. In The Routledge handbook of language and persuasion, pages 207–223. Routledge. 20th International Workshop on Semantic Evaluation (SemEval-2026). Association for Computational Linguistics. find
  5. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Ky… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  6. Piotr Szymański and Tomasz Kajdanowicz. 2017. A scikit-based python environment for performing multi-label classification. arXiv preprint arXiv:1702.01460. — [arXiv:1702.01460](https://arxiv.org/abs/1702.01460) find
  7. Marc Tanti, Lonneke van der Plas, Claudia Borg, and Albert Gatt. 2021. On the language-specificity of multilingual bert and the impact of fine-tuning. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 214–227. find
  8. Xinyang Zhang, Yury Malkov, Omar Florez, Serim Park, Brian McWilliams, Jiawei Han, and Ahmed El-Kishky. 2023. Twhin-bert: A socially-enriched pre-trained language model for multilingual tweet representations at twitter. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data… find
  9. Haytame Fallah, Patrice Bellot, Emmanuel Bruno, and Elisabeth Murisasco. 2022. Adapting transformers for multi-label text classification. In CIRCLE (Joint Conference of the Information Retrieval Communities in Europe) 2022. find
  10. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic bert sentence embedding. In Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: Long papers), pages 878–891.
  11. Yova Kementchedjhieva and Ilias Chalkidis. 2023. An exploration of encoder-decoder approaches to multi-label classification for legal and biomedical text. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5828–5843. find
  12. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, "Ozge Alacam, Cengiz Acar"urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chak… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  13. Political 0.6243 0.6612 0.6947 0.8210 0.0800 0.6154 0.3077 0.2692 0.8102 0.5871 Racial/Ethnic 0.1026 0.1333 0.0000 0.0000 0.6316 0.0000 0.1176 0.3958 0.5668 0.2164 Religious 0.1905 0.3333 0.0000 0.8619 0.0000 0.0000 0.0000 0.4507 0.6699 0.2785 find
  14. Other 0.2222 0.0000 0.4444 0.0000 0.0000 0.0000 0.0000 0.0833 0.6545 0.1560 *Table 1: Results from Baseline (Frozen Model + Classification Heads)* Language deu eng fas hin nep pol rus spa urd Avg Macro F1 0.4545 0.2734 0.6523 0.7379 0.7629 0.4646 0.4372 0.5339 0.7655 0.5647 Micro F1 0.5540 0.5466 0.… find
  15. Political Racial/Ethnic Religious Gender/Sexual Other Pol R/E Rel G/S Oth Language 2395 1916 1908 1816 1808 1916 1938 1814 1779 1772 1908 1814 1969 1775 1768 1816 1779 1775 1825 1763 1808 1772 1768 1763 1808 *Table 5: Label co-occurrences for the Urdu training set. Abbreviations: Pol (Political), R/… find
  16. Support deu eng fas hin nep pol rus spa urd 0.96 0.63 0.94 0.86 0.80 1.00 0.91 0.57 0.82 0.84 0.91 0.94 0.44 0.60 0.42 0.91 0.88 0.92 0.90 0.74 0.94 0.59 0.69 0.59 0.91 0.69 0.87 32 32 33 27 20 24 33 33 36 Accuracy Macro Average Weighted Average 0.83 0.83 0.79 0.76 0.79 0.77 0.78 270 270 270 *Table… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vandici-2026-multi-label-polarization-classification-twhin/