Elamur Research⌕
☰
home / papers / 2026 / mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection

mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection

Paper · 2026 · Dominik Macko, Alok Debnath and Jakub Simko

Summary

This paper describes a team's entry to SemEval-2026 Task 9 (POLAR), a shared task on spotting online polarization in 22 languages. The task has three subtasks: binary detection, polarization type, and rhetorical manifestation. The authors adapted mdok, their earlier machine-generated-text detector that won both PAN@CLEF2025 subtasks, so that it handles multi-label classification. They trained one model on all languages pooled together, using QLoRA with 4-bit quantization. For the binary subtask they enlarged the training data with altered copies of texts: anonymized, lowercased, uppercased, and homoglyph-substituted. The goal was a detector that responds to meaning rather than surface form.

A single shared model proved competitive in the binary subtask but struggled with the multi-label subtasks. Manifestation identification was the weakest. As an unsubmitted side study, the authors also tested emotion-appraisal features as polarization signals and found encouraging per-label AUC values. The work shows that one pooled multilingual model can be a practical baseline across many languages, including low-resource ones.

Key points

  • Base models and training: Qwen3-32B was used for subtask 1 and Gemma-3-27B-pt for subtasks 2–3, since multi-label finetuning of Qwen3 caused problems. Training ran for one epoch with a constant learning rate of 2e-5 and batch size 1. Checkpoints were chosen by AUC for subtask 1 and macro F1 for subtasks 2–3.
  • Data augmentation (subtask 1 only): The four augmentation techniques together added about 20% more training data (roughly 5% each). Train and dev sets were merged and deduplicated. A balanced validation set of 4,400 samples was built (100 per label per language).
  • Subtask 1 results: Macro F1 ranged from about 0.63 (Khmer) to 0.92 (Chinese).
  • Ranking against other teams: Out of 62 language–subtask combinations, the system was in the top 20% in 14 cases, the top 50% in 28, and the bottom 20% in 15.
    • It placed 1st for Italian in subtask 1.
    • Its best placements elsewhere were 3rd for Nepali in subtask 2 and 4th for Urdu in subtask 3.
  • Appraisal exploration:
    • An AppraisePLM-style regressor with an XLM-RoBERTa encoder was trained on enISEAR/deISEAR.
    • It annotated only the training set, which was split 80/20 for evaluation.
    • Per-language LogisticRegression classifiers were then trained on these appraisal annotations.
    • Code is available at github.com/kinit-sk/mdok-style-polar2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Trinity College DublinDublin · Ireland24 works · 1,231 citations🤝Kempelen Institute of Intelligent TechnologiesBratislava · Slovakia11 works · 82 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Dominik Macko, Alok Debnath and Jakub Simko. 2026. mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.46

@inproceedings{macko2026mdok,
  title = {{mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection}},
  author = {Dominik Macko, Alok Debnath and Jakub Simko},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.46},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.46}
}

References

17 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Janek Bevendorff, Yuxia Wang, Jussi Karlgren, Matti Wiegmann, Maik Fröbe, Akim Tsivgun, Jinyan Su, Zhuohan Xie, Mervat T. Abassy, Jonibek Mansurov, Rui Xing, Minh Ngoc Ta, Kareem Ashraf Elozeiri, Tianle Gu, Raj Vardhan Tomar, Jiahui Geng, Ekaterina Artemova, Artem Shelmanov, Nizar Habash, and 5 othe… find
  2. Lucio La Cava, Dominik Macko, Róbert Móro, Ivan Srba, and Andrea Tagarelli. 2025. Authorship attribution in multilingual machine-generated texts. Preprint, arXiv:2508.01656. find
  3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  4. Alok Debnath, Yvette Graham, and Owen Conlan. 2025. An appraisal theoretic approach to modelling affect flow in conversation corpora. In Proceedings of the 29th Conference on Computational Natural Language Learning, pages 233–250, Vienna, Austria. Association for Computational Linguistics. find
  5. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient fine-tuning of quantized LLMs. In Advances in Neural Information Processing Systems, volume 36, pages 10088–10115. Curran Associates, Inc. find
  6. Jan Hofmann, Enrica Troiano, Kai Sassenberg, and Roman Klinger. 2020. Appraisal theories for emotion classification in text. In Proceedings of the 28th International Conference on Computational Linguistics, pages 125–138, Barcelona, Spain (Online). International Committee on Computational Linguistic… find
  7. Dominik Macko. 2025. mdok of KInIT: Robustly fine-tuned LLM for binary and multiclass AI-generated text detection. In Working Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, CEUR-WS.org. find
  8. Dominik Macko, Robert Moro, and Ivan Srba. 2025. Increasing the robustness of the fine-tuned multilingual machine-generated text detectors. Preprint, arXiv:2503.15128. — [arXiv:2503.15128](https://arxiv.org/abs/2503.15128) find
  9. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, "Ozge Alacam, Cengiz Acar"urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chak… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  10. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Ky… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  11. Klaus R Scherer and Harald G Wallbott. 1994. Evidence for universality and cultural variation of differential emotion response patterning. Journal of personality and social psychology, 66(2):310. find
  12. Michal Spiegel and Dominik Macko. 2024a. IMGTB: A framework for machine-generated text detection benchmarking. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 172–179, Bangkok, Thailand. Association for Computational… find
  13. Michal Spiegel and Dominik Macko. 2024b. KInIT at SemEval-2024 task 8: Fine-tuned LLMs for multilingual machine-generated text detection. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 558–564, Mexico City, Mexico. Association for Computational Linguis… find
  14. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  15. Enrica Troiano, Sebastian Padó, and Roman Klinger. 2019a. Crowdsourcing and validating event-focused emotion corpora for German and English. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4005– 4011, Florence, Italy. Association for Computational Li… find
  16. Enrica Troiano, Sebastian Padó, and Roman Klinger. 2019b. Crowdsourcing and validating event-focused emotion corpora for german and english. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4005–4011. find
  17. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/macko-2026-mdok-style-semeval-2026-task/