Elamur Research⌕
☰
home / papers / 2026 / PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation

PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation

Paper · 2026 · Srikar Kashyap Pulipaka

Summary

This system paper tackles Subtask 1 of SemEval-2026 Task 9. The task asks whether a social media post in one of 22 languages is polarized or not, using the POLAR dataset and macro-F1. Polarization here covers stereotyping, vilification, dehumanization and intolerance.

The author trains a separate classifier per language by adapting Gemma 3 (12B and 27B) with LoRA and a classification head. The training sets are small (about 1,700–7,000 examples) and often imbalanced. Each one is enlarged with roughly 1,000 GPT-4o-mini samples, created by native generation, paraphrasing and contrastive minimal pairs, then filtered for label leakage and near-duplicates. Decision thresholds are tuned per language. For each language, the development set picks one of four options: either model alone, a plain average of both, or a weighted mix.

The final system reaches 0.811 mean macro-F1, second of 60 teams. The work matters as evidence that robustness across splits counts more than development scores. XLM-RoBERTa and Qwen3 looked competitive in development but collapsed on test. A topic coverage gap in POLAR's Italian data explains that language's weak score.

Key points

  • Leaderboard: 0.811 versus 0.818 for the winner. The system placed first for Amharic, Hindi and Swahili, top-3 in 8 languages and top-10 in 17.
  • Thresholds and ensembles:
    • Searching thresholds from 0.3 to 0.7 adds 2–4% F1 with no retraining.
    • The best values span 0.3 (Nepali) to 0.7 (Khmer, Amharic, Bengali), reflecting miscalibrated probabilities.
    • Ensembles win 14 of 22 languages. Khmer gains the most (+8.7%).
  • Synthetic data:
    • A 30% synthetic ratio works best (dev F1 0.822 vs. 0.812 without augmentation).
    • Khmer loses 6.4% at a 50% ratio.
    • Deduplication uses paraphrase-multilingual-MiniLM-L12-v2 with a 0.90 cosine cutoff.
  • Models and generalization:
    • Gemma 3 12B ran in 8-bit on one A100 with LoRA rank 16; Gemma 3 27B ran in 4-bit NF4 on two A100s.
    • A hybrid XLM-R/Qwen3 submission scored only 0.665. XLM-R fell from 0.856 dev F1 to 0.297 test F1 on Bengali.
  • Italian: It scored worst (0.563). The political and "other" topics never appear in train or dev, yet they make up 41% of the test set and about 87% of its positives.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Srikar Kashyap Pulipaka. 2026. PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.200

@inproceedings{pulipaka2026psk,
  title = {{PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation}},
  author = {Srikar Kashyap Pulipaka},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.200},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.200}
}

References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jan Cegin, Jakub Simko, and Peter Brusilovsky. 2025. LLMs vs established text augmentation techniques for classification: When do the benefits outweigh the costs? In Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  2. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized language models. In Advances in Neural Information Processing Systems, volume 36. find
  3. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500, Brussels, Belgium. Association for Computational Linguistics. find
  4. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the Tenth International Conference on Learning Representations (ICLR).
  5. Zachary C. Lipton, Charles Elkan, and Balakrishnan Narayanaswamy. 2014. Thresholding classifiers to maximize F1 score. Computing Research Repository, arXiv:1402.1892. — [arXiv:1402.1892](https://arxiv.org/abs/1402.1892) find
  6. Shrirang Mhalgi, Srikar Kashyap Pulipaka, and Sandra Kübler. 2024. IUCL at PAN 2024: Using data augmentation for conspiracy theory detection. In Working Notes of CLEF 2024 – Conference and Labs of the Evaluation Forum, Grenoble, France. find
  7. Aida Mostafazadeh Davani, Ali Omrani, Brendan Kennedy, Mohammad Atari, Xiang Ren, and Morteza Dehghani. 2021. Improving counterfactual generation for fair hate speech detection. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 92–101, Online. Association for Computatio… find
  8. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakr… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  9. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Ky… — POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
  10. Ignazio Pillai, Giorgio Fumera, and Fabio Roli. 2013. Threshold optimisation for multi-label classifiers. Pattern Recognition, 46(7):2055–2065. find
  11. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  12. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany. Association for Computational L… find
  13. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Sl… find
  14. Zheng Xin Yong, Cristina Menghini, and Stephen H. Bach. 2024. LexC-Gen: Generating data for extremely low-resource languages with large language models and bilingual lexicons. Computing Research Repository, arXiv:2402.14086. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pulipaka-2026-psk-semeval-2026-task-9/