Elamur Research⌕
☰
home / papers / 2026 / UPR at SemEval-2026 Task 9: Multi-Label Polarization and Manifestation Detection in Urdu

UPR at SemEval-2026 Task 9: Multi-Label Polarization and Manifestation Detection in Urdu

Paper · 2026 · Mtayyaba Shahzad, Inzmam Khadam, Zaufishan Mahmood et al.

Summary

This paper presents the University of Poonch Rawalakot (UPR) system for the Urdu track of SemEval-2026 Task 9, which targets polarization in social media posts. The authors treat two subtasks as multi-label classification. Subtask-2 assigns posts to five social dimensions: political, religious, racial/ethnic, gender/sexual and other. Subtask-3 tags six forms of polarized expression: stereotyping groups, vilifying them, denying their humanity, using extreme wording, showing no empathy, and invalidating others.

They fine-tune XLM-RoBERTa with a sigmoid head and Binary Cross-Entropy loss. Before training, they clean the Urdu text with Unicode normalization and remove non-Urdu content. They drop near-duplicates using TF-IDF cosine similarity. To counter class imbalance, they use dictionary-based synonym replacement and oversampling.

On the development sets, XLM-RoBERTa reaches Macro-F1 of 0.75 and 0.72, outperforming Linear SVM, XGBoost, DistilBERT and mBERT. Test-set scores are not given. Error analysis points to overlapping categories as the main weakness, such as political vs. religious or stereotype vs. vilification. The work provides a reproducible reference point for polarization detection in low-resource, Nastaliq-script Urdu.

Key points

  • Data: official SemEval-2026 Task 9 Urdu corpora with 3,561 training posts for Subtask-2 and 3,563 for Subtask-3. In Subtask-3, 2,409 posts carry several manifestation labels and 1,087 carry none.
  • Preprocessing:
    • Texts under 3 tokens are filtered out.
    • Unigram+bigram TF-IDF pairs with cosine similarity above 0.95 are treated as near-duplicates and removed.
    • Augmentation replaces at most two words per sentence (p = 0.1) using a hand-built Urdu synonym list.
    • Oversampling targets a 1:1 balance between label-free and labelled posts.
  • Training setup: AdamW, learning rate 2e-5, batch size 16, up to 10 epochs, max length 128 tokens, a fixed 0.5 decision threshold and seed 42.
  • Validation Macro-F1:
ModelSubtask-2Subtask-3
XLM-RoBERTa0.750.72
mBERT0.680.70
Linear SVM0.680.70
XGBoost0.670.71
DistilBERT0.660.68
  • Limitations named by the authors: a small dataset, possible annotation noise, no per-label threshold tuning, and no use of discourse structure or external knowledge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sejong UniversitySeoul · South Korea2 works · 1 citation🎓University of Poonch RawalakotRawlakot · Pakistan2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Mtayyaba Shahzad, Inzmam Khadam, Zaufishan Mahmood, Junaid Rashid, Shamaila Hayat and Fakhar Ayub. 2026. UPR at SemEval-2026 Task 9: Multi-Label Polarization and Manifestation Detection in Urdu. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.333

@inproceedings{shahzad2026upr,
  title = {{UPR at SemEval-2026 Task 9: Multi-Label Polarization and Manifestation Detection in Urdu}},
  author = {Mtayyaba Shahzad, Inzmam Khadam, Zaufishan Mahmood, Junaid Rashid, Shamaila Hayat and Fakhar Ayub},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.333},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.333}
}

References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Waqar Ashiq, Samra Kanwal, Adnan Rafique, Muhammad Waqas, Tahir Khurshaid, Elizabeth Caro Montero, Alicia Bustamante Alonso, and Imran Ashraf. 2024. Roman urdu hate speech detection using hybrid machine learning models and hyperparameter optimization. Scientific Reports, 14(1):28590. find
  2. Oleg Rogov, Aung Kyaw Htet, and 24 others. 2026b. Polar: A benchmark for multilingual, multicultural, and multi-event online polarization. Preprint, arXiv:2505.20624.
  3. Muhammad Bilal, Atif Khan, Salman Jan, Shahrulniza Musa, and Shaukat Ali. 2023. Roman urdu hate speech detection using transformer-based model for cyber security applications. Sensors, 23(8):3909. find
  4. Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021. Resources and benchmark corpora for hate speech detection: a systematic review. Language Resources and Evaluation, 55(2):477–523. find
  5. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794. find
  6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In ACL. find
  7. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20(3):273–297.
  8. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media, volume 11, pages 512–515.
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL.
  10. Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. 2021. A survey of data augmentation approaches for nlp. In Findings of the association for computational linguistics: ACL-IJCNLP 2021, pages 968–988.
  11. Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. Acm Computing Surveys (Csur), 51(4):1–30. find
  12. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, "Ozge Alacam, Cengiz Acar"urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chak… — SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
  13. Muhammad Owais Raza, Qaisar Khan, and Ghulam Muhammad Soomro. 2021. Urdu abusive language detection using machine learning. In FIRE (Working Notes), pages 774–783. find
  14. Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. 2009. Classifier chains for multi-label classification. In Joint European conference on machine learning and knowledge discovery in databases, pages 254–269. Springer. find
  15. Sebastian Ruder, Matthew E Peters, Swabha Swayamdipta, and Thomas Wolf. 2019. Transfer learning in natural language processing. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Tutorials, pages 15–18.
  16. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. find
  17. Grigorios Tsoumakas and Ioannis Katakis. 2007. Multi-label classification: An overview. International Journal of Data Warehousing and Mining (IJDWM), 3(3):1–13. find
  18. Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021. Learning from the worst: Dynamically generated datasets to improve online hate detection. In Proceedings of the 59th annual meeting of the Association for Computational Linguistics and the 11th international joint conference on nat…
  19. Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP…
  20. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acartürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shahzad-2026-upr-semeval-2026-task-9/