Elamur Research⌕
☰
home / papers / 2026 / NAMAA at SemEval-2026 Task 9: Comparing Generative, Retrieval-Augmented, and Discriminative Methods for Arabic Online Polarization Detection and Type Classification

NAMAA at SemEval-2026 Task 9: Comparing Generative, Retrieval-Augmented, and Discriminative Methods for Arabic Online Polarization Detection and Type Classification

Paper · 2026 · Abdelbasset Djamai, Sahara Al-Madi, Norah Al-Zaid et al.

Summary

The paper addresses Arabic online polarization detection in the SemEval-2026 Task 9 (POLAR) Arabic track. The task covers binary detection of "us vs. them" content (ST1) and multi-label classification into five polarization types (ST2). Dialect mixing, informal spelling and implicit framing make the task hard. The authors compare three families of methods on the POLAR Arabic split (3,380 train, 169 dev, 1,521 test):

  • fine-tuning six Arabic BERT-style encoders, with and without LLM-generated augmentation;
  • zero-shot prompting of several open LLMs;
  • retrieval-augmented in-context learning (RAG-ICL), where nearest labeled training examples serve as demonstrations.

For ST1, RAG-ICL with Gemma-3-27b-it scores best on test, but the best fine-tuned encoder comes very close at much lower cost. For ST2, the strongest system is a cascade: the ST1 encoder screens out non-polarized posts, and only the remaining posts go to RAG-ICL for type labeling. This sharply cuts spurious labels. Chain-of-thought, self-refinement and contrastive prompting give no improvement. The study offers a practical benchmark of cost versus accuracy for low-resource, dialect-heavy classification.

Key points

  • Six encoders were compared: AraBERT v02, AraBERT v02-Twitter, MARBERT v2, QARiB, CAMeLBERT-Mix and AraModernBERT. The best was AraBERT v02-Twitter with augmentation (dev macro F1 0.8516, test 0.8249). The effect of augmentation varied by model.
  • On ST1 test, RAG-ICL (Gemma-3-27b-it, English prompt, k=7) reached 0.8314 macro F1, while zero-shot with the same model reached only 0.5221.
  • On ST2, gating with the ST1 encoder lowered the zero-shot hallucination rate on non-polarized inputs from 0.61 to 0.17. Filtered RAG-ICL reached 0.6304 on dev and 0.6196 on test, against 0.5292 for the officially submitted unfiltered zero-shot run.
  • None of the advanced prompting variants beat standard filtered RAG-ICL on dev (0.6304):
    • CoT: 0.6196;
    • contrastive: 0.6062;
    • self-refinement: 0.5692.
  • Augmentation used DeepSeek V3.2. Whether English or Arabic prompts worked better depended on the model and the task. RAG-ICL took about 2.2 s per sample, compared with about 0.2 s for zero-shot.

Interesting ideas

  • I1. The augmentation creates "hard negatives": the LLM rewrites a positive example so that it keeps its surface form and tone but loses the target property. This forces the classifier to stop relying on vocabulary cues.
  • I2. Before generating data, a sample of the corpus is first condensed into a reusable style guide (dialect, tone, conventions), and that guide then conditions every generation call. This keeps synthetic data closer to the real distribution.
  • I3. A cheap binary gate decides whether the expensive fine-grained labeler runs at all; inputs that fail the gate get an empty label set, which removes invented labels on negative inputs.
  • I4. All type labels are predicted jointly in one decision rather than once per category, so the label combination stays internally consistent. The paper also tracks a hallucination rate on negative inputs as its own metric.
  • I5. The error audit found that the model fires on surface keywords: routine mentions of God trigger the Religious label, while posts without obvious keywords are missed. This exposes reliance on lexical cues over pragmatic intent.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver data for frames or irony could include minimally edited counterexamples that keep the wording but flip the role or intent, and these could double as negative controls in evaluation.
  • I2 (learn): An LLM teacher tier could first extract a genre or register profile of the target texts (fables, school tests) and condition all synthetic sentences on it.
  • I3 (frames): The irony and implicature modules could run only after a lightweight trigger detector fires, and literal sentences would get an explicit "no implicature" result instead of a forced guess.
  • I4 (frames): Semantic roles for one predicate could be assigned jointly with a consistency check, and the error rate on sentences that should yield no frame could be reported separately.
  • I5 (answer): Test sets for the readers could include keyword-bait items, where a trigger word appears but the meaning does not support it, to check that answers come from the meaning graph rather than from word matching.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Arab International UniversityGhabāghib · Syria13 works · 46 citations🎓Ain Shams UniversityCairo · Egypt12 works · 109 citations🎓King Khalid UniversityAbhā · Saudi Arabia4 works · 16 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksInformation retrieval & search · 473 works

Cite

Cite as Abdelbasset Djamai, Sahara Al-Madi, Norah Al-Zaid, Khloud Al Jallad and Mona Azim. 2026. NAMAA at SemEval-2026 Task 9: Comparing Generative, Retrieval-Augmented, and Discriminative Methods for Arabic Online Polarization Detection and Type Classification. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.439

@inproceedings{djamai2026namaa,
  title = {{NAMAA at SemEval-2026 Task 9: Comparing Generative, Retrieval-Augmented, and Discriminative Methods for Arabic Online Polarization Detection and Type Classification}},
  author = {Abdelbasset Djamai, Sahara Al-Madi, Norah Al-Zaid, Khloud {Al Jallad} and Mona Azim},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.439},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.439}
}

References

26 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish, and Younes Samih. 2021. Pre-training bert on arabic tweets: Practical considerations. Preprint, arXiv:2102.10684. ↗
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105. tional Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 10476– 10496. ↗
  3. Ibrahim Abu Farha, Silviu Vlad Oprea, Steven Wilson, and Walid Magdy. 2022. Semeval-2022 task 6: isarcasmeval, intended sarcasm detection in English and Arabic. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 802–814, Seattle, United States. Association for Computational Linguistics.
  4. Matteo Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. 2021. The echo chamber effect on social media. Proceedings of the National Academy of Sciences, 118(9):e2023301118. ↗
  5. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the WANLP 2021 shared task on sarcasm and sentiment detection in Arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 296–305, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  6. Nora Alturayeif, Hamzah Luqman, Zaid Alyafeai, and Asma Yamani. 2024. StanceEval 2024: The First Arabic Stance Detection Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 774–782, Bangkok, Thailand. Association for Computational Linguistics.
  7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9– 15. ↗
  8. Ahmet Okan Arık, Gizem Parlayandemir, and Serra Çelik. 2026. Llm-based data augmentation for text classification on imbalanced datasets: A case study on fake news detection. Egyptian Informatics Journal, 33:100886. find
  9. Pablo Barberá. 2015. Birds of the same feather tweet together: Bayesian ideal point estimation using twitter data. Political Analysis, 23(1):76– 91. ↗
  10. Jan Cegin, Jakub Simko, and Peter Brusilovsky. 2025. Llms vs established text augmentation techniques for classification: When do the benefits outweight the costs? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computa-DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025. Deepseek-v3.2: Pushing the frontier of open large language models. Preprint, arXiv:2512.02556. ↗
  11. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. ↗
  12. Omar Elshehy, Omer Nacar, Abdelbasset Djamai, Muhammed Ragab, Khloud Al Jallad, and Mona Abdelazim. 2026. AraModernBERT: Transtokenized initialization and long-context encoder modeling for Arabic. In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script, pages 313–321, Rabat, Morocco. Association for Computational Linguistics. ↗
  13. Injy Hamed, Caroline Sabty, Slim Abdennadher, Ngoc Thang Vu, Thamar Solorio, and Nizar Habash. 2025. A survey of code-switched Arabic NLP: Progress, challenges, and future directions. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4561–4585, Abu Dhabi, UAE. Association for Computational Linguistics. ↗
  14. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92–104.
  15. Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, and Philip Resnik. 2014. Political ideology detection using recursive neural networks. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1113–1122.
  16. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41.
  17. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2020. Overview of arabic offensive language and hate speech detection. arXiv preprint arXiv:2005.01845. ↗
  18. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, ”Ozge Alacam, Cengiz Acart”urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Kyaw Htet, and 24 others. 2026a. Polar: A benchmark for multilingual, multicultural, and multi-event online polarization. Preprint, arXiv:2505.20624. ↗
  19. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, ”Ozge Alacam, Cengiz Acart”urk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakraborty, and 15 others. 2026b. SemEval-2026 task 9: Detecting multilingual, multicultural and multievent online polarization. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). Association for Computational Linguistics.
  20. Manaranjan Pradhan, Naga Vemprala, and Naveen Gudigantala. 2026. Beyond zero-shot: Enhancing llm financial complaint classification with relevancy-driven rag-based few-shot prompting. find
  21. Juhi Singh, Ziqiao Ao, and Sebastian Antinome. 2025. Optimizing prompt refinement: Algorithmic strategies for llm-driven text classification tasks. Available at SSRN 5221492. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, and 23 others. 2025a. Fanar: An arabic-centric multimodal generative ai platform. Preprint, arXiv:2501.13944. ↗
  22. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, and 197 others. 2025b. Gemma 3 technical report. Preprint, arXiv:2503.19786. ↗
  23. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672. ↗
  24. Zhiqiang Wang, Yanbin Lin, Jiajun Shen, and Xingquan Zhu. 2025. A survey of large language models for text classification: What, why, when, where, and how. Authorea Preprints. ↗
  25. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. Wajdi Zaghouani, Hamdy Mubarak, and Md. ↗
  26. Rafiul Biswas. 2024. So hateful! building a multi-label hate speech annotated arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15044–15055, Torino, Italia. ELRA and ICCL.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/djamai-2026-namaa-semeval-2026-task-9/