Elamur Research⌕
☰
home / papers / 2026 / ProBel: Propaganda Detection with Techniques, Spans, and Explanations

ProBel: Propaganda Detection with Techniques, Spans, and Explanations

Paper · 2026 · Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Elisa Sartori et al.

Summary

Propaganda analysis is usually split into separate tasks. One task asks whether a sentence is propagandistic, another names the technique, and a third locates the responsible text. These tasks normally live in different datasets, languages and evaluation setups, so it is hard to see how supervision at one level affects another. The authors build ProBel, an Arabic–English resource where every news sentence or social media post carries four aligned annotations: a binary label, multi-label tags from 23 techniques grouped into six coarse categories, technique-labeled character spans, and a reference explanation. It extends PropXplain and greatly enlarges the English portion. Using a shared setup, they compare zero-shot prompting, in-context learning, task-specific fine-tuning, joint multi-task training and distillation. One bilingual multi-task model (MT-SFT) gives the best overall balance. Transfer between tasks depends on the supervision level: span-only training hurts sentence-level detection. Distillation does not replace direct fine-tuning. The work shows that a single deployable model can handle detection, localisation and explanation together.

Key points

  • Arabic splits are kept as in PropXplain (18,453/1,318/1,326). The English splits grow from 4,472/621/922 to 18,775/2,567/3,993, built from 347 articles across 42 news sources (97 newly added).
  • On the same test splits, MT-SFT (five tasks, two languages) reaches binary macro-F1 0.763 for Arabic and 0.685 for English. The fine-tuned Llama-3.1-8B from PropXplain scores 0.760 and 0.675.
  • On Arabic technique-labeled span detection (overlap-adjusted micro-F1), zero-shot GPT-4 scores 0.117. Task-specific SFT reaches 0.421 and MT-SFT 0.411.
  • In the cross-task study, span-only training weakens binary detection, while classification-only training cannot recover span boundaries. Joint bilingual training is the most stable setup, and monolingual fine-tuning reduces transfer to the other language.
  • Distillation results:
    • Off-policy distillation from GPT-5 traces lifts the zero-shot binary score from 0.561 to 0.697.
    • On-policy self-distillation (OPSD) gains little.
    • Neither approach reaches MT-SFT (0.749).
    • Replacing hard KL clipping with a saturating clip improves OPSD, with similar effects on HateXplain and GSM8K.

Interesting ideas

  • I1. Annotate each instance at several aligned levels (yes/no decision, category, evidence span, explanation) so that every positive decision is backed by located evidence.
  • I2. The reference explanations are produced from the gold labels and gold spans, not written freely, so the explanation stays grounded in the annotated structure.
  • I3. Train on one supervision level and measure every other level, to expose where joint training helps and where it causes negative transfer.
  • I4. Distillation loss clipped with a saturating function instead of a hard cutoff keeps gradients on the few decisive tokens where teacher and student disagree most.
  • I5. Spans are scored with overlap-adjusted micro-F1 and stored with character boundaries plus a label, so partial matches get graded credit.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): irony, implicature and speech-act decisions in prag could each carry the token span that triggers them, making every pragmatic label checkable against the text.
  • I2 (learn): an LLM teacher tier could be fed the gold UD tree or frame and asked only to verbalise it, which yields grounded rationales without inventing analysis.
  • I3 (learn): gparse or the tagger could be trained with and without auxiliary tasks (morphology, frames), and each run evaluated across all stages to catch hidden degradation.
  • I4 (learn): when distilling the transformer parser into a smaller model, the rare high-KL arcs (attachment ambiguities) would still drive learning instead of being silenced.
  • I5 (world): entity and event mentions in the text world could be evaluated the same way, giving credit for near-correct mention boundaries rather than all-or-nothing matches.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PaduaPadua · Italy7 works · 4,173 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Social Sciences · 744 works Sociology and Political Science · 221 works

Cite

Cite as Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Elisa Sartori, Giovanni Da San Martino and Firoj Alam. 2026. ProBel: Propaganda Detection with Techniques, Spans, and Explanations. arXiv:2608.22388.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2608.22388 · arXiv 2608.22388

@misc{kmainasi2026probel,
  title = {{ProBel: Propaganda Detection with Techniques, Spans, and Explanations}},
  author = {Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Elisa Sartori, Giovanni {Da San Martino} and Firoj Alam},
  year = {2026},
  doi = {10.48550/arxiv.2608.22388},
  eprint = {2608.22388},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2608.22388}
}

References

35 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. 2024. On-policy distillation of language models: Learning from self-generated mistakes. In The Twelfth International Conference on Learning Representations (ICLR). ↗ our card
  2. Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020. Generating fact checking explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352–7364, Online. Association for Computational Linguistics. ↗ our card
  3. Alberto Barrón-Cedeño, Giovanni Da San Martino, Israa Jaradat, and Preslav Nakov. 2019. Proppy: A system to unmask propaganda in online news. In Proceedings of the The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pages 9847–9848. AAAI Press. find
  4. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. In Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 9560–9572. ↗ our card
  5. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bangkok, Thailand. Association for Computational Linguistics. our card
  6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. ArXiv preprint, arXiv:2110.14168. ↗ our card
  7. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. SemEval-2020 task 11: Detection of propaganda techniques in news articles. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). International Committee for Computational Linguistics. ↗ our card
  8. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5636–5646, Hong Kong, China. Association for Computational Linguistics. find
  9. Dimitar Dimitrov, Firoj Alam, Maram Hasanain, Abul Hasnat, Fabrizio Silvestri, Preslav Nakov, and Giovanni Da San Martino. 2024. Semeval-2024 task 4: Multilingual detection of persuasion techniques in memes. In Proceedings of the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics. find
  10. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021. Task 6 at SemEval-2021: Detection of persuasion techniques in texts and images. In Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval ’21, Bangkok, Thailand. find
  11. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. ↗
  12. Maram Hasanain, Fatema Ahmad, and Firoj Alam. 2024. Can GPT-4 identify propaganda? annotation and detection of propaganda spans in news articles. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING ’24, pages 2724– 2744, Torino, Italia. ELRA and ICCL. find
  13. Maram Hasanain, Md Arid Hasan, Mohamed Bayan Kmainasi, Elisa Sartori, Ali Ezzat Shahroor, Giovanni Da San Martino, and Firoj Alam. 2025. PropXplain: Can LLMs enable explainable propaganda detection? In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23855– 23863, Suzhou, China. Association for Computational Linguistics. ↗
  14. Peter Hase and Mohit Bansal. 2021. When can models learn from explanations? a formal framework for understanding the roles of explanation data. ArXiv preprint, arXiv:2102.02201. ↗
  15. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada. ↗ our card
  16. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the 10th International Conference on Learning Representations, ICLR ’22. our card
  17. Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas Krause. 2026. Reinforcement learning via self-distillation. ArXiv preprint. ↗
  18. Simran Kaur, Narutatsu Ri, Yinghui He, Liam Fowl, and Sanjeev Arora. 2026. Rethinking on-policy self-distillation for thinking models. ArXiv preprint, arXiv:2607.05184. ↗
  19. Mikhail Lepekhin and Serge Sharoff. 2023. FTD at SemEval-2023 task 3: News genre and propaganda detection by comparing mono- and multilingual models with fine-tuning on additional data. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 549–555, Toronto, Canada. Association for Computational Linguistics. find
  20. Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 14867–14875. find
  21. Jakub Piskorski, Nicolas Stefanovitch, Giovanni Da San Martino, and Preslav Nakov. 2023. SemEval-2023 task 3: Detecting the category, the framing, and the persuasion techniques in online news in a multilingual setup. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2343–2361, Toronto, Canada. Association for Computational Linguistics. find
  22. Daniel Russo, Serra Sinem Tekiroğlu, and Marco Guerini. 2023. Benchmarking the generation of fact checking explanations. Transactions of the Association for Computational Linguistics, 11:1250–1264. ↗
  23. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. ArXiv preprint, arXiv:1707.06347. ↗
  24. Danil Semin, Ondřej Dušek, and Zdeněk Kasner. 2026. Strategies for span labeling with large language models. ArXiv preprint, arXiv:2601.16946. ↗
  25. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. 2025. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267. ↗ our card
  26. FANAR TEAM, Ummar Abbas, Mohammad Shahmeer Ahmad, Minhaj Ahmad, Abdulaziz Al-Homaid, Anas Al-Nuaimi, Enes Altinisik, Ehsaneddin Asgari, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Asim Ersoy, Masoomali Fatehkia, Mohammed Qusay Hashim, Majd Hawasly, Mohamed Hefeeda, Mus’ab Husaini, Keivin Isufaj, Soon-Gyo Jung, Houssam Lachemat, Ji Kim Lucas, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Mourad Ouzzani, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, and Yifan Zhang. 2026. Fanar 2.0: Arabic generative ai stack. ArXiv preprint, arXiv:2603.16397. ↗
  27. Gemma Team. 2026. Gemma 4 technical report. ArXiv preprint, arXiv:2607.02770. ↗
  28. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  29. Sarah Wiegreffe and Ana Marasović. 2021. Teach me to explain: A review of datasets for explainable natural language processing. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. ↗
  30. World Economic Forum. 2026. The global risks report 2026. Technical report, World Economic Forum. Published: 14 January 2026. find
  31. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 technical report. ArXiv preprint, arXiv:2505.09388. ↗ our card
  32. Shunyuan Yu, Giovanni Da San Martino, Mitra Mohtarami, James Glass, and Preslav Nakov. 2021. Interpretable propaganda detection in news articles. ArXiv preprint, arXiv:2108.12802. ↗
  33. Liudmila Zavolokina, Kilian Sprenkamp, Zoya Katashinskaya, Daniel Gordon Jones, and Gerhard Schwabe. 2024. Think fast, think slow, think critical: Designing an automated propaganda detection tool. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, Honolulu, HI, USA. Association for Computing Machinery. find
  34. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. ↗
  35. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. 2026. Self-distilled reasoner: On-policy self-distillation for large language models. In Forty-third International Conference on Machine Learning. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kmainasi-2026-probel-propaganda-detection-techniques-spans/