Elamur Research⌕
☰
home / papers / 2026 / KinyaProp: Fine-Grained Propaganda Annotation in Kinyarwanda

KinyaProp: Fine-Grained Propaganda Annotation in Kinyarwanda

Paper · 2026 · Manzi Fabrice Niyigaba, Ivory Yang and Soroush Vosoughi

Summary

Most work on identifying specific propaganda techniques in text covers only a few well-resourced languages such as English, Arabic and Chinese. It is therefore unclear whether reports of LLMs annotating as well as humans also hold for low-resource, culturally distinct languages. The authors built KinyaProp, a Kinyarwanda news dataset. It contains 608 articles from the Rwandan news corpus of Niyongabo et al. (2020), labeled at span level with the 18-technique scheme of Da San Martino et al. (2019). Three trained native-speaker students annotated it over about three months. A majority-vote gold subset serves as the benchmark.

The authors fine-tuned XLM-R-large and AfroXLM-R-large as baselines. They also tested GPT-4o, GPT-4.1 and Claude Sonnet 4.5 with zero-shot prompting, in-context learning and chain-of-thought, plus supervised fine-tuning for GPT-4o. The LLMs behaved as cautious, surface-oriented annotators. They missed much of the propaganda, scored very low and inconsistently on discourse-level techniques, and flagged emotionally colored but neutral words. This limits how far LLM-annotation successes in high-resource languages carry over, and it argues for hybrid human–LLM annotation pipelines.

Key points

  • KinyaProp is described as the first fine-grained propaganda resource for Kinyarwanda and for any Bantu language. Both the raw annotations and the gold subset are released at github.com/Manzi-Fabrice/KinyaProp.
  • Inter-annotator agreement (γ metric) is γ_s = 0.544 for spans and γ_sl = 0.548 for spans plus labels. This exceeds the original Da San Martino et al. figures (0.34/0.31). Disagreement comes mostly from span boundaries rather than technique labels.
  • The gold subset was built by clustering spans at IoU ≥ 0.5 and keeping only spans with agreement from at least two annotators:
    • 1,774 spans across 582 articles;
    • 526 articles (86.51%) contain propaganda and 56 are non-propaganda;
    • 26 articles were excluded for lack of majority agreement.
  • The label distribution is long-tailed. Appeal to Authority has 717 spans, Loaded Language 417 and Repetitions 256, while Whataboutism has just 1.
  • For Loaded Language, inflected forms of kwica ("to kill") alone produced 64, 54 and 44 false positives for GPT-4o, GPT-4.1 and Claude. The six most frequent false-positive root groups totaled 139, 107 and 90 respectively.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Manzi Fabrice Niyigaba, Ivory Yang and Soroush Vosoughi. 2026. KinyaProp: Fine-Grained Propaganda Annotation in Kinyarwanda. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1580

@inproceedings{niyigaba2026kinyaprop,
  title = {{KinyaProp: Fine-Grained Propaganda Annotation in Kinyarwanda}},
  author = {Manzi Fabrice Niyigaba, Ivory Yang and Soroush Vosoughi},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1580},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1580}
}

References

24 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, and Dietrich Klakow. 2022. Adapting pre-trained language models to African languages via multilingual adaptive fine-tuning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4336–4349, Gyeongju, Republic…
  2. Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, and Preslav Nakov. 2022a. A survey on multimodal disinformation detection. In Proceedings of the 29th International Conference on Computational Linguistics, page… find
  3. Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, Giovanni Da San Martino, and Preslav Nakov. 2022b. Overview of the WANLP 2022 shared task on propaganda detection in Arabic. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 108–118, Abu Dhabi, United Arab Emirates (…
  4. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 2020 Conference on Empirical Methods in N… find
  5. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. SemEval-2020 task 11: Detection of propaganda techniques in news articles. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). Internationa…
  6. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019. Fine-grained analysis of propaganda in news articles. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natura…
  7. Maram Hasanain, Fatema Ahmad, and Firoj Alam. 2024a. Can GPT-4 identify propaganda? annotation and detection of propaganda spans in news articles. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2724–… find
  8. Maram Hasanain, Fatema Ahmad, and Firoj Alam. 2024b. Large language models for propaganda span annotation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14522– 14532, Miami, Florida, USA. Association for Computational Linguistics. find
  9. Yuanyuan Lei and Ruihong Huang. 2023. Discourse structures guided fine-grained propaganda identification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 331–342, Singapore. Association for Computational Linguistics. find
  10. Yann Mathet, Antoine Widlöcher, and Jean-Philippe Métivier. 2015. The unified and holistic method gamma (γ) for inter-annotator agreement measure and alignment. Computational Linguistics, 41(3):437–479. 467–472, Bangkok, Thailand. Association for Computational Linguistics. find
  11. Jay Mohta, Kenan Ak, Yan Xu, and Mingwei Shen. 2023. Are large language models good annotators? In Proceedings on "I Can’t Believe It’s Not Better: Failure Modes in the Age of Foundation Models" at NeurIPS 2023 Workshops, volume 239 of Proceedings of Machine Learning Research, pages 38–48. PMLR. find
  12. Shivam Sharma, Firoj Alam, Md. Shad Akhtar, Dimitar Dimitrov, Giovanni Da San Martino, Hamed Firooz, Alon Halevy, Fabrizio Silvestri, Preslav Nakov, and Tanmoy Chakraborty. 2022. Detecting and understanding harmful memes: A survey. In Proceedings of the Thirty-First International Joint Conference on… find
  13. Rubungo Andre Niyongabo, Qu Hong, Julia Kreutzer, and Li Huang. 2020. KINNEWS and KIRNEWS: Benchmarking cross-lingual text classification for Kinyarwanda and Kirundi. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5507–5521, Barcelona, Spain (Online). Interna… find
  14. Antoine Nzeyimana. 2020. Morphological disambiguation from stemming data. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4649–4660, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  15. Antoine Nzeyimana. 2024. Low-resource neural machine translation with morphological modeling. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 182–195, Mexico City, Mexico. Association for Computational Linguistics. find
  16. Antoine Nzeyimana and Andre Niyongabo Rubungo. 2022. KinyaBERT: a morphology-aware Kinyarwanda language model. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5347–5363, Dublin, Ireland. Association for Computational Linguisti… find
  17. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931–2937, Copenhagen, Denmark. As…
  18. Lin Shen. 2023. Culture and explicitness of persuasion: Linguistic evidence from a 51-year corpus-based cross-cultural comparison of the united nations general debate speeches across 55 countries (1970-2020). Cross-Cultural Research, 57(2-3):166–192. find
  19. Meng-Hsien Shih, Ren-feng Duann, and Siaw-Fong Chung. 2020. The analysis and annotation of propaganda techniques in Chinese news texts. In Proceedings of the 32nd Conference on Computational Linguistics and Speech Processing (ROCLING 2020), pages 331–345, Taipei, Taiwan. The Association for Computat… find
  20. Jason Stanley. 2015. How Propaganda Works. Princeton University Press. find
  21. Prashanth Vijayaraghavan and Soroush Vosoughi. 2022. TWEETSPIN: Fine-grained propaganda detection in social media using multi-view representations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages… find
  22. Ruize Wang, Duyu Tang, Nan Duan, Wanjun Zhong, Zhongyu Wei, Xuanjing Huang, Daxin Jiang, and Ming Zhou. 2020. Leveraging declarative knowledge in text and first-order logic for fine-grained propaganda detection. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processin… find
  23. Ariana Sahitaj, Premtim Sahitaj, Veronika Solopova, Jiaao Li, Sebastian Möller, and Vera Schmitt. 2025. Hybrid annotation for propaganda detection: Integrating LLM pre-annotations with human intelligence. In Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI), pages 215–228, Vienn… find
  24. Uzair Shah, Md. Rafiul Biswas, Marco Agus, Mowafa Househ, and Wajdi Zaghouani. 2024. MemeMind at ArAIEval shared task: Generative augmentation and feature fusion for multimodal propaganda detection in Arabic memes through advanced language and vision models. In Proceedings of the Second Arabic Natur…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/niyigaba-2026-kinyaprop-fine-grained-propaganda-annotation/