Elamur Research⌕
☰
home / papers / 2026 / DiNO: Disinformation Narrative Observer

DiNO: Disinformation Narrative Observer

Paper · 2026 · Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska et al.

Summary

The paper asks how to recover the disinformation narratives that a specific news outlet spreads. Narratives here are broad recurring messages built from many related false claims. The authors propose DiNO, a four-stage pipeline:

  1. Split articles into candidate spans with a new segmentation scheme, DiNO-sgm, and match those spans to fact-checked false claims by semantic similarity.
  2. Use an LLM to check that the article actually endorses each matched claim rather than refuting it.
  3. Embed the confirmed segments, reduce their dimensionality and cluster them.
  4. Have an LLM condense each cluster into a short, neutral narrative statement.

DiNO was applied to coverage of the Ukraine War, COVID-19 and Migration from disinformation-prone outlets and from The Guardian. The extracted narratives were scored against a reference collection of known disinformation narratives. DiNO clearly beat the narrative-mining baselines Relatio and CaNarEx on both topic and stance agreement. It also found few narratives in the reputable outlet. This makes it a practical, auditable tool for monitoring disinformation at the outlet level.

Key points

  • Data: twelve datasets in total.
    • Nine contain about 130k news articles from Sputnik, Pravda, NaturalNews, Breitbart and The Guardian.
    • Three contain about 26k verified false claims, drawn from sources such as EUvsDisinfo and the Google Fact Check Tool API.
  • Main results: DiNO scored 0.75 on topical alignment, versus 0.52–0.53 for the baselines. On stance alignment it scored 0.85, versus 0.60–0.65. These correspond to gains of 41–44% and 30–41%.
  • Credible outlet: The Guardian produced only about two narratives per domain, and these had opposed or contextual stances.
  • Cost: a full run in the best configuration costs roughly $100 USD.
  • Release: code, prompts, annotations, extracted narratives and dataset materials are publicly available (GitHub wsosnowski/DiNO).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

Cite

Cite as Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska and Adam Wierzbicki. 2026. DiNO: Disinformation Narrative Observer. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2160

@inproceedings{sosnowski2026dino,
  title = {{DiNO: Disinformation Narrative Observer}},
  author = {Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska and Adam Wierzbicki},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2160},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2160}
}

References

102 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AAP FactCheck. 2024. Trump’s debunked dominion claim given another run. find
  2. Hervé Abdi and Lynne J. Williams. 2010. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2(4):433–459. find
  3. AFP Fact Check. 2021. COVID-19 vaccines do not contain magnetic microchips. find
  4. Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In Proceedings of the 10th international workshop on semantic evaluation.
  5. Nika Aleksejeva. 2023. Narrative warfare: How the Kremlin and Russian news outlets justified a war of aggression against Ukraine. Atlantic Council. find
  6. Nandini Anantharama, Simon Angus, and Lachlan O’Neill. 2022. Canarex: Contextually aware narrative extraction for semantically rich text-as-data applications. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3551– 3564. find
  7. Anthropic. 2025. Introducing Claude Sonnet 4.5. find
  8. Elliott Ash, Germain Gauthier, and Philine Widmer. 2024. Relatio: Text semantics capture political and economic narratives. Political Analysis, 32(1). find
  9. Michael Balsamo. 2020. Disputing trump, barr says no widespread election fraud. find
  10. Harry G. Barrow, Jay M. Tenenbaum, Robert C. Bolles, and Helen C. Wolf. 1977. Parametric correspondence and chamfer matching: Two new techniques for image matching. In Proceedings of the 5th International Joint Conference on Artificial Intelligence (IJCAI), pages 659–663. find
  11. Y Benkler, R Faris, and H Roberts. 2018. Immigration and islamophobia: Breitbart and the trump party. Network propaganda: Manipulation, disinformation, and radicalization in American politics. Oxford University Press. https://doi.org/10.1093/oso/9780190923624.003, 4. — [doi:10.1093/oso/9780190923624.003](https://doi.org/10.1093/oso/9780190923624.003) ↗
  12. Mack Blackburn, Ning Yu, John Berrie, Brian Gordon, David Longfellow, William Tirrell, and Mark Williams. 2020. Corpus development for studying online disinformation campaign: a narrative+ stance approach. In Proceedings for the First International Workshop on Social Threats in Online Conversations:… find
  13. Breitbart. 2025. Breitbart immigration. find
  14. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  15. CDC. 2024. Myths and facts about COVID-19 vaccines. find
  16. Tong Chen, Hongwei Wang, Sihao Chen, Wenhao Yu, Kaixin Ma, Xinran Zhao, Hongming Zhang, and Dong Yu. 2024. Dense x retrieval: What retrieval granularity should we use? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
  17. Eun Cheol Choi and Emilio Ferrara. 2024. Automated claim matching with large language models: empowering fact-checkers in the fight against misinformation. In Companion Proceedings of the ACM Web Conference 2024, pages 1441–1449. find
  18. EDMO. 2024. Edmo fact-checking network statement about methodology. find
  19. Mark Elsner, Grace Atkinson, Saadia Zahidi, and World Economic Forum. 2025. The global risks report 2025: 20th edition. Insight report, World Economic Forum, Cologny/Geneva, Switzerland. Published 15 January 2025. find
  20. Mercedes Esteban-Bravo, Jose M Vidal-Sanz, et al. 2024. Predicting the virality of fake news at the early stage of dissemination. Expert Systems with Applications, 248:123390. find
  21. Maria Hellman. 2024. Security, Disinformation and Harmful Narratives: RT and Sputnik News Coverage about Sweden. Palgrave Macmillan, Cham, Switzerland. find
  22. EUvsDisinfo. 2017. The pro-kremlin narrative about migrants. find
  23. Catalina Jaramillo. 2023. WHO ‘pandemic treaty’ draft reaffirms nations’ sovereignty to dictate health policy. find
  24. EUvsDisinfo. 2019. 5 common pro-kremlin disinformation narratives. find
  25. Myunghoon Kang and Greg Chih-Hsin Sheen. 2025. The making of the boy who cried wolf: fake news and media skepticism. Political Science Research and Methods, 13(2):465–474. find
  26. EUvsDisinfo. 2021. Ukrainians imbued with destructive nazi ideology. find
  27. EUvsDisinfo. 2022. Key narratives in pro-kremlin disinformation part 4: “The Imminent Collapse”. find
  28. EUvsDisinfo. 2024. Disinfo database. find
  29. Jesús M Fraile-Hernández, Anselmo Peńas, and Pablo Moral. 2024. Automatic identification of narratives: Evaluation framework, annotation methodology, and dataset creation. IEEE Access, 13:11734–11753. find
  30. Giuseppe Gallipoli and Luca Cagliero. 2025. It is not a piece of cake for gpt: Explaining textual entailment recognition in the presence of figurative language. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9656–9674.
  31. Pranav Goel, Jon Green, David Lazer, and Philip Resnik. 2023. Mainstream news articles co-shared with fake news buttress misinformation narratives. find
  32. Google. 2025a. Gemini 2.5 Pro model card. Model card. find
  33. Google. 2025b. Google fact check explorer. find
  34. Amanda Gray Meral. 2020. Learning the lessons from the EU–turkey deal: Europe’s renewed test. find
  35. Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794.
  36. Nuno Guimarães, Maria da Purificação Silvano, Ricardo Campos, AM Jorge, and Ana Filipa Pacheco. 2025. Narratex dataset: Explaining the dominant narratives in news texts. Findings of the Association for Computational Linguistics: EMNLP 2025. find
  37. Yang Liu. 2019. Fine-tune bert for extractive summarization. arXiv preprint arXiv:1903.10318. find
  38. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
  39. Jason Lucas, Adaku Uchendu, Michiharu Yamashita, Jooyoung Lee, Shaurya Rohatgi, and Dongwon Lee. 2023. Fighting fire with fire: The dual role of llms in crafting and detecting elusive disinformation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. find
  40. Rabeeh Karimi Mahabadi, Florian Mai, and James Henderson. 2019. Learning entailment-based sentence embeddings from natural language inference. find
  41. Lisa M. Martinez. Unraveling the giant ball of lies, myths, and disinformation about immigration. Center for Immigration Policy & Research (CIPR), University of Denver. find
  42. Leland McInnes, John Healy, and Steve Astels. 2017. hdbscan: Hierarchical density based clustering. Journal of Open Source Software, 2(11):205.
  43. Leland McInnes, John Healy, and James Melville. 2018. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.
  44. Media Bias/Fact Check. 2025. Natural news – bias and credibility. find
  45. Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. 2024. SFR-Embedding-2: Advanced text embedding with multi-stage training. Hugging Face model card. find
  46. Michael Hameleers. 2023. Disinformation as a context-bound phenomenon: toward a conceptual clarification integrating actors, intentions and techniques of creation and dissemination. Communication Theory. find
  47. Meta. 2024. Llama-3.3-70B-Instruct model card. Hugging Face. find
  48. John A Hartigan and Manchek A Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics), 28(1):100–108. find
  49. Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 404–411. find
  50. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. DeBERTa: Decoding-enhanced BERT with disentangled attention. arXiv preprint arXiv:2006.03654.
  51. Deborah Miori and Constantin Petrov. 2025. Narratives from gpt-derived networks of news and a link to financial markets dislocations. International Journal of Data Science and Analytics, 20(2):1105–1129. find
  52. Arkadiusz Modzelewski, Giovanni Da San Martino, Pavel Savov, Magdalena Anna Wilczyńska, and Adam Wierzbicki. 2024. Mipd: Exploring manipulation and intention in a novel corpus of polish disinformation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages… find
  53. Arkadiusz Modzelewski, Witold Sosnowski, Eleni Pa-padopulos, Elisa Sartori, Tiziano Labruna, Giovanni Da San Martino, and Adam Wierzbicki. 2026. Malicious intent dataset and inoculating llms for enhanced disinformation detection. In Proceedings of the 19th Conference of the European Chapter of the A…
  54. Pablo Moral, Jesús M Fraile, Guillermo Marco, Anselmo Peñas, and Julio Gonzalo. 2024. Overview of dipromats 2024: Detection, characterization and tracking of propaganda in messages from diplomats and authorities of world powers. Procesamiento del lenguaje natural, 73:347–358. find
  55. NaturalNews. 2025. Naturalnews: Covid-19 tag page. find
  56. Nic Newman, A Ross Arguedas, Craig T Robertson, Rasmus Kleis Nielsen, and Richard Fletcher. 2025. Digital news report 2025. Reuters Institute for the study of Journalism. find
  57. News Pravda. 2025. News pravda: Ukraine. find
  58. Nikolaos Nikolaidis, Nicolas Stefanovitch, Purificação Silvano, Dimitar Iliyanov Dimitrov, Roman Yangarber, Nuno Guimarães, Elisa Sartori, Ion Androutsopoulos, Preslav Nakov, Giovanni Da San Martino, et al. 2025. Polynarrative: A multilingual, multilabel, multi-domain dataset for narrative extractio… find
  59. Thomas A. O’Neill. 2017. An overview of interrater agreement on likert scales for researchers and practitioners. Frontiers in Psychology, 8:777. find
  60. Jakub Piskorski, Tarek Mahmoud, Nikolaos Nikolaidis, Ricardo Campos, Alipio Mario Jorge, Dimitar Dimitrov, Purificação Silvano, Roman Yangarber, Shivam Sharma, Tanmoy Chakraborty, et al. 2025. find
  61. Semeval 2025 task 10: Multilingual characterization and extraction of narratives from online news. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2610–2643. find
  62. PolitiFact. 2022. Russian has not been banned in ukraine, despite repeated claims. find
  63. PolitiFact. 2024. False claim resurfaces about WHO pandemic treaty and US sovereignty. find
  64. Maria Popp, Miriam Stegemann, Maria-Inti Metzen-dorf, Susan Gould, Peter Kranke, Patrick Mey-bohm, Nicole Skoetz, and Stephanie Weibel. 2021. Ivermectin for preventing and treating covid-19. Cochrane Database of Systematic Reviews, (7). find
  65. Justin Reedy, Benjamin Gonzalez O’Brien, and Elizabeth H Hurst. 2023. Pandemic politics: Immigration, framing, and covid-19. Journal of Race, Ethnicity, and Politics, 8(2):246–266. find
  66. Reuters. 2024. What is ISIS-K and why would it attack a Moscow concert hall? find
  67. Reuters Fact Check. 2024a. Posts wrongly claim 7,000 migrants arrived in lampedusa in 36 hours. find
  68. Reuters Fact Check. 2024b. Video shows boats arriving in spain in 2023, not britain in 2024. find
  69. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. find
  70. Michel Rose and John Irish. 2024. France’s Macron does not rule out Europeans sending troops to Ukraine. find
  71. Peter J. Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20:53–65. find
  72. Tim Sainburg, Leland McInnes, and Timothy Q. Gentner. 2021. Parametric UMAP embeddings for representation and semisupervised learning. Neural Computation, 33(11):2881–2907. find
  73. OpenAI. 2025b. GPT-5 model. find
  74. OpenAI. 2025c. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  75. Matúš Pikuliak, Ivan Srba, Robert Moro, Timo Hro-madka, Timotej Smoleň, Martin Melišek, Ivan Vykopal, Jakub Simko, Juraj Podroužek, and Maria Bielikova. 2023. Multilingual previously fact-checked claim retrieval. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process… find
  76. Rocío Sánchez del Vas and Jorge Tuñón Navarro. 2024. Disinformation on the covid-19 pandemic and the russia-ukraine war: Two sides of the same coin? Humanities and Social Sciences Communications, 11(1). find
  77. Fátima C. Carrilho Santos. 2023. Artificial intelligence in automated detection of disinformation: A thematic analysis. Journalism and Media, 4(2):679–687. find
  78. Shaden Shaar, Nikolay Babulkov, Giovanni Da San Martino, and Preslav Nakov. 2020. That is a known lie: Detecting previously fact-checked claims. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 3607–3618.
  79. Shaden Shaar, Nikola Georgiev, Firoj Alam, Giovanni Da San Martino, Aisha Mohamed, and Preslav Nakov. 2022. Assisting the human fact-checkers: Detecting all previously fact-checked claims in a document. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2069–2080. find
  80. Snopes. 2022. Zelenskyy swastika jersey pic is fake. find
  81. Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska, Jahna Otterbacher, and Adam Wierzbicki. 2024. Eu disinfotest: a benchmark for evaluating language models’ ability to detect disinformation narratives. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14702– 1472… find
  82. Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska, and Adam Wierzbicki. 2025. Dinam: Disinformation narrative mining with large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 30212–30239. find
  83. Saranac Hale Spencer. 2020. Sharpie ballots count in arizona. find
  84. Sputnik. 2025a. Sputnikglobe: Covid-19 topic page. find
  85. Sputnik. 2025b. Sputnikglobe search results for migration. find
  86. Sputnik. 2025c. Sputnikglobe search results for ukraine. find
  87. Kate Starbird, Ahmer Arif, and Tom Wilson. 2019. Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–26. find
  88. Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, and Han Xiao. 2024. jina-embeddings-v3: Multilingual embeddings with task lora. find
  89. Edson C Tandoc Jr. 2019. The facts of fake news: A research review. Sociology Compass, 13(9):e12724. find
  90. Kyilah Terry. 2021. The EU–turkey deal, five years on: A frayed and controversial but enduring blueprint. find
  91. The Guardian. 2025a. The guardian: Coronavirus outbreak. find
  92. The Guardian. 2025b. The guardian: Migration. find
  93. The Guardian. 2025c. The guardian: Ukraine. find
  94. United Nations. 2022. Black sea grain initiative joint coordination centre. find
  95. U.S. Customs and Border Protection. Southwest land border encounters. find
  96. VIGINUM. 2024. PORTAL KOMBAT: A structured and coordinated pro-Russian propaganda network. Technical report, Secrétariat général de la défense et de la sécurité nationale. find
  97. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv e-prints, pages arXiv–2212.
  98. World Health Organization. 2021. Who-convened global study of origins of SARS-CoV-2: China part. find
  99. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  100. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations. find
  101. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623.
  102. Liwen Zheng, Chaozhuo Li, Zheng Liu, Feiran Huang, Haoran Jia, Zaisheng Ye, and Xi Zhang. 2025. Fact in fragments: Deconstructing complex claims via llm-based atomic fact extraction and verification. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sosnowski-2026-dino-disinformation-narrative-observer/