Elamur Research⌕
☰
home / papers / 2026 / Population Replacement Conspiracy Theories Detection on Telegram and News Headlines: Benchmarking LLMs and BERT Models in Portuguese and Italian

Population Replacement Conspiracy Theories Detection on Telegram and News Headlines: Benchmarking LLMs and BERT Models in Portuguese and Italian

Paper · 2026 · Erik Bran Marino and Renata Vieira

Summary

The paper addresses automatic detection of Population Replacement Conspiracy Theories (PRCTs). These are narratives such as the Great Replacement, White Genocide or Kalergi Plan, which claim that immigration is an orchestrated demographic substitution. Detection resources so far have targeted mainly English, COVID-19 or generic conspiracies, so the authors build what they describe as the first systematic benchmark for Portuguese and Italian. They use two expert-annotated corpora: informal European Portuguese Telegram messages and formal Italian immigration headlines. The task is binary classification, examined in four phases:

  • zero-shot prompting of open-weights LLMs
  • single-domain fine-tuning (LoRA for LLMs, standard fine-tuning for language-specific encoders), tested in-domain and across domains
  • training on the merged corpus
  • a Pareto analysis balancing accuracy and throughput

DeepSeek-V3 is the strongest zero-shot model. DeepSeek-R1-14B fine-tuned on Telegram achieves the best overall scores, including through cross-domain transfer. Small encoders remain competitive for high-volume use. Because PRCTs are tied to extremist violence and polarization, the benchmark helps monitoring. However, the authors frame the cross-domain findings as tentative given the limited data.

Key points

  • Data:
    • Portuguese Telegram subset of Mute Cods: 919 train / 231 test, 15.7% PRCT, Krippendorff's α = 0.58.
    • Italian headlines from PartisanLens (collected via Media Cloud, 2020–2024): 434 train / 131 test, 28.5% PRCT, α = 0.774.
  • Models:
    • LLMs: Mistral-7B-Instruct-v0.3, Phi-4 14B, DeepSeek-R1-14B and DeepSeek-V3 (671B, accessed via API).
    • Encoders: XLM-RoBERTa-base, Albertina-280M (PT) and UmBERTo-110M (IT).
  • Zero-shot results: DeepSeek-V3 reached F1-macro 0.879 on Italian news and 0.758 on Portuguese Telegram, with DeepSeek-R1-14B close behind. Phi-4 was dropped from fine-tuning because of weak zero-shot results.
  • Setup:
    • LoRA used rank 8, alpha 16, learning rate 1e-5 and 600 iterations.
    • Encoders were trained with class-weighted loss at learning rate 2e-5.
    • Local runs used MLX/PyTorch on an Apple M3 Max with 64 GB of memory.
  • Limitations: single seed, a single English zero-shot prompt with no few-shot examples, and no diachronic analysis. The authors recommend human-in-the-loop use only, given the risk of false positives.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ÉvoraEvora · Portugal2 works

Also on the byline: Centro Interdisciplinar de História Culturas e Sociedades.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

Cite

Cite as Erik Bran Marino and Renata Vieira. 2026. Population Replacement Conspiracy Theories Detection on Telegram and News Headlines: Benchmarking LLMs and BERT Models in Portuguese and Italian. Proceedings of the 1st Workshop on Information Disorder (InDor) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4n7qd9ww7ete

@inproceedings{marino2026population,
  title = {{Population Replacement Conspiracy Theories Detection on Telegram and News Headlines: Benchmarking LLMs and BERT Models in Portuguese and Italian}},
  author = {Erik Bran Marino and Renata Vieira},
  booktitle = {Proceedings of the 1st Workshop on Information Disorder (InDor) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4n7qd9ww7ete},
  url = {https://doi.org/10.63317/4n7qd9ww7ete}
}

References

16 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Johannes Langguth, Daniel Thilo Schroeder, Petra Filkuková, Stefan Brenner, Jesper Phillips, and Konstantin Pogorelov. 2023. Coco: an annotated twitter dataset of covid-19 conspiracy theories. Journal of Computational Social Science, 6(2):443–484. find
  2. Davide Bassi, Erik Bran Marino, Renata Vieira, and Martin Pereira. 2025. Old but gold: LLM-based features and shallow learning methods for fine-grained controversy analysis in YouTube Michele Joshua Maggini, Davide Bassi, and Pablo Gamallo. 2025. Detecting hyperpartisanship and rhetorical bias in cl… find
  3. Michele Joshua Maggini, Paloma Piot, Anxo Pérez, Erik Bran Marino, Lúa Santamaría Montesinos, Ana Lisboa Cotovio, Marta Vázquez Abuín, Javier Parapar, and Pablo Gamallo. 2026. Parti-sanlens: A multilingual dataset of hyperpartisan and conspiratorial immigration narratives in european media. In Proce… find
  4. Erik Marino, Jesus M. Benitez-Baleato, and Ana Sofia Ribeiro. 2024. The polarization loop: How emotions drive propagation of disinformation in online media—the case of conspiracy theories and extreme right movements in southern europe. Social Sciences, 13(603). find
  5. Erik Bran Marino, Davide Bassi, and Renata Vieira. 2025. Linguistic markers of population replacement conspiracy theories in youtube immigration discourse. In Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), pages 670–679. find
  6. Ilia Markov, Nikola Ljubešić, Darja Fišer, and Walter Daelemans. 2021. Exploring stylometric and emotion-based features for multilingual cross-domain hate speech detection. In Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, page… find
  7. Ariadna Matamoros-Fernández and Johan Farkas. 2021. Racism, hate speech, and social media: A systematic review and critique. Television & new media, 22(2):205–224. find
  8. Barbara Plank. 2022. The “problem” of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  9. Konstantin Pogorelov, Daniel Thilo Schroeder, Petra Filkuková, Stefan Brenner, and Johannes Langguth. 2021. Wico text: a labeled dataset of conspiracy theory and 5g-corona misinformation tweets. In Proceedings of the 2021 workshop on open challenges in online social networks, pages 21–25. find
  10. Cagri Toraman, Furkan Şahinuç, and Eyup Yilmaz. 2022. Large-scale hate speech detection with cross-domain transfer. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2215–2225, Marseille, France. European Language Resources Association. find
  11. Claire Wardle and Hossein Derakhshan. 2017. Information disorder: Toward an interdisciplinary framework for research and policymaking, volume 27. Council of Europe Strasbourg. find
  12. Maxwell Weinzierl and Sanda Harabagiu. 2022. Vaccinelies: A natural language resource for learning to recognize misinformation about the covid-19 and hpv vaccines. arXiv preprint arXiv:2202.09449. find
  13. Karolina Wojtasik. 2020. Utøya–christchurch–halle. right-wing extremists’ terrorism. Security Dimensions. International and National Studies, (33):84–97. find
  14. Alessandro Miani, Thomas Hills, and Adrian Bangerter. 2022. Loco: The 88-million-word language of conspiracy corpus. Behavior research methods, 54(4):1794–1817. find
  15. Milan Obaidi, Jonas Kunst, Simon Ozer, and Sasha Y Kimel. 2022. The “great replacement” conspiracy: How the perceived ousting of whites can evoke violent extremism and islamophobia. Group Processes & Intergroup Relations, 25(7):1675–1695. find
  16. Endang Wahyu Pamungkas and Viviana Patti. 2019. Cross-domain and cross-lingual abusive language detection: A hybrid approach with deep learning and a multilingual lexicon. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 363… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marino-2026-population-replacement-conspiracy-theories-detection/