Elamur Research⌕
☰
home / papers / 2026 / PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media

PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media

Paper · 2026 · Michele Joshua Maggini, Paloma Piot, Anxo Pérez et al.

Summary

Tools for spotting hyperpartisan reporting and Population Replacement Conspiracy Theories (PRCT) mostly rely on English, US-focused data. They also tend to model partisanship, stance and rhetorical manipulation as unrelated tasks. The authors address this with PartisanLens, a corpus of 1617 immigration-related news headlines in Spanish, Italian and Portuguese. Headlines were gathered from Media Cloud through keyword queries and pre-screened with Llama 3.1 8B so that rare categories would appear in the sample. Native-speaker experts then labelled them jointly for hyperpartisanship, PRCT, stance and three rhetorical bias types. The benchmark is used to test LLM classifiers in zero- and few-shot settings, to check how well LLM annotators reproduce human labels, and to condition models on annotators' socio-economic and ideological profiles. Few-shot prompting beats zero-shot. PRCT proves the easiest task and stance the hardest. LLM annotators stay below human agreement, and persona conditioning helps mostly for PRCT. The resource fills a gap for European, non-English research on polarising migration narratives.

Key points

  • Headlines come from 329 outlets in Spain, Italy and Portugal, published 2020–2024. They were selected from about 400,000 deduplicated entries, of which under 1% were PRCT-related.
  • Labels cover hyperpartisan, PRCT, stance, and the rhetorical biases loaded language, appeal to fear and name-calling. Nine native-speaker annotators spent roughly 66 hours and reached substantial agreement on the main tasks.
  • There are clear language differences: Italian carries the strongest PRCT signal, while Portuguese has more hyperpartisan headlines.
  • Larger models perform better, but carefully selected few-shot examples can close that gap. Even Llama 3.3-70B does not reach human-level agreement as an annotator.
  • Persona prompts derived from political compass and prism tests improve alignment with individual annotators on PRCT, give mixed results for hyperpartisanship, and barely change stance. Data, code and prompts are available at github.com/MichJoM/PartisanLens.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Universidad de La RiojaLogroño · Spain24 works · 125 citations🎓Universidade da CoruñaA Coruña · Spain14 works · 328 citations🎓Universidade de Santiago de CompostelaSantiago de Compostela · Spain13 works · 364 citations🎓University of ÉvoraEvora · Portugal2 works

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Michele Joshua Maggini, Paloma Piot, Anxo Pérez, Erik Bran Marino, Lúa Santamaría Montesinos, Ana Lisboa Cotovio, Marta Vázquez Abuín, Javier Parapar and Pablo Gamallo. 2026. PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.53

@inproceedings{maggini2026partisanlens,
  title = {{PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media}},
  author = {Michele Joshua Maggini, Paloma Piot, Anxo Pérez, Erik Bran Marino, Lúa Santamaría Montesinos, Ana Lisboa Cotovio, Marta Vázquez Abuín, Javier Parapar and Pablo Gamallo},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.53},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.53}
}

References

54 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xavier Aragall. 2018. 2018: Migration in the Mediterranean; Situation, Context and Evolution. https://www.iemed.org/publication/2018-migration-in-the-mediterranean-situation-context-and-evolution. ↗
  2. Dennis Assenmacher, Paloma Piot, Katarina Laken, David Jurgens, and Claudia Wagner. 2025. Beyond the explicit: A bilingual dataset for dehumanization detection in social media. Preprint, arXiv:2510.18582. — [arXiv:2510.18582](https://arxiv.org/abs/2510.18582) find
  3. Joachim Baumann, Paul Röttger, Aleksandra Urman, Albert Wendsjö, Flor Miriam Plaza del Arco, Johannes B. Gruber, and Dirk Hovy. 2025. Large language model hacking: Quantifying the hidden risks of using llms for text annotation. Preprint, arXiv:2509.08825. — [arXiv:2509.08825](https://arxiv.org/abs/2509.08825) find
  4. Tilman Beck, Hendrik Schuff, Anne Lauscher, and Iryna Gurevych. 2024. Sensitivity, performance, robustness: Deconstructing the effect of sociodemographic prompting. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers),…
  5. Emily M Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. find
  6. Laura Biester, Vanita Sharma, Ashkan Kazemi, Naihao Deng, Steven Wilson, and Rada Mihalcea. 2022. Analyzing the effects of annotator gender across NLP tasks. In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022, pages 10–19, Marseille, France. European Language Resources A… find
  7. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). Internationa…
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Preprint, arXiv:1810.04805.
  9. Stephan Dochow-Sondershaus and Céline Teney. 2024. Opinion polarization of immigration and eu attitudes between social classes–the limiting role of working class dissensus. European Societies, 26(5):1363– 1394. find
  10. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, ..., and Zhiwei Zhao. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  11. Soufiane Essahli, Oussama Sarsar, Ahmed Bentajer, Anas Motii, and Imane Fouad. 2025. Fakezero: Real-time, privacy-preserving misinformation detection for facebook and x. Preprint, arXiv:2510.25932. — [arXiv:2510.25932](https://arxiv.org/abs/2510.25932) find
  12. Leon Fröhling, Gianluca Demartini, and Dennis Assenmacher. 2025. Personas with attitudes: Controlling LLMs for diverse data annotation. In Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH), pages 468–481, Vienna, Austria. Association for Computational Linguistics. find
  13. Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. 2024. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094.
  14. Jennifer Golbeck, Matthew Mauriello, Brooke Auxier, Keval H. Bhanushali, Christopher Bonk, Mohamed Amine Bouzaghrane, Cody Buntain, Riya Chanduka, Paul Cheakalos, Jennine B. Everett, Waleed Falak, Carl Gieringer, Jack Graney, Kelly M. Hoffman, Lindsay Huth, Zhenya Ma, Mayanka Jha, Misbah Khan, Varsh… find
  15. Michael Heseltine and Bernhard Clemm von Hohenberg. 2024. Large language models as a substitute for human experts in annotating political text. Research & Politics, 11(1):20531680241236239. find
  16. Tomáš Horych, Christoph Mandl, Terry Ruas, Andre Greiner-Petter, Bela Gipp, Akiko Aizawa, and Timo Spinde. 2025. The promises and pitfalls of LLM annotations in dataset labeling: a case study on media bias detection. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 137… find
  17. Md Zobaer Hossain, Md Ashraful Rahman, Md Saiful Islam, and Sudipta Kar. 2020. BanFakeNews: A dataset for detecting fake news in Bangla. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2862–2871, Marseille, France. European Language Resources Association. find
  18. Johannes Kiesel, Maria Mestre, Rishabh Shukla, Emmanuel Vincent, Payam Adineh, David Corney, Benno Stein, and Martin Potthast. 2019. SemEval-2019 task 4: Hyperpartisan news detection. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 829–839, Minneapolis, Minnesota, USA…
  19. Jongin Kim, Byeo Rhee Bak, Aditya Agrawal, Jiaxi Wu, Veronika Wirtz, Traci Hong, and Derry Wijaya. 2023. COVID-19 vaccine misinformation in middle income countries. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3903–3915, Singapore. Association for… find
  20. Nadejda Komendantova, Dmitry Erokhin, and Teresa Albano. 2023. Misinformation and its impact on contested policy issues: The example of migration discourses. Societies, 13(7). find
  21. Alex Kulesza. 2012. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning, 5(2–3):123–286. find
  22. Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700.
  23. J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159. find
  24. Johannes Langguth, Daniel Thilo Schroeder, Petra Filkuková, Stefan Brenner, Jesper Phillips, and Konstantin Pogorelov. 2023. Coco: an annotated twitter dataset of covid-19 conspiracy theories. Journal of Computational Social Science, 6(2):443–484. find
  25. João A. Leite, Olesya Razuvayevskaya, Kalina Bontcheva, and Carolina Scarton. 2024. Euvsdisinfo: A dataset for multilingual detection of pro-kremlin disinformation in news articles. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM ’24, page 5380–5… find
  26. Xuannan Liu, Zekun Li, Peipei Li, Huaibo Huang, Shuhan Xia, Xing Cui, Linzhi Huang, Weihong Deng, and Zhaofeng He. 2025. Mmfakebench: A mixed-source multimodal misinformation detection benchmark for lvlms. In ICLR. find
  27. Yiwei Luo, Dallas Card, and Dan Jurafsky. 2020. Detecting stance in media on global warming. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3296–3315, Online. Association for Computational Linguistics. find
  28. Michele Joshua Maggini, Davide Bassi, and Pablo Gamallo. 2025a. Detecting hyperpartisanship and rhetorical bias in climate journalism: A sentence-level Italian dataset. In Proceedings of the 2nd Workshop on Natural Language Processing Meets Climate Change (ClimateNLP 2025), pages 168–187, Vienna, Au… find
  29. Michele Joshua Maggini, Davide Bassi, Paloma Piot, Gaël Dias, and Pablo Gamallo Otero. 2025b. A systematic review of automated hyperpartisan news detection. PLOS ONE, 20(2):1–39. find
  30. Erik Bran Marino, Jesus M Benitez-Baleato, and Ana Sofia Ribeiro. 2024. The polarization loop: How emotions drive propagation of disinformation in online media—the case of conspiracy theories and extreme right movements in southern europe. Social Sciences, 13(11):603. find
  31. Mark Mets, Andres Karjus, Indrek Ibrus, and Maximilian Schich. 2024. Automated stance detection in complex topics and small languages: The challenging case of immigration in polarizing news media. PLOS ONE, 19(4):1–16.
  32. Alessandro Miani, Thomas Hills, and Adrian Bangerter. 2021. Loco: The 88-million-word language of conspiracy corpus. Behavior research methods, pages 1–24. find
  33. Mistral AI team. 2025. Mistral NeMo \| Mistral AI. https://mistral.ai/news/mistral-nemo. Accessed: 09/01/2025. ↗
  34. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Computational Ling…
  35. Saif M Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. 2017. Stance and sentiment in tweets. ACM Transactions on Internet Technology (TOIT), 17(3):1–23. find
  36. Salar Mohtaj, Ata Nizamoglu, Premtim Sahitaj, Vera Schmitt, Charlott Jakob, and Sebastian Möller. 2024. Newspolyml: Multi-lingual european news fake assessment dataset. In Proceedings of the 3rd ACM International Workshop on Multimedia AI against Disinformation, pages 82–90. find
  37. Sagnik Mukherjee, Muhammad Farid Adilazuarda, Sunayana Sitaram, Kalika Bali, Alham Fikri Aji, and Monojit Choudhury. 2024. Cultural conditioning or placebo? on the effectiveness of socio-demographic prompting. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,… find
  38. Dan S. Nielsen and Ryan McConville. 2022. Mumin: A large-scale multilingual multimodal fact-checked misinformation social network dataset. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’22, page 3141–3153, New York, NY, USA.… find
  39. OECD. 2023. Indicators of Immigrant Integration 2023 — oecd.org. https://www.oecd.org/en/publications/ indicators-of-immigrant-integration-2023_ 1d5020a6-en.html. [Accessed 06-06-2025]. ↗
  40. Paloma Piot, Patricia Martín-Rodilla, and Javier Parapar. 2025a. Personalisation or prejudice? addressing geographic bias in hate speech detection using debias tuning in large language models. Preprint, arXiv:2505.02252. — [arXiv:2505.02252](https://arxiv.org/abs/2505.02252) find
  41. Paloma Piot, David Otero, Patricia Martín-Rodilla, and Javier Parapar. 2025b. Can llms evaluate what they cannot annotate? revisiting llm reliability in hate speech detection. Preprint, arXiv:2512.09662.
  42. Paloma Piot and Javier Parapar. 2025. Towards Efficient and Explainable Hate Speech Detection via Model Distillation, page 376–392. Springer Nature Switzerland. find
  43. Jakub Piskorski, Nicolas Stefanovitch, Nikolaos Nikolaidis, Giovanni Da San Martino, and Preslav Nakov. 2023. Multilingual multifaceted understanding of online news in terms of genre, framing, and persuasion techniques. In Proceedings of the 61st Annual Meeting of the Association for Computational L… find
  44. Barbara Plank. 2022. The “problem” of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  45. Konstantin Pogorelov, Daniel Thilo Schroeder, Petra Filkuková, Stefan Brenner, and Johannes Langguth. 2021. Wico text: A labeled dataset of conspiracy theory and 5g-corona misinformation tweets. In Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks, OASIS ’21, page 21–25,… find
  46. Oscar Sainz, Jon Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. 2023. NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10776–10787, Singap…
  47. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Ling…
  48. Johannes Schäfer, Aidan Combs, Christopher Bagdon, Jiahui Li, Nadine Probol, Lynn Greschner, Sean Papay, Yarik Menchaca Resendiz, Aswathy Velutharam-bath, Amelie Wuehrl, Sabine Weber, and Roman Klinger. 2025. Which demographics do LLMs default to during annotation? In Proceedings of the 63rd Annual… find
  49. Mark Sedgwick. 2024. The great replacement narrative: fear, anxiety and loathing across the west. Politics, Religion & Ideology, 25(4):548–562. find
  50. Jennifer Stromer-Galley, Brian McKernan, Saklain Zaman, Chinmay Maganur, and Sampada Regmi. 2025. The efficacy of large language models and crowd annotation for accurate content analysis of political social media messages. Social Science Computer Review, 0(0):08944393251334977. find
  51. Petter Törnberg. 2024. Large language models outperform expert coders and supervised classifiers at annotating political social media messages. Social Science Computer Review, 0(2024):08944393241286471. find
  52. Livia Van Vliet, Petter Törnberg, and Justus Uitermark. 2020. The twitter parliamentarian database: Analyzing twitter politics across 26 countries. PLoS one, 15(9):e0237073. find
  53. Inna Vogel and Peter Jiang. 2019. Fake news detection with the new german dataset "germanfakenc". In Digital Libraries for Open Knowledge - 23rd International Conference on Theory and Practice of Digital Libraries, TPDL 2019, Oslo, Norway, September 9-12, 2019, Proceedings, pages 288–295. find
  54. Jiang Zhang, Qiong Wu, Yiming Xu, Cheng Cao, Zheng Du, and Konstantinos Psounis. 2024. Efficient toxic content detection by bootstrapping and distilling large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(19):21779– 21787. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maggini-2026-partisanlens-multilingual-dataset-hyperpartisan-conspiratorial/