Elamur Research⌕
☰
home / papers / 2026 / Who's Behind It? Annotating and Extracting Conspiratorial Actors from German Telegram Posts

Who's Behind It? Annotating and Extracting Conspiratorial Actors from German Telegram Posts

Paper · 2026 · Helena Mihaljevic, Jolanda Beer, Mareike Lisker et al.

Summary

Most computational work on conspiracy theories labels whole documents. It rarely asks who a narrative blames. This paper treats that question as a span-level task. The authors define conspiratorial actors by role: real or imagined people, groups or institutions that a text presents as deliberately pursuing harmful aims. This makes the task harder than NER, because such actors can be pronouns, vague collectives or implicit references. They wrote detailed guidelines over several iterative rounds, with explicit rules for span boundaries. Using these, they added actor spans to the CT-positive part of the German TelCovACT Telegram dataset. Two annotators reached agreement scores of 0.73–0.79. They then fine-tuned German transformers for BIO token classification and ran the resulting model over more than a million posts in the Schwurbelarchiv. The analysis shows a long-tailed actor distribution with a stable core. Antisemitic actors are prominent and persistent, while event-specific actors rise and fade. The work shows that actor extraction is feasible and supports narrative analysis that goes beyond detecting whether a post is conspiratorial.

Key points

  • The corpus contains the 1,251 CT-propagating posts (out of 3,663) from TelCovACT, collected 03/2020–12/2021. 922 of these posts (73.7%) contain at least one actor span, with a mean of 3.45 spans per post.
  • Agreement was measured on 100 extra posts using Gamma, strict SeqEval F1, relaxed overlap F1 and character-overlap F1, giving scores of 0.73–0.79. Of the disagreements, 83% concerned whether something was an actor at all and only 17% concerned span boundaries.
  • For comparison, the PsyCoMark shared task reported actor F1 of 0.28 for its baseline and about 0.40 for its best systems. Its overall marker agreement was very low (α = 0.15).
  • In the Schwurbelarchiv, antisemitic actors account for 15.7% of spans, misogynistic actors for 4.8% and racialized actors for 0.5%. At least 8.5% of actor-bearing posts name only implicit actors.
  • The single most frequent actor is the implicit "they" (German 'sie'), at roughly 6–10% of monthly mentions. 'Media', 'government' and 'politicians' stay stable over time.

Interesting ideas

  • I1. Span annotation targets the smallest self-contained phrase that identifies the actor, rather than marking a head word and expanding it through dependency parsing, which keeps the labels largely independent of syntax.
  • I2. The actor category is defined by narrative role rather than entity type, so the same word can be an actor in one text and not in another; a country name, for instance, may act as an agent or merely name a place.
  • I3. Disagreements are split into "is it an actor at all" versus "where does the span end", and many presence/absence cases turn out to be simple omissions, especially of pronouns and repeated mentions.
  • I4. Agreement is reported with several metrics at once (Gamma, strict, relaxed, character-overlap), which separates disagreement about the meaning region from disagreement about exact boundaries.
  • I5. In preliminary tests, LLMs proposed plausible actors that never appeared in the text, even when instructed to extract only textual spans.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Treebank conversion could store both the minimal phrase and the head-plus-subtree span, so that mismatches between them flag parser attachment errors around modifiers, appositions and coordination.
  • I2 (frames): Agent-like roles in prag frames could be judged against the whole text's storyline rather than a single verb, separating instigators from helpers and from affected groups.
  • I3 (learn): When witness parsers disagree in silver data, the disagreement could be classified the same way, with recall misses (overlooked pronoun mentions) sent to targeted repair rules rather than treated as genuine ambiguity.
  • I4 (learn): Evaluating frame-argument and coreference spans with both strict and overlap scores would show whether a regression comes from wrong boundaries or from wrong content.
  • I5 (learn): Every span from an LLM teacher tier should be mechanically checked as a substring of the input before it enters training data, and invented spans should be logged as a negative control.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Goethe University FrankfurtFrankfurt am Main · Germany18 works · 373 citations🎓HTW Berlin - University of Applied SciencesBerlin · Germany3 works · 1 citation

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Helena Mihaljevic, Jolanda Beer, Mareike Lisker and Katharina Soemer. 2026. Who's Behind It? Annotating and Extracting Conspiratorial Actors from German Telegram Posts. arXiv:2607.04962.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2607.04962 · arXiv 2607.04962

@misc{mihaljevic2026who,
  title = {{Who's Behind It? Annotating and Extracting Conspiratorial Actors from German Telegram Posts}},
  author = {Helena Mihaljevic, Jolanda Beer, Mareike Lisker and Katharina Soemer},
  year = {2026},
  doi = {10.48550/arxiv.2607.04962},
  eprint = {2607.04962},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2607.04962}
}

References

34 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mathias Angermaier, Elisabeth Höldrich, João Pinheiro Neto, and Jana Lasser. 2026. The Schwurbelarchiv: a German-language dataset derived from multimodal Telegram content for the study of conspiracy theories. EPJ Data Science, 15(1). ↗
  2. Michael Barkun. 2003. A Culture of Conspiracy: Apocalyptic Visions in Contemporary America. University of California Press. find
  3. Luke Bates, Max Glockner, Preslav Nakov, and Iryna Gurevych. 2025. ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety. arXiv preprint. ↗
  4. Veronika Batzdorfer. 2024. Conspiracy Narratives on Voat: A Longitudinal Analysis of Cognitive Activation and Evolutionary Psychology Features. In Proceedings of the 16th ACM Web Science Conference, WEBSCI ’24, pages 42–47, New York, NY, USA. Association for Computing Machinery. ↗
  5. Michael Butter. 2018. "Nichts ist, wie es scheint." Über Verschwörungstheorien. Suhrkamp Verlag, Berlin. find
  6. Michael Butter. 2021. Conspiracy Theories – Conspiracy Narratives. DIEGESIS, 10(1). find
  7. Noam Chomsky. 1970. Remarks on Nominalization. Linguistics Club, Indiana University. find
  8. Francesco Corso, Francesco Pierri, and Gianmarco De Francisci Morales. 2025. Conspiracy Theories and Where to Find Them on TikTok. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8346–8362, Vienna, Austria. Association for Computational Linguistics. find
  9. Karen M. Douglas and Robbie M. Sutton. 2023. What Are Conspiracy Theories? A Definitional Approach to Their Correlates, Consequences, and Communication. Annual Review of Psychology, 74(1):271–298. find
  10. Karen M. Douglas, Joseph E. Uscinski, Robbie M. Sutton, Aleksandra Cichocka, Turkay Nefes, Chee Siang Ang, and Farzin Deravi. 2019. Understanding conspiracy theories. Political Psychology, 40(S1):3–35. ↗
  11. Rui Feng, Jie Yuan, and Chao Zhang. 2020. Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction. arXiv preprint. ↗
  12. Mark Fenster. 2008. Conspiracy Theories: Secrecy and Power in American Culture. University of Minnesota Press. find
  13. Max Horkheimer and Theodor W. Adorno. 2002. Elements of Anti-Semitism: Limits of Enlightenment. In Gunzelin Schmid Noerr, editor, Dialectic of Enlightenment: Philosophical Fragments, pages 137– 172. Stanford University Press, Stanford, CA. find
  14. Ray Jackendoff. 1977. X syntax: a study of phrase structure. Cambridge, Mass. : MIT Press. find
  15. Johannes Langguth, Daniel Thilo Schroeder, Petra Filkuková, Stefan Brenner, Jesper Phillips, and Konstantin Pogorelov. 2023. COCO: an annotated Twitter dataset of COVID-19 conspiracy theories. Journal of Computational Social Science, 6(2):443–484. ↗
  16. Grace E. Lee and Aixin Sun. 2019. A Study on Agreement in PICO Span Annotations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1149–1152, Paris France. ACM. find
  17. Zhiwei Liu, Boyang Liu, Paul Thompson, Kailai Yang, and Sophia Ananiadou. 2024. ConspEmoLLM: Conspiracy Theory Detection Using an Emotion-Based Large Language Model. arXiv preprint. ↗
  18. Yann Mathet, Antoine Widlöcher, and Jean-Philippe Métivier. 2015. The Unified and Holistic Method Gamma (γ) for Inter-Annotator Agreement Measure and Alignment. Computational Linguistics, 41(3):437–479. ↗
  19. J. D. Moffitt, Catherine King, and Kathleen M. Carley. 2021. Hunting Conspiracy Theories During the COVID-19 Pandemic. Social Media + Society, 7(3). ↗
  20. Heidi Campana Piva, Shaina Ashraf, Maziar Kianimoghadam Jouneghani, Arianna Longo, Rossana Damiano, Lucie Flek, and Marco Antonio Stranisci. 2026. Conspiracy Frame: a Semiotically-Driven Approach for Conspiracy Theories Detection. arXiv preprint. ↗
  21. Milena Pustet, Elisabeth Steffen, and Helena Mihaljevic. 2024. Detection of Conspiracy Theories Beyond Keyword Bias in German-Language Telegram Using Large Language Models. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 13–27. Association for Computational Linguistics. find
  22. Lance Ramshaw and Mitch Marcus. 1995. Text Chunking using Transformation-Based Learning. In Third Workshop on Very Large Corpora.
  23. Giuseppe Russo, Niklas Stoehr, and Manoel Horta Ribeiro. 2023. ACTI at EVALITA 2023: Automatic Conspiracy Theory Identification Task Overview. In EVALITA 2023: 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, Parma, IT. find
  24. Mattia Samory and Tanushree Mitra. 2018. Conspiracies Online: User Discussions in a Conspiracy Community Following Dramatic Events. Proceedings of the International AAAI Conference on Web and Social Media, 12(1). find
  25. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. SemEval-2026 Task 10: PsyCoMark – Psycholinguistic Conspiracy Marker Extraction and Detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 3628–3639, San Diego, California, USA. Association for Computational Linguistics.
  26. Heidi Schulze, Julian Hohner, Simon Greipl, Maximilian Girgnhuber, Isabell Desta, and Diana Rieger. 2022. Far-right conspiracy groups on fringe platforms: a longitudinal analysis of radicalization dynamics on Telegram. Convergence: The International Journal of Research into New Media Technologies, 28(4):1103–1126. ↗
  27. Shadi Shahsavari, Pavan Holur, Tianyi Wang, Timothy R. Tangherlini, and Vwani Roychowdhury. 2020. Conspiracy in the time of corona: automatic detection of emerging COVID-19 conspiracy theories in social media and the news. Journal of Computational Social Science, 3(2):279–317. ↗
  28. Cécile Simmons, Anna Provan, and Thays Moreira. 2025. Analysing the Spread of Gendered Disinformation in the Run-Up to the 2024 French and US Elections. Policy Brief, Centre for Feminist Foreign Policy, Berlin. find
  29. Elisabeth Steffen. 2025. More than Memes: A Multimodal Topic Modeling Approach to Conspiracy Theories on Telegram. Proceedings of the International AAAI Conference on Web and Social Media, 19:1831– 1844. find
  30. Elisabeth Steffen, Helena Mihaljevic, Milena Pustet, Nyco Bischoff, Maria Do Mar Castro Varela, Yener Bayramoglu, and Bahar Oghalai. 2023. Codes, Patterns and Shapes of Contemporary Online Antisemitism and Conspiracy Narratives – an Annotation Guide and Labeled German-Language Dataset in the Context of COVID-19. In Proceedings of the International AAAI Conference on Web and Social Media, volume 17, pages 1082–1092. find
  31. Timothy R. Tangherlini, Shadi Shahsavari, Behnam Shahbazi, Ehsan Ebrahimzadeh, and Vwani Roychowdhury. 2020. An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, Pizzagate and storytelling on the web. PLOS ONE, 15(6):e0233879. ↗
  32. Jan-Willem van Prooijen and Karen M Douglas. 2017. Conspiracy theories as part of history: The role of societal crisis situations. Memory Studies, 10(3):323– 333. ↗
  33. Matteo Vergani, Alfonso Martinez Arranz, Ryan Scrivens, and Liliana Orellana. 2022. Hate Speech in a Telegram Conspiracy Channel During the First Year of the COVID-19 Pandemic. Social Media + Society, 8(4). ↗
  34. Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2022. Transfer Learning from Semantic Role Labeling to Event Argument Extraction with Template-based Slot Querying. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2627–2647, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mihaljevic-2026-who-s-behind-annotating-extracting/