Elamur Research⌕
☰
home / papers / 2026 / TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse

TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse

Paper · 2026 · Samuel Richter, Mounika Marreddy and Alexander Mehler

Summary

Conspiracy content on social media is hard to detect from vocabulary alone. It relies on psychological framing, such as blaming powerful agents or hinting at hidden motives. The TTLab team addressed this in SemEval-2026 Task 10 (PsyCoMark). The task's dataset contains Reddit submissions with document-level conspiracy labels and span annotations for five mechanisms: Actor, Action, Effect, Victim and Evidence. The authors dropped "can't tell" items to make the task binary. They fine-tuned BERT, DistilBERT, RoBERTa, XLNet and ELECTRA on three inputs: the raw post, a labeled list of only the marker spans, and both combined. They also stacked the five models under a logistic regression meta-learner. Marker inputs clearly beat raw text, raising the best macro F1 from 0.759 to 0.817 (both BERT). Adding text to markers mostly helped weaker models, and the ensemble scored 0.842. The test set had no gold markers, so the marker results are oracle upper bounds, and the leaderboard run used text only. The study shows narrative structure is a strong signal and motivates predicting markers automatically.

Key points

  • PsyCoMark data: over 4,800 annotated Reddit submissions from more than 190 subreddits, labeled 1,715 conspiracy, 2,263 not conspiracy and 877 can't tell. Annotator agreement was moderate (Krippendorff's alpha 0.58).
  • Text-only macro F1 ranged from 0.727 (XLNet) to 0.759 (BERT). With markers only, BERT reached 0.817.
  • Combining text and markers lifted RoBERTa from 0.761 to 0.805 and XLNet from 0.782 to 0.806, while BERT stayed flat (0.816).
  • The stacking ensemble scored best (P 0.848, R 0.835, F1 0.842). Base models were trained on 80% of the data, and the meta-learner used the remaining 20%.
  • In error analysis, text models over-relied on trigger words like "vaccine" or "UFO". Marker models labeled any Actor–Action–Victim pattern as conspiracy. Activist political language was often misclassified as conspiratorial.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Goethe University FrankfurtFrankfurt am Main · Germany18 works · 373 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

🧭 Topics 1Research subjects and languages the work is about.Psycholinguistics & language acquisition · 102 works

Cite

Cite as Samuel Richter, Mounika Marreddy and Alexander Mehler. 2026. TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.103

@inproceedings{richter2026ttlab,
  title = {{TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse}},
  author = {Samuel Richter, Mounika Marreddy and Alexander Mehler},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.103},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.103}
}

References

9 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  3. Anastasia Giachanou, Bilal Ghanem, and Paolo Rosso. 2023. Detection of conspiracy propagators using psycho-linguistic characteristics. Journal of Information Science, 49:3–17. find
  4. Anastasia Giachanou, Bilal Ghanem, Esteban A. Ríssola, Paolo Rosso, Fabio Crestani, and Daniel Oberski. 2022. The impact of psycholinguistic patterns in discriminating between fake news spreaders and fact checkers. Data Knowledge Engineering, 138:101960. find
  5. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  6. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. SemEval-2026 task 10: Psy-CoMark – psycholinguistic conspiracy marker extraction and detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). find
  7. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. find
  8. Marco Valentino, Leonardo Ranaldi, Giulia Pucci, Federico Ranaldi, and André Freitas. 2026. Semeval-2026 task 11: Disentangling content and formal reasoning in large language models. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  9. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pre-training for language understanding. Advances in neural information processing systems, 32. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/richter-2026-ttlab-semeval-2026-task-10/