Elamur Research⌕
☰
home / papers / 2026 / GUNLP at SemEval-2026 Task 10: Emotion-Aware Multi-Task Learning for Conspiracy Detection

GUNLP at SemEval-2026 Task 10: Emotion-Aware Multi-Task Learning for Conspiracy Detection

Paper · 2026 · Rojin Ziaei, Mahsa Khoshnoodi and Nazli Goharian

Summary

This SemEval 2026 Task 10 (PsyCoMark) system paper from Georgetown targets Subtask 2: labelling Reddit posts as expressing a conspiratorial belief (Yes), not expressing one (No), or Can't Tell. The authors fine-tune COVID-Twitter-BERT v2 (CT-BERT-v2), a BERT-large model pre-trained on COVID-era social media. They add two classification heads on the [CLS] token and train them together, one for conspiracy stance and one for emotion as an auxiliary task.

To supply emotion supervision, GPT-5 assigned one of six emotion labels to each post using chain-of-thought prompting. GPT-5 also wrote a paraphrase of every post, roughly doubling the ~4,300 labelled examples to ~8,600. Classes were balanced by oversampling.

On the authors' internal validation split, the emotion-aware setup scored 0.87 macro-F1 versus 0.82 for text only. However, the authors treat 0.54 macro-F1 on the public dev set as the more trustworthy estimate. The official test score was 0.34 under a revised protocol that included labels missing from their training data.

The work suggests affective signals may help separate endorsement from mere discussion of conspiracies, though the gap between internal and official scores is large.

Key points

  • Model and training: CT-BERT-v2 with a dual-head multi-task setup and a combined cross-entropy loss. Hyperparameters were AdamW, learning rate 2e-5, batch size 8, 3 epochs, maximum 128 tokens, and no frozen layers.
  • Data pipeline:
    • Posts were rehydrated with the organizers' script.
    • Preprocessing lowercased text and stripped URLs.
    • Classes were balanced by random oversampling with sklearn, followed by a stratified 90/10 train/validation split.
    • Emotion labels were Happy, Sad, Anger, Fear, Disgust, and Surprise.
  • Emotion label check: 150 training examples were annotated by two humans and GPT-5. Cohen's κ was low across pairs: 0.271 between the humans, 0.357 and 0.414 between each human and GPT-5. All three agreed on only 28% of examples.
  • Reported scores:
    • Internal validation: 0.82 macro-F1 (text only) and 0.87 (text + emotion), computed with the pre-revision Codabench pipeline.
    • Public dev set: 0.54 macro-F1, with per-class F1 of Yes 0.655, No 0.752, and Can't Tell 0.222.
    • Official test set: 0.34.
  • Errors and limitations:
    • Can't Tell is the weakest class.
    • Implicit framing, sarcasm, and rhetorical questions cause errors.
    • The emotion labels are LLM-generated rather than gold human labels.
    • Only English Reddit data and only Subtask 2 were covered.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgetown UniversityWashington · United States90 works · 4,732 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSociology and Political Science · 626 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 1,035 works

Cite

Cite as Rojin Ziaei, Mahsa Khoshnoodi and Nazli Goharian. 2026. GUNLP at SemEval-2026 Task 10: Emotion-Aware Multi-Task Learning for Conspiracy Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.342

@inproceedings{ziaei2026gunlp,
  title = {{GUNLP at SemEval-2026 Task 10: Emotion-Aware Multi-Task Learning for Conspiracy Detection}},
  author = {Rojin Ziaei, Mahsa Khoshnoodi and Nazli Goharian},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.342},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.342}
}

References

24 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Oluwaseun Ajao, Deepayan Bhowmik, and Shahrzad Zargari. 2019. Sentiment aware fake news detection on online social networks. In ICASSP 2019 – IEEE International Conference on Acoustics, Speech and Signal Processing, pages 2507–2511. IEEE. find
  2. Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni, Antonio Scala, Guido Caldarelli, H Eugene Stanley, and Walter Quattrociocchi. 2016. The spreading of misinformation online. Proceedings of the national academy of Sciences, 113(3):554–559. find
  3. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4040–4054. Association for Computational Linguist… find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  5. Karen M Douglas and Robbie M Sutton. 2023. What are conspiracy theories? a definitional approach to their correlates, consequences, and communication. Annual review of psychology, 74(1):271–298. find
  6. Ullrich KH Ecker, Stephan Lewandowsky, John Cook, Philipp Schmid, Lisa K Fazio, Nadia Brashier, Panayiota Kendeou, Emily K Vraga, and Michelle A Amazeen. 2022. The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1):13–29. find
  7. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Associatio…
  8. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computatio… find
  9. Pavan Holur, Tianyi Wang, Shadi Shahsavari, Timothy Tangherlini, and Vwani Roychowdhury. 2022. Which side are you on? insider-outsider classification in conspiracy-theoretic social media. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper… find
  10. Martin Müller, Marcel Salathé, and Per E. Kummervold. 2023. Covid-twitter-bert: A natural language processing model to analyse covid-19 content on twitter. Frontiers in Artificial Intelligence, 6:1023281. find
  11. OpenAI. 2025. GPT-5 system card. Technical report, OpenAI. find
  12. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, and 1 others. 2011. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830. find
  13. Lotte Pummerer, Theofilos Gkinopoulos, Karen M. Douglas, Daniel Jolley, and Kai Sassenberg. 2024. The appraisal model of conspiracy theories: Applying appraisal theories to understand emotional and behavioral reactions to conspiracy theories. Psychological Inquiry, 35(3-4):159–178. find
  14. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  15. Damir Korenčić, Berta Chulvi, Xavier Bonet Casals, Alejandro Toselli, Mariona Taulé, and Paolo Rosso. 2024. What distinguishes conspiracy from critical narratives? a computational analysis of oppositional discourse. Expert Systems, 41(11):e13671. find
  16. Timothy R Tangherlini, Shadi Shahsavari, Behnam Shahbazi, Ehsan Ebrahimzadeh, and Vwani Roychowdhury. 2020. An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, pizzagate and storytelling on the web. PloS one, 15(6):e0233879. find
  17. Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019a. Multi-task deep neural networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.
  18. Joseph Uscinski, Adam Enders, Amanda Diekman, John Funchion, Casey Klofstad, Sandra Kuebler, Manohar Murthi, Kamal Premaratne, Michelle Seelig, Daniel Verdear, and 1 others. 2022. The psychological and political correlates of conspiracy theory beliefs. Scientific reports, 12(1):21672. find
  19. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  20. Jan-Willem Van Prooijen and Karen M Douglas. 2017. Conspiracy theories as part of history: The role of societal crisis situations. Memory studies, 10(3):323– 333. find
  21. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. International Conference on Learning Representations (ICLR). find
  22. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. science, 359(6380):1146–1151. find
  23. Jitendra Singh Malik, Hezhe Qiao, Guansong Pang, and Anton van den Hengel. 2025. Deep learning for hate speech detection: a comparative study. International Journal of Data Science and Analytics, 20(4):3053– 3068. find
  24. Saif M. Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 174–184, Melbourne, Australia. Association for Computational…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ziaei-2026-gunlp-semeval-2026-task-10/