Elamur Research⌕
☰
home / papers / 2026 / Unibuc-NLP at SemEval-2026 Task 10: Unmasking Conspiracies with Pre-Trained Language Models

Unibuc-NLP at SemEval-2026 Task 10: Unmasking Conspiracies with Pre-Trained Language Models

Paper · 2026 · Teodor-George Marchitan and Liviu Dinu

Summary

The paper presents team Unibuc-NLP's entry for SemEval-2026 Task 10 (PsyCoMark), Subtask 2. The task is to decide whether a Reddit comment expresses belief in a conspiracy. The data covers a range of topics and subreddits, and instead of topical keywords, the labels are grounded in psycholinguistic markers, namely Actor, Victim, Action, Threat, Effect and Evidence.

The authors compare three families of systems:

  • frozen embeddings (Jina-embeddings-v3, HateBERT, BERT-Sentiment), optionally combined with textstat readability features and fed to Optuna-tuned classical classifiers;
  • end-to-end fine-tuning of DeBERTa-v3-base and DistilBERT with a small 128-unit head and seven pooling variants;
  • QLoRA adaptation of the decoder-only models Qwen3-0.6B and Mistral-7B-v0.3.

Fine-tuned DeBERTa-v3-base with [CLS] pooling was their strongest submission, reaching weighted F1 0.74 and placing 29th of 52. After the deadline, they found that weighted layer pooling raised F1 to 0.78 on the validation split. This suggests that cues signalling conspiracy belief are spread across intermediate layers rather than concentrated in the final one.

The study is useful as evidence that compact encoders beat much larger QLoRA-tuned LLMs on a small dataset (about 4K comments).

Key points

  • Data: 4,361 training comments, reduced to 4,316 after removing "Can't tell" labels and unavailable posts. The official validation set has 100 items and the test set 938.
  • Official results: DeBERTa [128] scored 0.74 (rank 29/52). Jina-v3 + textstat scored 0.73 (rank 35), and HateBERT + BERT-Sentiment + textstat scored 0.71 (rank 50).
  • Pooling ablation: Weighted Layer pooling (0.78) outperformed [CLS] and MHAP (both 0.75) and Mean (0.73). Single-head Attention and GlobalMHAP lagged at 0.65.
  • Other ablations:
    • textstat features improved Jina-v3 by only +0.02 and did nothing for the other embedding models.
    • Removing label smoothing or differential learning rates each cost 0.09 F1.
    • The QLoRA models performed poorly (Qwen3-0.6B 0.18, Mistral-7B-v0.3 0.51), which the authors attribute to overfitting.
  • Error analysis:
    • Neutral posts about UFOs or Epstein were wrongly flagged as conspiracy beliefs.
    • Implicit beliefs in general-interest subreddits were often missed.
    • Short comments (<50 words) reached only 0.76 accuracy.
    • Recall was 0.63 for "Yes" versus 0.88 for "No".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BucharestBucharest · Romania21 works · 516 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Teodor-George Marchitan and Liviu Dinu. 2026. Unibuc-NLP at SemEval-2026 Task 10: Unmasking Conspiracies with Pre-Trained Language Models. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.178

@inproceedings{marchitan2026unibuc,
  title = {{Unibuc-NLP at SemEval-2026 Task 10: Unmasking Conspiracies with Pre-Trained Language Models}},
  author = {Teodor-George Marchitan and Liviu Dinu},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.178},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.178}
}

References

14 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tommaso Caselli, Valerio Basile, Jelena Mitrović, and Michael Granitzer. 2021. HateBERT: Retraining BERT for abusive language detection in English. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 17–25, Online. Association for Computational Linguistics.
  2. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Preprint, arXiv:2305.14314.
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Preprint, arXiv:1810.04805.
  4. Debanjan Ghosh, Kai North, Ekaterina Kochmar, Mamoru Komachi, and Marcos Zampieri, editors. 2026. Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). Association for Computational Linguistics, San Diego, United States. 2023. Jina embeddings: A novel set of high-perf… find
  5. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. Preprint, arXiv:2006.03654.
  6. Benjamin D. Horne and Sibel Adali. 2017. This just in: Fake news packs a lot in title, uses simpler, repetitive content in text body, more similar to satire than real news. CoRR, abs/1703.09398. — [arXiv:1703.09398](https://arxiv.org/abs/1703.09398) find
  7. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.
  8. Miquel India, Pooyan Safari, and Javier Hernando. 2019. Self multi-head attention for speaker recognition. Preprint, arXiv:1906.09890. — [arXiv:1906.09890](https://arxiv.org/abs/1906.09890) find
  9. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  10. Ludovic Mompelat, Zuoyu Tian, Amanda Kessler, Matthew Luettgen, Aaryana Rajanala, Sandra Kübler, and Michelle Seelig. 2022. How “loco” is the LOCO corpus? annotating the language of conspiracy theories. In Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022, pages 111–11… find
  11. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  12. Malliga Subramanian, Premjith B, Kogilavani Shanmugavadivel, Santhiya Pandiyan, Balasubramanian Palani, and Bharathi Raja Chakravarthi. 2025. Overview of the shared task on fake news detection in Dravidian languages-DravidianLangTech@NAACL 2025. In Proceedings of the Fifth Workshop on Speech, Vision… find
  13. Zhiming Wang, Kaisheng Yao, Xiaolong Li, and Shuo Fang. 2020. Multi-resolution multi-head attention in deep speaker embedding. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6464–6468. find
  14. Michael Günther, Louis Milliken, Jonathan Geuter, Georgios Mastrapas, Bo Wang, and Han Xiao. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marchitan-2026-unibuc-nlp-semeval-2026-task/