Elamur Research⌕
☰
home / papers / 2026 / CuriosAI at SemEval-2026 Task 10:Hybrid approaches to conspiracy span extraction and conspiracy detection

CuriosAI at SemEval-2026 Task 10:Hybrid approaches to conspiracy span extraction and conspiracy detection

Paper · 2026 · Hiroki Takushima, Fumika Beppu, Aiswariya Manoj Kumar et al.

Summary

This paper describes SoftBank's CuriosAI entry to SemEval-2026 Task 10, which covers two subtasks. The first is extracting conspiracy markers (Actor, Action, Effect, Victim, Evidence) as possibly overlapping text spans. The second is labeling Reddit posts as conspiratorial or not.

For span extraction, the authors cast the problem as multi-label token classification, using a DeBERTa-v3-large encoder with independent sigmoid outputs for each marker type. They also tested several alternatives: other encoders, Empath lexical features, retrieval-based span transfer, dictionary matching and a LoRA-tuned Qwen3-14B. None of these beat the encoder, which the authors partly attribute to the decoder's lack of right-side context.

For detection, they built seven diverse classifiers spanning lexical, embedding, kNN, transformer and LLM approaches. These were combined by a weighted sum of log-odds.

The main finding is that the single model scoring best on the development set did not perform best on the test set, while the ensemble gave the top test score. This points to model diversity and calibration-aware aggregation as safer choices than picking one dev-optimal model under distribution shift.

Key points

  • Marker extraction is scored with token-level IoU (threshold 0.5). Multi-label DeBERTa-v3-large reached macro F1 0.2297 (dev) and 0.1813 (test), compared with 0.2033 and 0.1400 for the single-label baseline. RoBERTa-base and ELECTRA-base scored lower.
  • The best single detector on dev was DeBERTa (pretrained) + kNN (weighted F1 0.8318), but it dropped to 0.7372 on test.
  • The final logit ensemble scored lower on dev (0.8182) but reached the best test weighted F1 of 0.7620. In this ensemble, Qwen3-14B received weight 0.05, the other six models shared the rest equally, and the decision threshold was 0.55.
  • Relabeling the training data with GPT-5 (confidence above 0.95) changed only 14 training instances, and models trained on it gave no clear gain.
  • The error analysis identifies three main problems: span boundary mismatches, confusion between Action and Effect, and false positives caused by conspiracy vocabulary appearing without the post endorsing it.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢SoftBank Corp.Tokyo · Japan2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Hiroki Takushima, Fumika Beppu, Aiswariya Manoj Kumar, Yuki Shibata, Takayuki Hori and Daichi Yamaga. 2026. CuriosAI at SemEval-2026 Task 10:Hybrid approaches to conspiracy span extraction and conspiracy detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.71

@inproceedings{takushima2026curiosai,
  title = {{CuriosAI at SemEval-2026 Task 10:Hybrid approaches to conspiracy span extraction and conspiracy detection}},
  author = {Hiroki Takushima, Fumika Beppu, Aiswariya Manoj Kumar, Yuki Shibata, Takayuki Hori and Daichi Yamaga},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.71},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.71}
}

References

13 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations (ICLR). find
  2. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Ivan Petrov, and Preslav Nakov. 2020. Semeval-2020 task 11: Detection of propaganda techniques in news articles. In Proceedings of the 14th International Workshop on Semantic Evaluation (SemEval).
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT.
  4. Ethan Fast, Binbin Chen, and Michael S. Bernstein. 2016. Empath: Understanding topic signals in large-scale text. In Proceedings of CHI. find
  5. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In International Conference on Machine Learning (ICML). find
  6. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. In International Conference on Learning Representations (ICLR).
  7. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
  8. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS). find
  9. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  10. John Pavlopoulos, Jeffrey Sorensen, Léo Laugier, and Ion Androutsopoulos. 2021. Semeval-2021 task 5: Toxic spans detection. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval).
  11. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  12. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS).
  13. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/takushima-2026-curiosai-semeval-2026-task-10/