Elamur Research⌕
☰
home / papers / 2026 / VARH-AI at SemEval-2026 Task 10: Exploiting Architectural Diversity with Transformer-SSM Ensembles and Confidence-Based Iterative Refinement for Conspiracy Detection

VARH-AI at SemEval-2026 Task 10: Exploiting Architectural Diversity with Transformer-SSM Ensembles and Confidence-Based Iterative Refinement for Conspiracy Detection

Paper · 2026 · Hritav Solanki, Shubham Sharma, Manish Prasad et al.

Summary

This paper presents the VARH-AI system for Subtask 2 of SemEval-2026 Task 10 (PsyCoMark), which asks whether an English Reddit submission statement is conspiracy-related. The authors assume that attention-based and state-space architectures make different kinds of errors, so they combine them in one committee. The ensemble has four sources: DeBERTa-v3-large, Twitter-RoBERTa-large, DeBERTa-v3-small, and a Bidirectional Mamba (BiMamba) classifier. BiMamba uses masked sequence reversal and attention pooling.

The system adds three further techniques:

  • Safety-switched multi-task learning: a BIO marker-tagging loss is applied only to samples with gold annotations.
  • Conservative pseudo-labeling: a committee assigns labels only to examples it scores with high confidence.
  • Test-time augmentation: predictions are averaged over the original and lowercased text.

The final system reached 0.78 weighted F1 and placed 4th on the CodaBench leaderboard; the best team scored 0.89. Ablations indicate that architectural diversity contributes the largest improvement. A non-Transformer SSM proves useful as a complementary ensemble member even though it is not stronger than the Transformers on its own.

Key points

  • Ensemble design: sources are weighted 2.0 (DeBERTa-v3-large), 1.5 (Twitter-RoBERTa), 1.0 (BiMamba, built on mamba-130m-hf), and 1.0 (DeBERTa-v3-small). Each source contributes 5 cross-validation folds, giving 20 committee members.
  • Pseudo-labeling: a 13-checkpoint committee kept only predictions with confidence of at least 0.80 or at most 0.20. It accepted 399 of the 938 unlabeled test-pool items (95 "Yes", 304 "No"), producing a training set of 3,552 instances.
  • Ablation results:
    • Single models score 0.73–0.74 weighted F1.
    • Pseudo-labels add about 0.01.
    • 1,000 LLM-generated synthetic posts give no improvement.
    • The ensemble grows from 0.76 to 0.77 to 0.78 as sources are added.
    • SS-MTL alone improves F1 from 0.735 to 0.744.
    • A decision threshold of 0.55 works best.
  • Exploratory CoT distillation (not used in the final system): Llama-3.1-70B matched 98.8% of the gold "No" labels, whereas its agreement on gold "Yes" labels was just 31.4%. Too few positives remained to fine-tune Phi-2 effectively.
  • Stated limitations: the system is evaluated only on English Reddit data, has a high computational cost, and Subtask 1 is not evaluated. Code is published on GitHub (MambaEnsembleSE26).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Birla Institute of Technology and Science, PilaniPilāni · India16 works · 41,879 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Hritav Solanki, Shubham Sharma, Manish Prasad, Rakhi Agrawal and Yashvardhan Sharma. 2026. VARH-AI at SemEval-2026 Task 10: Exploiting Architectural Diversity with Transformer-SSM Ensembles and Confidence-Based Iterative Refinement for Conspiracy Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.138

@inproceedings{solanki2026varh,
  title = {{VARH-AI at SemEval-2026 Task 10: Exploiting Architectural Diversity with Transformer-SSM Ensembles and Confidence-Based Iterative Refinement for Conspiracy Detection}},
  author = {Hritav Solanki, Shubham Sharma, Manish Prasad, Rakhi Agrawal and Yashvardhan Sharma},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.138},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.138}
}

References

13 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Karen M. Douglas, Joseph E. Uscinski, Robbie M. Sutton, Aleksandra Cichocka, Turkay Nefes, Chee Siang Ang, and Farzin Deravi. 2019. Understanding conspiracy theories. Political Psychology, 40(S1):3–35. find
  2. M. A. Ganaie, Minghui Hu, A. K. Malik, M. Tanveer, and P. N. Suganthan. 2022. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151. find
  3. Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. ArXiv:2312.00752. find
  4. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021a. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. Preprint, arXiv:2111.09543.
  5. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021b. DeBERTa: Decoding-enhanced BERT with disentangled attention. In International Conference on Learning Representations.
  6. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. ArXiv:1907.11692.
  7. Daniel Loureiro, Kiamehr Rezaee, Talayeh Riahi, Francesco Barbieri, Leonardo Neves, Luis Espinosa Anke, and Jose Camacho-Collados. 2023. Tweet insights: A visualization platform to extract temporal insights from twitter. arXiv preprint arXiv:2308.02142. ArXiv:2308.02142. — [arXiv:2308.02142](https://arxiv.org/abs/2308.02142) find
  8. Jonathan D. Moffitt, Christopher King, and Kathleen M. Carley. 2021. Hunting for conspiracy theories on social media. Journal of Information Technology & Politics, 18(3):304–319. find
  9. Konstantin Pogorelov, Daniel Thilo Schroeder, Stefan Brenner, and Johannes Langguth. 2021. WICO graph: A labeled dataset of conspiracy theories and its graph analysis. In Proceedings of the 2021 Workshop on Open Challenges in Online Social Networks, pages 29–34. ACM. find
  10. Mattia Samory, Felix Soldner, and Veronika Batzdor-fer. 2026. SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
  11. Fatima Shahid and Omar Alonso. 2022. Detecting and understanding conspiracy theories on social media. In Proceedings of the ACM Web Conference 2022, pages 2684–2694. ACM. find
  12. Jan-Willem van Prooijen and Michele Acker. 2015. The psychology of conspiracy theories. Current Directions in Psychological Science, 24(6):425–430. find
  13. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/solanki-2026-varh-ai-semeval-2026-task/