Elamur Research⌕
☰
home / papers / 2026 / KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection

KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection

Paper · 2026 · Archie Sage and Salvatore Greco

Summary

This system paper describes the KCLarity team's entry to CLARITY, a SemEval 2026 shared task on automatically judging how clearly politicians answer interview questions. The task has two parts: a three-way clarity label (Task 1) and a nine-way evasion technique label (Task 2). The data is the QEvasion dataset of US presidential interviews, where the two label sets are linked by a hierarchical taxonomy.

The authors compare two approaches:

  • Fine-tuned encoders (RoBERTa, DeBERTa-v3). They test predicting clarity directly against predicting evasion and mapping it upward to clarity. They also try class-weighted loss, two input formats, two data-splitting schemes and person-name masking.
  • Zero-shot decoder-only LLMs, both open-weight and commercial.

Both clarity formulations score about the same, so the team prefers the evasion-first setup, because one model then covers both tasks. RoBERTa-large performs best on the public test split. Zero-shot GPT-5.2, however, holds up better on the hidden official test set, which points to a trade-off between in-domain fit and robustness. Most auxiliary training tweaks gave no consistent improvement. The authors present their simple, reproducible pipelines as baselines for this difficult, annotator-disputed problem.

Key points

  • Data: QEvasion contains 3,448 training and 308 test instances. Each test item has three independent evasion annotations. Ambivalent dominates the clarity labels at 59.2%, against 30.5% Clear Reply and 10.3% Clear Non-Reply.
  • Official results: the best submission was zero-shot GPT-5.2. It ranked 22nd of 44 in Task 1 (macro F1 = 0.74) and 13th of 33 in Task 2 (macro F1 = 0.50).
  • Encoder results: on the public test set, RoBERTa-large with evasion-based clarity reached macro F1 of about 0.661 for clarity. Its average per-annotator evasion F1 was about 0.37.
  • Evaluation: evasion predictions are scored separately against each annotator, plus an ACC_match metric counting a prediction as correct if it agrees with any one annotator.
  • Main error sources: confusion centres on Ambivalent versus the other clarity classes, and among the evasion labels Implicit, General, Deflection and Partial. These areas mirror where human annotators also disagree.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓King's College LondonLondon · United Kingdom42 works · 1,617 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

Cite

Cite as Archie Sage and Salvatore Greco. 2026. KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.408

@inproceedings{sage2026kclarity,
  title = {{KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection}},
  author = {Archie Sage and Salvatore Greco},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.408},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.408}
}

References

27 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. Preprint, arXiv:2111.09543.
  2. AI@Meta. 2024. Llama 3 model card. find
  3. Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, and Alexandra Uma. 2021. We need to consider disagreement in evaluation. In Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, pages 15–21, Online. Association for Computa…
  4. Janet Beavin Bavelas, Alex Black, Lisa Bryson, and Jennifer Mullett. 1988. Political equivocation: A situational explanation. Journal of Language and Social psychology, 7(2):137–145. find
  5. Janet Beavin Bavelas, Alex Black, Nicole Chovil, and Jennifer Mullett. 1990. Equivocal communication. Sage Publications, Inc. find
  6. Peter Bull. 1998. Equivocation theory and news interviews. Journal of Language and Social Psychology, 17(1):36–51. find
  7. Peter Bull. 2003. The microanalysis of political communication: Claptrap and ambiguity. Routledge. find
  8. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  10. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107–1128, Miami…
  11. Eve Fleisig, Rediet Abebe, and Dan Klein. 2023. When the majority is wrong: Modeling annotator disagreement for subjective tasks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6715–6726, Singapore. Association for Computational Linguistics. find
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, and et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  13. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  14. King’s College London. 2025. King’s computational research, engineering and technology environment (create). https://doi.org/10.18742/rnvf-m076. Retrieved December, 2025. — [doi:10.18742/rnvf-m076](https://doi.org/10.18742/rnvf-m076) ↗
  15. Khaled Al Nuaimi, Gautier Marti, Alexis Marchal, and Andreas Henschel. 2025. Detecting evasive answers in financial Q&A: A psychological discourse taxonomy and lightweight baselines. In Proceedings of The 10th Workshop on Financial Technology and Natural Language Processing, pages 191–196, Suzhou, C… find
  16. Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivière, Alina Beygelzimer, Florence d’Alché Buc, Emily Fox, and Hugo Larochelle. 2020. Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program). Preprint, arXiv:2003.12206. find
  17. Parameswary Rasiah. 2010. A framework for the systematic analysis of evasion in parliamentary discourse. Journal of Pragmatics, 42(3):664–680. find
  18. Archie Sage, Jeroen Keppens, and Helen Yannakoudakis. 2025. A survey of cognitive distortion detection and classification in NLP. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 14884–14899, Suzhou, China. Association for Computational Linguistics. find
  19. Harald Semmelrock, Tony Ross-Hellauer, Simone Kopeinik, Dieter Theiler, Armin Haberl, Stefan Thalmann, and Dominik Kowald. 2025. Reproducibility in machine learning-based research: Overview, barriers and drivers. Preprint, arXiv:2406.14325. find
  20. Sagarika Shreevastava and Peter Foltz. 2021. Detecting cognitive distortions from patient-therapist interactions. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pages 151–158, Online. Association for Computational Linguistics. find
  21. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksandra Spyra, Alex Baker-Whitcomb, Alex Beutel, Alex Karpenko, and 465 others. 2025… — OpenAI GPT-5 System Card
  22. Gemma Team. 2025a. Gemma 3. find
  23. Qwen Team. 2025b. Qwen3 technical report. Preprint, arXiv:2505.09388.
  24. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2024. "i never said that": A dataset, taxonomy and baselines on response clarity classification. Preprint, arXiv:2409.13879. find
  25. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2026. Semeval-2026 task 6: Clarity – unmasking political question evasions. Preprint, arXiv:2603.14027.
  26. Paul Watzlawick, Janet Beavin Bavelas, and Don D Jackson. 2011. Pragmatics of human communication: A study of interactional patterns, pathologies and paradoxes. WW Norton & Company. find
  27. Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. 2021. A robustly optimized BERT pre-training approach with post-training. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, pages 1218–1227, Huhhot, China. Chinese Information Processing Society of China.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sage-2026-kclarity-semeval-2026-task-6/