Elamur Research⌕
☰
home / papers / 2026 / CLaC at SemEval-2026 Task 6: Response Clarity Detection in Political Discourse

CLaC at SemEval-2026 Task 6: Response Clarity Detection in Political Discourse

Paper · 2026 · Nawar Turk, Lucas Miquet-Westphal and Leila Kosseim

Summary

The CLaC team built a system for SemEval-2026 Task 6 (CLARITY), which asks how clearly U.S. presidents answer interview questions. Task 1 sorts answers into 3 Clarity classes. Task 2 assigns 9 finer Evasion labels. The data are 3,993 English question-answer pairs, and the labels are strongly imbalanced.

The authors compared three families of approaches:

  • eight transformer encoders, each tuned through a four-stage pipeline;
  • a Longformer variant for inputs longer than 512 tokens;
  • prompted LLMs, tested under several prompting strategies.

The strongest setup was a majority-vote ensemble of GPT-5, Gemini-3-Flash-Preview and Qwen3-235B-Instruct. It used 27 class-balanced examples, the interviewer's full turn as context, and label definitions extended with their Evasion subcategories. Because time ran out, this Task 1 setup was reused unchanged for Task 2.

The ensemble scored 80 macro-F1 on Task 1 (9th of 41) and 59 on Task 2 (3rd of 33). Prompting without any fine-tuning beat the tuned encoders, especially on minority classes. The work suggests that for small, hierarchically labelled tasks, prompt and ensemble design can matter more than updating model weights.

Key points

  • Dataset: 3,448 train, 308 dev and 237 test pairs. The dev set carries three independent Evasion annotations per instance.
  • Encoder tuning: Unfreezing only the top 25% of layers averaged 59.1 dev macro-F1, versus 41.2 for full fine-tuning. DeBERTa-v3-base was the best single encoder (65.1).
  • Dev-set comparison: Mixing English and multilingual encoders gave 70.5, Longformer-base-4096 reached 64.3, and the LLM ensemble reached 78.1. Extra interviewer context helped the LLMs but slightly hurt the encoders.
  • Prompt refinement: Adding Level-2 subcategory descriptions to the label definitions raised GPT-5 from 64.4 to 71.5. Among open-weight LLMs, model size did not predict accuracy.
  • Main error source: Both ensembles most often confused Clear Reply with Ambivalent, the same boundary where human annotators agree least (κ = 0.65). Code and prompts are released on GitHub (CLaC-Lab).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Concordia UniversityMontreal · Canada9 works · 673 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Nawar Turk, Lucas Miquet-Westphal and Leila Kosseim. 2026. CLaC at SemEval-2026 Task 6: Response Clarity Detection in Political Discourse. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.238

@inproceedings{turk2026clac,
  title = {{CLaC at SemEval-2026 Task 6: Response Clarity Detection in Political Discourse}},
  author = {Nawar Turk, Lucas Miquet-Westphal and Leila Kosseim},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.238},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.238}
}

References

15 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, and 132 others. 2025. Llama-Nemotron: Efficient Reasoning Models. Preprint, arXiv:2505.00949.
  2. Peter Bull. 2003. The Microanalysis of Political Communication: Claptrap and Ambiguity. Routledge. find
  3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  5. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, and 557 others. 2024. The Llama 3 Herd of Models. Preprint, arXiv:2407.21783.
  6. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In Proceedings of Eleventh International Conference on Learning Representations.
  7. Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large Language Models are Zero-Shot Reasoners. In Advances in Neural Information Processing Systems, volume 35, pages 22199–22213. Curran Associates, Inc.
  8. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Preprint, arXiv:1907.11692.
  9. Parameswary Rasiah. 2010. A Framework for the Systematic Analysis of Evasion in Parliamentary Discourse. Journal of Pragmatics, 42(3):664–680. find
  10. Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, and 27 others. 2025. The Prompt Report: A Systematic Survey of Prompt Engineering Techniques. Preprint, arXiv:2406.06608.
  11. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, and 480 others. 2025. OpenAI GPT-5 System Card. Preprint, arXiv:2601.03267.
  12. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2024. “I Never Said That”: A Dataset, Taxonomy and Baselines on Response Clarity Classification. In Findings of the Association for Computational Linguistics (EMNLP 2024), pages 5204–5233, Miami, Florid… find
  13. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2026. SemEval-2026 Task 6: CLARITY – Unmasking Political Question Evasions. Preprint, arXiv:2603.14027.
  14. Xiaohan Xu, Chongyang Tao, Tao Shen, Can Xu, Hongbo Xu, Guodong Long, Jian-Guang Lou, and Shuai Ma. 2024. Re-Reading Improves Reasoning in Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), pages 15549– 15575, Miami, Florida… find
  15. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, and 56 others. 2025. Qwen3 Technical Report. Preprint, arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/turk-2026-clac-semeval-2026-task-6/