Elamur Research⌕
☰
home / papers / 2026 / ChulaNLP at SemEval-2026 Task 6: A Hybrid BERT-LLM Framework for Political Response Clarity and Evasion Detection

ChulaNLP at SemEval-2026 Task 6: A Hybrid BERT-LLM Framework for Political Response Clarity and Evasion Detection

Paper · 2026 · Wisarut Peerachaidecho and Attapol Rutherford

Summary

The paper addresses SemEval-2026 Task 6 (CLARITY). The task asks systems to tell whether politicians' interview responses actually answer the question. Subtask 1 labels each response with a three-way clarity level. Subtask 2 identifies one of nine evasion techniques.

The authors work with English question–answer pairs from White House presidential interview transcripts. A large share of these pairs exceeds the 512-token limit of standard encoders. They compare fine-tuned encoders (DeBERTa, RooseBERT and BigBird) and find that DeBERTa-large, trained on truncated rather than discarded long inputs, works best. Its five most probable labels are then given to an LLM (Kimi-K2, prompted few-shot) as a restricted choice set. The LLM's evasion label is mapped back to a clarity level for Subtask 1.

This combination outperforms either component alone. It ranked 7th in Subtask 1 and 2nd in Subtask 2 of the final evaluation. The work shows that cheap, task-tuned encoders can usefully narrow the decision space for general-purpose LLMs in fine-grained pragmatic classification.

Key points

  • Data cleaning: Of 3,448 training pairs, duplicate annotations with conflicting labels were removed, leaving 3,364 instances. The development set has 308 pairs. About 36.6% of training pairs and 43.5% of development pairs exceed 512 tokens.
  • Encoder comparison: Truncated DeBERTa-large was the strongest encoder on the development set, with Subtask 2 macro F1 of 0.46 and top-5 accuracy of 0.97. BigBird-RoBERTa-base and RooseBERT-base-cased lagged behind.
  • Hybrid on the development set: Few-shot Kimi-K2 restricted to the top-5 candidates reached macro F1 of 0.52 on Subtask 2. Using all nine labels, Kimi-K2 and DeepSeek scored roughly 0.39–0.45.
  • Hybrid on the official test set: The system scored 0.82 on Subtask 1 and 0.61 on Subtask 2. The organizers' fine-tuned LLM baseline scored 0.82 and 0.57 respectively.
  • Per-class difficulty: Explicit answers were easiest to detect (F1 0.63). Deflection (0.22), Implicit (0.26) and General (0.33) were hardest, which the authors attribute to their subtle pragmatic cues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chulalongkorn UniversityBangkok · Thailand6 works · 304 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

Cite

Cite as Wisarut Peerachaidecho and Attapol Rutherford. 2026. ChulaNLP at SemEval-2026 Task 6: A Hybrid BERT-LLM Framework for Political Response Clarity and Evasion Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.236

@inproceedings{peerachaidecho2026chulanlp,
  title = {{ChulaNLP at SemEval-2026 Task 6: A Hybrid BERT-LLM Framework for Political Response Clarity and Evasion Detection}},
  author = {Wisarut Peerachaidecho and Attapol Rutherford},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.236},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.236}
}

References

7 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. Jim T. Dillon. 2025. The practice of questioning. find
  3. Deborah Dore, Elena Cabrio, and Serena Villata. 2025. Roosebert: A new deal for political language modelling. Preprint, arXiv:2508.03250. — [arXiv:2508.03250](https://arxiv.org/abs/2508.03250) find
  4. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhanced bert with disentangled attention. In Proceedings of the 9th International Conference on Learning Representations (ICLR).
  5. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2024. “I never said that”: A dataset, taxonomy and baselines on response clarity classification. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5204–5233, Miami, Florida… find
  6. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2026. Semeval-2026 task 6: Clarity – unmasking political question evasions. Preprint, arXiv:2603.14027.
  7. Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. Big bird: transformers for longer sequences. In Proceedings of the 34th International Conference on Neural Information Processing Sys…

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/peerachaidecho-2026-chulanlp-semeval-2026-task-6/