Elamur Research⌕
☰
home / papers / 2026 / HU at SemEval-2026 Task 6: A Hybrid Discriminative Modeling of Political Clarity and Evasion

HU at SemEval-2026 Task 6: A Hybrid Discriminative Modeling of Political Clarity and Evasion

Paper · 2026 · Taha Munawar, Basil Khan, Arsal Jangda et al.

Summary

This SemEval-2026 Task 6 (CLARITY) system from Habib University labels political interview question–answer pairs at two levels. Subtask 1 assigns a coarse clarity label, and Subtask 2 assigns a fine-grained evasion technique. The data come from 287 White House presidential interviews (2006–2023), split by ChatGPT-3.5 Turbo into single QA pairs: 3,450 for training and 308 for testing. Both label sets are heavily imbalanced.

The authors compared six families of models on limited hardware:

  • long-input ModernBERT classifiers
  • attention-pooled multiple instance learning (MIL) over answer chunks
  • an NLI reformulation
  • a QLoRA-tuned Llama-3.2-3B
  • a hierarchical multi-task gating model
  • HiAGM

For clarity, the best system treats the answer as a premise and checks whether it entails a hypothesis that the question was explicitly answered. Adversarial weight perturbation and blended probability calibration further improve this model. On the hidden test set, the submission reached macro-F1 0.81 (Subtask 1) and 0.45 (Subtask 2).

The work shows that entailment framing and chunk-level reasoning outperform generative and strongly hierarchy-constrained models under tight compute. Data augmentation did not help.

Key points

  • NLI model for clarity: MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli with inputs up to 2048 tokens. NLI labels map to clarity classes:
    • Entailment → Clear Reply
    • Neutral → Ambivalent Reply
    • Contradiction → Clear Non-Reply
  • Gains on the development set: AWP lifted macro-F1 from 0.70 to 0.74. Blending Isotonic Regression with Temperature Scaling (T = 1.23, w_iso = 0.59) raised it to 0.79.
  • Best Subtask 2 model: attention-based MIL with a frozen encoder, reaching 0.43 dev macro-F1. Answers are split into 256-token chunks with a 160-token stride, up to 16 chunks per answer.
  • Weaker alternatives on Subtask 1: Llama-3.2-3B (QLoRA, LoRA r=16, alpha=32) scored 0.57 and overfit early. MTL gating scored 0.64 and HiAGM 0.52.
  • No-gain experiments:
    • None of the augmentation methods (word deletion, oversampling, contextual replacement, back-translation via Gemini 2.5 Flash, few-shot synthetic data) beat the unaugmented best models.
    • Domain pre-training on Canadian parliamentary QA data also gave no measurable improvement.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

Cite

Cite as Taha Munawar, Basil Khan, Arsal Jangda, Sarfaraz Baig, Sandesh Kumar and Abdul Samad. 2026. HU at SemEval-2026 Task 6: A Hybrid Discriminative Modeling of Political Clarity and Evasion. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.34

@inproceedings{munawar2026hu,
  title = {{HU at SemEval-2026 Task 6: A Hybrid Discriminative Modeling of Political Clarity and Evasion}},
  author = {Taha Munawar, Basil Khan, Arsal Jangda, Sarfaraz Baig, Sandesh Kumar and Abdul Samad},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.34},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.34}
}

References

20 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Michael Alvarez and Jacob Morrier. 2026. Measuring the quality of answers in political q&as with large language models. Political Analysis, 34(1):78–95. find
  2. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computati…
  3. Katerina Chatsiou and Slava Jankin Mikhaylov. 2020. Deep learning for political science. In The SAGE Handbook of Research Methods in Political Science and International Relations. SAGE Publications, London, UK. find
  4. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems, volume 36.
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  6. Jonas Gabrielsen, Heidi Jønch-Clausen, and Christina Pontoppidan. 2020. Answering without answering: Shifting as an evasive rhetorical strategy. Journalism, 21(9):1355–1370. find
  7. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach.
  8. ICLR 2020 OpenReview submission. find
  9. Meta AI. 2024. Llama 3.2 3B Instruct Model Card. https://www.llama.com/docs/ model-cards-and-prompt-formats/ llama3_2/. ↗
  10. Zhengyan Shi, Giuseppe Castellucci, Simone Filice, Saar Kuzi, Elad Kravi, Eugene Agichtein, Oleg Rokhlenko, and Shervin Malmasi. 2025. Ambiguity detection and uncertainty calibration for question answering with large language models. In Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 20… find
  11. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2024. “I never said that”: A dataset, taxonomy and baselines on response clarity classification. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5204–5233, Miami, Florida… find
  12. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2026. SemEval-2026 Task 6: CLARITY – unmasking political question evasions. Preprint, arXiv:2603.14027.
  13. Ariel Vázquez Carranza. 2018. Evading and resisting answering: An analysis of mexican spanish news interviews. In Maj-Britt Mosegaard Hansen and Rosina Márquez Reiter, editors, The Pragmatics of Sensitive Activities in Institutional Discourse, pages 65–89. John Benjamins Publishing Company. find
  14. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with disentangled attention. In International Conference on Learning Representations.
  15. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  16. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In International Conference on Learning Representations.
  17. Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, and Yejin Choi. 2023. We’re afraid language models aren’t modeling ambiguity. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 790–807, Si…
  18. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Griffin Thomas Adams, Jeremy Howard, and Iacopo Poli. 2025. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory…
  19. Dongxian Wu, Shu-Tao Xia, and Yisen Wang. 2020. Adversarial weight perturbation helps robust generalization. In Advances in Neural Information Processing Systems. find
  20. Jie Zhou, Chunping Ma, Dingkun Long, Guangwei Xu, Ning Ding, Haoyu Zhang, Pengjun Xie, and Gongshen Liu. 2020. Hierarchy-aware global model for hierarchical text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1106–1117, Online. Assoc… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/munawar-2026-hu-semeval-2026-task-6/