Elamur Research⌕
☰
home / papers / 2026 / TTLab at StanceEval-2026: A Cloze-Style Prompting Approach for Arabic-Language Stance Detection (CLASP-Ar)

TTLab at StanceEval-2026: A Cloze-Style Prompting Approach for Arabic-Language Stance Detection (CLASP-Ar)

Summary

CLASP-Ar is the TTLab entry to StanceEval-2026. The task is to decide whether an Arabic post from X supports a given target topic, opposes it, or takes no position. Earlier top systems for Arabic stance relied on multitask heads, ensembles or target-specific models. The authors instead train a single encoder, aubmindlab/bert-base-arabertv02-twitter, to fill a masked slot. The prompt contains the target, a predicted sentiment label and the text. The MLM head may only choose among three one-character label tokens.

Training has two stages:

  • Intermediate pre-training on ArabicStance-X, with sentiment labels supplied by the SARF classifier.
  • Fine-tuning on Mawqif-XT, extended with synthetic None examples and trained with a class-weighted loss.

The system scores F_avg2 of 71.36% on Track 1 (seen targets) and 74.14% on Track 2 (unseen targets). The authors credit the stronger unseen-target result mainly to the diverse pre-training topics. The work suggests that auxiliary signals can be added cheaply as prompt text rather than as extra task heads, keeping one transferable model.

Key points

  • Prompt and labels: the prompt is laid out as Target / Sentiment / Stance [MASK] / Text. A verbalizer maps Favor, Against and None to the tokens 'F', 'A' and 'N', and the softmax runs only over these three. This avoids class names being split into several subwords.
  • Data:
    • Mawqif-XT has 3,502 training and 619 dev instances, and Favor heavily outnumbers None.
    • ArabicStance-X has 14,477 tweets over 17 topics, of which 11,500 were used for two epochs of intermediate pre-training.
    • 1,815 extra None examples were created by pairing texts with unrelated targets.
  • Sentiment as input: adding sentiment raised dev F_avg2 by 1.38 points (84.90 → 86.28). Adding sarcasm as well gave the best F_avg3 (72.50), but sentiment alone was the most consistent gain.
  • Imbalance fixes:
    • Class weighting lifted F_none by roughly 7 points and F_avg3 by roughly 2 points over the baseline, and by up to 9 F1 points on None per the conclusion.
    • The synthetic None examples mostly helped the Against class while hurting None.
  • Sentiment source: SARF, a MARBERTv2-based multi-view model, beat a frozen XLM-RoBERTa sentiment model as the label provider (dev F_avg2 81.50 vs 80.96). The None class remains weak and unstable, with standard deviation above 10 points in some setups.

Interesting ideas

  • I1. Restricting a masked-LM prediction to a tiny set of single-token labels turns classification into reuse of the pretrained output layer, so no new head is learned and the label decision stays readable as a token choice. — For Mova (learn): when an LLM or encoder teacher labels silver data, forcing its answer into a closed token set would make teacher verdicts comparable across witnesses and easy to audit.
  • I2. Feeding a predicted auxiliary label (sentiment) into the input as plain text, instead of training a parallel task head, lets one model use a side signal without architectural changes. — For Mova (frames): a cheap upstream polarity or irony guess could be passed as an explicit, logged feature into speech-act or implicature decisions, keeping the reason for each decision traceable.
  • I3. Building "no relation" examples by pairing an input with a mismatched target is an almost free negative generator. The paper also shows its cost: it sharpened one boundary but blurred the very class it was meant to help. — For Mova (learn): shuffled text–question or text–frame pairs could serve as negative controls in evaluation, while checking per-class effects rather than trusting the aggregate.
  • I4. An intermediate training pass on a broader, multi-topic dataset before task fine-tuning was credited with better transfer to unseen targets. — For Mova (answer): readers for fables or school tests could first be tuned on a varied pool of question types, so new topics are less of a shock.
  • I5. Reporting per-class F1 with multi-seed spread exposed that the minority class was highly volatile, which a single averaged score would hide. — For Mova (learn): evaluation reports for rare frame roles or rare UD relations should include seed variance, so apparent gains on small classes are not mistaken for real ones.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2609.29733 · arXiv 2609.29733

@misc{verma2026ttlab,
  title = {{TTLab at StanceEval-2026: A Cloze-Style Prompting Approach for Arabic-Language Stance Detection (CLASP-Ar)}},
  author = {Verma, Bhuvanesh and Abusaleh, Ali and Mehler, Alexander},
  year = {2026},
  doi = {10.48550/arxiv.2609.29733},
  eprint = {2609.29733},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2609.29733}
}

References

23 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ali Abusaleh, Bhuvanesh Verma, and Alexander Mehler. 2026. TTLab at AraSentEval: okSARF sentiment analysis via root-based fusion for multi-dialectal Arabic. In The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks, pages 262–268, Palma, Mallorca (Spain). Association for Computational Linguistics. find
  2. Rasha Albalawi, Nuha Albadi, Hamzah Luqman, Saad Ezzini, Maram Kurdi, Asma Yamani, Ahmed Ashraf, Maged Al-Shaibani, and Nora Alturayeif. 2026a. Stanceeval-2026: The second stance detection shared task. In Proceedings of the Fourth Arabic Natural Language Processing Conference (ArabicNLP 2026), Budapest, Hungary. Association for Computational Linguistics. find
  3. Rasha Albalawi, Nuha Albadi, Hamzah Luqman, Maram Kurdi, Saad Ezzini, Asma Yamani, and Ahmed Ashraf. 2026b. Mawqif-xt: An arabic benchmark dataset for cross-target stance detection. Preprint, arXiv:2608.09539. ↗
  4. Mamoun Alghaslan and Khaled Almutairy. 2024. MGKM at StanceEval2024 fine-tuning large language models for Arabic stance detection. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 816–822, Bangkok, Thailand. Association for Computational Linguistics. find our card
  5. Ali Alkhathlan, Faris Alahmadi, Faris Kateb, and Hend Al-Khalifa. 2025. Constructing and evaluating ArabicStanceX: a social media dataset for arabic stance detection. Front. Artif. Intell., 8:1615800. find
  6. Nora Alturayeif, Hamzah Luqman, Zaid Alyafeai, and Asma Yamani. 2024. Stanceeval 2024: The first arabic stance detection shared task. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 774–782. find our card
  7. Nora Saleh Alturayeif, Hamzah Abdullah Luqman, and Moataz Aly Kamaleldin Ahmed. 2022. Mawqif: A multi-label Arabic dataset for target-specific stance detection. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 174–184, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find
  8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. find our card
  9. Mohamed Badran, Mo’men Hamdy, Marwan Torki, and Nagwa M El-Makky. 2024. Alexunlp-bh at stanceeval2024: Multiple contrastive losses ensemble strategy with multi-task learning for stance detection in arabic. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 823–827. find
  10. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017. Stance classification of context-dependent claims. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261. find
  11. Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. 2022. XLM-T: Multilingual language models in Twitter for sentiment analysis and beyond. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 258–266, Marseille, France. European Language Resources Association. find our card
  12. Saptarshi Ghosh, Kripabandhu Ghosh, Debasis Ganguly, Tanmoy Chakraborty, Gareth JF Jones, Marie-Francine Moens, and Muhammad Imran. 2018. Exploitation of social media for emergency relief and preparedness: Recent research and trends. Information Systems Frontiers, 20(5):901–907. find
  13. Ahmed Hasanaath and Aisha Alansari. 2024. Stancecrafters at stanceeval2024: Multi-task stance detection using bert ensemble with attention based aggregation. In Proceedings of the Second Arabic Natural Language Processing Conference, pages 811–815. find our card
  14. Emily Kubin and Christian Von Sikorski. 2021. The role of (social) media in political polarization: a systematic review. Annals of the International Communication Association, 45(3):188–206. find
  15. Veronica Lynn, Salvatore Giorgi, Niranjan Balasubramanian, and H Andrew Schwartz. 2019. Tweet classification without the tweet: An empirical examination of user versus document attributes. In Proceedings of the third workshop on natural language processing and computational social science, pages 18–28. find
  16. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pages 31–41. find our card
  17. Kamalich Muniz-Rodriguez, Sylvia K Ofori, Lauren C Bayliss, Jessica S Schwind, Kadiatou Diallo, Manyun Liu, Jingjing Yin, Gerardo Chowell, and Isaac Chun-Hai Fung. 2020. Social media use in emergency response to natural disasters: a systematic review with a public health perspective. Disaster medicine and public health preparedness, 14(1):139–149. find
  18. Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th conference of the European chapter of the association for computational linguistics: main volume, pages 255–269. find our card
  19. Jay J Van Bavel, Steve Rathje, Elizabeth Harris, Claire Robertson, and Anni Sternisko. 2021. How social media shapes polarization. Trends in cognitive sciences, 25(11):913–916. find
  20. Wan Wei, Xiao Zhang, Xuqin Liu, Wei Chen, and Tengjiao Wang. 2016. pkudblab at semeval-2016 task 6: A specific convolutional neural network system for effective stance detection. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pages 384–388. find our card
  21. Brandon T Willard and Rémi Louf. 2023. Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702. ↗
  22. Ruifeng Xu, Yu Zhou, Dongyin Wu, Lin Gui, Jiachen Du, and Yun Xue. 2016. Overview of nlpcc shared task 4: Stance detection in chinese microblogs. In International Conference on Computer Processing of Oriental Languages, pages 907–916. Springer. find
  23. Bowen Zhang, Genan Dai, Fuqiang Niu, Nan Yin, Xiaomao Fan, Senzhang Wang, Xiaochun Cao, and Hu Huang. 2024. A survey of stance detection on social media: New directions and perspectives. arXiv preprint arXiv:2409.15690. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/verma-2026-ttlab-stanceeval-2026-cloze-style/