Elamur Research⌕
☰
home / papers / 2026 / Codezone Research Group at AraSentEval Shared Task: Arabic Sentiment Swap beyond Negation Prepending, Benchmarking Multilingual T5 against Large Language Models on the MA'AKS Corpus

Codezone Research Group at AraSentEval Shared Task: Arabic Sentiment Swap beyond Negation Prepending, Benchmarking Multilingual T5 against Large Language Models on the MA'AKS Corpus

Paper · 2026 · Abdulkadir Shehu Bichi and Sarah Yassine

Summary

This shared-task paper addresses Arabic sentiment swap: rewriting a sentence so that its polarity flips while the remaining content stays intact. The authors built ASBN-MT5, a fine-tuned mT5-small (300M parameters), for AraSentEval 2026 Subtask 2. The task data derive from MA'AKS, a parallel corpus of sentiment-swapped Arabic covering MSA plus Egyptian and Gulf dialects. The model receives a prompt written in Arabic script that names the target polarity. The training setup is designed to fit on a single NVIDIA T4. Evaluation goes beyond exact match to include:

  • a negation-marker heuristic for detecting flips,
  • character-level similarity,
  • the rates of empty outputs and of outputs that simply copy the source.

On validation, about 59% of outputs flip in each direction, and similarity to the source stays near 0.955. However, more than a third of outputs are unchanged copies. On the official test leaderboard, the system placed fourth with low style accuracy but a relatively strong BLEU. The authors take this as evidence that shallow edits, such as prepending negation particles, are not enough, and they suggest Arabic-centric LLMs as the next step.

Key points

  • Data. The dataset has 6,263 training, 1,315 validation and 646 test pairs; the paper gives the total as 7,578. Texts come from product reviews, book critiques and hospitality feedback, with an average source length of 11 words.
  • Training setup.
    • Optimisation: AdamW at 5×10⁻⁵, 5 epochs, maximum length of 96 tokens.
    • Memory: per-device batch 2 with 8-step gradient accumulation (effective batch 16). Gradient checkpointing roughly halves peak memory, from about 4.5 GB to about 2.2 GB.
    • Decoding: beam size 4 with a no-repeat bigram constraint.
  • Validation results.
    • Flip rate: 59.46% for Pos→Neg and 58.48% for Neg→Pos.
    • Exact match: only 1.53% and 0.80%.
    • Similarity: average about 0.955, median 0.977.
    • Failures: no empty outputs, but 37.64% of outputs are identical to the source.
  • Test leaderboard. The team (asbichi362) ranked 4th of 4 with SSA 0.237, BLEU 30.79 and chrF 54.08. The winner, yumnahamdy, reached SSA 0.755, BLEU 43.00 and chrF 65.36. Teams ranked 2nd and 3rd had higher SSA but lower BLEU (27.22 and 20.33).
  • Stated limitations. The flip heuristic captures only lexical negation changes. mT5-small is far smaller than current Arabic LLMs. Test references were withheld by the organisers.

Interesting ideas

  • I1. Writing the control token (the desired polarity) in the same script as the data lets the encoder reuse its pretrained subword knowledge instead of learning an unfamiliar English label.
  • I2. Copy-of-source rate and empty-output rate are reported as standalone quality indicators, because they expose a model that "succeeds" on similarity metrics by doing nothing.
  • I3. The paper shows surface metrics and meaning metrics pulling apart: BLEU stays competitive while a classifier reports that polarity was rarely changed.
  • I4. Flip success is judged cheaply by checking whether negation markers appear or vanish in the expected direction, but the authors admit this misses non-lexical sentiment shifts.
  • I5. Combining gradient checkpointing with gradient accumulation brought a seq2seq transformer down to about 2 GB on a single T4 without shrinking the effective batch.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Mova's generator could express control signals (tense, polarity, register) using the vocabulary of the target language, so a pretrained surface realiser can treat them as meaningful input.
  • I2 (learn): every rewrite or repair evaluation could include an identity baseline as a negative control, flagging any system whose scores are matched by trivially copying the input.
  • I3 (back): the text → graph → text compressor should be scored on the recovered meaning graph (for example, the polarity of each event), not only on n-gram overlap with the source.
  • I4 (frames): a frame-level check, asking whether the event or evaluation in the frame now carries the opposite polarity, would replace the marker heuristic with an explainable test of meaning change.
  • I5 (tree): the same pair of tricks could let gparse train larger encoders on modest GPUs while keeping batch statistics stable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Lebanese UniversityBeirut · Lebanon4 works · 49 citations🎓Vivekananda Global UniversityJaipur · India2 works

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Abdulkadir Shehu Bichi and Sarah Yassine. 2026. Codezone Research Group at AraSentEval Shared Task: Arabic Sentiment Swap beyond Negation Prepending, Benchmarking Multilingual T5 against Large Language Models on the MA'AKS Corpus. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2qp7m87fbeti

@inproceedings{bichi2026codezone,
  title = {{Codezone Research Group at AraSentEval Shared Task: Arabic Sentiment Swap beyond Negation Prepending, Benchmarking Multilingual T5 against Large Language Models on the MA'AKS Corpus}},
  author = {Abdulkadir Shehu Bichi and Sarah Yassine},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/2qp7m87fbeti},
  url = {https://doi.org/10.63317/2qp7m87fbeti}
}

References

19 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Fahad Abdu, Raed Mughaus, Shadi Abudalfa, Mohammed Ahmed, and Ahmed Abdelali. 2025. An empirical evaluation of Arabic text formality transfer: A comparative study. Language Resources and Evaluation. ↗
  2. Fayez Aljomah, Lama Aldhafeeri, Mohammed Alfadel, Saad Alshahrani, Qaisar Abbas, and 1 others. 2025. Enhancing Arabic sentiment analysis with pre-trained CAMeLBERT: A case study on noisy texts. Computers, Materials & Continua, 84(3):5317–5335. find
  3. Hamzah Almaqtari, Fang Zeng, and Abdulaziz Mohammed. 2024. Enhancing Arabic sentiment analysis of consumer reviews: Machine learning and deep learning methods based on NLP. Algorithms, 17(11):495. ↗
  4. Saad Ezzini, Paul Rayson, Shadi Abudalfa, Maram Alharbi, and Mo El-Haj. 2026. AraSentEval: A shared task on sentiment analysis and swapping in arabic. In Proceedings of the 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7), co-located with the Language Resources and Evaluation Conference (LREC 2026), Palma, Mallorca, Spain. find
  5. Ibrahim Abu Farha and Walid Magdy. 2024. Arabic sentiment analysis: Challenges and opportunities. ACM Computing Surveys. find
  6. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Mosen Alharthi, Bang An, and 1 others. 2024. AceGPT, localizing large language models in Arabic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 8139–8163, Mexico City, Mexico. Association for Computational Linguistics.
  7. Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: A survey. Computational Linguistics, 48(1):155–205. ↗
  8. Anis Koubaa, Adel Ammar, Lahouari Ghouti, Omer Nacar, and Serry Sibaee. 2025. ArabianGPT: An Arabic language adaptation of GPT-2. In Generative AI and Large Language Models: Opportunities, Challenges, and Applications, volume 1214 of Studies in Computational Intelligence. Springer, Cham. ↗
  9. Yuning Lyu, Tongshuang Luo, Jinliang Shi, Todd Hollon, and Honglak Lee. 2023. Fine-grained text style transfer with diffusion-based language models. In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023), pages 65–74. Association for Computational Linguistics. find
  10. Sarah Mankarious and Ayah Zirikly. 2025. Style transfer as bias mitigation: Diffusion models for synthetic mental health text for Arabic. In arXiv preprint arXiv:2601.14124. ↗
  11. Raed Mughaus, Shadi Abudalfa, Hamzah Luqman, Fahad Abdu, Mohammed AlAli, Nawaf Al-Dowayan, and Ahmed Abdelali. 2026. Ma’aks: manually-curated parallel dataset for Arabic text sentiment swap. Language Resources and Evaluation, 60(1):1. Sourabrata Mukherjee, Atul Kr. ↗
  12. Ojha, Pruthwik Mujadia, and Ondrej Dusek. 2024. Are large language models actually good at text style transfer? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand. Association for Computational Linguistics. find
  13. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730–27744.
  14. Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W. Black. 2018. Style transfer through back-translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 866–876, Melbourne, Australia. Association for Computational Linguistics.
  15. Neha Sengupta, Sunil Kumar Siddiqui, Basil Bore-gowda, and 1 others. 2023. Jais and Jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. arXiv preprint arXiv:2308.16149. ↗
  16. Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in Neural Information Processing Systems, volume 30, pages 6830– 6841. ↗
  17. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 483–498, Online. Association for Computational Linguistics. ↗
  18. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In Proceedings of the 8th International Conference on Learning Representations (ICLR). ↗
  19. Mohamed Zouidine and Mohamed Khalil. 2025. Large language models for Arabic sentiment analysis and machine translation. Engineering, Technology & Applied Science Research, 15(2):20737–20742. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bichi-2026-codezone-research-group-arasenteval-shared/